The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Mental Models of AI Agents in a Cooperative Game Setting
Abstract
As more and more forms of AI become prevalent, it becomes increasingly important to understand how people develop mental models of these systems. In this work we study people's mental models of AI in a cooperative word guessing game. We run think-aloud studies in which people play the game with an AI agent; through thematic analysis we identify features of the mental models developed by participants. In a large-scale study we have participants play the game with the AI agent online and use a post-game survey to probe their mental model. We find that those who win more often have better estimates of the AI agent's abilities. We present three components for modeling AI systems, propose that understanding the underlying technology is insufficient for developing appropriate conceptual models (analysis of behavior is also necessary), and suggest future work for studying the revision of mental models over time.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“it more importantly shows how the AI's confidence is a simple, yet salient feature via which users create a mental model of the AI's global behavior [23].”
From Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2021) · cites this paper as [23]
In the citing paper
“It has been empirically shown that users with better mental models of their AI teammates are more successful; in particular, knowledge of the error boundary enables the user to know when to trust automation and when to override it [3,12].”
From Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative (2021) · cites this paper as [12]
In the citing paper
“It has been empirically shown that users with better mental models of their AI teammates are more successful; in particular, knowledge of the error boundary enables the user to know when to trust automation and when to override it [3,12].”
From Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative (2021) · cites this paper as [12]
Citations
Cites
Philosophical Investigations: Philosophische Untersuchungen
Burnheim, J., & Philosophical Studies, W. S. F. T. N. U. O. I. (1954) · Philosophical Studies · doi:10.5840/philstudies1954421 · Cited by 2,902
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11164 · Cited by 2,312
The University of South Florida free association, rhyme, and word fragment norms
Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195588 · Cited by 1,929
Some observations on mental models
Norman, D. A. (1983) · Medical Entomology and Zoology · Cited by 1,549
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03975 · Cited by 758
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 1×
“Yin et. al [29] look at the effect of stated and observed accuracy of machine learning models on people’s trust of the system, finding that the effect of stated accuracy can change depending on the observed accuracy.”
BackgroundMental models, conceptual models, and modelling
Greca, I. M., & Moreira, M. A. (2000) · International Journal of Science Education · doi:10.1080/095006900289976 · Cited by 518
Research works printed but not linked: 5
- Nolan Bard, Jakob N Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, and others. 2019. The Hanabi Challenge: A New Frontier for AI Research. arXiv preprint arXiv:1902.00506 (2019). · arXiv
- Virginia Braun and Victoria Clarke. 2012. Thematic analysis. In APA handbook of research methods in psychology, Vol. 2. American Psychological Association, 57--71.
- Tim Miller. 2018. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence (2018).
- Judy Hanwen Shen, Matthias Hofer, Bjarke Felbo, and Roger Levy. 2018. Comparing Models of Associative Meaning: An Empirical Investigation of Reference in Simple Language Games. arXiv preprint arXiv:1810.03717 (2018). · arXiv
- Ludwig Wittgenstein. 2009. Philosophical investigations. John Wiley & Sons.
Cited by
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
“” While this observation empirically shows that end-users develop mental model of trust in AI-assisted decision making [5], it more importantly shows how the AI’s confdence is a simple, yet salient feature via which users create a mental model of the AI’s global behavior [23].”
BackgroundDesign Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2021) · arXiv · doi:10.48550/arxiv.2109.06977 · Cited by 635
“demonstrated that people can construct mental models of AI through repeated interactions with a model that help them understand the AI’s knowledge distributions, local behavior, and global behavior.[13] Using prompts to generate image evidence of AI knowledge is also a way of reducing uncertainty with AI, one of the f…”
BackgroundDesign Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501825 · Cited by 639
“demonstrated that people can construct mental models of AI through repeated interactions with a model that help them understand the AI’s knowledge distributions, local behavior, and global behavior.[13] Using prompts to generate image evidence of AI knowledge is also a way of reducing uncertainty with AI, one of the f…”
BackgroundLai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
“The most common objective metric is time taken on the task [1, 8, 20, 23, 36, 38, 41, 56, 60, 63, 70, 71, 79, 91, 96, 106, 108].”
BackgroundMethod
Metadata
- Authors
- Gero, K. I., Ashktorab, Z., Dugan, C., Pan, Q., Johnson, J., Geyer, W., Ruiz, M., Miller, S., Millen, D. R., Campbell, M., Kumaravel, S., & Zhang, W.
- Affiliations
- Columbia University (US)
- IBM (United States) (US)
- IBM Research - Cambridge (United States) (US)
- ORCID
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-12
- OpenAlex
- W3030109276
- Cited by
- 121
- References
- 26
- Citations by year
- 2026: 11 · 2025: 30 · 2024: 21 · 2023: 23 · 2022: 20 · 2021: 15 · 2020: 1
- FWCI
- 20.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- AI agents, cooperative games, word guessing game, human-AI interaction, think-aloud study, conceptual model, model revision, human-AI collaboration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mental Models of AI Agents in a Cooperative Game Setting. https://app.cochat.ai/papers/10.1145/3313831.3376316
- The paper
- Gero, K. I., Ashktorab, Z., Dugan, C., Pan, Q., Johnson, J., Geyer, W., Ruiz, M., Miller, S., Millen, D. R., Campbell, M., Kumaravel, S., & Zhang, W. (2020). Mental Models of AI Agents in a Cooperative Game Setting. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376316
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.