The DOI matches its Crossref record; 71 of the 76 references it cites were found in a registry or an index; 23 of the 38 sources its claims cite were checked for support.
Improving Human-AI Collaboration With Descriptions of AI Behavior
Open accessAbstract
People work with AI systems to improve their decision making, but often under- or over-rely on AI predictions and perform worse than they would have unassisted. To help people appropriately rely on AI aids, we propose showing them behavior descriptions, details of how AI systems perform on subgroups of instances. We tested the efficacy of behavior descriptions through user studies with 225 participants in three distinct domains: fake review detection, satellite image classification, and bird classification. We found that behavior descriptions can increase human-AI accuracy through two mechanisms: helping people identify AI failures and increasing people's reliance on the AI when it is more accurate. These findings highlight the importance of people's mental models in human-AI collaboration and show that informing people of high-level AI behaviors can significantly improve AI-assisted decision making.
Overview
CoChat Summary
TL;DR
This paper tests whether showing people short, subgroup-level descriptions of an AI’s behavior helps them collaborate better with the AI. In three Amazon Mechanical Turk classification studies with 225 participants, behavior descriptions improved performance in two tasks, but did not change participants’ reported trust, helpfulness, or willingness to reuse the AI.
Key findings
- Across the three tasks, behavior descriptions significantly improved overall accuracy in 2 of 3 domains: reviews and birds; the satellite task showed no significant overall improvement.
- In the reviews task, accuracy differed across conditions with F(2,64) = 9.18, p < 0.001; the only significant pairwise difference was between No AI and AI + BD (p < 0.001, 95% CI [0.06, 0.22]).
- In the birds task, accuracy differed across conditions with F(2,65) = 3.98, p = 0.023; the only significant pairwise difference was between No AI and AI + BD (p = 0.048, 95% CI [0.00, 0.16]), and this produced complementary human-AI accuracy.
- The proposed mechanism in H2 was only partially supported: in reviews, there were no accuracy differences for the behavior-description groups, while in birds, group 2 showed a significant difference (F(2,65) = 6.81, p = 0.002) and supported H2; overall, increased accuracy also came from greater reliance on the AI in the main group.
Methods
- Three between-subjects human-subject experiments were run on Amazon Mechanical Turk with 225 U.S. participants total, across fake review detection, satellite image classification, and bird classification.
- Each participant labeled 30 instances: 20 from the main group and 5 each from two behavior-description groups; in the AI + BD condition, participants saw a behavior description for those subgroup instances.
- Main analyses used ANOVA with Tukey post-hoc tests for accuracy, Mann-Whitney U tests with Bonferroni corrections for Likert items, and linear models to test learning effects.
Limitations
- The paper states that participants were Amazon Mechanical Turk workers with limited domain expertise, so results may differ for domain experts/professionals; the authors explicitly say future studies should test that.
- The study was a controlled simulated setting, so effects may differ in real-world consequential settings such as radiology; the paper explicitly notes this.
- The paper says it tested one specific type of behavior description—subgroup accuracy—so the generality to other behavior-description formats is not established; the text makes evident this is a design-scope limitation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig. 1 Don Norman's mental model framework [43] describes how designers use their mental models to implement systems. End-users then interact with the systems and develop their own mental models of how they believe the systems work. While this process is similar for AI models, a key difference is that an AI is not a direct representation of a developer's intent, but a stochastic model learned from data. This means that (1) AI developers themselves have to make sense of what an AI system has learned through testing and iteration. Subsequently, they can encode these insights as (2) behavior descriptions , details of how an AI performs on subgroups of instances, that can be shown to end-users to improve human-AI collaboration.](https://figures.cochat.ai/paper-figures/8a25e79489564f9f8cbc1e6321036c0b/improving-human-ai-collaboration-with-descriptions-of-ai-behavior-figure-1.png)









Verification
Claims the paper makes
14 Valid · 3 Not valid · 12 Not checked · 84 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“First, modern AI systems are often black-box models for which humans cannot see how or why the model made a prediction [54].”
What the sources say
- Supportedcites [54] [54] Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, 5 (May 2019), 206-215. https://doi.org/10.1038/s42256-019-0048-x - source
The Introduction says that many machine-learning models used for high-stakes predictions are black boxes that do not explain their predictions in a way humans can understand. This supports the citing sentence’s paraphrase. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite the insights post-hoc model explanations can provide end-users, they have been found to have a small or even detrimental effect on the performance of AI-assisted decision making [17, 34].”
What the sources say
- Partly supportedcites [17] [17] Eric Chu, Deb Roy, and Jacob Andreas. 2020. Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction. arXiv:2007.12248 [cs, stat] (July 2020). http://arxiv.org/abs/2007.12248 arXiv: 2007.12248. - sourceExperimental — accuracy not yet measured
Chu et al. (2020) support the 'small effect' half of the claim: Section 3.1 (p. 5) states 'The addition of explanations did not improve accuracy' and that explanation-without-prediction treatments 'resulted in small and non statistically significant effects', and the Conclusion (p. 8) reports that showing explanations 'did not have a significant effect on accuracy, trust, or understanding'. However, our copy contains no finding that explanations were detrimental to performance in their own trial — the ordering explain-strong < explain-spurious < explain-random is explicitly called small and non-significant (p. 5), and even faulty explanations 'did not significantly decrease human trust'. The detrimental/hindering results the claim bundles in appear only as reports of other work in Related Work (p. 8), e.g. accuracy decreases in [49] and transparency hindering error detection in [34]. (LLM, conf=0.72)
- Partly supportedcites [34] [34] Vivian Lai and Chenhao Tan. 2019. On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, Atlanta GA USA, 29-38. https://doi.org/10.1145/3287560.3287590 - sourceExperimental — accuracy not yet measured
Section 4.1 (p. 6) reports that explanations alone slightly improved human accuracy in the paper’s deception-detection task, supporting the ‘small effect’ part. It does not report a detrimental effect on decision-making performance; the Concluding Discussion (p. 9) raises a separate concern that irrelevant explanations can increase trust in machine predictions. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In another empirical study, Hopkins and Booth [23] found that practitioners would avoid or limit model updates due to concerns about breaking their models and introducing more failures.”
What the sources say
- Supportedcites Hopkins and Booth [23] [23] Aspen Hopkins and Serena Booth. 2021. Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development. In AIES 2021 - Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. Association for Computing Machinery, Inc, 134-145. https://doi.org/10.1145/3461702.3462527 - source
Section 4.2.1 (Bias Mitigation Through Diversity or Personalization) states that teams delayed introducing new, diverse data 'until user requirements deemed it strictly necessary,' and that 'By doing so, they minimized how frequently they updated-and potentially broke-their models,' describing this slow, cautious strategy as 'common in production' — directly supporting avoidance/limitation of updates for fear of breaking models. Section 4.1.3 corroborates the failure concern: R13's retraining on each new data iteration produced 'unpredictable performance drops' and they weighed decomposing the model to scale 'without the risk of compromising past performance.' Section 4.5.4 adds that less experienced organizations 'might not afford retraining and replacing models,' with R3 calling ML development 'notably slow and defensive.' (LLM, conf=0.88)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“We chose this task since it has been used in previous studies of human-AI collaboration to test the effect of explanations [34] and tutorials [33].”
What the sources say
- Supportedcites [34] [34] Vivian Lai and Chenhao Tan. 2019. On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, Atlanta GA USA, 29-38. https://doi.org/10.1145/3287560.3287590 - source
The paper describes using deceptive-review detection to study human decisions assisted by machine-learning models (Related Work, p. 3). It tests explanations of machine predictions as a form of assistance (ACM Reference Format, p. 2). The tutorials portion of the citing sentence has a separate citation and is not attributed to this source. (LLM, conf=0.93)
- Supportedcites [33] [33] Vivian Lai, Han Liu, and Chenhao Tan. 2020. "Why is 'Chicago' Deceptive?" Towards Building Model-Driven Tutorials for Humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1-13. https://doi.org/10.1145/3313831.3376873 - source
The citation to this paper is attached to “tutorials,” not “explanations.” The Introduction (p. 2) says the authors used deceptive review detection as a testbed for experiments on whether tutorials improve human performance, and the Abstract (p. 1) reports that the tutorials did improve it. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“ML practitioners interviewed by Holstein et al. [22] would often try to fix one problem that would cause the model to start failing in other unrelated ways.”
What the sources say
- Partly supportedcites Holstein et al. [22] [22] Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé, Miro Dudik, and Hanna Wallach. 2019. Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1-16. https://doi.org/10.1145/3290605.3300830 - sourceExperimental — accuracy not yet measured
In “Addressing Detected Issues” (p. 11), interviewees describe fairness fixes that sometimes produced unexpected harms or other forms of unfairness. This supports the idea that fixing one issue could create another, but not the claim that practitioners often caused models to fail in unrelated ways. (LLM, conf=0.85)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
54 more claims · 3 with their cited sources checked
Create a free account to see all 113 claims and to verify the 3 references still open.
- It may state two separate claims as one. 34
- Probably not a substantial claim. 24
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 1×
“LIME [50] is one such method, which fits linear models to a neighborhood of instances to highlight which features most influenced a model’s prediction.”
MethodAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 2×
“Glass-box models can be easier to debug and trace, improving people’s ability to understand AI predictions [54].”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 4×
“A detailed and accurate mental model allows a person to effectively complement an AI system by appropriately relying [37] on its output, while an overly simple or wrong mental model can lead to blind spots and systematic failures [3, 8].”
BackgroundInfluentialRemote Sensing Image Scene Classification: Benchmark and State of the Art
Cheng, G., Han, J., & Lu, X. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2675998 · Cited by 2,926 · cited 2×
“The satellite images come from the NWPU-RESISC45 dataset [16], which has 31,500 satellite images across 45 classes.”
MethodHuman trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 1×
“Although unexpected, the effect is corroborated by studies on trust in AI systems that found that low reliability, AI output that violates people’s over time, decreases trust and reliance in an AI [20].”
ResultSemantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 1×
“Despite the growing capability of NLP models, many state-of-the-art systems still encode serious biases and harms [13] and fail basic logical tests [52] that developers have been aware of for years, but have not been able to fix.”
BackgroundAnchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“Instead of using all input features to explain a prediction, follow-up techniques such as Anchors [51] and MUSE [35] learn sets of rules on features that are sufficient to explain predictions for a subset of instances.”
Method
Research works printed but not linked: 5
- Ruogu Kang, Laura Dabbish, Nathaniel Fruchter, and Sara Kiesler. 2015. My Data Just Goes Everywhere: User Mental Models of the Internet and Implications for Privacy and Security. In Eleventh Symposium On Usable Privacy and Security (SOUPS 2015). USENIX Association, Ottawa, 39--52. https://www.usenix.org/conference/soups2015/proceedings/presentation/kang
- 10.1038/s41586-019--1138-y · doi:10.1038/s41586-019--1138-y
- 10.18653/v1 · doi:10.18653/v1
- 10.1007/s10648-010--9128--5 · doi:10.1007/s10648-010--9128--5
- P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona. 2010. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001. California Institute of Technology.
Cited by
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
“The studies in our dataset, though, come from different samples of people—some look at doctors 37–39 , others at crowdworkers 4,6,34 and still others at students 13,40,41 —and this variation can limit the comparability of the effect sizes to a degree 36 .”
Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
Williamson, S. M., & Prybutok, V. (2024) · Information · doi:10.3390/info15060299 · Cited by 146
The Impact of Imperfect XAI on Human-AI Decision-Making
Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641022 · Cited by 121
Self-citationMa, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642671 · Cited by 94
Metadata
- Authors
- Cabrera, Á. A., Perer, A., & Hong, J. I.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2023-04-14
- Volume
- 7
- Issue
- CSCW1
- Pages
- 1-21
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4317464833
- Cited by
- 87
- References
- 76
- Citations by year
- 2026: 37 · 2025: 30 · 2024: 18 · 2023: 2
- FWCI
- 25.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (IIS-2040942)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- human-AI collaboration, behavioral description, AI-assisted decision-making, trust in AI, mental models, fake review detection, satellite image classification, bird classification, explainable AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Improving Human-AI Collaboration With Descriptions of AI Behavior. https://app.cochat.ai/papers/10.1145/3579612
- The paper
- Cabrera, Á. A., Perer, A., & Hong, J. I. (2023). Improving Human-AI Collaboration With Descriptions of AI Behavior. Proceedings of the ACM on Human-Computer Interaction. https://doi.org/10.1145/3579612
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.