The DOI matches its Crossref record; 104 of the 104 references it cites were found in a registry or an index; 44 of the 63 sources its claims cite were checked for support.
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Open accessAbstract
AI explanations are often mentioned as a way to improve human-AI decision-making, but empirical studies have not found consistent evidence of explanations' effectiveness and, on the contrary, suggest that they can increase overreliance when the AI system is wrong. While many factors may affect reliance on AI support, one important factor is how decision-makers reconcile their own intuition---beliefs or heuristics, based on prior knowledge, experience, or pattern recognition, used to make judgments---with the information provided by the AI system to determine when to override AI predictions. We conduct a think-aloud, mixed-methods study with two explanation types (feature- and example-based) for two prediction tasks to explore how decision-makers' intuition affects their use of AI predictions and explanations, and ultimately their choice of when to rely on AI. Our results identify three types of intuition involved in reasoning about AI predictions and explanations: intuition about the task outcome, features, and AI limitations. Building on these, we summarize three observed pathways for decision-makers to apply their own intuition and override AI predictions. We use these pathways to explain why (1) the feature-based explanations we used did not improve participants' decision outcomes and increased their overreliance on AI, and (2) the example-based explanations we used improved decision-makers' performance over feature-based explanations and helped achieve complementary human-AI performance. Overall, our work identifies directions for further development of AI decision-support systems and explanation methods that help decision-makers effectively apply their intuition to achieve appropriate reliance on AI.
Overview
CoChat Summary
TL;DR
This think-aloud mixed-methods study (N=26) examines how decision-makers’ intuition shapes reliance on AI predictions with feature- versus example-based explanations across income prediction and biography classification. It identifies three intuition-driven pathways to override AI and finds example-based explanations supported complementary human-AI performance, while feature-based explanations did not improve outcomes and increased overreliance on incorrect AI.
Key findings
- The paper identifies three types of intuition involved in reasoning about AI predictions and explanations—intuition about the task outcome, features, and AI limitations—and summarizes three pathways to override AI: strong outcome intuition disagreeing with AI, applying feature intuition to discredit the explanation, and recognizing signals of prediction unreliability.
- Example-based explanations led to complementary performance: income prediction accuracy was 61.1% ± 2.9% with no AI, 60.6% ± 4.2% with feature-based explanations, and 71.1% ± 2.9% with example-based explanations; biography classification was 60.0% ± 4.4%, 64.4% ± 4.3%, and 71.2% ± 4.6%, respectively, while the AI alone achieved 62.5% on the presented instances.
- Feature-based explanations did not significantly improve overall accuracy over No AI (SE = 0.00, p = 0.88 for income prediction; SE = 0.04, p = 0.51 for biography classification) and decreased accuracy when the AI was incorrect (SE = −0.23, p = 0.09 for income prediction; SE = −0.24, p < 0.01 for biography classification); example-based explanations had no such negative effect and were significant for biography classification (p = 0.04) and marginally significant for income prediction (p = 0.07).
- Subjective preferences diverged from performance: 11 participants preferred feature-based explanations, 10 preferred example-based, and 5 were impartial; 7 of the 11 who preferred feature-based explanations performed better with example-based explanations. Among the 5 participants with lowest domain knowledge (average phase 2 accuracy = 43.8%), example-based explanations improved accuracy to 70.0% while feature-based explanations yielded 52.5%.
Methods
- Mixed-methods think-aloud study with N=26 participants, recruited via Twitter and internal message boards at a large international technology company, limited to US-based participants and selected to diversify ML and XAI backgrounds; within-subjects design where each participant was randomly assigned to one of two tasks but engaged with both explanation types in randomized order.
- Two prediction tasks: income prediction (>$50,000 from Folktables 2018 US Census data, random forest with 80.8% held-out accuracy) and biography classification (five professions from the BIOS dataset, bag-of-words random forest with 75.4% held-out accuracy). Phase 2 had 16 instances with no AI; Phase 3 showed the same 16 instances with AI support, 8 per explanation type, sampled as 5 AI-correct and 3 AI-incorrect per explanation type, with think-aloud during Phase 3 and a post-study interview.
- Analysis combined exploratory mixed-effects regression on decision accuracy with participants as a random effect and explanation type as a fixed effect (No AI as reference), plus qualitative open coding informed by Grounded Theory on think-aloud and interview data, with coders initially blind to decision correctness.
Limitations
- The paper states that think-aloud may not resemble realistic decision-making and may increase cognitive engagement; think-aloud data are observational and cannot isolate the effect of different intuition types or identify precise causes; the two-phase repeated-instance design may strengthen prior intuition; and the study is limited by small sample size, low-stakes tasks, use of an unfaithful post-hoc explanation method (LIME), and a sample skewed toward highly educated and ML-experienced individuals, so the identified intuition types may not be complete or fully generalizable.
- Evident from the text: the convenience sample from a technology company and Twitter, restricted to US participants, further limits generalizability; the paper itself cautions that the quantitative analysis is exploratory and that p-values should be interpreted with caution given the small sample size.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
18 Valid · 5 Not valid · 8 Not checked · 179 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Our exploratory quantitative analysis shows that, consistent with prior work [4,82,87], featurebased explanations did not improve participants' decision outcomes compared to their performance without AI support, and led to overreliance on incorrect AI predictions.”
- Also stated in: § Abstract · plain
“We use these pathways to explain why (1) the feature-based explanations we used did not improve participants' decision outcomes and increased their overreliance on AI”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 1 INTRODUCTION
What the sources say
- Disputedcites [4] 4. Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-16. sourceExperimental — accuracy not yet measured
Section 5.1 (p. 8) says all team conditions, including explanation conditions, achieved complementary performance—exceeding unassisted human performance. The paper's finding of no significant improvement concerns explanations compared with AI assistance that showed confidence scores, not compared with no AI support (Section 5.1, p. 7; Conclusions, p. 14). (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [82] 82. Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318-328. source
- Partly supportedcites [87] 87. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295-305. sourceExperimental — accuracy not yet measured
In §4.2.2 Results (page unavailable), the paper reports no significant improvement in AI-assisted accuracy when explanations were shown. But its comparison is across AI-information conditions, not against participants’ performance without AI support; it reports participants’ unaided accuracy separately as 63%. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“On the contrary, several studies found that showing feature-based explanations increases people's tendency to over-rely on the model when it is wrong, compared to showing only the AI predictions [4,68,82,87].”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 2.2.1 Can XAI methods improve appropriate reliance?
What the sources say
- Supportedcites [4] 4. Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-16. source
The pilot study (Section 3.2, p. 4) compared feature-based explanations with a condition showing only the model prediction and confidence. It reports increased reliance on incorrect recommendations when the predicted class was explained. The main study (Section 5.1, pp. 8–9) found the effect varied by dataset, but the pilot directly supports the claim attributed to this paper. (LLM, conf=0.91)
- Partly supportedcites [68] 68. Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1-52. sourceExperimental — accuracy not yet measured
The Overview of experiments (§1.2; page not available) reports that participants shown a clear model were less able to correct sizable mistakes on unusual apartments than participants shown a black-box model. But this paper compares access to model internals, not a distinct feature-based explanation alongside an otherwise identical prediction-only display; §5.2 also reports that the effect did not appear in the third experiment. The claim therefore captures a finding but describes the intervention and consistency more broadly than the source supports. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [82] 82. Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318-328. source
- Disputedcites [87] 87. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295-305. sourceExperimental — accuracy not yet measured
In §4.2 Results (page unavailable), the paper compares feature-based explanations with a prediction-only baseline and finds no significant difference in participants’ willingness to switch to the AI’s prediction (p = .66). It concludes that explanations did not perceptibly affect trust, contrary to the claimed increase in over-reliance when the model was wrong. The paper discusses increased acceptance of incorrect predictions in another study, but does not find that effect in its own comparison. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, many empirical studies on human-AI decision-making did not observe complementary performance [6,16,37,52,58,61,68,82,84,87].”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 2.2 Human-AI Decision-Making
What the sources say
- Partly supportedcites [6] 6. Emma Beede, Elizabeth Baylor, Fred Hersch, Anna Iurchenko, Lauren Wilcox, Paisan Ruamviboonsuk, and Laura M Vardoulakis. 2020. A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1-12. sourceExperimental — accuracy not yet measured
The “Challenges evaluating AI in clinical settings” section reports that mammography computer-aided detection generally did not improve radiologists’ diagnostic accuracy. That supports an example of human–AI assistance failing to improve performance, but this paper does not establish the broader claim that many empirical studies found no complementary performance, nor does it report a comparative performance result for its own deployment. Page numbers were not available in our copy. (LLM, conf=0.76)
- Supportedcites [16] 16. Samuel Carton, Qiaozhu Mei, and Paul Resnick. 2020. Feature-Based Explanations Don't Help People Detect Misclas- sifications of Online Toxicity. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 14. 95-106. sourceExperimental — accuracy not yet measured
In Accuracy and Agreement, the paper reports that showing participants the model’s prediction slightly reduced accuracy, while explanations did not significantly change that result. Experiment Design for Evaluating Interpretability also describes a gap between hoped-for and demonstrated gains in human–model performance across this and similar studies. The paper therefore supports the claim as one example among the studies cited, rather than as evidence about every human–AI decision task. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [37] 37. Ben Green and Yiling Chen. 2019. The principles and limits of algorithm-in-the-loop decision making. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-24. source
- Supportedcites [52] 52. Vivian Lai, Han Liu, and Chenhao Tan. 2020. " Why is' Chicago'deceptive?" Towards Building Model-Driven Tutorials for Humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-13. sourceExperimental — accuracy not yet measured
The Results (p. 8) show that human participants rarely outperformed the ML model: 0.4% in Experiment 1 and 8.1% with effective real-time assistance in Experiment 2. This study therefore supports the cited sentence as one example of limited complementary performance, while acknowledging that a minority of participants did outperform the model. (LLM, conf=0.88)
- Supportedcites [58] 58. Han Liu, Vivian Lai, and Chenhao Tan. 2021. Understanding the effect of out-of-distribution examples and interactive explanations on human-AI decision making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1-45. sourceExperimental — accuracy not yet measured
Section 2.1 reports that, across many studies of challenging prediction tasks, human–AI teams seldom outperform AI alone; it also identifies a notable exception. Because the paper defines complementary performance as outperforming both AI alone and humans alone, those reported studies support the citing sentence’s claim. (LLM, conf=0.91)
- Supportedcites [61] 61. Scott M Lundberg, Bala Nair, Monica S Vavilala, Mayumi Horibe, Michael J Eisses, Trevor Adams, David E Liston, Daniel King-Wai Low, Shu-Fang Newman, Jerry Kim, et al. 2018. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature biomedical engineering 2, 10 (2018), 749-760. source
In the Results section, the paper reports that anaesthesiologists assisted by Prescience performed better than unaided anaesthesiologists but slightly worse than Prescience alone. It therefore supports this citation as one example of a study that did not observe complementary performance; it does not itself establish how many studies found that result. (LLM, conf=0.90)
- Supportedcites [68] 68. Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1-52. source
This paper supports its use as one example in the citing sentence’s list of empirical studies. Section 3.2 (page unavailable) reports that, for typical apartments, participants using a model made less accurate predictions than the model alone, although they did better than participants without a model. The paper does not itself establish how many studies found this pattern. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [82] 82. Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318-328. source
- Not checked — no full text of the sourcecites [84] 84. Fumeng Yang, Zhuanyi Huang, Jean Scholtz, and Dustin L Arendt. 2020. How do visual explanations foster end users' appropriate trust in machine learning?. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 189-201. source
- Supportedcites [87] 87. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295-305. sourceExperimental — accuracy not yet measured
In 3.2 Results (page not provided), the paper reports that human–AI joint decisions were not more accurate than AI decisions alone, even in the low-confidence cases where the authors expected a benefit. This study therefore supports the cited sentence as one example of an empirical study that did not observe complementary performance. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Our exploratory quantitative analysis shows that, consistent with prior work [4,82,87], featurebased explanations did not improve participants' decision outcomes compared to their performance without AI support, and led to overreliance on incorrect AI predictions.”
- Also stated in: § Abstract · plain
“increased their overreliance on AI”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 1 INTRODUCTION
What the sources say
- Supportedcites [4] 4. Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-16. source
Section 3.1 (p. 4) describes the study’s feature-based explanations as highlights showing the importance of input words. The pilot-study findings (Section 3.2, p. 4) report increased reliance on AI recommendations even when they were incorrect; Section 5.1 (p. 8) reports a similar effect for explanations of the top prediction, though the effect varied by task. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [82] 82. Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318-328. source
- Partly supportedcites [87] 87. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295-305. sourceExperimental — accuracy not yet measured
In 4.2 Results (page unavailable), the paper reports that local feature-based explanations did not improve trust calibration or AI-assisted accuracy. It does not show that they caused greater reliance on incorrect predictions: agreement was similar with and without explanations. Its discussion of increased acceptance of incorrect predictions concerns Lai and Tan’s study, not its own experiment. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
Every source the paper cites for this claim was checked and supports it.
In the paper
“This suggests that example-based explanations can provide additional context and support inductive reasoning [11] to help people form additional intuition about features.”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > Applying intuition about features to reason about explanations.
What the sources say
- Supportedcites [11] 11. Zana Buçinca, Phoebe Lin, Krzysztof Z Gajos, and Elena L Glassman. 2020. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In Proceedings of the 25th international conference on intelligent user interfaces. 454-464. source
Section 3.1 (p. 3) describes example-based explanations that show four additional food images and ask participants to recognize ingredients contributing to fat content and draw their own conclusions. It explicitly identifies this as inductive reasoning, supporting the paraphrase about context and intuition about features. (LLM, conf=0.91)
hedgedstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
101 more claims · 1 with their cited sources checked
Create a free account to see all 210 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 52
- It may state two separate claims as one. 63
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These findings suggest that a reflective approach toward the use of GenAI tools, which can lead to what prior work refers to as”
From The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers (2025) · cites this paper as [20]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Conversely, with the important caveat that users' self-confidence is a subjective measure of their knowledge, experiences, and abilities on the tasks [20,59,85], higher self-confidence is associated with more critical thinking, even though workers who are confident in their own skills tend to perceive greater effort in these tasks, particularly when evaluating and applying AI responses.”
From The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers (2025) · cites this paper as [20]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This can lead to disastrous outcomes through overreliance [19, 25, 83, 102], when people take actions based on incorrect outputs.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Vice versa, people may -and often do -trust an untrustworthy model (Chen, Liao, Vaughan, & Bansal, 2023).”
From Trust, distrust, and appropriate reliance in (X)AI: A conceptual clarification of user trust and survey of its empirical evaluation (2025) · cites this paper as Chen, Liao, Vaughan, & Bansal, 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They show a number of studies that suggest that explanations can increase users' reliance on AI predictions even if the AI system's predictions are wrong.”
From Trust, distrust, and appropriate reliance in (X)AI: A conceptual clarification of user trust and survey of its empirical evaluation (2025) · cites this paper as Chen et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 4.3.1 Subjective preferences of explanation types.
“For example, P17 said "when the data was super consistent, I was gonna go with it"”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 4.3.1 Subjective preferences of explanation types.
“"when the data was super consistent, I was gonna go with it" (P32).”
Citations
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 2×
“Explanations can also be classified as global or local [1, 35].”
BackgroundMethodLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 2×
“First, we overview related work on explainable AI (XAI), focusing on XAI for ML models—what is sometimes referred to as interpretability in the ML literature [26, 53].”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Research works printed but not linked: 1
- Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning. PMLR, 2668--2677.
Cited by
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
“But some studies show that augmenting humans with AI can lead to better outcomes than humans or AI working alone 24–26 , while others show the opposite 4,7,9 .”
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713778 · Cited by 490
Mersha, M., Lam, K., Wood, J., AlShami, A. K., & Kalita, J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128111 · Cited by 237
“Human-AI decision-making (collaboration): In recent autonomous vehicle systems, machine learning models are utilized to assist users in making final judgments or decisions, representing a form of collaboration between humans and AI systems [183].”
MethodA literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180
“The data was obtained using queries that include keywords such as ‘Explainable AI,’ ’XAI,’ and related terms through Google Scholar’s advanced search [111], [120].”
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147
Self-citation
Metadata
- Authors
- Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G.
- Affiliations
- Carnegie Mellon University (US)
- Microsoft Research Montréal (Canada) (CA)
- Microsoft Research, New York, NY, USA
- Microsoft (United States) (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2023-09-28
- Volume
- 7
- Issue
- CSCW2
- Pages
- 1-32
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4387344906
- Cited by
- 220
- References
- 74
- Citations by year
- 2026: 90 · 2025: 72 · 2024: 47 · 2023: 10
- FWCI
- 15.59
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Microsoft Research
- Institut National Du Cancer
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- human intuition, AI explanations, human-AI decision-making, overreliance on AI, feature-based explanations, example-based explanations, human-AI complementarity, decision support system, think-aloud
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations. https://app.cochat.ai/papers/10.1145/3610219
- The paper
- Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023). Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations. Proceedings of the ACM on Human-Computer Interaction. https://doi.org/10.1145/3610219
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.