Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Open accessThe DOI matches its Crossref record; 105 of the 105 references it cites were found in a registry or an index; 44 of the 63 sources its claims cite were checked for support.
Abstract
AI explanations are often mentioned as a way to improve human-AI decision-making, but empirical studies have not found consistent evidence of explanations' effectiveness and, on the contrary, suggest that they can increase overreliance when the AI system is wrong. While many factors may affect reliance on AI support, one important factor is how decision-makers reconcile their own intuition---beliefs or heuristics, based on prior knowledge, experience, or pattern recognition, used to make judgments---with the information provided by the AI system to determine when to override AI predictions. We conduct a think-aloud, mixed-methods study with two explanation types (feature- and example-based) for two prediction tasks to explore how decision-makers' intuition affects their use of AI predictions and explanations, and ultimately their choice of when to rely on AI. Our results identify three types of intuition involved in reasoning about AI predictions and explanations: intuition about the task outcome, features, and AI limitations. Building on these, we summarize three observed pathways for decision-makers to apply their own intuition and override AI predictions. We use these pathways to explain why (1) the feature-based explanations we used did not improve participants' decision outcomes and increased their overreliance on AI, and (2) the example-based explanations we used improved decision-makers' performance over feature-based explanations and helped achieve complementary human-AI performance. Overall, our work identifies directions for further development of AI decision-support systems and explanation methods that help decision-makers effectively apply their intuition to achieve appropriate reliance on AI.
Overview
CoChat Summary
TL;DR
This think-aloud mixed-methods study (N=26) examines how decision-makers’ intuition shapes reliance on AI predictions with feature- versus example-based explanations across income prediction and biography classification. It identifies three intuition-driven pathways to override AI and finds example-based explanations supported complementary human-AI performance, while feature-based explanations did not improve outcomes and increased overreliance on incorrect AI.
Key findings
- The paper identifies three types of intuition involved in reasoning about AI predictions and explanations—intuition about the task outcome, features, and AI limitations—and summarizes three pathways to override AI: strong outcome intuition disagreeing with AI, applying feature intuition to discredit the explanation, and recognizing signals of prediction unreliability.
- Example-based explanations led to complementary performance: income prediction accuracy was 61.1% ± 2.9% with no AI, 60.6% ± 4.2% with feature-based explanations, and 71.1% ± 2.9% with example-based explanations; biography classification was 60.0% ± 4.4%, 64.4% ± 4.3%, and 71.2% ± 4.6%, respectively, while the AI alone achieved 62.5% on the presented instances.
- Feature-based explanations did not significantly improve overall accuracy over No AI (SE = 0.00, p = 0.88 for income prediction; SE = 0.04, p = 0.51 for biography classification) and decreased accuracy when the AI was incorrect (SE = −0.23, p = 0.09 for income prediction; SE = −0.24, p < 0.01 for biography classification); example-based explanations had no such negative effect and were significant for biography classification (p = 0.04) and marginally significant for income prediction (p = 0.07).
- Subjective preferences diverged from performance: 11 participants preferred feature-based explanations, 10 preferred example-based, and 5 were impartial; 7 of the 11 who preferred feature-based explanations performed better with example-based explanations. Among the 5 participants with lowest domain knowledge (average phase 2 accuracy = 43.8%), example-based explanations improved accuracy to 70.0% while feature-based explanations yielded 52.5%.
Methods
- Mixed-methods think-aloud study with N=26 participants, recruited via Twitter and internal message boards at a large international technology company, limited to US-based participants and selected to diversify ML and XAI backgrounds; within-subjects design where each participant was randomly assigned to one of two tasks but engaged with both explanation types in randomized order.
- Two prediction tasks: income prediction (>$50,000 from Folktables 2018 US Census data, random forest with 80.8% held-out accuracy) and biography classification (five professions from the BIOS dataset, bag-of-words random forest with 75.4% held-out accuracy). Phase 2 had 16 instances with no AI; Phase 3 showed the same 16 instances with AI support, 8 per explanation type, sampled as 5 AI-correct and 3 AI-incorrect per explanation type, with think-aloud during Phase 3 and a post-study interview.
- Analysis combined exploratory mixed-effects regression on decision accuracy with participants as a random effect and explanation type as a fixed effect (No AI as reference), plus qualitative open coding informed by Grounded Theory on think-aloud and interview data, with coders initially blind to decision correctness.
Limitations
- The paper states that think-aloud may not resemble realistic decision-making and may increase cognitive engagement; think-aloud data are observational and cannot isolate the effect of different intuition types or identify precise causes; the two-phase repeated-instance design may strengthen prior intuition; and the study is limited by small sample size, low-stakes tasks, use of an unfaithful post-hoc explanation method (LIME), and a sample skewed toward highly educated and ML-experienced individuals, so the identified intuition types may not be complete or fully generalizable.
- Evident from the text: the convenience sample from a technology company and Twitter, restricted to US participants, further limits generalizability; the paper itself cautions that the quantitative analysis is exploratory and that p-values should be interpreted with caution given the small sample size.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
18 Valid · 5 Not valid · 8 Not checked · 179 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Our exploratory quantitative analysis shows that, consistent with prior work [4,82,87], featurebased explanations did not improve participants' decision outcomes compared to their performance without AI support, and led to overreliance on incorrect AI predictions.”
- Also stated in: § Abstract · plain
“We use these pathways to explain why (1) the feature-based explanations we used did not improve participants' decision outcomes and increased their overreliance on AI”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 1 INTRODUCTION
What the sources say
- Disputedcites [4] 4. Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-16. sourceExperimental — accuracy not yet measured
Section 5.1 (p. 8) says all team conditions, including explanation conditions, achieved complementary performance—exceeding unassisted human performance. The paper's finding of no significant improvement concerns explanations compared with AI assistance that showed confidence scores, not compared with no AI support (Section 5.1, p. 7; Conclusions, p. 14). (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [82] 82. Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318-328. source
- Partly supportedcites [87] 87. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295-305. sourceExperimental — accuracy not yet measured
In §4.2.2 Results (page unavailable), the paper reports no significant improvement in AI-assisted accuracy when explanations were shown. But its comparison is across AI-information conditions, not against participants’ performance without AI support; it reports participants’ unaided accuracy separately as 63%. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“On the contrary, several studies found that showing feature-based explanations increases people's tendency to over-rely on the model when it is wrong, compared to showing only the AI predictions [4,68,82,87].”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 2.2.1 Can XAI methods improve appropriate reliance?
What the sources say
- Supportedcites [4] 4. Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-16. source
The pilot study (Section 3.2, p. 4) compared feature-based explanations with a condition showing only the model prediction and confidence. It reports increased reliance on incorrect recommendations when the predicted class was explained. The main study (Section 5.1, pp. 8–9) found the effect varied by dataset, but the pilot directly supports the claim attributed to this paper. (LLM, conf=0.91)
- Partly supportedcites [68] 68. Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1-52. sourceExperimental — accuracy not yet measured
The Overview of experiments (§1.2; page not available) reports that participants shown a clear model were less able to correct sizable mistakes on unusual apartments than participants shown a black-box model. But this paper compares access to model internals, not a distinct feature-based explanation alongside an otherwise identical prediction-only display; §5.2 also reports that the effect did not appear in the third experiment. The claim therefore captures a finding but describes the intervention and consistency more broadly than the source supports. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [82] 82. Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318-328. source
- Disputedcites [87] 87. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295-305. sourceExperimental — accuracy not yet measured
In §4.2 Results (page unavailable), the paper compares feature-based explanations with a prediction-only baseline and finds no significant difference in participants’ willingness to switch to the AI’s prediction (p = .66). It concludes that explanations did not perceptibly affect trust, contrary to the claimed increase in over-reliance when the model was wrong. The paper discusses increased acceptance of incorrect predictions in another study, but does not find that effect in its own comparison. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, many empirical studies on human-AI decision-making did not observe complementary performance [6,16,37,52,58,61,68,82,84,87].”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 2.2 Human-AI Decision-Making
What the sources say
- Partly supportedcites [6] 6. Emma Beede, Elizabeth Baylor, Fred Hersch, Anna Iurchenko, Lauren Wilcox, Paisan Ruamviboonsuk, and Laura M Vardoulakis. 2020. A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1-12. sourceExperimental — accuracy not yet measured
The “Challenges evaluating AI in clinical settings” section reports that mammography computer-aided detection generally did not improve radiologists’ diagnostic accuracy. That supports an example of human–AI assistance failing to improve performance, but this paper does not establish the broader claim that many empirical studies found no complementary performance, nor does it report a comparative performance result for its own deployment. Page numbers were not available in our copy. (LLM, conf=0.76)
- Supportedcites [16] 16. Samuel Carton, Qiaozhu Mei, and Paul Resnick. 2020. Feature-Based Explanations Don't Help People Detect Misclas- sifications of Online Toxicity. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 14. 95-106. sourceExperimental — accuracy not yet measured
In Accuracy and Agreement, the paper reports that showing participants the model’s prediction slightly reduced accuracy, while explanations did not significantly change that result. Experiment Design for Evaluating Interpretability also describes a gap between hoped-for and demonstrated gains in human–model performance across this and similar studies. The paper therefore supports the claim as one example among the studies cited, rather than as evidence about every human–AI decision task. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [37] 37. Ben Green and Yiling Chen. 2019. The principles and limits of algorithm-in-the-loop decision making. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-24. source
- Supportedcites [52] 52. Vivian Lai, Han Liu, and Chenhao Tan. 2020. " Why is' Chicago'deceptive?" Towards Building Model-Driven Tutorials for Humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-13. sourceExperimental — accuracy not yet measured
The Results (p. 8) show that human participants rarely outperformed the ML model: 0.4% in Experiment 1 and 8.1% with effective real-time assistance in Experiment 2. This study therefore supports the cited sentence as one example of limited complementary performance, while acknowledging that a minority of participants did outperform the model. (LLM, conf=0.88)
- Supportedcites [58] 58. Han Liu, Vivian Lai, and Chenhao Tan. 2021. Understanding the effect of out-of-distribution examples and interactive explanations on human-AI decision making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1-45. sourceExperimental — accuracy not yet measured
Section 2.1 reports that, across many studies of challenging prediction tasks, human–AI teams seldom outperform AI alone; it also identifies a notable exception. Because the paper defines complementary performance as outperforming both AI alone and humans alone, those reported studies support the citing sentence’s claim. (LLM, conf=0.91)
- Supportedcites [61] 61. Scott M Lundberg, Bala Nair, Monica S Vavilala, Mayumi Horibe, Michael J Eisses, Trevor Adams, David E Liston, Daniel King-Wai Low, Shu-Fang Newman, Jerry Kim, et al. 2018. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature biomedical engineering 2, 10 (2018), 749-760. source
In the Results section, the paper reports that anaesthesiologists assisted by Prescience performed better than unaided anaesthesiologists but slightly worse than Prescience alone. It therefore supports this citation as one example of a study that did not observe complementary performance; it does not itself establish how many studies found that result. (LLM, conf=0.90)
- Supportedcites [68] 68. Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1-52. source
This paper supports its use as one example in the citing sentence’s list of empirical studies. Section 3.2 (page unavailable) reports that, for typical apartments, participants using a model made less accurate predictions than the model alone, although they did better than participants without a model. The paper does not itself establish how many studies found this pattern. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [82] 82. Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318-328. source
- Not checked — no full text of the sourcecites [84] 84. Fumeng Yang, Zhuanyi Huang, Jean Scholtz, and Dustin L Arendt. 2020. How do visual explanations foster end users' appropriate trust in machine learning?. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 189-201. source
- Supportedcites [87] 87. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295-305. sourceExperimental — accuracy not yet measured
In 3.2 Results (page not provided), the paper reports that human–AI joint decisions were not more accurate than AI decisions alone, even in the low-confidence cases where the authors expected a benefit. This study therefore supports the cited sentence as one example of an empirical study that did not observe complementary performance. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Our exploratory quantitative analysis shows that, consistent with prior work [4,82,87], featurebased explanations did not improve participants' decision outcomes compared to their performance without AI support, and led to overreliance on incorrect AI predictions.”
- Also stated in: § Abstract · plain
“increased their overreliance on AI”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 1 INTRODUCTION
What the sources say
- Supportedcites [4] 4. Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-16. source
Section 3.1 (p. 4) describes the study’s feature-based explanations as highlights showing the importance of input words. The pilot-study findings (Section 3.2, p. 4) report increased reliance on AI recommendations even when they were incorrect; Section 5.1 (p. 8) reports a similar effect for explanations of the top prediction, though the effect varied by task. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [82] 82. Xinru Wang and Ming Yin. 2021. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces. 318-328. source
- Partly supportedcites [87] 87. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295-305. sourceExperimental — accuracy not yet measured
In 4.2 Results (page unavailable), the paper reports that local feature-based explanations did not improve trust calibration or AI-assisted accuracy. It does not show that they caused greater reliance on incorrect predictions: agreement was similar with and without explanations. Its discussion of increased acceptance of incorrect predictions concerns Lai and Tan’s study, not its own experiment. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract · plain
Every source the paper cites for this claim was checked and supports it.
In the paper
“This suggests that example-based explanations can provide additional context and support inductive reasoning [11] to help people form additional intuition about features.”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > Applying intuition about features to reason about explanations.
What the sources say
- Supportedcites [11] 11. Zana Buçinca, Phoebe Lin, Krzysztof Z Gajos, and Elena L Glassman. 2020. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In Proceedings of the 25th international conference on intelligent user interfaces. 454-464. source
Section 3.1 (p. 3) describes example-based explanations that show four additional food images and ask participants to recognize ingredients contributing to fat content and draw their own conclusions. It explicitly identifies this as inductive reasoning, supporting the paraphrase about context and intuition about features. (LLM, conf=0.91)
hedgedstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
101 more claims · 1 with their cited sources checked
Create a free account to see all 210 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 52
- It may state two separate claims as one. 63
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These findings suggest that a reflective approach toward the use of GenAI tools, which can lead to what prior work refers to as”
From The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers (2025) · cites this paper as [20]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Conversely, with the important caveat that users' self-confidence is a subjective measure of their knowledge, experiences, and abilities on the tasks [20,59,85], higher self-confidence is associated with more critical thinking, even though workers who are confident in their own skills tend to perceive greater effort in these tasks, particularly when evaluating and applying AI responses.”
From The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers (2025) · cites this paper as [20]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This can lead to disastrous outcomes through overreliance [19, 25, 83, 102], when people take actions based on incorrect outputs.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Vice versa, people may -and often do -trust an untrustworthy model (Chen, Liao, Vaughan, & Bansal, 2023).”
From Trust, distrust, and appropriate reliance in (X)AI: A conceptual clarification of user trust and survey of its empirical evaluation (2025) · cites this paper as Chen, Liao, Vaughan, & Bansal, 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They show a number of studies that suggest that explanations can increase users' reliance on AI predictions even if the AI system's predictions are wrong.”
From Trust, distrust, and appropriate reliance in (X)AI: A conceptual clarification of user trust and survey of its empirical evaluation (2025) · cites this paper as Chen et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 4.3.1 Subjective preferences of explanation types.
“For example, P17 said "when the data was super consistent, I was gonna go with it"”
§ Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations > 4.3.1 Subjective preferences of explanation types.
“"when the data was super consistent, I was gonna go with it" (P32).”
Citations
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 2×
“Explanations can also be classified as global or local [1, 35].”
BackgroundMethodLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 2×
“First, we overview related work on explainable AI (XAI), focusing on XAI for ML models—what is sometimes referred to as interpretability in the ML literature [26, 53].”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Research works printed but not linked: 1
- Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning. PMLR, 2668--2677.
Cited by
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
“But some studies show that augmenting humans with AI can lead to better outcomes than humans or AI working alone 24–26 , while others show the opposite 4,7,9 .”
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713778 · Cited by 490
Mersha, M., Lam, K., Wood, J., AlShami, A. K., & Kalita, J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128111 · Cited by 237
“Human-AI decision-making (collaboration): In recent autonomous vehicle systems, machine learning models are utilized to assist users in making final judgments or decisions, representing a form of collaboration between humans and AI systems [183].”
MethodA literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180
“The data was obtained using queries that include keywords such as ‘Explainable AI,’ ’XAI,’ and related terms through Google Scholar’s advanced search [111], [120].”
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147
Self-citationFok, R., & Weld, D. S. (2024) · AI Magazine · doi:10.1002/aaai.12182 · Cited by 81
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
Schoeffer, J., De-Arteaga, M., & Kühl, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642621 · Cited by 68
“It has been argued that explanations are an enabler for better AI-assisted decision-making [8, 35, 46, 62, 102].”
BackgroundDigital Twin Models: Functions, Challenges, and Industry Applications
Rayhana, R., Bai, L., Xiao, G. G., Liao, M., & Liu, Z. (2024) · IEEE Journal of Radio Frequency Identification · doi:10.1109/jrfid.2024.3387996 · Cited by 81
“In earlier times, decision-making processes were influenced by intuition, heuristics, or basic computations [401].”
BackgroundThe Impact of Imperfect XAI on Human-AI Decision-Making
Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641022 · Cited by 60
“These types of scenarios should not be overlooked when investigating human-AI collaborations in work settings.”
BackgroundInfluentialHerrera, F. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103133 · Cited by 79
Kim, S. S. Y., Vaughan, J. W., Liao, Q. V., Lombrozo, T., & Russakovsky, O. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714020 · Cited by 58
“…128, 129] and (un)certainty [6, 14, 17, 41, 132], explanations of outputs [6, 14, 17, 40, 41, 69, 132], and onboarding materials [18, 68] impact user reliance, as well as the roles played by human intuition [20], task complexity [101, 102], and other human, AI, and context-related factors [63].”
Self-citationVisser, R., Peters, T. M., Scharlau, I., & Hammer, B. (2025) · Cognitive Systems Research · doi:10.1016/j.cogsys.2025.101357 · Cited by 58
“…on trust (Papenmeier et al., 2022), acceptance (Bansal et al., 2021; Cramer et al., 2008; Jiang et al., 2022; Wanner et al., 2020), switch rate (Chen et al., 2023; Leffrang & Müller, 2021; Naiseh et al., 2023), advice adoption (Bansal et al., 2021; He et al., 2023; Jiang et al., 2022;…”
InfluentialFok, R., & Weld, D. S. (2023) · arXiv · doi:10.48550/arxiv.2305.07722 · Cited by 54
AXNav: Replaying Accessibility Tests from Natural Language
Taeb, M., Swearngin, A., Schoop, E., Cheng, R., Jiang, Y., & Nichols, J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642777 · Cited by 52
“These are areas we hope to improve in future work.”
BackgroundEscalation Risks from Language Models in Military and Diplomatic Decision-Making
Rivera, J.-P., Mukobi, G., Reuel, A., Lamparth, M., Smith, C., & Schneider, J. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658942 · Cited by 77
Ansari, Z. A., Tripathi, M. M., & Ahmed, R. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00307-8 · Cited by 64
Arif, H., Kumar, A., Fahad, M., & Hussain, H. K. (2024) · International Journal of Multidisciplinary Sciences and Arts · doi:10.47709/ijmdsa.v2i2.3452 · Cited by 83
Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making
Lu, Z., Wang, D., & Yin, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653708 · Cited by 39
To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models
Bo, J. Y., Wan, S., & Anderson, A. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714097 · Cited by 36
“Recent research has investigated the role of LLMs in human-LLM collaborations for applications such as writing [47], communication [27], coding [35, 60], education [7, 43, 52], fact-checking [38, 71, 84], information search [65, 69, 73], and other specialized domains [15, 18, 87].”
Alawamleh, M., Shammas, N., Alawamleh, K. J., & Ismail, L. B. (2024) · Journal of Open Innovation Technology Market and Complexity · doi:10.1016/j.joitmc.2024.100338 · Cited by 67
Capinding, A. T., & Dumayas, F. T. (2024) · Problems of Education in the 21st Century · doi:10.33225/pec/24.82.630 · Cited by 71
“However, if AI is misused the students may not trust themselves, but rather it may lead to overreliance or dependency on AI (Chen et al., 2023).”
Li, J., Yang, Y., Liao, Q. V., Zhang, J., & Lee, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713336 · Cited by 88
Self-citationSpatola, N. (2024) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2024.100099 · Cited by 73
Uncertainty in XAI: Human Perception and Modeling Approaches
Chiaburu, T., Haußer, F., & Bießmann, F. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6020055 · Cited by 53
“In this respect, other studies [53] have investigated how human subjects can correctly use their intuition to reduce their overreliance on AI’s assistance.”
BackgroundDesigning Co-Creative Systems: Five Paradoxes in Human–AI Collaboration
Salma, Z., Hijón‐Neira, R., & Pizarro, C. (2025) · Information · doi:10.3390/info16100909 · Cited by 43
“AI-powered technologies like ChatGPT, Midjourney, and Autodesk Dreamcatcher promote iterative, non-linear collaboration, combining human intuition and AI-generated insights [15,44].”
Göndöcs, D., Horváth, S., & Dörfler, V. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103622 · Cited by 52
A Decision Theoretic Framework for Measuring AI Reliance
Guo, Z., Wu, Y., Hartline, J. D., & Hullman, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658901 · Cited by 27
Can large language models help predict results from a complex behavioural science study?
Lippert, S., Dreber, A., Johannesson, M., Tierney, W., Cyrus-Lai, W., Uhlmann, E. L., Emotion Expression Collaboration, & Pfeiffer, T. (2024) · Royal Society Open Science · doi:10.1098/rsos.240682 · Cited by 45
Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information
Spitzer, P., Holstein, J., Hemmer, P., Vössing, M., Kühl, N., Martin, D., & Satzger, G. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710999 · Cited by 18
Jung, J., Kang, S., Choi, J., El-Kareh, R. E., Lee, H., & Kim, H. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105943 · Cited by 23
Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking
Warren, G., Shklovski, I., & Augenstein, I. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713277 · Cited by 66
“As with any complex decision, despite all the evidence it is often difficult for fact-checkers to articulate how, in the end, the decision is actually made [27, 96].”
From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis
Li, Z., Zhu, H., Lu, Z., Xiao, Z., & Yin, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713133 · Cited by 38
“To help decision makers interact with and rely on the AI model more appropriately, a wide range of approaches was recently developed [7, 9, 25, 48, 49, 52, 53].”
When Are Combinations of Humans and AI Useful?Preprint
Vaccaro, M. A., Almaatouq, A., & Malone, T. W. (2024) · doi:10.31234/osf.io/mk46p · Cited by 16
Arroyo-Machado, W., Ma, J., Chen, T., Johnson, T. P., Islam, S., Michalegko, L., & Welch, E. (2025) · PLOS One · doi:10.1371/journal.pone.0330416 · Cited by 15
What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI
Choudhuri, R., Trinkenreich, B., Pandita, R., Kalliamvakou, E., Steinmacher, I., Gerosa, M., Sanchez, C., & Sarma, A. (2025) · arXiv · doi:10.48550/arxiv.2505.17418 · Cited by 14
Eckhardt, S., Kühl, N., Dolata, M., & Schwabe, G. (2025) · ACM Computing Surveys · doi:10.1145/3776528 · Cited by 45
The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
Goyal, N., Baumler, C., Nguyen, T. T., & Daumé, H. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645210 · Cited by 27
“However, these studies find varying utility of explanations.”
BackgroundTowards Automated Accessibility Report Generation for Mobile Apps
Swearngin, A., Wu, J., Zhang, X., Gómez, E., Coughenour, J., Stukenborg, R., Garg, B., Hughes, G., Hilliard, A., Bigham, J. P., et al. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3674967 · Cited by 25
Explainable AI for Audio and Visual Affective Computing: A Scoping Review
Johnson, D. S., Hakobyan, O., Paletschek, J., & Drimalla, H. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3505269 · Cited by 27
Nullius in Explanans: an ethical risk assessment for explainable AI
Nannini, L., Huyskes, D., Panai, E., Pistilli, G., & Tartaro, A. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09800-7 · Cited by 18
“…through quantitative measures, often without sufficient involvement from all stakeholders affected by these explanations (Kaur et al., 2020; Schem-mer et al., 2022) or unclear benefits for their usefulness (Bertrand et al., 2022; Chen et al., 2023; Schemmer et al., 2022; Vasconcelos et al., 2023).”
Swathi Kumari, H., & Veeramanju, K. T. (2024) · International Journal of Management Technology and Social Sciences · doi:10.47992/ijmts.2581.6012.0346 · Cited by 10
Wu, W., Zuo, E., Zhang, W., & Meng, X. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1492375 · Cited by 11
“Despite advancements in artificial intelligence, which excel in logical reasoning, memory, and computational power (23), AI systems lack the emotional recognition capabilities inherent to human decision-making (24).”
Counterfactual Explanations May Not Be the Best Algorithmic Recourse Approach
Upadhyay, S., Lakkaraju, H., & Gajos, K. Z. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712095 · Cited by 20
“These explanations aim to indicate how each feature in the input data contributes to an algorithmic decision [12].”
Breckner, K., Neumayr, T., Mara, M., Streit, M., & Augstein, M. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2482742 · Cited by 16
How Can I Signal You To Trust Me: Investigating AI Trust Signalling in Clinical Self-Assessments
Kollerup, N. K., Wester, J., Skov, M. B., & van Berkel, N. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661612 · Cited by 15
“…the image and generated text-based recommendation, participants were asked to either accept or decline the recommendation provided by the AI dermatologist as to following or not following up with a doctor consultation, an established approach to studying trust in Human-AI collaboration [18, 65].”
MethodThe Amplifying Effect of Explainability in AI-assisted Decision-making in Groups
Duarte, R. D. B., Abreu, M. C., Campos, J., & Paiva, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713534 · Cited by 18
“Additionally, using example-based explanations has proven effective in reducing over-reliance and improving decision-making by fostering a complementary dynamic between AI and the user [11].”
InfluentialThe Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 15
Exploring Trust and Transparency in Retrieval-Augmented Generation for Domain Experts
Ravi, D., & Sindhgatta, R. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719985 · Cited by 11
Swinger, N., Baseman, C. M., Ryu, M., Abdullah, S., Wiese, C. W., Sherrill, A. M., & Arriaga, R. I. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710917 · Cited by 15
Mersha, M. A., Lamb, K., Wood, J., Alshami, A. M., & Kalita, J. K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4715286 · Cited by 7
Zhang, Z. T., Buchner, F., Liu, Y., & Butz, A. (2024) · Proceedings of Mensch und Computer 2024 · doi:10.1145/3670653.3670660 · Cited by 6
“This task has been used in previous studies on AI-assisted decision-making [8, 24, 29, 35].”
Schäfer, U., Sipos, L., & Müller-Birn, C. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757451 · Cited by 13
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 15
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
A New AI Approach by Acquisition of Characteristics in Human Decision-Making Process
Zhou, Y., & Khatibi, S. (2024) · Applied Sciences · doi:10.3390/app14135469 · Cited by 8
Evaluating the Explainability of Attributes and Prototypes for a Medical Classification Model
Gallée, L., Lisson, C. S., Lisson, C. G., Drees, D., Weig, F., Vogele, D., Beer, M., & GOETZ, M. A. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-63787-2_3 · Cited by 11
“…”were less disruptive of people’s natural intuition about outcomes, they better promoted inductive reasoning about features and the decision task generally, and, in particular, they provided strong and accurate signals of prediction unreliability” [12] compared to feature-based explanations.”
BackgroundLi, L., Wang, T., Wang, Y., Wang, Y., & Zeng, X. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2618563 · Cited by 10
“More strikingly, experimental research indicates that such reliance can persist when CAI’s suggestions are suboptimal or erroneous, with users sometimes adopting AI-provided solutions over their own judgments (Buc¸inca et al., 2021; V. Chen et al., 2023).”
Tambwekar, P., & Gombolay, M. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1375490 · Cited by 10
What Else Do I Need to Know? The Effect of Background Information on Users’ Reliance on QA Systems
Goyal, N., Briakou, E., Liu, A., Baumler, C., Bonial, C., Micher, J. C., Voss, C. R., Carpuat, M. J., & Daumé, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.201 · Cited by 19
From friction to synergy: The complex interplay of human creativity and AI
Deschryver, M., Henriksen, D., & Leahy, S. (2025) · Possibility Studies & Society · doi:10.1177/27538699251350483 · Cited by 13
“Given these implications, it is unsurprising that recent research on hybrid intelligence increasingly highlights the strengths of human intuition (e.g. Chen et al., 2023).”
From Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Raees, M., & Papangelis, K. (2026) · arXiv · doi:10.48550/arxiv.2604.23896 · Cited by 5
Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making
Lee, M. H., & Tok, M. Z. Y. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732155 · Cited by 14
Khosravi, M., Izadi, R., Sarvestani, M. A., Bouzarjomehri, H., Marzaleh, M. A., & Ravangard, R. (2025) · Health Informatics Journal · doi:10.1177/14604582251381269 · Cited by 7
“68,69 In conclusion, arti fi cial intelligence is most effectively employed as a supportive instrument that augments expert policymaking and decision-making through collaborative integration, rather than replacing human expertise.”
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 20
“…on AI decision support have focused on feature-based explanations (Lai et al. 2023a), and evidence remains inconclusive regarding their effectiveness in improving user accuracy or reducing overreliance (Zhang et al. 2020; Wang and Yin 2021; Ma et al. 2023; Cau et al. 2023b; Chen et al. 2023).”
InfluentialEvaluating What Others Say: The Effect of Accuracy Assessment in Shaping Mental Models of AI Systems
Do, H. J., Brachman, M., Dugan, C., Pan, Q., Rai, P., Johnson, J. M., & Thawani, R. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686912 · Cited by 12
“Among different types of explanations, many XAI studies have shown the benefits of ‘example-based explanation’, which shows AI predictions for different input examples [15, 20].”
When Are Two Lists Better than One?: Benefits and Harms in Joint Decision-Making
Donahue, K., Gollapudi, S., & Kollias, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28866 · Cited by 15
Conformal Inverse Optimization for Adherence-aware Prescriptive AnalyticsPreprint
Chan, T. C. Y., Delage, E., & Lin, B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4970102 · Cited by 4
A cognitive science framework could prevent harmful public policy decisions involving AI
Rooy, D. V. (2025) · Behavioral Science & Policy · doi:10.1177/23794607251336099 · Cited by 7
Understanding the Effects of Explaining Predictive but Unintuitive Features in Human-XAI Interaction
Qu, J., Arguello, J., & Wang, Y. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732021 · Cited by 7
A proposed framework for data-driven human factors evaluation
Ormerod, I., Dybvik, H., Fraser, M., & Snider, C. M. (2024) · Proceedings of the Design Society · doi:10.1017/pds.2024.11 · Cited by 4
“The development of explainable AI (XAI) systems in this context is pivotal, and the managing of over-reliance on AI systems must be carefully considered to ensure the sensible decisions are being made (Chen et al., 2023).”
BackgroundPascual-Triana, J. D., Fernández, A., Ser, J. D., & Herrera, F. (2025) · Information Sciences · doi:10.1016/j.ins.2024.121844 · Cited by 4
Intuitive Human–Artificial Intelligence Theranostic Complementarity
Turner, J. H. (2025) · Cancer Biotherapy and Radiopharmaceuticals · doi:10.1089/cbr.2025.0021 · Cited by 5
“9”
Erlei, A., Cau, F. M., Georgiev, R., Kumar, S. C., Bizer, K., & Gadiraju, U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791420 · Cited by 7
“Furthermore, additional information may also fail to improve human reliance if they cannot interpret the signals correctly or experience information overload [112].”
Qu, J., Arguello, J., & Wang, Y. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658547 · Cited by 5
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-MakingPreprint
Schoeffer, J., De‐Arteaga, M., & Kuehl, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.11812 · Cited by 3
“It has been argued that explanations are an enabler for better AI-assisted decision-making [8, 35, 46, 62, 102].”
BackgroundExplain it like I'm me: User Characteristics in XAI Decision-Making
Khadar, M., Timma, A., Ludden, L., & Kaur, H. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732116 · Cited by 7
Wang, Y., Zhao, J. L., Ba, Y., Mancenido, M. V., Chiou, E. K., & Maciejewski, R. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2564272 · Cited by 3
“For example, Chen et al. (2023) found that presenting example-based explanations can significantly improve decision performance compared to conditions without XAI, while feature-based explanations cannot; Humer et al. (2022) showed that example-based explanations are better for improving…”
An Empirical Examination of the Evaluative AI Framework
Kornowicz, J. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2581260 · Cited by 3
“This has led to a situation where users often engage superficially with explanations and develop an overreliance on AI (Bansal et al., 2021; Buc¸inca et al., 2021; Chen et al., 2023; Chromik et al., 2021), shifting from the original problem of underreliance.”
Trust to Reliance: Measurement Constructs for Human-AI Appropriate Reliance
Raees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798835 · Cited by 8
(De)Noise: Moderating the Inconsistency Between Human Decision-Makers
Grgić-Hlača, N., Ali, J., Gummadi, K. P., & Vaughan, J. W. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686987 · Cited by 5
“People are also more likely to take advice from decision aids that are explainable [17, 80, 99].”
Self-citationChang, M., Srinivasan, A., & Palani, S. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789077 · Cited by 5
Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making
Chappidi, S., Singh, J., & Krauze, A. V. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791428 · Cited by 8
LLM or Human? Perceptions of Trust and Quality in Research Summaries
Akpinar, N.-J., Avula, S., Lee, C., Dang, B., Razat, K., & Murdock, V. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3793386 · Cited by 7
“Transparency and explanations can strengthen trust but show mixed effects on calibrated reliance [7].”
LL.me: Supporting Identity Work through Human-AI Alignment
Hall, K., Ohsawa, M. S., Swain, V. D., & Kim, J. G. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790890 · Cited by 1
Jung, J., Kang, S., Choi, J., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5017467 · Cited by 2
Johnson, D. S. (2026) · Proceedings of the 34th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3774935.3806657 · Cited by 2
Towards Understanding Human-AI Reliance Patterns Through Explanation Styles
Casolin, E. R., & Salim, F. D. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678996 · Cited by 2
Bao, H., Zhang, Y., Hu, X., & Zhang, J. (2025) · Chinese Management Studies · doi:10.1108/cms-04-2025-0425 · Cited by 4
Stracke, L., Burggräf, P., Sauer, C. R., Schütz, M., & Kaiser, J. (2026) · Production Engineering · doi:10.1007/s11740-025-01416-0 · Cited by 3
Calibrated Trust in Dealing with LLM Hallucinations: A Qualitative Study
Ryser, A., Allwein, F., & Schlippe, T. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11391250 · Cited by 7
“D, intuition in LLM use takes various forms—such as quick judgments based on linguistic coherence, internal consistency, or output logic [36].”
Introducing Social Explainable AI
Rohlfing, K. J., & Lim, B. Y. (2026) · Springer eBooks · doi:10.1007/978-981-96-5290-7_1 · Cited by 3
A Review of Taxonomies of Explainable AI (XAI) Evaluation
Speith, T. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806537 · Cited by 3
“As deployment pressure has intensified, however, so has skepticism [18, 32, 37]: do current explanations actually align with AI models, benefit users, or improve human–AI decision making?”
Nanda, P. K. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2697344 · Cited by 1
Kim, M. J., Jo, H., Park, H., & Cho, Y. (2024) · Interpretation · doi:10.1190/int-2024-0019.1 · Cited by 2
Xie, Y., Meisel, J. D., Meisel, C. A., Betancourt-Lozano, J., Yan, J., & Bugiolacchi, R. (2024) · Applied Sciences · doi:10.3390/app14209461 · Cited by 2
Emmanouilidis, C., Waschull, S., & Zotelli, J. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-80775-6_26 · Cited by 5
1 + 1 > 2? Information, Humans, and MachinesPreprint
Lu, T., & Zhang, Y. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4045718 · Cited by 1
Ibidunmoye, A. F., Eneano, G. A., Ikeakaonwu, O. M., Kolawole, L. O., Omigbodun, F. T., Olawale, R. A., & Chuwa, C. C. (2026) · Research Square · doi:10.21203/rs.3.rs-8605675/v1 · Cited by 3
Integrating Complementary Feature Sets for Human-AI Decision-Making
Lage, I., & Wintersteen, B. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789096
“[8 ] characterizes 3 pathways through which users’ intuitions can override AI predictions, including intuitions about outcomes, how features are used, and the AI’s reliability.”
User Reliance on AI Support for Collaborative Partner Selection
Hrkalović, T. M., Dudzik, B., Hao, C., & Willemsen, M. C. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789082
Metadata
- Authors
- Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G.
- Affiliations
- Carnegie Mellon University (US)
- Microsoft Research Montréal (Canada) (CA)
- Microsoft Research, New York, NY, USA
- Microsoft (United States) (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2023-09-28
- Volume
- 7
- Issue
- CSCW2
- Pages
- 1-32
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4387344906
- Cited by
- 220
- References
- 74
- Citations by year
- 2026: 90 · 2025: 72 · 2024: 47 · 2023: 10
- FWCI
- 15.59
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Microsoft Research
- Institut National Du Cancer
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- human intuition, AI explanations, human-AI decision-making, overreliance on AI, feature-based explanations, example-based explanations, human-AI complementarity, decision support system, think-aloud
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.