In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making
Open accessAbstract
The current literature on AI-advised decision making -- involving explainable AI systems advising human decision makers -- presents a series of inconclusive and confounding results. To synthesize these findings, we propose a simple theory that elucidates the frequent failure of AI explanations to engender appropriate reliance and complementary decision making performance. We argue explanations are only useful to the extent that they allow a human decision maker to verify the correctness of an AI's prediction, in contrast to other desiderata, e.g., interpretability or spelling out the AI's reasoning process. Prior studies find in many decision making contexts AI explanations do not facilitate such verification. Moreover, most tasks fundamentally do not allow easy verification, regardless of explanation method, limiting the potential benefit of any type of explanation. We also compare the objective of complementary performance with that of appropriate reliance, decomposing the latter into the notions of outcome-graded and strategy-graded reliance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
As task complexity increases, the verifiability and plausibility of AI advice tend to decrease.
“Moreover, as task complexity increases, the verifability [40] and plausibility [57, 60] of AI advice tend to decrease.”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [40]
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Explainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967 · see citing works
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · see citing works
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · see citing works
Hackman, J. R., & Morris, C. G. (1975) · Advances in experimental social psychology · doi:10.1016/s0065-2601(08)60248-8 · Cited by 1,346 · see citing works
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · see citing works
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024 · see citing works
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · see citing works
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Group process and productivity
Washburn, P. V. (1973) · OhioLink ETD Center (Ohio Library and Information Network) · Cited by 869 · see citing works
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794 · see citing works
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · see citing works
Self-citationThe structure and function of explanations.
Lombrozo, T. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.08.004 · Cited by 729 · see citing works
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
Group versus individual performance: Are N + 1 heads better than one?
Hill, G. W. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.91.3.517 · Cited by 692 · see citing works
Demonstrability and social combination processes on mathematical intellective tasks
Laughlin, P. R., & Ellis, A. L. (1986) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(86)90022-3 · Cited by 534 · see citing works
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466 · see citing works
Self-citationExplanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402 · see citing works
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392 · cited 4× · see citing works
“Unfortunately, achieving this notion of appropriate reliance through explanations has been elusive [3, 7, 16, 18, 31, 35, 37, 39, 50, 57–59, 63, 72, 76].”
BackgroundInfluentialLaughlin, P. R., Hatch, E. C., Silver, J. S., & Boh, L. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.90.4.644 · Cited by 390 · see citing works
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · see citing works
Impact of a deep learning assistant on the histopathologic classification of liver cancer
Kiani, A., Uyumazturk, B., Rajpurkar, P., Wang, A., Gao, R., Jones, E., Yu, Y., Langlotz, C. P., Ball, R. L., Montine, T. J., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0232-8 · Cited by 325 · see citing works
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320 · see citing works
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · see citing works
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · see citing works
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012429 · Cited by 284 · see citing works
Self-citationThe challenge of crafting intelligible intelligence
Weld, D. S., & Bansal, G. (2019) · Communications of the ACM · doi:10.1145/3282486 · Cited by 253 · see citing works
Self-citationExplaining Recommendations: Satisfaction vs. Promotion
Bilgic, M., & Mooney, R. J. (2005) · Cited by 252 · see citing works
Jacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247 · see citing works
Davis, J. H. (1992) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(92)90044-8 · Cited by 222 · see citing works
Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584066 · Cited by 208 · see citing works
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · see citing works
Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581001 · Cited by 196 · see citing works
Miller, T. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594001 · Cited by 172 · see citing works
Evaluating saliency map explanations for convolutional neural networks: a user study
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Berthouze, N. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377519 · Cited by 163 · see citing works
The Impact of Placebic Explanations on Trust in Intelligent Systems
Eiband, M., Buschek, D., Kremer, A., & Hussmann, H. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312787 · Cited by 152 · see citing works
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 145 · see citing works
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI
Chromik, M., Eiband, M., Buchner, F., Krüger, A., & Butz, A. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450644 · Cited by 144 · see citing works
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141 · cited 1× · see citing works
“Beyond simply improving the performance of a human-AI team, we are interested in complementary performance , where the human-AI system performs better (for example, its accuracy is higher or decisions are made more cheaply) than either the human or AI separately.”
BackgroundDo People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning
Gajos, K. Z., & Mamykina, L. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511138 · Cited by 127 · see citing works
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123 · see citing works
How can I choose an explainer?: An Application-grounded Evaluation of Post-hoc Explanations
Jesus, S., Belém, C., Balayan, V., Bento, J., Saleiro, P., Bizarro, P., & Gama, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445941 · Cited by 112 · see citing works
Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems
Nourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E., & Gogate, V. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450639 · Cited by 107 · cited 3× · see citing works
“…✗ ✗ Wang and Yin [71] Recidivism prediction Feature importance ✗ ✗ Wang and Yin [71] Forest cover prediction Feature importance ✗ ✗ Nourani et al. [57] Kitchen policy verification Feature importance ✗ ✗ Poursabzi-Sangdeh et al. [59] Apartment price prediction Feature importance ✗ ✗ Jesus et al.…”
BackgroundAre explanations always important?: a study of deployed, low-cost intelligent interactive systems
Bunt, A., Lount, M., & Lauzon, C. (2012) · Proceedings of the 2012 ACM international conference on Intelligent User Interfaces · doi:10.1145/2166966.2166996 · Cited by 106 · see citing works
Does Explainable Artificial Intelligence Improve Human Decision-Making?
Alufaisan, Y., Marusich, L. R., Bakdash, J. Z., Zhou, Y., & Kantarcioglu, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16819 · Cited by 106 · see citing works
Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Ghai, B., Liao, Q. V., Zhang, Y., Bellamy, R., & Mueller, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432934 · Cited by 106 · see citing works
Bell, A., Solano-Kamaiko, I., Nov, O., & Stoyanovich, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533090 · Cited by 103 · see citing works
Laughlin, P. R., & Adamopoulos, J. (1980) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.38.6.941 · Cited by 95 · see citing works
What can AI do for me?: evaluating machine learning interpretations in cooperative play
Feng, S., & Boyd-Graber, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302265 · Cited by 92 · see citing works
Human-Centric Justification of Machine Learning Predictions
Biran, O., & McKeown, K. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/202 · Cited by 84 · see citing works
Examining the effect of explanation on satisfaction and trust in AI diagnostic systems
Alam, L., & Mueller, S. T. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01542-6 · Cited by 82 · see citing works
Aligning Faithful Interpretations with their Social Attribution
Jacovi, A., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00367 · Cited by 74 · see citing works
HIVE: Evaluating the Human Interpretability of Visual Explanations
Kim, S. S. Y., Meister, N., Ramaswamy, V. V., Fong, R., & Russakovsky, O. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19775-8_17 · Cited by 72 · see citing works
Do I trust my machine teammate?: an investigation from perception to decision
Yu, K., Berkovsky, S., Taib, R., Zhou, J., & Chen, F. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302277 · Cited by 71 · see citing works
Evaluating Human-Language Model Interaction
Lee, M., Srivastava, M., Hardy, A., Thickstun, J., Durmus, E., Paranjape, A., Gerard-Ursin, I., Li, X. L., Ladhak, F., Rong, F., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09746 · Cited by 61 · see citing works
Do explanations make VQA models more predictable to a human?
Chandrasekaran, A., Prabhu, V., Yadav, D., Chattopadhyay, P., & Parikh, D. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1128 · Cited by 59 · see citing works
Feature-Based Explanations Don't Help People Detect Misclassifications of Online Toxicity
Carton, S., Mei, Q., & Resnick, P. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7282 · Cited by 58 · see citing works
Collective Choice, Judgment, and Problem Solving
Stasser, G. L., & Dietz-Uhler, B. L. (2001) · Wiley eBooks · doi:10.1002/9780470998458.ch2 · Cited by 57 · see citing works
Group and individual performance on a complementary task as a function of initial ability level
Laughlin, P. R., & Johnson, H. H. (1966) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(66)90032-1 · Cited by 53 · see citing works
Leveraging rationales to improve human task performance
Das, D., & Chernova, S. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377512 · Cited by 52 · see citing works
Rating Reliability and Bias in News Articles: Does AI Assistance Help Everyone?
Horne, B. D., Nevo, D., O’Donovan, J., Cho, J.-H., & Adalı, S. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3226 · Cited by 49 · see citing works
A Human-Grounded Evaluation of SHAP for Alert ProcessingPreprint
Weerts, H., van Ipenburg, W., & Pechenizkiy, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03324 · Cited by 45 · see citing works
Legibility and the Legacy of Racialized Dispossession in Digital Agriculture
Liu, J., & Sengers, P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479867 · Cited by 45 · see citing works
Machine Explanations and Human Understanding
Chen, C.-C., Feng, S., Sharma, A., & Tan, C. (2022) · Trans. Mach. Learn. Res. · doi:10.1145/3593013.3593970 · Cited by 39 · see citing works
Measuring Progress on Scalable Oversight for Large Language Models
Bowman, S. R., Hyun, J., Perez, E., Chen, E., Pettit, C., Heiner, S., Lukošiūtė, K., Askell, A., Jones, A., Chen, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.03540 · Cited by 34 · see citing works
Nguyen, G., Kim, D., & Nguyen, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.14944 · Cited by 32 · see citing works
On the Diversity and Limits of Human Explanations
Tan, C. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.158 · Cited by 27 · see citing works
González, A. V., Bansal, G., Fan, A., Mehdad, Y., Jia, R., & Iyer, S. V. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.95 · Cited by 19 · see citing works
Carey, H. R., & Laughlin, P. R. (2011) · Group Processes & Intergroup Relations · doi:10.1177/1368430211419174 · Cited by 18 · see citing works
HILDIF: Interactive Debugging of NLI Models Using Influence Functions
Zylberajch, H., Lertvittayakumjorn, P., & Toni, F. (2021) · INTERNLP · doi:10.18653/v1/2021.internlp-1.1 · Cited by 18 · see citing works
Visual correspondence-based explanations improve AI robustness and human-AI team accuracyPreprint
Nguyen, G., Taesiri, M. R., & Nguyen, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.00780 · Cited by 8 · see citing works
Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions
Parrish, A., Trivedi, H. K., Perez, E., Chen, A., Nangia, N., Phang, J., & Bowman, S. R. (2022) · doi:10.18653/v1/2022.lnls-1.3 · Cited by 3 · see citing works
Assisting Human Decisions in Document Matching
Kim, J. S., Chen, V., Pruthi, D., Shah, N. B., & Talwalkar, A. (2023) · arXiv · doi:10.48550/arxiv.2302.08450 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Explainable Generative AI (GenXAI): a survey, conceptualization, and research agenda
Schneider, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10916-x · Cited by 184 · see citing works
Hou, C., Zhu, G., Sudarshan, V., Lim, F. S., & Ong, Y. S. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105329 · Cited by 70 · see citing works
A cognitive approach to human–AI complementarity in dynamic decision-making
González, C., & Heidari, H. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-025-00499-x · Cited by 69 · see citing works
Hou, C., Zhu, G., & Sudarshan, V. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13613 · Cited by 72 · see citing works
Kim, S. S. Y., Vaughan, J. W., Liao, Q. V., Lombrozo, T., & Russakovsky, O. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714020 · Cited by 58 · see citing works
“However, prior work suggests that in many settings, the very presence of an explanation can increase trust and reliance, whether or not it is warranted [6, 33, 93, 97, 117, 132].”
Who Determines What Is Relevant? Humans or AI? Why Not Both?
Faggioli, G., Dietz, L., Clarke, C. L. A., Demartini, G., Hagen, M., Hauff, C., Kando, N., Kanoulas, E., Potthast, M., Stein, B., et al. (2024) · Communications of the ACM · doi:10.1145/3624730 · Cited by 32 · see citing works
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41 · see citing works
Müller, R. (2024) · International journal of human computer interactions · doi:10.1080/10447318.2024.2381929 · Cited by 39 · see citing works
Principles and Guidelines for the Use of LLM Judges
Dietz, L., Zendel, O., Bailey, P., Clarke, C. L. A., Cotterill, E., Dalton, J., Hasibi, F., Sanderson, M., & Craswell, N. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744588 · Cited by 28 · see citing works
Reicherts, L., Zhang, Z. T., von Oswald, E., Liu, Y., Rogers, Y., & Hassib, M. E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713295 · Cited by 28 · see citing works
Large Language Models Help Humans Verify Truthfulness – Except When They Are Convincingly Wrong
Si, C., Goyal, N., Wu, T., Zhao, C., Feng, S., Daumé Iii, H., & Boyd-Graber, J. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.81 · Cited by 26 · see citing works
Amaliah, N. R., Tjahjono, B., & Palade, V. (2025) · Electronics · doi:10.3390/electronics14173384 · Cited by 34 · see citing works
Measuring the effect of mental workload and explanations on appropriate AI reliance using EEG
Zhang, Z. T., Argın, S. K., Bilen, M. B., Urgun, D., Deniz, S. M., Liu, Y., & Hassib, M. E. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2431055 · Cited by 19 · see citing works
Pafla, M., Larson, K. S., & Hancock, M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642934 · Cited by 17 · see citing works
Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
Zhang, Z. T., Feger, S. S., Dullenkopf, L., Liao, R., Süsslin, L. E., Liu, Y., & Butz, A. M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687024 · Cited by 16 · see citing works
Diagnosing AI Explanation Methods with Folk Concepts of Behavior
Jacovi, A., Bastings, J., Gehrmann, S., Goldberg, Y., & Filippova, K. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593993 · Cited by 15 · see citing works
Why AI may undermine phronesis and what to do about it
Tsai, C., & Ku, H.-L. (2024) · AI and Ethics · doi:10.1007/s43681-024-00617-0 · Cited by 11 · see citing works
Abels, C. M., Lopez-Lopez, E., Burton, J. W., Holford, D. L., Brinkmann, L. U. L., Herzog, S. M., & Lewandowsky, S. (2025) · Behavioral Science & Policy · doi:10.1177/23794607251347020 · Cited by 8 · see citing works
Zhang, Z. T., Buchner, F., Liu, Y., & Butz, A. (2024) · Proceedings of Mensch und Computer 2024 · doi:10.1145/3670653.3670660 · Cited by 6 · see citing works
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 6 · see citing works
Müller, R. (2024) · doi:10.31219/osf.io/ms2kv · Cited by 6 · see citing works
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 9 · see citing works
Where Are We and Where Can We Go on the Road to Reliance-Aware Explainable User Interfaces?
Filho, J. C. D. S., Belloum, R., & de Oliveira, K. M. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00038 · Cited by 5 · see citing works
Koçyiğit, E., Rossi, A., Sergeeva, A., Negri-Ribalta, C., Farjami, A., & Lenzini, G. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732129 · Cited by 5 · see citing works
What Else Do I Need to Know? The Effect of Background Information on Users’ Reliance on QA Systems
Goyal, N., Briakou, E., Liu, A., Baumler, C., Bonial, C., Micher, J. C., Voss, C. R., Carpuat, M. J., & Daumé, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.201 · Cited by 10 · see citing works
Can I Trust You? Exploring the Impact of Misleading AI Suggestions on User's Trust
Pretolesi, D., Zechner, O., Schrom-Feiertag, H., & Tscheligi, M. (2024) · IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) · doi:10.1109/metroxraine62247.2024.10796402 · Cited by 6 · see citing works
Hecksteden, A., Kempe, M., & Berger, J. (2025) · Science and Medicine in Football · doi:10.1080/24733938.2025.2517056 · Cited by 5 · see citing works
Lau, G. R., Kasturiratna, K. T. A. S., Goh, A. Y. H., Tong, E. M. W., & Hartanto, A. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101186 · Cited by 7 · see citing works
Using Large Language Models in Behavioral Science Interventions: Promise & Risk
Hecht, C. A., Ong, D. C., Clapper, M., Jones, M., Demszky, D., Yang, D., Eichstaedt, J. C., Bryan, C. J., & Yeager, D. S. (2025) · Behavioral Science & Policy · doi:10.1177/23794607251344698 · Cited by 7 · see citing works
Erlei, A., Cau, F. M., Georgiev, R., Kumar, S. C., Bizer, K., & Gadiraju, U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791420 · Cited by 4 · see citing works
Understanding the Effects of AI-Assisted Critical Thinking on Human-AI Decision Making
Tian, H. Y., Amin, H., & Yin, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790785 · Cited by 5 · see citing works
Demystifying XAI: Requirements for Understandable XAI Explanations
Stodt, J., Reich, C., & Knahl, M. (2024) · Studies in health technology and informatics · doi:10.3233/shti240477 · Cited by 3 · see citing works
Molaei, S., Robert, L., & Banović, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757437 · Cited by 2 · see citing works
The Enterprising and Elusive Prospects of Human-AI Collaboration
Gadiraju, U., & Balayn, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_7 · Cited by 2 · see citing works
Improving Human Verification of LLM Reasoning through Interactive Explanation Interfaces
Zhou, R., Nguyen, G., Kharya, N., Nguyen, A., & Agarwal, C. (2026) · Proceedings of the 31st International Conference on Intelligent User Interfaces · doi:10.1145/3742413.3789134 · Cited by 2 · see citing works
Efficiency Pitfalls of Explainable AI in Clinical Diagnostic and Treatment Human-AI Workflows
Hunsicker, T., Schulz, A., Leist, R. A., Kiefer, S., Boden, K., Rothaus, K., König, C. J., & Langer, M. (2026) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208261443764 · Cited by 2 · see citing works
Hansel, J., & Zdravković, M. (2026) · Anaesthesia · doi:10.1111/anae.70302 · Cited by 2 · see citing works
AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of InterventionsPreprint
Schoeffer, J., Jakubik, J., Vössing, M., Kühl, N., & Satzger, G. (2024) · doi:10.31219/osf.io/cekm9 · Cited by 2 · see citing works
Confidence Contours: Uncertainty-Aware Annotation for Medical Semantic Segmentation
Ye, A., Chen, Q. Z., & Zhang, A. X. (2023) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v11i1.27559 · Cited by 1 · see citing works
Appropriate Reliance in Human-AI Collaborative Problem-Solving: The Role of Critical Thinking
Hou, C., Zhu, G., & Sudarshan, V. K. (2025) · Computer-supported collaborative learning/The Computer-Supported Collaborative Learning Conference · doi:10.22318/cscl2025.608996 · Cited by 1 · see citing works
Disentangling Stakeholder Role and Expertise in User-Centered Explainable AI
Szymanski, M., Abeele, V. V., & Verbert, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728351 · Cited by 1 · see citing works
Machine Learning in Smart Manufacturing: Challenges and Solutions
Adalat, O., Polovina, N., & Konur, S. (2026) · Applied Sciences · doi:10.3390/app16178599 · Cited by 1 · see citing works
Integrating XAI into Safe AI Development and AI Certification Processes: An Expert-Based Analysis
Frész, B., Göbels, V. P., Omri, S., Brajovic, D., Aichele, A., Kutz, J., Neuhüttler, J., & Huber, M. F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30849-8_13 · Cited by 6 · see citing works
Schoeffer, J., De‐Arteaga, M., & Kühl, N. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5029746
Schoeffer, J., De‐Arteaga, M., & Elmer, J. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732070
MAGIX: A Unified Framework for the Use of XAI in Enterprises
Sovrano, F., & Miller, T. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_6
Critical thinking in human-AI collaborationsPreprint
Bilen, E., & Hervé, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6304755
Braca, D., Shahu, A., Talypova, D., Wintersberger, P., & Hubig, N. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789102
Degachi, S., Niforatos, E., & Kortuem, G. (2026) · Behaviour & Information Technology · doi:10.1080/0144929x.2026.2638907 · Cited by 1 · see citing works
Understanding the Affordances of Control in AI Reasoning for Human-AI Decision-Making
Moon, J., Huang, C., & Xiao, Z. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798555
AI Error Difficulty Modulates the Effectiveness of Explainability in Decision Support Systems
Rieger, T., Schindler, H., Koch, K., & Onnasch, L. (2026) · ACM Trans. Comput. Hum. Interact. · doi:10.1145/3817603
Rieger, T., & Onnasch, L. (2026) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208261468618
Explainable AI, Academic Confidence, and Decision-Making Quality in Physics Learning
Hidayah, N., & Sodikin, S. (2026) · Digital Learning Social Science and Life-course Studies · doi:10.70211/disolife.v2i1.777 · Cited by 3 · see citing works
Data-Centric ML for Regulatory Compliance: A Survey and Taxonomy
Brajovic, D., & Huber, M. F. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-32732-1_43
Li, M., Shu, J., Chang, F., Peng, T., Chen, X., Wu, Y., & Tao, T. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2739769
Rehman, U. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1924913
- Authors
- Fok, R., & Weld, D. S.
- Affiliations
- University of Washington (US)
- Allen Institute for Artificial Intelligence (US)
- Corresponding author
- Fok, Raymond
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Volume
- 45
- Issue
- 3
- Pages
- 317-332
- ISSN
- 0738-4602 (ISSN-L)
- arXiv
- 2305.07722
- OpenAlex
- W4400195536
- Cited by
- 54
- References
- 76
- Citations by year
- 2026: 18 · 2025: 20 · 2024: 13 · 2023: 3
- FWCI
- 7.56
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- AI Magazine (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- University of Washington
- Allen Institute for Artificial Intelligence
- Office of Naval Research (N00014‐21‐1‐2707, N00014)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- explainable AI, AI explanations, verifiability, human-AI collaboration, interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
