How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Open accessThe DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Abstract
The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to complex AI systems. Although it is usually considered an essentially technical field, effort has been made recently to better understand users' human explanation methods and cognitive constraints. Despite these advances, the community lacks a general vision of what and how cognitive biases affect explainability systems. To address this gap, we present a heuristic map which matches human cognitive biases with explainability techniques from the XAI literature, structured around XAI-aided decision-making. We identify four main ways cognitive biases affect or are affected by XAI systems: 1) cognitive biases affect how XAI methods are designed, 2) they can distort how XAI techniques are evaluated in user studies, 3) some cognitive biases can be successfully mitigated by XAI techniques, and, on the contrary, 4) some cognitive biases can be exacerbated by XAI techniques. We construct this heuristic map through the systematic review of 37 papers-drawn from a corpus of 285-that reveal cognitive biases in XAI systems, including the explainability method and the user and task types in which they arise. We use the findings from our review to structure directions for future XAI systems to better align with people's cognitive processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“First, a battery of empirical studies found that people rarely trigger analytical thinking when directly presented with AI's suggestions [11, 15, 105].”
From Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making (2025) · cites this paper as [11]
In the citing paper
“Human-centered evaluations often measure subjective qualities such as user satisfaction or opinion by directly questioning participants.While they provide a holistic perspective on the explanation as they are embedded in the AI system and context, they also introduce additional noise.They do not directly measure the desired qualities but have to rely on proxy questions [42] which introduce human b”
From Study on the Helpfulness of Explainable Artificial Intelligence (2024) · cites this paper as [4]
In the citing paper
“Prior research has shown that certain cognitive biases can be amplified by specific explanation styles and designs [43]”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [43]
In the citing paper
“This can counter possible human cognitive biases [8,97] and avoid making people directly deal with probabilities.”
From Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making (2023) · cites this paper as [8]
In the citing paper
“XAI has been a main subject of reviews as well, not only to structure existing techniques (Sheu and Pardeshi, 2022), but also to investigate human factors that arise during decision making, including cognitive biases (Bertrand et al., 2022) and cognitive processes supported by interactive explanations (Bertrand et al., 2023).”
From Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review (2025) · cites this paper as Bertrand et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Judgment under Uncertainty: Heuristics and Biases
(1982) · doi:10.1017/cbo9780511809477 · Cited by 8,240
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 1×
“To address this gap, researchers [12]–[14] have been using insights about people’s cognition and behaviors such as dual-process theory [15], which states that people rely on heuristics and cognitive biases to process information and make decisions.”
BackgroundExplanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
When Corrections Fail: The Persistence of Political Misperceptions
Nyhan, B., & Reifler, J. (2010) · Political Behavior · doi:10.1007/s11109-010-9112-2 · Cited by 3,173
Research works printed but not linked: 18
- H. Lakkaraju , E. Kamar , R. Caruana , and J. Leskovec , " Interpretable & Explorable Approximations of Black Box Models," ArXiv170701154 Cs , Jul. 2017 . H. Lakkaraju, E. Kamar, R. Caruana, and J. Leskovec, "Interpretable & Explorable Approximations of Black Box Models," ArXiv170701154 Cs, Jul. 2017.
- 10.1145/3233231 · doi:10.1145/3233231 · Paper page
- D. Kahneman , Thinking , fast and slow . New York, NY, US : Farrar, Straus and Giroux , 2011 , p. 499. D. Kahneman, Thinking, fast and slow. New York, NY, US: Farrar, Straus and Giroux, 2011, p. 499.
- A. Springer and S. Whittaker , " Progressive disclosure: empirically motivated approaches to designing effective transparency," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY , USA , Mar. 2019 , pp. 107 -- 120 . A. Springer and S. Whittaker, "Progressive disclosure: empirically motivated approaches to designing effective transparency," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY, USA, Mar. 2019, pp. 107--120.
- D. J. Hilton , "Logic and causal attribution," in Contemporary science and natural explanation : Commonsense conceptions of causality , New York, NY, US : New York University Press , 1988 , pp. 33 -- 65 . D. J. Hilton, "Logic and causal attribution," in Contemporary science and natural explanation: Commonsense conceptions of causality, New York, NY, US: New York University Press, 1988, pp. 33--65.
- M. M. A. de Graaf and B. F. Malle , " How People Explain Action (and Autonomous Intelligent Systems Should Too)," presented at the 2017 AAAI Fall Symposium Series , Oct. 2017 . M. M. A. de Graaf and B. F. Malle, "How People Explain Action (and Autonomous Intelligent Systems Should Too)," presented at the 2017 AAAI Fall Symposium Series, Oct. 2017.
- U. Bhatt : Measuring, Communicating, and Using Uncertainty," in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, New York, NY , USA , Jul. 2021 , pp. 401 -- 413 . U. Bhatt et al., "Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty," in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, New York, NY, USA, Jul. 2021, pp. 401--413.
- T. Kulesza , S. Stumpf , M. Burnett , S. Yang , I. Kwan , and W. Wong , " Too much, too little, or just right? Ways explanations impact end users' mental models," in 2013 IEEE Symposium on Visual Languages and Human Centric Computing , Sep. 2013 , pp. 3 -- 10 . T. Kulesza, S. Stumpf, M. Burnett, S. Yang, I. Kwan, and W. Wong, "Too much, too little, or just right? Ways explanations impact end users' mental models," in 2013 IEEE Symposium on Visual Languages and Human Centric Computing, Sep. 2013, pp. 3--10.
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
- N. M. Barbosa and M. Chen , " Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY , USA , May 2019 , pp. 1 -- 12 . N. M. Barbosa and M. Chen, "Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, May 2019, pp. 1--12.
- V. Danry , P. Pataranutaporn , Y. Mao , and P. Maes , " Wearable Reasoner: Towards Enhanced Human Rationality Through A Wearable Device With An Explainable AI Assistant," in Proceedings of the Augmented Humans International Conference, New York, NY , USA , Mar. 2020 , pp. 1 -- 12 . V. Danry, P. Pataranutaporn, Y. Mao, and P. Maes, "Wearable Reasoner: Towards Enhanced Human Rationality Through A Wearable Device With An Explainable AI Assistant," in Proceedings of the Augmented Humans International Conference, New York, NY, USA, Mar. 2020, pp. 1--12.
- L. Coba , L. Rook , M. Zanker , and P. Symeonidis , " Decision making strategies differ in the presence of collaborative explanations: two conjoint studies," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY , USA , Mar. 2019 , pp. 291 -- 302 . L. Coba, L. Rook, M. Zanker, and P. Symeonidis, "Decision making strategies differ in the presence of collaborative explanations: two conjoint studies," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY, USA, Mar. 2019, pp. 291--302.
- A. Simkute , E. Luger , M. Evans , and R. Jones , " Experts in the Shadow of Algorithmic Systems: Exploring Intelligibility in a Decision-Making Context," in Companion Publication of the 2020 ACM Designing Interactive Systems Conference, New York, NY , USA , Jul. 2020 . A. Simkute, E. Luger, M. Evans, and R. Jones, "Experts in the Shadow of Algorithmic Systems: Exploring Intelligibility in a Decision-Making Context," in Companion Publication of the 2020 ACM Designing Interactive Systems Conference, New York, NY, USA, Jul. 2020.
- G. A. Klein , Sources of Power : How People Make Decisions . Nature , 1988 . G. A. Klein, Sources of Power: How People Make Decisions. Nature, 1988.
- T. Kim and H. Song , " The Effect of Message Framing and Timing on the Acceptance of Artificial Intelligence's Suggestion," in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems , New York, NY, USA , Apr. 2020 , pp. 1 -- 8 . T. Kim and H. Song, "The Effect of Message Framing and Timing on the Acceptance of Artificial Intelligence's Suggestion," in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, Apr. 2020, pp. 1--8.
- H. Suresh , S. R. Gomez , K. K. Nam , and A. Satyanarayan , " Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, New York , NY, USA , May 2021 , pp. 1 -- 16 . H. Suresh, S. R. Gomez, K. K. Nam, and A. Satyanarayan, "Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, May 2021, pp. 1--16.
- G. Bansal AI Explanations on Complementary Team Performance ," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , New York, NY, USA , May 2021 , pp. 1 -- 16 . G. Bansal et al., "Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, May 2021, pp. 1--16.
- M. McKenna and D. J. Coates , " Compatibilism," in The Stanford Encyclopedia of Philosophy , Fall 2021 ., E. N. Zalta, Ed . Metaphysics Research Lab, Stanford University , 2021. M. McKenna and D. J. Coates, "Compatibilism," in The Stanford Encyclopedia of Philosophy, Fall 2021., E. N. Zalta, Ed. Metaphysics Research Lab, Stanford University, 2021.
Cited by
Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
AI and the quest for diversity and inclusion: a systematic literature review
Shams, R. A., Zowghi, D., & Bano, M. (2025) · AI and Ethics · doi:10.1007/s43681-023-00362-w · Cited by 192
Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Romeo, G., & Conti, D. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02422-7 · Cited by 209
“While their intuition is more refined than non-experts’ one, it also makes experts more susceptible to belief perseverance and algorithmic aversion (Bertrand et al. 2022).”
InfluentialMa, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143
“This can counter possible human cognitive biases [8, 97] and avoid making people directly deal with probabilities.”
BackgroundHuman-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130
Rams, hounds and white boxes: investigating human–AI collaboration protocols in medical diagnosis
Cabitza, F., Campagner, A., Ronzio, L., Cameli, M., Mandoli, G. E., Pastore, M. C., Sconfienza, L. M., Folgado, D., Barandas, M., & Gamboa, H. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102506 · Cited by 117
Sokol, K., Fackler, J., & Vogt, J. E. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01725-9 · Cited by 105
“Rudimen-tary research in this direction has explored the influence of selected AI explanation types on human cognitive biases, constraints and reasoning faults, demonstrating that while some can be mitigated, others may inadvertently be exacerbated 131–133 .”
Explainable artificial intelligence for spectroscopy data: a review
Contreras, J., & Bocklitz, T. W. (2024) · Pflügers Archiv - European Journal of Physiology · doi:10.1007/s00424-024-02997-y · Cited by 139
Brasse, J., Broder, H. R., Förster, M., Klier, M., & Sigler, I. (2023) · Electronic Markets · doi:10.1007/s12525-023-00644-5 · Cited by 87
“The final group, lay users , includes individuals who are affected by AI decisions (Bertrand et al., 2022; Cooper, 2004; Ribera & Lapedriza, 2019), e.g., the bank customer who was approved or denied a credit line based on an AI system’s recommendation (Mittelstadt et al., 2019).”
InfluentialOpening the Black Box: A systematic review on explainable artificial intelligence in remote sensing
Höhl, A., Obadić, I., Fernández‐Torres, M., Najjar, H., Oliveira, D. A. B., Akata, Z., Dengel, A. R., & Zhu, X. X. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3467001 · Cited by 142
Performance and Metacognition Disconnect when Reasoning in Human-AI Interaction
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2024) · arXiv · doi:10.48550/arxiv.2409.16708 · Cited by 71
Medical artificial intelligence for clinicians: the lost cognitive perspective
Tikhomirov, L., Semmler, C., McCradden, M., Searston, R., Ghassemi, M., & Oakden-Rayner, L. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00095-5 · Cited by 63
Martins, T., de Almeida, A., Cardoso, E., & Nunes, L. (2023) · IEEE Access · doi:10.1109/access.2023.3347028 · Cited by 98
“…literature review, the authors systematically analyze papers looking for ways to tackle the problem of cognitive bias or the ‘‘systematic error in judgment and decision-making common to all human beings’’ (as defined in [21]) that has been found in XAI methods used in decision-making systems [17].”
BackgroundThe Review of Studies on Explainable Artificial Intelligence in Educational Research
Türkmen, G. (2024) · Journal of Educational Computing Research · doi:10.1177/07356331241310915 · Cited by 97
“Finally, Bertrand et al. (2022) analyzed the impact of cognitive biases in XAI systems and stated that methods that using XAI.”
Ma, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642671 · Cited by 54
Ma, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2403.09552 · Cited by 56
Braca, A., & Dondio, P. (2023) · Italian Journal of Marketing · doi:10.1007/s43039-023-00077-0 · Cited by 137
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
Bach, A. K. P., Nørgaard, T. M., Brok, J. C., & van Berkel, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581513 · Cited by 47
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 115
“Prior studies found that human factors like expertise and domain knowledge [18, 83] and cognitive bias [7, 46] can greatly affect user trust [117] and appropriate reliance [101] on the AI system.”
Understanding plant phenotypes in crop breeding through explainable AI
Danilevicz, M. F., Upadhyaya, S. R., Batley, J., Bennamoun, M., Bayer, P. E., & Edwards, D. (2025) · Plant Biotechnology Journal · doi:10.1111/pbi.70208 · Cited by 56
“…on its application for understanding plant phenotypes, but a wealth of more general tutorials have been previously published (Bayer et al ., 2021a; Bertrand et al ., 2022; Das and Rad, 2020; Kaur et al ., 2020; Molnar, 2019; Schemmer et al ., 2022), with a noteworthy mention to a couple of plant…”
Müller, R. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2381929 · Cited by 54
“Several review articles have collected such countermeasures that are largely based on human-machine interaction research (e.g., Bertrand et al., 2022; de Visser et al., 2020).”
BackgroundInfluentialTo Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models
Bo, J. Y., Wan, S., & Anderson, A. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714097 · Cited by 36
“Additionally, explainable AI (XAI) research has found it effective to focus on increasing the user’s cognitive engagement in the task, such as through simple cognitive forcing functions that add frictions and delays to the interface or removing direct recommendations [9, 28].”
Cabitza, F., Fregosi, C., Campagner, A., & Natali, C. (2024) · Communications in Computer and Information Science · doi:10.1007/978-3-031-63803-9_14 · Cited by 35
“Evidence Evidence strongly confirms and supports the utility of XAI in decision-making processes, highlighting its contribution to improved accuracy [10, 2, 3].”
BackgroundResultInfluentialWhy we do need explainable AI for healthcare
Cinà, G., Röber, T. E., Goedhart, R., & Birbil, Ş. İ. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00209-4 · Cited by 53
Advice from artificial intelligence: a review and practical implications
Baines, J. I., Dalal, R. S., Ponce, L. P., & Tsai, H.-C. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1390182 · Cited by 28
“Therefore, despite the intelligibility of simpler explanations, it has been recommended that AI advisor developers should focus on providing coherent and broad explanations, with a focus on scope over simplicity (Bertrand et al., 2022).”
BackgroundInfluentialWang, K., Hou, W., Hong, L., & Guo, J. (2025) · Electronics · doi:10.3390/electronics14030420 · Cited by 40
New XAI tools for selecting suitable 3D printing facilities in ubiquitous manufacturing
Wang, Y.-C., & Chen, T. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-023-01104-5 · Cited by 27
PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation
Liu, Y., Sharma, P., Oswal, M., Xia, H., & Huang, Y. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735789 · Cited by 87
Bertrand, A., Eagan, J., & Maxwell, W. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594053 · Cited by 61
“This makes it particularly challenging to make explanations both simple and appealing to lay users, while encouraging cognitive engagement and skepticism [4, 36].”
BackgroundSelf-citationBergomi, L., Nicora, G., Orlowska, M. A., Podrecca, C., Bellazzi, R., Fregosi, C., Salinaro, F., Bonzano, M., Crescenzi, G., Speciale, F., et al. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03045-0 · Cited by 34
Operationalizing Explainable Artificial Intelligence in the European Union Regulatory Ecosystem
Nannini, L., Alonso, J. M., Catalá, A., Lama, M., & Barro, S. (2024) · IEEE Intelligent Systems · doi:10.1109/mis.2024.3383155 · Cited by 43
“3,5 Failure to incorporate contextual factors could jeopar-dize the effectiveness of AI explanations.”
BackgroundInfluentialHuman–machine collaboration for enhanced decision-making in governance
Rooy, D. V. (2024) · Data & Policy · doi:10.1017/dap.2024.72 · Cited by 29
“However, it is worth mentioning that while xAI clarifies AI processes, it does not inherently correct for, and can in fact exacerbate decision bias (Bertrand et al., 2022).”
How Do HCI Researchers Study Cognitive Biases? A Scoping Review
Boonprakong, N., Tag, B., Gonçalves, J., & Dingler, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713450 · Cited by 40
“Researchers have also explored cognitive biases in AI-assisted decision-making [14, 69, 104, 200].”
Mapping the landscape of ethical considerations in explainable AI research
Nannini, L., Manerba, M. M., & Beretta, I. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09773-7 · Cited by 38
“Identifying appropriate standards requires normative deliberation, not just computational evaluation, always cognizant that XAI techniques can perpetuate biases if not carefully designed (Bertrand et al., 2022; Chaudhuri & Salakhutdi-nov, 2019; Shamsabadi et al., 2022).”
Do Expressions Change Decisions? Exploring the Impact of AI's Explanation Tone on Decision-Making
Okoso, A., Yang, M., & Baba, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713744 · Cited by 32
“Repeated absence of discrepancies may also lead participants to fully rely on AI predictions, exhibiting automation bias [6].”
Invisible to Machines: Designing AI that Supports Vision Work in Radiology
Anichini, G., Natali, C., & Cabitza, F. (2024) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-024-09491-0 · Cited by 18
“…computational decision aids are stipulated (van Berkel et al. 2021), including the modality in which the AI output is presented to the user and the decision-making step at which the result is provided (e.g., before or after a first decision was made by the clinician, as in Bertrand et al. 2022).”
Jung, I.-C., Schuler, K., Zerlik, M., Grummt, S., Sedlmayr, M., & Sedlmayr, B. (2025) · Digital Health · doi:10.1177/20552076241308298 · Cited by 18
Kaur, H., Conrad, M. R., Rule, D., Lampe, C. A. C., & Gilbert, É. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637354 · Cited by 33
Liel, Y., & Zalmanson, L. (2025) · Journal of Management Information Systems · doi:10.1080/07421222.2025.2561390 · Cited by 24
Bhusal, D., Shin, H. J., Shewale, A. A., Veerabhadran, M. K., Clifford, M., Rampazzi, S., & Rastogi, N. (2023) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1145/3600160.3600193 · Cited by 36
Deceptive XAI: Typology, Creation and Detection
Schneider, J., Meske, C., & Vlachos, M. (2023) · SN Computer Science · doi:10.1007/s42979-023-02401-z · Cited by 16
Framework for human–XAI symbiosis: extended self from the dual-process theory perspective
Litvinova, Y., Mikalef, P., & Luo, X. R. (2024) · Journal of Business Analytics · doi:10.1080/2573234x.2024.2396366 · Cited by 20
“Acknowledging D. Wang et al. (2019) contribution, Bertrand et al. (2022) conducted a comprehensive review of human biases discussed in XAI literature.”
InfluentialSaliency-Bench: A Comprehensive Benchmark for Evaluating Visual Explanations
Zhang, Y., Song, J., Gu, S., Jiang, T., Pan, B., Bai, G., & Zhao, L. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737414 · Cited by 29
The Human Side of XAI: Bridging the Gap between AI and Non-expert Audiences
Severes, B., Carreira, C., Vieira, A. B., Gomes, E. R., Aparício, J. T., & Pereira, I. (2023) · ACM International Conference on Design of Communication (SIGDOC) · doi:10.1145/3615335.3623062 · Cited by 20
Nullius in Explanans: an ethical risk assessment for explainable AI
Nannini, L., Huyskes, D., Panai, E., Pistilli, G., & Tartaro, A. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09800-7 · Cited by 18
“…through quantitative measures, often without sufficient involvement from all stakeholders affected by these explanations (Kaur et al., 2020; Schem-mer et al., 2022) or unclear benefits for their usefulness (Bertrand et al., 2022; Chen et al., 2023; Schemmer et al., 2022; Vasconcelos et al., 2023).”
Robot Navigation in Risky, Crowded Environments: Understanding Human Preferences
Suresh, A., Taylor, A. M., Riek, L. D., & Martínez, S. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3290533 · Cited by 15
Offloading Wisdom: Four Technological Relations that Mediate Phronesis
Zelny, A. S. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00889-2 · Cited by 16
“…with the fact that many of our AI-assisted decisions are affected by cognitive biases such as confirmation, anchoring, and availability heuristics (Bertrand et al., 2022; Rastogi et al., 2022) there is a worry that our deliberative capacities might be negatively influenced by technological…”
Let Me Think! Investigating the Effect of Explanations Feeding Doubts About the AI Advice
Cabitza, F., Campagner, A., Famiglini, L., Natali, C., Caccavella, V. M., & Gallazzi, E. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40837-3_10 · Cited by 20
Selvakumar, P., Haashni, S., SudhaAnanthi, G., & Manjunath, T. C. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-2484-5.ch004 · Cited by 10
Color Shadows 2: Assessing the Impact of XAI on Diagnostic Decision-Making
Natali, C., Famiglini, L., Campagner, A., Maida, G. A. L., Gallazzi, E., & Cabitza, F. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_33 · Cited by 12
Avoiding Decision Fatigue with AI-Assisted Decision-Making
Echterhoff, J. M., Melkote, A., Kancherla, S., & McAuley, J. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659569 · Cited by 15
The Amplifying Effect of Explainability in AI-assisted Decision-making in Groups
Duarte, R. D. B., Abreu, M. C., Campos, J., & Paiva, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713534 · Cited by 19
“One possible explanation for this effect is the amplification of cognitive biases such as the anchoring bias [7, 43, 61] and automation bias [35] in group dynamics.”
LLM-Driven APT Detection for 6G Wireless Networks: A Systematic Review and Taxonomy
Golec, M., Khamayseh, Y., Melhem, S. B., & Alwarafy, A. (2025) · IEEE Access · doi:10.1109/access.2025.3595665 · Cited by 13
“In addition, XAI methods for decision-making mechanisms are another frequently investigated method [44].”
The Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 15
Jackson, N. J., Brown, K. E., Rn, M., Murrow, M., Cauley, M. R., Collins, B. X., Novak, L. L., Benda, N. C., & Ancker, J. S. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag002 · Cited by 19
“84 However, we also identified conflicting evidence where xAI led to more appropriate trust.”
Ceva, E., & Jiménez, M. C. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09670-x · Cited by 14
“XAI techniques may indeed exacerbate cognitive bias amongst users, including the “illusion of explanatory depth” triggered by an automated explanation in conditions of alleged process transparency (Bertrand et al., 2022).”
BackgroundWorkshop on Understanding and Mitigating Cognitive Biases in Human-AI Collaboration
Boonprakong, N., He, G., Gadiraju, U., van Berkel, N., Wang, D., Chen, S., Liu, J., Tag, B., Gonçalves, J., & Dingler, T. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3611284 · Cited by 13
SIDEs: Separating Idealization from Deceptive 'Explanations' in xAI
Sullivan, E. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658999 · Cited by 17
Current Trends, Challenges and Techniques in XAI Field; A Tertiary Study of XAI Research
Brdnik, S., & Šumak, B. (2024) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro60963.2024.10569528 · Cited by 21
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4772689 · Cited by 8
Study on the Helpfulness of Explainable Artificial Intelligence
Labarta, T., Kulicheva, E., Froelian, R., Geißler, C., Melman, X., & von Klitzing, J. (2024) · Communications in Computer and Information Science · doi:10.1007/978-3-031-63803-9_16 · Cited by 8
Wijaya, E. K., Gaol, F. L., & Matsuo, T. (2024) · Emerging Science Journal · doi:10.28991/esj-2024-08-01-011 · Cited by 10
“Decision support models may not always yield completely accurate outcomes due to their reliance on a technology that emulates human decision-making processes [39].”
BackgroundBertrand, A., Eagan, J., Maxwell, W., & Brand, J. L. M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642326 · Cited by 10
Self-citationHolstein, J., Böcking, L., Spitzer, P., Kühl, N., Vössing, M., & Satzger, G. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3762813 · Cited by 17
“These prior beliefs remain relevant in the context of decision-support from AI systems and warrant further investigation [9, 36, 93, 102].”
Chen, T. (2024) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-031-54935-9_1 · Cited by 6
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
“Existing reviews in HCAI can be broadly categorized into XAI [2, 6, 92], trust [100, 162], interactions (i.e., interactive Machine Learning (ML)) [45, 121], or related domains [1, 7, 49, 81].”
Increasing system transparency through confidence information in cooperative, automated driving
Peintner, J. B., Manger, C., & Riener, A. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2407018 · Cited by 7
“Bertrand et al. (2022) stress the importance of explicitly stated confidence levels when humans have to contribute to and cooperate with machine-learning-assisted decision-making.”
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 20
“…black-box AI models, eXplainable AI techniques have emerged to help users understand how the AI reached a specific decision in low-and high-stakes situations, including high-uncertainty and safety-critical contexts (Bertrand et al. 2022; Lai et al. 2023a; Rong et al. 2024; Subramanian et al. 2024).”
Guo, Y. Q., Tang, Q., Wang, H., Jia, M., & Wang, W. (2024) · Industrial Management & Data Systems · doi:10.1108/imds-12-2023-0905 · Cited by 11
Where Are We and Where Can We Go on the Road to Reliance-Aware Explainable User Interfaces?
Filho, J. C. D. S., Belloum, R., & de Oliveira, K. M. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00038 · Cited by 8
Self-citationSmart Technologies for Fall Detection and Prevention
Chen, T., & Lee, Y. (2024) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-031-59443-4_5 · Cited by 5
From Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Raees, M., & Papangelis, K. (2026) · arXiv · doi:10.48550/arxiv.2604.23896 · Cited by 5
“Bertrand et al. [5] analyzed how explanations affect decision-making, examining users’ cognitive biases and trust.”
Szymczyk, N., Szczuka, J. M., Krämer, N. C., & Wischnewski, M. (2025) · Mensch und Computer (MuC) · doi:10.1145/3743049.3743072 · Cited by 6
Kokje, E., Lermer, E., Kleine, A., & Gaube, S. (2026) · Cognitive Research Principles and Implications · doi:10.1186/s41235-026-00707-z · Cited by 5
“…lack of improvement in performance here resulted from underreliance on AI advice, whereas studies that report adverse effects of explainable advice widely attribute it to increased overreliance due to increased trust in AI (Bertrand et al., 2022; Rosenbacke et al., 2024; Schem-mer et al., 2022).”
Effects of AI explanations on trust and reliance: a study in job shop scheduling
Saßmannshausen, T. M., Burggräf, P., Hassenzahl, M., & Sauer, C. R. (2026) · Ergonomics · doi:10.1080/00140139.2026.2634115 · Cited by 4
Human-Agent Interaction and Human Dependency: Possible New Approaches for Old Challenges
Carli, R., Najjar, A., & Al‐Thani, D. (2024) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3687272.3688308 · Cited by 8
Rubegni, E., Ayoub, O., Rizzo, S., Barbero, M., Bernegger, G., Faraci, F. D., Mangili, F., Soldini, E., Trimboli, P., & Facchini, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60606-9_16 · Cited by 9
Van Dort, B. A., Engelsma, T., Sneekes, P., Peute, L., & Medlock, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001417 · Cited by 9
“…the explanation (5) Express risk as whole-number percentages to reduce confusion Prior research has shown that certain cognitive biases can be amplified by specific explanation styles and designs [43] and that providing too much detail may reduce user performance due to information overload [44].”
Folgado, D., Famiglini, L., Campagner, A., Dores, H., Barandas, M., Gamboa, H. F. S., & Cabitza, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-95838-0_14 · Cited by 6
Design Principles for Human-Centred Explainable AI: A Scoping Review
Hughes, N. G. J., Jia, Y., Sujan, M. A., Lawton, T., Habli, I., & McDermid, J. A. (2026) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3771720 · Cited by 6
Erlei, A., Cau, F. M., Georgiev, R., Kumar, S. C., Bizer, K., & Gadiraju, U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791420 · Cited by 7
“While mechanisms such as reputation systems, third-party certi-fications, and disclosure labels (e.g., Apple’s privacy “nutrition” labels) aim to bridge this gap [60, 126, 127, 132], they often fail in practice due to selective reporting, high cognitive load, or strategic complexity that undermines…”
Heuristics and Biases in AI Decision-Making: Implications for Responsible AGI
Saeedi, P., Goodarzi, M., & Canbaz, M. A. (2025) · International Conference on Artificial Intelligence, Robotics and Control (AIRC) · doi:10.1109/airc64931.2025.11077505 · Cited by 5
WordIllusion: An adversarial text generation algorithm based on human cognitive system
Fu, H., Wang, C., Sun, J., Zhao, Y., Lin, H., Sun, J., & Zhang, B. (2023) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101179 · Cited by 3
Human-Centered AI (Also) for Humanistic Management
Facchini, A., & Mangili, F. (2024) · Humanism in business series · doi:10.1007/978-3-031-67155-5_11 · Cited by 5
Ethical Challenges in Explainable AI: A Review on Cultural and Social Bias
Thalpage, N. S., Epa, D., & Jayawardhana, S. (2025) · Journal of Digital Art & Humanities · doi:10.33847/2712-8148.6.1_3 · Cited by 4
The Enterprising and Elusive Prospects of Human-AI Collaboration
Gadiraju, U., & Balayn, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_7 · Cited by 5
Maximizing impact of explainable artificial intelligence in radiotherapy: a critical review
Heising, L. M., Wolfs, C. J. A., Ou, C. X. J., Hoebers, F. J. P., van Limbergen, E. J., Verhaegen, F., & Jacobs, M. J. G. (2026) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ae25b2 · Cited by 2
“Confirmation bias occurs when individuals favor information that confirms their pre-existing beliefs, which can be exacerbated by incomplete or improper XAI (Bertrand et al 2022).”
Mitigating Bias in AI Recruitment: Leveraging LIME for Fair and Transparent Hiring Models
Fares, N. Y., Moore, S. A., & Jammal, M. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-99958-1_29 · Cited by 4
Abreu, M. C., Duarte, R. D. B., Neves, B. D., Silva, N. A., Santos, R., & Campos, J. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3788685 · Cited by 4
Hoang, A., Phan, H., & Nguyen, V.-D. (2026) · Studies in big data · doi:10.1007/978-3-032-06179-9_9 · Cited by 7
Maathuis, H., Stalenhoef, M., van Otterloo, S., Zwaal, R., van Montfort, K., & Sent, D. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1668029 · Cited by 6
Charting the Ethical Landscape: A Holistic Examination of AI Ethics and Bias in the Financial Sector
Pria, S., Rubaie, I. A., & Prasad, B. V. V. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-6215-0.ch015 · Cited by 2
Raulino, N. L. G., Andrade, R. M. C., & Santos, I. D. S. (2024) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3702038.3702097 · Cited by 3
Designing the interaction between humans and autonomous systems: The role of behavioral science.
Rooy, D. V. (2023) · International Association of Societies of Design Research Congress (IASDR) · doi:10.21606/iasdr.2023.457 · Cited by 3
Explainability in Process Mining: A Framework for Improved Decision-Making
Nannini, L. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604729 · Cited by 5
Chen, L., Fang, Z., Jin, X., & Cai, X. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.104185 · Cited by 2
The Effects of Citations and Confirmation Bias on Trust in Chatbots
Lenz, T., Brackey, R., & Liu, J. (2025) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813251357884 · Cited by 1
“…trustworthiness of AI chatbot responses appears to depend on the topic of their content: user queries about political topics may be phrased in a way that elicits a response they desire, and so the response may be deemed more trustworthy (Bertrand et al., 2022; Ding et al., 2025; Shi et al., 2024).”
Murcia, A. R. S., & Torres, J. P. C. (2025) · CITAS · doi:10.15332/24224529.10549 · Cited by 3
Weaponizing cognitive bias in autonomous systems: a framework for black-box inference attacks
Chu, S., & Chen, Y. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1623573 · Cited by 1
Metadata
- Authors
- Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W.
- Affiliations
- Télécom Paris (FR)
- Venue
- Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2022-07-26
- Pages
- 78-91
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W4283032682
- Cited by
- 129
- References
- 66
- Citations by year
- 2026: 26 · 2025: 46 · 2024: 36 · 2023: 20 · 2022: 1
- FWCI
- 14.24
- Type
- conference paper
- Language
- en
- Open access
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- HAL (submittedVersion, other-oa)
- Funders
- Agence Nationale de la Recherche (ANR-20-CHIA-0023-01)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- cognitive biases, explainable AI, bias mitigation, heuristic maps, user study, human-AI interaction, cognitive processes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.