The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Open accessAbstract
The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to complex AI systems. Although it is usually considered an essentially technical field, effort has been made recently to better understand users' human explanation methods and cognitive constraints. Despite these advances, the community lacks a general vision of what and how cognitive biases affect explainability systems. To address this gap, we present a heuristic map which matches human cognitive biases with explainability techniques from the XAI literature, structured around XAI-aided decision-making. We identify four main ways cognitive biases affect or are affected by XAI systems: 1) cognitive biases affect how XAI methods are designed, 2) they can distort how XAI techniques are evaluated in user studies, 3) some cognitive biases can be successfully mitigated by XAI techniques, and, on the contrary, 4) some cognitive biases can be exacerbated by XAI techniques. We construct this heuristic map through the systematic review of 37 papers-drawn from a corpus of 285-that reveal cognitive biases in XAI systems, including the explainability method and the user and task types in which they arise. We use the findings from our review to structure directions for future XAI systems to better align with people's cognitive processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“First, a battery of empirical studies found that people rarely trigger analytical thinking when directly presented with AI's suggestions [11, 15, 105].”
From Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making (2025) · cites this paper as [11]
In the citing paper
“Human-centered evaluations often measure subjective qualities such as user satisfaction or opinion by directly questioning participants.While they provide a holistic perspective on the explanation as they are embedded in the AI system and context, they also introduce additional noise.They do not directly measure the desired qualities but have to rely on proxy questions [42] which introduce human b”
From Study on the Helpfulness of Explainable Artificial Intelligence (2024) · cites this paper as [4]
In the citing paper
“Prior research has shown that certain cognitive biases can be amplified by specific explanation styles and designs [43]”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [43]
In the citing paper
“This can counter possible human cognitive biases [8,97] and avoid making people directly deal with probabilities.”
From Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making (2023) · cites this paper as [8]
In the citing paper
“XAI has been a main subject of reviews as well, not only to structure existing techniques (Sheu and Pardeshi, 2022), but also to investigate human factors that arise during decision making, including cognitive biases (Bertrand et al., 2022) and cognitive processes supported by interactive explanations (Bertrand et al., 2023).”
From Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review (2025) · cites this paper as Bertrand et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Judgment under Uncertainty: Heuristics and Biases
(1982) · doi:10.1017/cbo9780511809477 · Cited by 8,240
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
When Corrections Fail: The Persistence of Political Misperceptions
Nyhan, B., & Reifler, J. (2010) · Political Behavior · doi:10.1007/s11109-010-9112-2 · Cited by 3,173
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Research works printed but not linked: 19
- H. Lakkaraju , E. Kamar , R. Caruana , and J. Leskovec , " Interpretable & Explorable Approximations of Black Box Models," ArXiv170701154 Cs , Jul. 2017 . H. Lakkaraju, E. Kamar, R. Caruana, and J. Leskovec, "Interpretable & Explorable Approximations of Black Box Models," ArXiv170701154 Cs, Jul. 2017.
- 10.1145/3233231 · doi:10.1145/3233231 · Paper page
- D. Kahneman , Thinking , fast and slow . New York, NY, US : Farrar, Straus and Giroux , 2011 , p. 499. D. Kahneman, Thinking, fast and slow. New York, NY, US: Farrar, Straus and Giroux, 2011, p. 499.
- A. Springer and S. Whittaker , " Progressive disclosure: empirically motivated approaches to designing effective transparency," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY , USA , Mar. 2019 , pp. 107 -- 120 . A. Springer and S. Whittaker, "Progressive disclosure: empirically motivated approaches to designing effective transparency," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY, USA, Mar. 2019, pp. 107--120.
- D. J. Hilton , "Logic and causal attribution," in Contemporary science and natural explanation : Commonsense conceptions of causality , New York, NY, US : New York University Press , 1988 , pp. 33 -- 65 . D. J. Hilton, "Logic and causal attribution," in Contemporary science and natural explanation: Commonsense conceptions of causality, New York, NY, US: New York University Press, 1988, pp. 33--65.
- M. M. A. de Graaf and B. F. Malle , " How People Explain Action (and Autonomous Intelligent Systems Should Too)," presented at the 2017 AAAI Fall Symposium Series , Oct. 2017 . M. M. A. de Graaf and B. F. Malle, "How People Explain Action (and Autonomous Intelligent Systems Should Too)," presented at the 2017 AAAI Fall Symposium Series, Oct. 2017.
- U. Bhatt : Measuring, Communicating, and Using Uncertainty," in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, New York, NY , USA , Jul. 2021 , pp. 401 -- 413 . U. Bhatt et al., "Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty," in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, New York, NY, USA, Jul. 2021, pp. 401--413.
- T. Kulesza , S. Stumpf , M. Burnett , S. Yang , I. Kwan , and W. Wong , " Too much, too little, or just right? Ways explanations impact end users' mental models," in 2013 IEEE Symposium on Visual Languages and Human Centric Computing , Sep. 2013 , pp. 3 -- 10 . T. Kulesza, S. Stumpf, M. Burnett, S. Yang, I. Kwan, and W. Wong, "Too much, too little, or just right? Ways explanations impact end users' mental models," in 2013 IEEE Symposium on Visual Languages and Human Centric Computing, Sep. 2013, pp. 3--10.
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
- N. M. Barbosa and M. Chen , " Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY , USA , May 2019 , pp. 1 -- 12 . N. M. Barbosa and M. Chen, "Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, May 2019, pp. 1--12.
- V. Danry , P. Pataranutaporn , Y. Mao , and P. Maes , " Wearable Reasoner: Towards Enhanced Human Rationality Through A Wearable Device With An Explainable AI Assistant," in Proceedings of the Augmented Humans International Conference, New York, NY , USA , Mar. 2020 , pp. 1 -- 12 . V. Danry, P. Pataranutaporn, Y. Mao, and P. Maes, "Wearable Reasoner: Towards Enhanced Human Rationality Through A Wearable Device With An Explainable AI Assistant," in Proceedings of the Augmented Humans International Conference, New York, NY, USA, Mar. 2020, pp. 1--12.
- L. Coba , L. Rook , M. Zanker , and P. Symeonidis , " Decision making strategies differ in the presence of collaborative explanations: two conjoint studies," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY , USA , Mar. 2019 , pp. 291 -- 302 . L. Coba, L. Rook, M. Zanker, and P. Symeonidis, "Decision making strategies differ in the presence of collaborative explanations: two conjoint studies," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY, USA, Mar. 2019, pp. 291--302.
- A. Simkute , E. Luger , M. Evans , and R. Jones , " Experts in the Shadow of Algorithmic Systems: Exploring Intelligibility in a Decision-Making Context," in Companion Publication of the 2020 ACM Designing Interactive Systems Conference, New York, NY , USA , Jul. 2020 . A. Simkute, E. Luger, M. Evans, and R. Jones, "Experts in the Shadow of Algorithmic Systems: Exploring Intelligibility in a Decision-Making Context," in Companion Publication of the 2020 ACM Designing Interactive Systems Conference, New York, NY, USA, Jul. 2020.
- G. A. Klein , Sources of Power : How People Make Decisions . Nature , 1988 . G. A. Klein, Sources of Power: How People Make Decisions. Nature, 1988.
- T. Kim and H. Song , " The Effect of Message Framing and Timing on the Acceptance of Artificial Intelligence's Suggestion," in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems , New York, NY, USA , Apr. 2020 , pp. 1 -- 8 . T. Kim and H. Song, "The Effect of Message Framing and Timing on the Acceptance of Artificial Intelligence's Suggestion," in Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, Apr. 2020, pp. 1--8.
- H. Suresh , S. R. Gomez , K. K. Nam , and A. Satyanarayan , " Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, New York , NY, USA , May 2021 , pp. 1 -- 16 . H. Suresh, S. R. Gomez, K. K. Nam, and A. Satyanarayan, "Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, May 2021, pp. 1--16.
- G. Bansal AI Explanations on Complementary Team Performance ," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , New York, NY, USA , May 2021 , pp. 1 -- 16 . G. Bansal et al., "Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, May 2021, pp. 1--16.
- M. McKenna and D. J. Coates , " Compatibilism," in The Stanford Encyclopedia of Philosophy , Fall 2021 ., E. N. Zalta, Ed . Metaphysics Research Lab, Stanford University , 2021. M. McKenna and D. J. Coates, "Compatibilism," in The Stanford Encyclopedia of Philosophy, Fall 2021., E. N. Zalta, Ed. Metaphysics Research Lab, Stanford University, 2021.
- J. Schaffer , J. O'Donovan , J. Michaelis , A. Raglin , and T. Höllerer , " I can do better than your AI: expertise and explanations," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY , USA , Mar. 2019 , pp. 240 -- 251 . J. Schaffer, J. O'Donovan, J. Michaelis, A. Raglin, and T. Höllerer, "I can do better than your AI: expertise and explanations," in Proceedings of the 24th International Conference on Intelligent User Interfaces, New York, NY, USA, Mar. 2019, pp. 240--251.
Cited by
Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
AI and the quest for diversity and inclusion: a systematic literature review
Shams, R. A., Zowghi, D., & Bano, M. (2025) · AI and Ethics · doi:10.1007/s43681-023-00362-w · Cited by 192
Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Romeo, G., & Conti, D. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02422-7 · Cited by 209
“While their intuition is more refined than non-experts’ one, it also makes experts more susceptible to belief perseverance and algorithmic aversion (Bertrand et al. 2022).”
InfluentialMa, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143
“This can counter possible human cognitive biases [8, 97] and avoid making people directly deal with probabilities.”
BackgroundHuman-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130
Metadata
- Authors
- Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W.
- Affiliations
- Télécom Paris (FR)
- Venue
- Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2022-07-26
- Pages
- 78-91
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W4283032682
- Cited by
- 129
- References
- 66
- Citations by year
- 2026: 26 · 2025: 46 · 2024: 36 · 2023: 20 · 2022: 1
- FWCI
- 14.24
- Type
- conference paper
- Language
- en
- Open access
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- HAL (submittedVersion, other-oa)
- Funders
- Agence Nationale de la Recherche (ANR-20-CHIA-0023-01)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- cognitive biases, explainable AI, bias mitigation, heuristic maps, user study, human-AI interaction, cognitive processes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review. https://app.cochat.ai/papers/10.1145/3514094.3534164
- The paper
- Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022). How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review. Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3514094.3534164
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.