Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Open accessAbstract
Several strands of research have aimed to bridge the gap between artificial intelligence (AI) and human decision-makers in AI-assisted decision-making, where humans are the consumers of AI model predictions and the ultimate decision-makers in high-stakes applications. However, people's perception and understanding are often distorted by their cognitive biases, such as confirmation bias, anchoring bias, availability bias, to name a few. In this work, we use knowledge from the field of cognitive science to account for cognitive biases in the human-AI collaborative decision-making setting, and mitigate their negative effects on collaborative performance. To this end, we mathematically model cognitive biases and provide a general framework through which researchers and practitioners can understand the interplay between cognitive biases and human-AI accuracy. We then focus specifically on anchoring bias, a bias commonly encountered in human-AI collaboration. We implement a time-based de-anchoring strategy and conduct our first user experiment that validates its effectiveness in human-AI collaborative decision-making. With this result, we design a time allocation strategy for a resource-constrained setting that achieves optimal human-AI collaboration under some assumptions. We, then, conduct a second user experiment which shows that our time allocation strategy with explanation can effectively de-anchor the human and improve collaborative performance when the AI model has low confidence and is incorrect.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In AI-assisted decision-making, the algorithmic recommendation functions as an anchor.
“In AIassisted decision-making, the algorithmic recommendation functions as such an anchor (Rastogi et al. 2022).”
From Limits of Interventions in AI-Assisted Decision Making (2026) · cites this paper as Rastogi et al. 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · see citing works
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 1× · see citing works
“…1998; Klayman, 1995; Oswald and Grosjean, 2004), anchoring bias (Furnham and Boo, 2011; Epley and Gilovich, 2006, 2001), availability bias (Tversky and Kahneman, 1973), etc. Recently, as AI systems are increasingly embedded into high stakes human decisions, understanding human behavior…”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 4× · see citing works
“…1973), etc. Recently, as AI systems are increasingly embedded into high stakes human decisions, understanding human behavior, and reliance on technology have become critical, “Poor partnerships between people and automation will become increasingly costly and catastrophic” (Lee and See, 2004).”
BackgroundInfluentialPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338 · see citing works
The Anchoring-and-Adjustment Heuristic: Why the Adjustments Are Insufficient
Epley, N., & Gilovich, T. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01704.x · Cited by 920 · cited 7× · see citing works
“The anchoring-and-adjustment heuristic, introduced by Tversky and Kahneman in [55] and studied in [20, 33] suggests that after being anchored, humans tend to adjust insufficiently because adjustments are effortful and tend to stop once a plausible estimate is reached.”
BackgroundMethodInfluentialZhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Self-citationDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · see citing works
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
Englich, B., Mussweiler, T., & Strack, F. (2006) · Personality and Social Psychology Bulletin · doi:10.1177/0146167205282152 · Cited by 554 · see citing works
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · see citing works
Self-citationThe Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392 · cited 3× · see citing works
“Research has shown that people do not calibrate their reliance on the AI based on its accuracy (Green and Chen, 2019).”
BackgroundMethodLai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · see citing works
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012429 · Cited by 284 · see citing works
Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings
Payzan-LeNestour, E., & Bossaerts, P. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1001048 · Cited by 267 · see citing works
Adaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239 · see citing works
The anchoring bias reflects rational use of cognitive resources
Lieder, F., Griffiths, T. L., M. Huys, Q. J., & Goodman, N. D. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1286-8 · Cited by 230 · see citing works
Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186 · cited 3× · see citing works
“…in several directions, such as human trust in algorithmic systems, interpretability, and explainability of machine learning models (Arnold et al., 2019; Zhang et al., 2020; Tomsett et al., 2020; Siau and Wang, 2018; Doshi-Velez and Kim, 2017; Lipton, 2018; Adadi and Berrada, 2018; Preece, 2018).”
BackgroundSelf-citation"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123 · see citing works
A Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy
Park, J. S., Barber, R., Kirlik, A., & Karahalios, K. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359204 · Cited by 65 · see citing works
Dice in the Black Box: User Experiences with an Inscrutable Algorithm
Springer, A., Hollis, V., & Whittaker, S. (2018) · arXiv · doi:10.48550/arxiv.1812.03219 · Cited by 29 · see citing works
Flagged first, then by citations
Research works printed but not linked: 34
- David Arnott. 2006. Cognitive biases and decision support systems development: a design science approach. Information Systems Journal 16, 1 (2006), 55-78.
- James H. Barnes JR. 1984. Cognitive biases and their impact on strategic planning. Strategic Management Journal 5, 2 (1984), 129-137. https://doi.org/10.1002/smj.4250050204 · DOI
- Thomas Baudel, Manon Verbockhaven, Guillaume Roy, Victoire Cousergue, and Rida Laarach. 2020. Addressing Cognitive Biases in Augmented Business Decision Systems. arXiv preprint arXiv:2009.08127 (2020). · arXiv
- Rainer Bromme, Friedrich W. Hesse, and Hans Spada. 2010. Barriers and Biases in Computer-Mediated Knowledge Communication: And How They May Be Overcome (1st ed.). Springer Publishing Company, Incorporated.
- Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. 2021. To Trust or to Think: Cognitive Forcing Functions CanReduce Overreliance on AI in AI-assisted Decision-making. Proc. ACM Hum.-Comput. Interact. 5, CSCW1 5 (2021), 21.
- Jerome R Busemeyer, Peter D Kvam, and Timothy J Pleskac. 2020. Comparison of Markov versus quantum dynamical models of human decision making. Wiley Interdisciplinary Reviews: Cognitive Science (2020).
- Valérian Chambon, Héloïse Théro, Marie Vidal, Henri Vandendriessche, Patrick Haggard, and Stefano Palminteri. 2020. Information about action outcomes differentially affects learning from self-determined versus imposed choices. Nature Human Behavior (2020). https://doi.org/10.1101/637157 · DOI
- Nick Chater, Joshua B. Tenenbaum, and Alan Yuille. 2006. Probabilistic models of cognition: Conceptual foundations. Trends in Cognitive Sciences 10, 7 (2006), 287 -291. https://doi.org/10.1016/j.tics.2006.05.007 Special issue: Probabilistic models of cognition. · DOI
- TK Das and Bing-Sheng Teng. 1999. Cognitive biases and strategic decision processes: An integrative perspective. Journal of Management Studies 36, 6 (1999), 757-778.
- Amit Dhurandhar, Bruce Graves, Rajesh Kumar Ravi, Gopikrishnan Maniachari, and Markus Ettl. 2015. Big Data System for Analyzing Risky Procurement Entities. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, August 10-13, 2015. ACM, 1741-1750.
- Joyce Ehrlinger, W.O. Readinger, and Bora Kim. 2016. Decision-Making and Cognitive Biases. Encyclopedia of Mental Health (12 2016). https://doi.org/10.1016/B978-0-12-397045-9.00206-8 · DOI
- Nicholas Epley and Thomas Gilovich. 2001. Putting adjustment back in the anchoring and adjustment heuristic: Differential processing of self-generated and experimenter-provided anchors. Psychological science 12, 5 (2001), 391-396.
- Philip Fernbach, Adam Darlow, and Steven Sloman. 2011. When good evidence goes bad: The weak evidence effect in judgment and decision-making. Cognition 119 (02 2011), 459-67. https://doi.org/10.1016/j.cognition.2011.01.013 · DOI
- Adrian Furnham and Hua Boo. 2011. A Literature Review of the Anchoring Effect. The Journal of Socio-Economics 40 (02 2011), 35-42. https://doi.org/10.1016/j.socec.2010.10.008 · DOI · Paper page
- Johannes Fürnkranz, Tomáš Kliegr, and Heiko Paulheim. 2020. On cognitive preferences and the plausibility of rule-based models. Machine Learning 109, 4 (2020), 853-898.
- Thomas L. Griffiths and Joshua B. Tenenbaum. 2006. Optimal Predictions in Everyday Cognition. Psychological Science 17, 9 (2006), 767-773. https://doi.org/10.1111/j.1467-9280.2006.01780.x · DOI
- Jeroen Janssen and Paul Kirschner. 2020. Applying collaborative cognitive load theory to computer-supported collaborative learning: towards a research agenda. Educational Technology Research and Development (01 2020), 1-23. https://doi.org/10.1007/s11423-019-09729-5 · DOI
- Joshua Klayman. 1995. Varieties of confirmation bias. In Psychology of learning and motivation. Vol. 32. Elsevier, 385-418.
- Tomáš Kliegr, Štěpán Bahník, and Johannes Fürnkranz. 2018. A review of possible effects of cognitive biases on interpretation of rule-based machine learning models. arXiv preprint arXiv:1804.02969 (2018). · arXiv
- Kaosu Matsumori, Yasuharu Koike, and Kenji Matsumoto. 2018. A Biased Bayesian Inference for Decision-Making and Cognitive Control. Frontiers in Neuroscience 12 (2018), 734. https://doi.org/10.3389/fnins.2018.00734 · DOI
- R. S. Nickerson. 1998. Confirmation Bias: A Ubiquitous Phenomenon in Many Guises. Review of General Psychology 2 (1998), 175 -220.
- M. Oswald and Stefan Grosjean. 2004. Confirmation bias. In R. F. Pohl (Ed.). Cognitive Illusions. A Handbook on Fallacies and Biases in Thinking, Judgement and Memory. Hove and N.Y.: Psychology Press, 79-96. https://doi.org/10.13140/2.1. 2068.0641 · DOI
- Elise Payzan-LeNestour and Peter Bossaerts. 2012. Do not Bet on the Unknown Versus Try to Find Out More: Estimation Uncertainty and "Unexpected Uncertainty" Both Modulate Exploration. Frontiers in Neuroscience 6 (2012), 150. https://doi.org/10.3389/fnins.2012.00150 · DOI
- Gloria Phillips-Wren, Daniel J. Power, and Manuel Mora. 2019. Cognitive bias, decision styles, and risk attitudes in decision making and DSS. Journal of Decision Systems 28, 2 (2019), 63-66. https://doi.org/10.1080/12460125.2019.1646509 · DOI
- Alun Preece. 2018. Asking 'Why'in AI: Explainability of intelligent systems-perspectives and challenges. Intelligent Systems in Accounting, Finance and Management 25, 2 (2018), 63-72.
- Keng Siau and Weiyu Wang. 2018. Building Trust in Artificial Intelligence, Machine Learning, and Robotics. Cutter Business Technology Journal 31 (03 2018), 47-53.
- Barry G. Silverman. 1992. Human-Computer Collaboration. Human-Computer Interaction 7, 2 (1992), 165-196.
- Herbert A Simon. 1956. Rational choice and the structure of the environment. Psychological review 63, 2 (1956), 129.
- Herbert A Simon. 1972. Theories of bounded rationality. Decision and Organization 1, 1 (1972), 161-176.
- Jacob Solomon. 2014. Customization bias in decision support systems. In Proceedings of the SIGCHI conference on human factors in computing systems. 3065-3074.
- Joshua B Tenenbaum. 1999. Bayesian modeling of human concept learning. In Advances in neural information processing systems. 59-68.
- Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. 2019. Designing Theory-Driven User-Centric Explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, 1-15. https://doi.org/10.1145/3290605.3300831 · DOI · Paper page
- Samuel Yeom and Michael Carl Tschantz. 2018. Discriminative but not discriminatory: A comparison of fairness definitions under different worldviews. arXiv preprint arXiv:1808.08619 (2018). · arXiv
- Yunfeng Zhang, Rachel KE Bellamy, and Wendy A Kellogg. 2015. Designing information for remediating cognitive biases in decision-making. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 2211-2220.
Cited by
Sorted by citations
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570 · see citing works
“These factors have received much attention in recent years 4,24,26,35 .”
Three Challenges for AI-Assisted Decision-Making
Steyvers, M., & Kumar, A. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231181102 · Cited by 217 · see citing works
Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Romeo, G., & Conti, D. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02422-7 · Cited by 209 · see citing works
“Consistent with this, Rastogi et al. (2022) demonstrated that time is a useful resource for de-anchoring.”
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129 · see citing works
Inkpen, K., Chappidi, S., Mallari, K., Nushi, B., Ramesh, D., Michelucci, P., Mandava, V., Vepřek, L. H., & Quinn, G. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3534561 · Cited by 132 · see citing works
The impact of AI errors in a human-in-the-loop process
Agudo, U., Liberal, K. G., Arrese, M., & Matute, H. (2024) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-023-00529-3 · Cited by 66 · see citing works
Bach, A. K. P., Nørgaard, T. M., Brok, J. C., & van Berkel, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581513 · Cited by 47 · see citing works
How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration
Cao, S., Gomez, C., & Huang, C.-M. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610068 · Cited by 48 · see citing works
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48 · see citing works
IdeationWeb: Tracking the Evolution of Design Ideas in Human-AI Co-Creation
Shen, H., Shen, L., Wu, W., & Zhang, K. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713375 · Cited by 24 · see citing works
He, G., Demartini, G., & Gadiraju, U. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713218 · Cited by 23 · see citing works
CreAItive Collaboration? Users' Misjudgment of AI-Creativity Affects Their Collaborative Performance
Bangerl, M. M., Disch, L., David, T., & Pammer-Schindler, V. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713886 · Cited by 13 · see citing works
Team dynamics in human-AI collaboration: effects on confidence, satisfaction, and accountability
Yousefi, M., Shahi, A., Sharifi, M., J Jorge Romera, A., Hoermann, S., & Piumsomboon, T. (2025) · Proceedings of the 27th International Conference on Multimodal Interaction · doi:10.1145/3716553.3750776 · Cited by 7 · see citing works
Mastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6 · see citing works
“Finally, if delays in provider decision making are leading to delays in intervention performance and patient deterioration, recommendations may need to be provided before initial decisions are made by providers.”
InfluentialRaees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5 · see citing works
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3 · see citing works
Limits of Interventions in AI-Assisted Decision Making
Shrestha, S., Ahn, E., & Vorobeychik, Y. (2026) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v9i4.43309
- Authors
- Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R.
- Affiliations
- Carnegie Mellon University (US)
- Twitter (United States) (US)
- IBM (United States) (US)
- Onfido, Greater Southampton Area, United Kingdom
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2022-03-30
- Volume
- 6
- Issue
- CSCW1
- Pages
- 1-22
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3093066786
- Cited by
- 216
- References
- 25
- Citations by year
- 2026: 66 · 2025: 85 · 2024: 34 · 2023: 21 · 2022: 10
- FWCI
- 31.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- NSF (CIF 1763734)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- cognitive biases, anchoring effect, human-AI collaboration, AI-assisted decision-making, model confidence, explanation, resource-constrained decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
