“If I Had All the Time in the World”: Ophthalmologists’ Perceptions of Anchoring Bias Mitigation in Clinical AI Support
Open accessAbstract
Clinical needs and technological advances have resulted in increased use of Artificial Intelligence (AI) in clinical decision support. However, such support can introduce new and amplify existing cognitive biases. Through contextual inquiry and interviews, we set out to understand the use of an existing AI support system by ophthalmologists. We identified concerns regarding anchoring bias and a misunderstanding of the AI’s capabilities. Following, we evaluated clinicians’ perceptions of three bias mitigation strategies as integrated into their existing decision support system. While clinicians recognised the danger of anchoring bias, we identified a concern around the impact of bias mitigation on procedure time. Our participants were divided in their expectations of any positive impact on diagnostic accuracy, stemming from varying reliance on the decision support. Our results provide insights into the challenges of integrating bias mitigation into AI decision support.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When presented with an AI suggestion, people tend to anchor their prediction to the suggestion.
“When presented with an AI suggestion, people tend to anchor their prediction to the suggestion [1,74]”
From Cognitive Forcing for Better Decision-Making: Reducing Overreliance on AI Systems Through Partial Explanations (2025) · cites this paper as [1]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · cited 3× · see citing works
“are biased toward an initially presented value” [60].”
InfluentialGulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697 · see citing works
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837 · see citing works
Braun, V., Clarke, V., Hayfield, N., & Terry, G. (2019) · Handbook of Research Methods in Health Social Sciences · doi:10.1007/978-981-10-5251-4_103 · Cited by 2,910 · see citing works
Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0040-6 · Cited by 1,638 · see citing works
A literature review of the anchoring effect
Furnham, A., & Boo, H. C. (2011) · The Journal of Socio-Economics · doi:10.1016/j.socec.2010.10.008 · Cited by 1,179 · see citing works
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · see citing works
Roy, A. G., Conjeti, S., Karri, S. P. K., Sheet, D., Katouzian, A., Wachinger, C., & Navab, N. (2017) · Biomedical Optics Express · doi:10.1364/boe.8.003627 · Cited by 614 · see citing works
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614 · see citing works
Norman, G. R., Monteiro, S. D., Sherbino, J., Ilgen, J. S., Schmidt, H. G., & Mamede, S. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001421 · Cited by 594 · see citing works
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · see citing works
From Mindless to Mindful Practice — Cognitive Bias and Clinical Decision Making
Croskerry, P. (2013) · New England Journal of Medicine · doi:10.1056/nejmp1303712 · Cited by 572 · see citing works
Mussweiler, T., Strack, F., & Pfeiffer, T. (2000) · Personality and Social Psychology Bulletin · doi:10.1177/01461672002611010 · Cited by 556 · cited 5× · see citing works
“Although mitigation strategies are frequently discussed in the literature [2, 24, 30, 37], relatively few studies have explored how bias mitigation can be incorporated into AI decision-support systems.”
BackgroundInfluentialThe Brief Implicit Association Test.
Sriram, N., & Greenwald, A. G. (2009) · Experimental Psychology · doi:10.1027/1618-3169.56.4.283 · Cited by 525 · see citing works
The Encyclopedia of Clinical Psychology
Huibers, M. J. H., & Cuijpers, P. J. (2014) · Wiley eBooks · doi:10.1002/9781118625392 · Cited by 416 · see citing works
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405 · see citing works
Niemeijer, M., van Ginneken, B., Russell, S. R., et al. (2007) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.06-0996 · Cited by 402 · see citing works
Artificial intelligence for diabetic retinopathy screening: a review
Grzybowski, A., Brona, P., Lim, G., Ruamviboonsuk, P., Tan, G. S. W., Abramoff, M., & Ting, D. S. W. (2020) · Eye · doi:10.1038/s41433-019-0566-0 · Cited by 376 · cited 1× · see citing works
“state that even if AI assessment systems increase in quality, clinical experts will still be required to assess atypical situations and low-quality images, as well as for quality monitoring [21].”
BackgroundCognitive bias in clinical medicine
O'Sullivan, E., & Schofield, S. (2018) · Journal of the Royal College of Physicians of Edinburgh · doi:10.4997/jrcpe.2018.306 · Cited by 334 · see citing works
CHI Conference on Human Factors in Computing Systems
Barbosa, S. D. J., Lampe, C. A. C., Appert, C., Shamma, D. A., Drucker, S. M., Williamson, J., & Yatani, K. (2022) · doi:10.1145/3491102 · Cited by 309 · see citing works
Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C., Korris, J., & Kassam, K. (2015) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/2372732215600886 · Cited by 259 · see citing works
Yang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247 · see citing works
Royce, C. S., Hayes, M. M., & Schwartzstein, R. M. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002518 · Cited by 242 · see citing works
The anchoring bias reflects rational use of cognitive resources
Lieder, F., Griffiths, T. L., M. Huys, Q. J., & Goodman, N. D. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1286-8 · Cited by 230 · see citing works
Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512930 · Cited by 216 · see citing works
Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review
Lambe, K. A., O'Reilly, G., Kelly, B. D., & Curristan, S. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004417 · Cited by 215 · cited 3× · see citing works
“Although mitigation strategies are frequently discussed in the literature [2, 24, 30, 37], relatively few studies have explored how bias mitigation can be incorporated into AI decision-support systems.”
BackgroundWang, R., Harper, F. M., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376813 · Cited by 211 · see citing works
Wang, D., Wang, L., Zhang, Z., Wang, D., Zhu, H., Gao, Y., Fan, X., & Tian, F. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445432 · Cited by 209 · see citing works
The current state of artificial intelligence in ophthalmology
Kapoor, R., Walters, S. P., & Al‐Aswad, L. A. (2018) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2018.09.002 · Cited by 198 · see citing works
Retinal Imaging Techniques for Diabetic Retinopathy Screening
Goh, J. C. H., Cheung, C. Y.-L., Sim, S. S., Tan, P. C., Tan, G. S. W., & Wong, T. Y. (2016) · Journal of Diabetes Science and Technology · doi:10.1177/1932296816629491 · Cited by 176 · see citing works
How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
Piorkowski, D., Park, S., Wang, A. Y., Wang, D., Muller, M., & Portnoy, F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449205 · Cited by 151 · see citing works
Diabetic Retinopathy Detection using Deep Learning
Nguyen, Q. H., Muthuraman, R., Singh, L., et al. (2020) · International Conference on Machine Learning and Soft Computing (ICMLSC) · doi:10.1145/3380688.3380709 · Cited by 144 · see citing works
A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445472 · Cited by 137 · see citing works
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129 · see citing works
Automated detection of exudates and macula for grading of diabetic macular edema
Akram, M. U., Tariq, A., Khan, S. A., et al. (2014) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2014.01.010 · Cited by 120 · see citing works
Understanding Decision Making in Critical Care
Lighthall, G. K., & Guillamet, C. V. (2015) · Clinical Medicine & Research · doi:10.3121/cmr.2015.1289 · Cited by 115 · see citing works
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
Fogliato, R., Chappidi, S., Lungren, M. P., Fisher, P., Wilson, D., Fitzke, M., Parkinson, M., Horvitz, E., Inkpen, K. M., & Nushi, B. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533193 · Cited by 115 · see citing works
Sherbino, J., Kulasegaram, K., Howey, E., & Norman, G. (2014) · CJEM · doi:10.2310/8000.2013.130860 · Cited by 108 · see citing works
Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems
Nourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E., & Gogate, V. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450639 · Cited by 107 · see citing works
Yang, Q., Zimmerman, J., Steinfeld, A., Carey, L., & Antaki, J. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858373 · Cited by 105 · see citing works
Ogdie, A., Reilly, J. B., Pang, W. G., Keddem, S., Barg, F. K., Feldt, J. M. V., & Myers, J. S. (2012) · Academic Medicine · doi:10.1097/acm.0b013e31826742c9 · Cited by 100 · see citing works
How neurologists think: A cognitive psychology perspective on missed diagnoses
Vickrey, B. G., Samuels, M. A., & Ropper, A. H. (2009) · Annals of Neurology · doi:10.1002/ana.21907 · Cited by 96 · see citing works
Schmidt, H. G., Mamede, S., van den Berge, K., van Gog, T., van Saase, J. L. C. M., & Rikers, R. M. J. P. (2013) · Academic Medicine · doi:10.1097/acm.0000000000000107 · Cited by 91 · see citing works
Individual differences in cognitive biases: Evidence against one-factor theory of rationality
Teovanović, P., Knežević, G., & Stankov, L. (2015) · Intelligence · doi:10.1016/j.intell.2015.02.008 · Cited by 77 · see citing works
Cabitza, F., & Zeitoun, J.-D. (2019) · Annals of Translational Medicine · doi:10.21037/atm.2019.04.07 · Cited by 72 · see citing works
Training in the mitigation of anchoring bias: A test of the consider-the-opposite strategy
Adame, B. J. (2015) · Learning and Motivation · doi:10.1016/j.lmot.2015.11.002 · Cited by 61 · see citing works
Activating reflective thinking with decision justification and debiasing training
İşler, O., Yılmaz, O., & Doğruyol, B. (2020) · Judgment and Decision Making · doi:10.1017/s1930297500008147 · Cited by 55 · see citing works
Designing Information for Remediating Cognitive Biases in Decision-Making
Zhang, Y., Bellamy, R. K. E., & Kellogg, W. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702239 · Cited by 51 · see citing works
Rieger, A., Draws, T., Theune, M., & Tintarev, N. (2021) · Proceedings of the 32st ACM Conference on Hypertext and Social Media · doi:10.1145/3465336.3475101 · Cited by 48 · see citing works
Charting Subtle Interaction in the HCI Literature
Pohl, H., Muresan, A., & Hornbæk, K. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300648 · Cited by 44 · see citing works
Retention and Transfer of Cognitive Bias Mitigation Interventions: A Systematic Literature Study
Korteling, J. E., Gerritsma, J. Y. J., & Toet, A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.629354 · Cited by 44 · see citing works
Tao, L., Zhang, C., Zeng, L., Zhu, S., Li, N., Li, W., Zhang, H., Zhao, Y., Zhan, S., & Ji, H. (2020) · JMIR Medical Informatics · doi:10.2196/16912 · Cited by 42 · see citing works
Kruse, C. S., & Ehrbar, N. (2020) · JMIR Medical Informatics · doi:10.2196/17283 · Cited by 42 · see citing works
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
(2016) · International Conference on Human Factors in Computing Systems · doi:10.1145/2858036 · Cited by 39 · see citing works
van Berkel, N., Bellio, M., Skov, M. B., & BLANDFORD, A. E. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3571815 · Cited by 37 · see citing works
Burst Your Bubble! An Intelligent System for Improving Awareness of Diverse Social Opinions
Gao, M., Do, H. J., & Fu, W. (2018) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3172944.3172970 · Cited by 35 · see citing works
Subtle gaze manipulation for improved mammography training
Sridharan, S., Bailey, R., McNamara, A., et al. (2012) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/2168556.2168568 · Cited by 33 · see citing works
AI-Moderated Decision-Making: Capturing and Balancing Anchoring Bias in Sequential Decision Tasks
Echterhoff, J. M., Yarmand, M., & McAuley, J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517443 · Cited by 31 · see citing works
Errors/Biases in Clinical Decision Making
Lilienfeld, S. O., & Lynn, S. J. (2014) · The Encyclopedia of Clinical Psychology · doi:10.1002/9781118625392.wbecp567 · Cited by 27 · see citing works
Designing Visual Markers for Continuous Artificial Intelligence Support
van Berkel, N., Ahmad, O. F., Stoyanov, D. V., Lovat, L. B., & BLANDFORD, A. E. (2020) · ACM Transactions on Computing for Healthcare · doi:10.1145/3422156 · Cited by 23 · see citing works
van Berkel, N., Opie, J., Ahmad, O. F., Lovat, L. B., Stoyanov, D. V., & BLANDFORD, A. E. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3480247 · Cited by 22 · see citing works
Veinott, E. S., Leonard, J., Papautsky, E. L., Perelman, B. S., Stankovic, A., Lorince, J., Hotaling, J. M., Ross, T. L., Todd, P. M., Castronova, E., et al. (2013) · IEEE International Games Innovation Conference (IGIC) · doi:10.1109/igic.2013.6659170 · Cited by 20 · see citing works
Customization bias in decision support systems
Solomon, J. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557211 · Cited by 20 · see citing works
Studying the Effects of Cognitive Biases in Evaluation of Conversational Agents
Santhanam, S., Karduni, A., & Shaikh, S. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376318 · Cited by 20 · see citing works
Understanding Design for Automated Image Analysis in Digital Pathology
Molin, J., Woźniak, P. W., Lundström, C. F., et al. (2016) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2971485.2971561 · Cited by 15 · see citing works
Jo, E., Ryu, M., Kenderova, G., So, S., Shapiro, B. B., Papoutsaki, A., & Epstein, D. A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502206 · Cited by 15 · see citing works
Method for Appropriating the Brief Implicit Association Test to Elicit Biases in Users
Dingler, T., Tag, B., Eccles, D., van Berkel, N., & Kostakos, V. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517570 · Cited by 11 · see citing works
COVID-19 and the Consequences of Anchoring Bias
Horowitz, H. W., Behar, C. F., & Greene, J. (2021) · Emerging infectious diseases · doi:10.3201/eid2708.211107 · Cited by 5 · see citing works
A machine-learning approach to retrieving diabetic retinopathy images
Chandakkar, P. S., Venkatesan, R., Li, B., et al. (2012) · doi:10.1145/2382936.2383030 · Cited by 4 · see citing works
26th International Conference on Intelligent User Interfaces
(2021) · doi:10.1145/3397481 · Cited by 4 · see citing works
Retention and Transfer of Cognitive Bias Mitigation Interventions: A Systematic Literature StudyPreprint
Korteling, J. E., J.Y.J.gerritsma, & Toet, A. (2021) · doi:10.31234/osf.io/ed43s · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [20]20. Jerome E Groopman. 2007. How doctors think. Vol. 82. Houghton Mifin Boston.
- [24]24. Ozan Isler, Onurcan Yilmaz, and Burak Dogruyol. 2020. Activating refective thinking with decision justifcation and debiasing training. Judgment and Decision Making 15, 6 (2020), 926-938.
- [60]60. Amos Tversky and Daniel Kahneman. 1974. Judgment under Uncertainty: Heuris- tics and Biases. Science 185, 4157 (1974), 1124-1131. https://doi.org/10.1126/ science.185.4157.1124
Cited by
Sorted by citations
de Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32 · see citing works
Self-citationRosbach, E., Ammeling, J., Krügel, S., Kießig, A., Fritz, A., Ganz, J., Puget, C., Donovan, T., Klang, A., Köller, M. C., et al. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713319 · Cited by 18 · see citing works
Mastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6 · see citing works
Zhu, Y., Sui, D., Wang, Z., Hu, X., Gu, L., Qi, Y., Wu, T., Wang, L., Wei, Y., Tang, W., et al. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791458 · Cited by 3 · see citing works
From Oracles to Productive Friction: Designing for Meaningful Human Oversight in Clinical AI
Rubegni, E., Natali, C., Valsecchi, C., Rizzo, S., Ayoub, O., & Facchini, A. (2026) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v9i3.43300
- Authors
- Bach, A. K. P., Nørgaard, T. M., Brok, J. C., & van Berkel, N.
- Affiliations
- Aalborg University (DK)
- Department of Computer Science, Aalborg University, Denmark
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-14
- OpenAlex
- W4323651244
- Cited by
- 47
- References
- 71
- Citations by year
- 2026: 12 · 2025: 18 · 2024: 11 · 2023: 6
- FWCI
- 8.48
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- VBN Forskningsportal (Aalborg Universitet) (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Carlsbergfondet (Algorithmic Explainability for Everyday Citizens)
- Innovationsfonden (DIREC Explain-Me)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- anchoring effect, bias mitigation, clinical decision support, artificial intelligence in healthcare, ophthalmologists, clinician perceptions, diagnostic accuracy, AI reliance, procedure time, cognitive biases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
