Artificial intelligence suppression as a strategy to mitigate artificial intelligence automation bias
Open accessAbstract
Background: Incorporating artificial intelligence (AI) into clinics brings the risk of automation bias, which potentially misleads the clinician’s decision-making. The purpose of this study was to propose a potential strategy to mitigate automation bias. Methods: This was a laboratory study with a randomized cross-over design. The diagnosis of anterior cruciate ligament (ACL) rupture, a common injury, on magnetic resonance imaging (MRI) was used as an example. Forty clinicians were invited to diagnose 200 ACLs with and without AI assistance. The AI’s correcting and misleading (automation bias) effects on the clinicians’ decision-making processes were analyzed. An ordinal logistic regression model was employed to predict the correcting and misleading probabilities of the AI. We further proposed an AI suppression strategy that retracted AI diagnoses with a higher misleading probability and provided AI diagnoses with a higher correcting probability. Results: The AI significantly increased clinicians’ accuracy from 87.2%±13.1% to 96.4%±1.9% (P < .001). However, the clinicians’ errors in the AI-assisted round were associated with automation bias, accounting for 45.5% of the total mistakes. The automation bias was found to affect clinicians of all levels of expertise. Using a logistic regression model, we identified an AI output zone with higher probability to generate misleading diagnoses. The proposed AI suppression strategy was estimated to decrease clinicians’ automation bias by 41.7%. Conclusion: Although AI improved clinicians’ diagnostic performance, automation bias was a serious problem that should be addressed in clinical practice. The proposed AI suppression strategy is a practical method for decreasing automation bias.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous work on automation bias suggests a tendency toward overreliance on LLM output, which may lead to poor compliance with manual review.
“Furthermore, previous work on automation bias suggests a tendency toward overreliance on LLM output, which may lead to poor compliance with manual review.28, 29, 30”
From Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters (2025) · cites this paper as [30]
The Introduction (page not provided) defines automation bias as over-accepting automated advice despite contrary data, and the Discussion (page not provided) reports clinicians being misled by faulty AI diagnoses. The study concerns AI-assisted MRI diagnosis, not LLM output, and our copy does not establish poor compliance with manual review. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Prospect Theory: An Analysis of Decision Under Risk
Kahneman, D., & Tversky, A. (2013) · World Scientific Handbook in Financial Economics Series · doi:10.1142/9789814417358_0006 · Cited by 3,091 · see citing works
The Threshold Approach to Clinical Decision Making
Pauker, S. G., & Kassirer, J. P. (1980) · New England Journal of Medicine · doi:10.1056/nejm198005153022003 · Cited by 1,371 · see citing works
Incidence of Anterior Cruciate Ligament Tears and Reconstruction: A 21-Year Population-Based Study
Sanders, T. L., Maradit Kremers, H., Bryan, A. J., Larson, D. R., Dahm, D. L., Levy, B. A., Stuart, M. J., & Krych, A. J. (2016) · The American Journal of Sports Medicine · doi:10.1177/0363546516629944 · Cited by 1,325 · see citing works
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · see citing works
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024 · see citing works
Deep neural network improves fracture detection by clinicians
Lindsey, R., Daluiski, A., Chopra, S., Lachapelle, A. S., Mozer, M. C., Sicular, S., Hanel, D. P., Gardner, M. J., Gupta, A., Hotchkiss, R. J., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806905115 · Cited by 654 · see citing works
Mori, Y., Kudo, S.-E., Misawa, M., Saito, Y., Ikematsu, H., Hotta, K., Ohtsuka, K., Urushibara, F., Kataoka, S., Ogawa, Y., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0249 · Cited by 509 · see citing works
McBee, M. P., Awan, O. A. R., Colucci, A. T., Ghobadi, C. W., Kadom, N., Kansagra, A. P., Tridandapani, S., & Auffermann, W. F. (2018) · Academic Radiology · doi:10.1016/j.acra.2018.02.018 · Cited by 502 · see citing works
Automation bias and verification complexity: a systematic review
Lyell, D., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw105 · Cited by 451 · see citing works
Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419 · see citing works
Impact of a deep learning assistant on the histopathologic classification of liver cancer
Kiani, A., Uyumazturk, B., Rajpurkar, P., Wang, A., Gao, R., Jones, E., Yu, Y., Langlotz, C. P., Ball, R. L., Montine, T. J., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0232-8 · Cited by 325 · see citing works
Human Factors Challenges for the Safe Use of Artificial Intelligence in Patient Care
Sujan, M., Furniss, D., Grundy, K., Grundy, H., Nelson, D., Elliott, M., White, S., Habli, I., & Reynolds, N. (2019) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100081 · Cited by 144 · see citing works
Bond, R. R., Novotny, T., Andrsova, I., Koc, L., Sisakova, M., Finlay, D., Guldenring, D., McLaughlin, J., Peace, A., McGilligan, V., et al. (2018) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2018.08.007 · Cited by 127 · see citing works
Eng, D. K.-M., Khandwala, N., Long, J., Fefferman, N. R., Lala, S. V., Strubel, N. A., Milla, S. S., Filice, R. W., Sharp, S. E., Towbin, A. J., et al. (2021) · Radiology · doi:10.1148/radiol.2021204021 · Cited by 84 · see citing works
Deep Automation Bias: How to Tackle a Wicked Problem of AI?
Strauß, S. (2021) · Big Data and Cognitive Computing · doi:10.3390/bdcc5020018 · Cited by 73 · see citing works
de Siqueira, V. S., Borges, M. M., Furtado, R. G., Dourado, C. N., & da Costa, R. M. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102165 · Cited by 59 · see citing works
Plasencia, C. M., Alderman, B. W., Barón, A. E., Rolfs, R. T., & Boyko, E. J. (1992) · Medical Decision Making · doi:10.1177/0272989x9201200306 · Cited by 19 · see citing works
Knoery, C., Bond, R., Iftikhar, A., Rjoob, K. W. Y., McGilligan, V., Peace, A. J., Heaton, J., & Leslie, S. J. (2019) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2019.08.006 · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- McBee MP, Awan OA, Colucci AT, et al. Deep learning in radiol- ogy. Acad Radiol 2018; 25 (11): 1472-80.
- Cummings ML. Automation bias in intelligent time critical decision support systems. In: AIAA 1st Intelligent Systems Technical Con- ference, Illinois, Chicago; 2004: 289-294.
- Xie S, Girshick R, Doll ar P, et al. Aggregated residual transforma- tions for deep neural networks. In: 2017 IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR); Honolulu, HI: IEEE; 2016.
- Chopra S, Hadsell R, LeCun Y. Learning a similarity metric dis- criminatively, with application to face verification. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05); San Diego, CA: IEEE; 2005.
Cited by
Sorted by citations
Anderson, T. N., Mohan, V., Dorr, D. A., Ratwani, R. M., Biro, J. M., & Gold, J. A. (2025) · Mayo Clinic proceedings. Digital health · doi:10.1016/j.mcpdig.2025.100292 · Cited by 17 · see citing works
“27 30 In a cross-sectional simulation study evaluating provider editing of AI-generated draft responses to patient portal messages, participants frequently failed to recognize clinically relevant note errors.”
- Authors
- Wang, D.-Y., Ding, J., Sun, A.-L., Liu, S.-G., Jiang, D., Li, N., & Yu, J.-K.
- Affiliations
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University , Beijing, China
- Beijing Yizhun Medical AI Co., Ltd , Beijing, China
- Peking University Third Hospital (CN)
- Corresponding author
- Li, Nan
- Yu, Jia-Kuo
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-08-10
- Volume
- 30
- Issue
- 10
- Pages
- 1684-1692
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC10531198
- OpenAlex
- W4385716169
- Cited by
- 54
- References
- 18
- Citations by year
- 2026: 28 · 2025: 18 · 2024: 6 · 2023: 2
- FWCI
- 2.16
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (U22A20283)
- Beijing-Tianjin-Hebei Basic Research Cooperation Project (V1640331014004)
- Fundamental Research Funds for the Central Universities (PKU2023XGK002)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- automation bias, AI-assisted diagnosis, clinical decision-making, anterior cruciate ligament rupture, magnetic resonance imaging, diagnostic accuracy, clinician expertise, clinical implementation of AI
- MeSH terms
- Clinical Decision-Making, Anterior Cruciate Ligament Injuries, Artificial Intelligence, Automation, Diagnosis, Computer-Assisted, Humans, Magnetic Resonance Imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
