Appropriate Reliance on Artificial Intelligence in Radiology Education
Abstract
Users of artificial intelligence (AI) can become overreliant on AI, negatively affecting the performance of human-AI teams. For a future in which radiologists use interpretive AI tools routinely in clinical practice, radiology education will need to evolve to provide radiologists with the skills to use AI appropriately and wisely. In this work, we examine how overreliance on AI may develop in radiology trainees and explore how this problem can be mitigated, including through the use of AI-augmented education. Radiology trainees will still need to develop the perceptual skills and mastery of knowledge fundamental to radiology to use AI safely. We propose a framework for radiology trainees to use AI tools with appropriate reliance, drawing on lessons from human-AI interactions research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reliance behaviours operate within a dynamic feedback loop, where users adjust their reliance patterns based on ongoing assessments of the AI's reliability and performance.
“prior research has proposed that reliance behaviours operate within a dynamic feedback loop, where users adjust their reliance patterns based on ongoing assessments of the AI's reliability and performance (Li & Little, 2023).”
From The role of critical thinking on undergraduates' reliance behaviours on generative <scp>AI</scp> in problem‐solving (2025) · cites this paper as Li & Little, 2023
AI users, regardless of subject matter expertise, are susceptible to overreliance.
“Li and Little (2023) argued that AI users, regardless of their subject matter expertise, are susceptible to overreliance.”
From Measuring undergraduate students’ reliance on generative AI during problem-solving: Scale development and validation. (2025) · cites this paper as Li and Little (2023)
Cites
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · see citing works
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · see citing works
Influence of Computer-Aided Detection on Performance of Screening Mammography
Fenton, J. J., Taplin, S. H., Carney, P. A., Abraham, L., Sickles, E. A., D’Orsi, C. J., Berns, E. A., Cutter, G., Hendrick, R. E., Barlow, W. E., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa066099 · Cited by 619 · see citing works
Do as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543 · see citing works
Truong, H. M. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.014 · Cited by 531 · see citing works
Visual Scanning, Pattern Recognition and Decision-making in Pulmonary Nodule Detection
Kundel, H. L., Nodine, C. F., & Carmody, D. P. (1978) · Investigative Radiology · doi:10.1097/00004424-197805000-00001 · Cited by 451 · see citing works
Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging
Arun, N., Gaw, N., Singh, P., Chang, K., Aggarwal, M., Chen, B., Hoebel, K., Gupta, S., Patel, J., Gidwani, M., et al. (2021) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2021200267 · Cited by 342 · see citing works
Self-citationWill Artificial Intelligence Replace Radiologists?
Langlotz, C. P. (2019) · Radiology Artificial Intelligence · doi:10.1148/ryai.2019190058 · Cited by 254 · see citing works
Visual scanning patterns of radiologists searching mammograms
Krupinski, E. A. (1996) · Academic Radiology · doi:10.1016/s1076-6332(05)80381-2 · Cited by 234 · see citing works
Artificial intelligence for precision education in radiology
Duong, M. T., Rauschecker, A. M., Rudie, J. D., Chen, P.-H., Cook, T. S., Bryan, R. N., & Mohan, S. (2019) · The British Journal of Radiology · doi:10.1259/bjr.20190389 · Cited by 205 · see citing works
Using eye movements to study visual search and to improve tumor detection.
Nodine, C. F., & Kundel, H. L. (1987) · Radiographics · doi:10.1148/radiographics.7.6.3423330 · Cited by 194 · see citing works
Drew, T. W., Evans, K. K., Võ, M. L. ‐., Jacobson, F. L., & Wolfe, J. M. (2012) · Radiographics · doi:10.1148/rg.331125023 · Cited by 182 · see citing works
Pusic, M. V., Pecaric, M. R., & Boutis, K. (2011) · Academic Medicine · doi:10.1097/acm.0b013e3182178c3c · Cited by 120 · see citing works
I can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119 · see citing works
Noninterpretive Uses of Artificial Intelligence in Radiology
Richardson, M. L., Garwood, E. R., Lee, Y. Z., Li, M., Lo, H. S., Nagaraju, A., Nguyen, X. V., Probyn, L. J., Rajiah, P. S., Sin, J. M., et al. (2020) · Academic Radiology · doi:10.1016/j.acra.2020.01.012 · Cited by 105 · see citing works
Self-citationRubin, G. D., Roos, J. E., Täll, M., Harrawood, B. P., Bag, S., Ly, D. L., Seaman, D. M., Hurwitz, L. M., Napel, S. A., & Choudhury, K. R. (2014) · Radiology · doi:10.1148/radiol.14132918 · Cited by 102 · see citing works
Computer-Displayed Eye Position as a Visual Aid to Pulmonary Nodule Interpretation
Kundel, H. L., Nodine, C. F., & Krupinski, E. A. (1990) · Investigative Radiology · doi:10.1097/00004424-199008000-00004 · Cited by 96 · see citing works
Tajmir, S., & Alkasab, T. K. (2018) · Academic Radiology · doi:10.1016/j.acra.2018.03.007 · Cited by 94 · see citing works
Yoo, H., Lee, S. H., Arru, C. D., Khera, R. D., Singh, R., Siebert, S. M., Kim, D., Lee, Y., Park, J. H., Eom, H. J., et al. (2021) · European Radiology · doi:10.1007/s00330-021-08074-7 · Cited by 86 · see citing works
Wang, M., Sun, Z., Jia, M., Wang, Y., Wang, H., Zhu, X., Chen, L., & Ji, H. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01797-7 · Cited by 52 · see citing works
The Evolving Importance of Artificial Intelligence and Radiology in Medical Trainee Education
Fischetti, C., Bhatter, P., Frisch, E., Sidhu, A., Helmy, M., Lungren, M., & Duhaime, E. (2022) · Academic Radiology · doi:10.1016/j.acra.2021.03.023 · Cited by 47 · see citing works
Shah, C., Davtyan, K., Nasrallah, I. M., Bryan, R. N., & Mohan, S. (2022) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-022-00713-9 · Cited by 44 · see citing works
The Need for a Machine Learning Curriculum for Radiologists
Wood, M. J., Tenenholtz, N., Geis, J. R., Michalski, M. H., & Andriole, K. P. (2018) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2018.10.008 · Cited by 38 · see citing works
Artificial Intelligence and the Trainee Experience in Radiology
Simpson, S. A., & Cook, T. S. (2020) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2020.09.028 · Cited by 38 · see citing works
A Perceptually Based Method for Enhancing Pulmonary Nodule Recognition
Krupinski, E. A., Nodine, C. F., & Kundel, H. L. (1993) · Investigative Radiology · doi:10.1097/00004424-199304000-00004 · Cited by 38 · see citing works
García-Peñalvo, F. J., García-Peñalvo, F. J., Spain, T. 4. I. C. O. T. E. F. E. M. 2. S., & Association for Computing Machinery-Digital Library (2016) · doi:10.1145/3012430 · Cited by 31 · see citing works
Shiang, T., Garwood, E., & Debenedectis, C. M. (2022) · Clinical Imaging · doi:10.1016/j.clinimag.2022.09.003 · Cited by 25 · see citing works
Ruberto, A. J., Rodenburg, D., Ross, K., Sarkar, P., Hungler, P. C., Etemad, A., Howes, D., Clarke, D., McLellan, J., Wilson, D., et al. (2021) · AEM Education and Training · doi:10.1002/aet2.10605 · Cited by 22 · see citing works
Visual Search and Lung Nodule Detection on CT Scans
Kundel, H. L. (2014) · Radiology · doi:10.1148/radiol.14142247 · Cited by 19 · see citing works
Chest “gestalt” and detectability of lung lesions
Oestmann, J. W., Greene, R. E., Bourgouin, P. M., Linetsky, L., & Llewellyn, H. J. (1993) · European Journal of Radiology · doi:10.1016/0720-048x(93)90015-f · Cited by 16 · see citing works
Dao, L., Harmouch, S. S., Chin, A. S.-L., Dao, T., Thian, Z., Chartrand‐Lefebvre, C., & Cohen, J. (2022) · medRxiv · doi:10.1101/2022.07.11.22277278 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Zhai, C., Wibowo, S., & Li, L. D. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00316-7 · Cited by 1,567 · see citing works
Demystifying AI: Current State and Future Role in Medical Education Assessment
Turner, L., Hashimoto, D. A., Vasisht, S., & Schaye, V. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005598 · Cited by 70 · see citing works
Wang, F., Li, N., Cheung, A. C. K., & Wong, G. K. W. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00547-9 · Cited by 69 · see citing works
Hou, C., Zhu, G., Sudarshan, V., Lim, F. S., & Ong, Y. S. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105329 · Cited by 70 · see citing works
Hou, C., Zhu, G., & Sudarshan, V. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13613 · Cited by 72 · see citing works
Naseer, M. A., Saeed, S., Afzal, A., Ali, S., & Malik, M. G. R. (2025) · BMC Medical Education · doi:10.1186/s12909-024-06552-2 · Cited by 60 · see citing works
Doherty, G., McLaughlin, L., Hughes, C., McConnell, J., Bond, R., & McFadden, S. L. (2024) · Radiography · doi:10.1016/j.radi.2023.12.019 · Cited by 39 · see citing works
Recognising errors in AI implementation in radiology: A narrative review
Stogiannos, N., Cuocolo, R., D’Antonoli, T. A., dos Santos, D. P., Harvey, H., Huisman, M., Koçak, B., Kotter, E., Lekadir, K., Shelmerdine, S. C., et al. (2025) · European Journal of Radiology · doi:10.1016/j.ejrad.2025.112311 · Cited by 31 · see citing works
AI’s Influence on Human Creative Agency
Watkins, R., & Barak-Medina, E. (2024) · Creativity Research Journal · doi:10.1080/10400419.2024.2437264 · Cited by 27 · see citing works
Students’ Reliance on AI in Higher Education: Identifying Contributing Factors
Pitts, G., Rani, N., Mildort, W., & Cook, E.-M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-12773-0_9 · Cited by 52 · see citing works
Student self-reflection as a tool for managing GenAI use in large class assessment
Combrinck, C., & Loubser, N. (2025) · Discover Education · doi:10.1007/s44217-025-00461-2 · Cited by 26 · see citing works
Leadership in radiology in the era of technological advancements and artificial intelligence
Wichtmann, B. D., Paech, D., Pianykh, O. S., Huang, S. Y., Seltzer, S. E., Brink, J. A., & Fennessy, F. M. (2025) · European Radiology · doi:10.1007/s00330-025-11745-4 · Cited by 16 · see citing works
To Solicit or Not to Solicit? Impact of AI Assistance Delivery Mechanisms on Decision-Making
Nair, P., Gressel, G., Anand, M. N., & Achuthan, K. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2536617 · Cited by 12 · see citing works
Hadi, Y. H., Hakami, A. M., Ajeebi, A., Faqihi, A., Sendi, E. T., Masrahi, M. A., Alalawi, S. A., Musharri, F. M., Hakami, A. N., Bani-Ahmad, M., et al. (2026) · European Journal of Radiology Artificial Intelligence · doi:10.1016/j.ejrai.2026.100073 · Cited by 11 · see citing works
Volin, J., van Assen, M., Bala, W., Safdar, N. M., & Balthazar, P. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.07.004 · Cited by 10 · see citing works
The role of patient outcomes in shaping moral responsibility in AI-supported decision making
Edwards, C. R., Murphy, A., Singh, A., Daniel, S., & Chamunyonga, C. (2025) · Radiography · doi:10.1016/j.radi.2025.102948 · Cited by 9 · see citing works
AI-generated videos in medical education: systematic review
Artsi, Y., Sorin, V., Glicksberg, B. S., Korfiatis, P. D., Thomas, D. C., Nadkarni, G. N., & Klang, E. (2025) · BMJ Open Quality · doi:10.1136/bmjoq-2025-003704 · Cited by 16 · see citing works
Gaze Assistance for Efficient Segmentation Correction of Medical Images
Khaertdinova, L., Shmykova, T., Pershin, I., Laryukov, A. V., Khanov, A., Ильшатович, З. Д., & Ibragimov, B. (2025) · IEEE Access · doi:10.1109/access.2025.3530701 · Cited by 6 · see citing works
Federico, L. C. M. D., Fusaro, D. D., Coppola, G., Gregori, M., & Durante, S. (2025) · Radiography · doi:10.1016/j.radi.2025.102972 · Cited by 5 · see citing works
A Narrative Review of Artificial Intelligence in Medical Diagnostics
Hirosawa, T., & Shimizu, T. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.063803 · Cited by 5 · see citing works
Zhang, P., Ye, Z., Wei, R., Niu, H., Li, H., Wang, M., Xue, P., & Riggs, S. L. (2025) · Applied Ergonomics · doi:10.1016/j.apergo.2025.104663 · Cited by 4 · see citing works
Mapping Human–AI Relationships: Intellectual Structure and Conceptual Insights
Gómez-Cruz, N. A., Castro, D. Y. R., Rey-Sarmiento, C. F., Torres, R. A. Z., & Niño, Á. M. (2026) · Technologies · doi:10.3390/technologies14020083 · Cited by 5 · see citing works
del Litoral., C. E., de la Cruz, J. P. C., García, A. E. P., del Caribe, U. A., Cárdenas, J. E. C., del Caribe, U. A., Gómez, L. M. P., & del Caribe, U. A. (2024) · Ad-gnosis · doi:10.21803/adgnosis.13.13.667 · Cited by 6 · see citing works
AI in Vocational Education and Training
Thakur, G. S. M., Banerjee, A., & Sarkar, P. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-8252-3.ch003 · Cited by 7 · see citing works
Radiologische Zukunft gestalten: KI aus Sicht junger Expert*innen
Beste, N. C., Busch, F., Frisch, A., Gassert, F. T., Hoffmann, E., Molwitz, I., Müller, L., Rischen, R., Salhöfer, L., Sieren, M. M., et al. (2025) · RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren · doi:10.1055/a-2532-5417 · Cited by 2 · see citing works
Cheung, L. M. E., Lo, O., & Shi, H. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100516 · Cited by 5 · see citing works
Xie, G., Liao, J., Tang, X., Fu, H., Liu, D., Deng, L., Wang, T., & Hong, X. (2025) · BMC Medical Education · doi:10.1186/s12909-026-09458-3 · Cited by 3 · see citing works
Pal, O., Duddu, V., Goyal, A., Goel, D., & Saha, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799386 · Cited by 2 · see citing works
From Experience to Experience: Key Insights for Improved Interaction with AI in Radiology
Fang, N., Pluyter, J. R., Bakker, S., Jacobs, I., Luyer, M. D. P., Nederend, J., Raijmakers, J., Chen, L.-L., & Funk, M. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650779 · Cited by 1 · see citing works
Delegation or Satisfaction? Explaining AI Dependence in College Students’ Learning
Long, Y., Wang, N., & Zhang, H. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-94184-9_8 · Cited by 1 · see citing works
Appropriate Reliance in Human-AI Collaborative Problem-Solving: The Role of Critical Thinking
Hou, C., Zhu, G., & Sudarshan, V. K. (2025) · Computer-supported collaborative learning/The Computer-Supported Collaborative Learning Conference · doi:10.22318/cscl2025.608996 · Cited by 1 · see citing works
Alternative Approaches to Resident Education
Chien, B., Pearce, J., Qureshi, M., Bhargava, P., & Mathur, M. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.08.054 · Cited by 1 · see citing works
Andreas, S., Pana, Z. D., Larentzakis, A. V., Kondylis, S., Maina, A. M., Ziogas, N. K., & Ntourakis, D. (2026) · Global Surgical Education - Journal of the Association for Surgical Education · doi:10.1007/s44186-025-00411-4 · Cited by 1 · see citing works
Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators
Pitts, G., Rani, N., & Mildort, W. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29763-1_28 · Cited by 4 · see citing works
Artificial Intelligence in Breast Imaging Daily Clinical Practice: Counterpoint—Proceed With Caution
Mullen, L. A., & Ambinder, E. B. (2024) · American Journal of Roentgenology · doi:10.2214/ajr.24.30962 · Cited by 1 · see citing works
Introduction to the Teaching and Education Focus Issue
Schaefer, P. W., & Bedi, H. S. (2023) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2023.08.029
Pearls and Pitfalls of Integrating Artificial Intelligence Into Radiology Education
Slanetz, P. J., Bala, W., Doo, F. X., & Cook, T. S. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.06.026
Who Will Chart the Ethics of Radiology Education?
Abbasian, H. (2025) · The Clinical Teacher · doi:10.1111/tct.70239
Wang, J., Wang, L., Yang, Z., Zou, Q., & Liu, Y. (2025) · Acta Radiologica · doi:10.1177/02841851251393507
Training and Education for AI in Health Care
Hirosawa, T. (2025) · Springer eBooks · doi:10.1007/978-981-95-4338-0_14
A Critical Narrative Synthesis Toward Precision Medical Training in Radiology Education
Alkhalili, K., & Ali, S. (2025) · Inquisiva Open · doi:10.64551/001c.150391
Skills Training in the Age of AI
Blakely, C., & Rose, R. L. (2026) · Springer eBooks · doi:10.1007/978-981-95-1622-3_5
AI in Projectional Radiography
McNally, C. S., Rainey, C., & Szczepura, K. R. (2026) · Springer eBooks · doi:10.1007/978-3-032-05080-9_5
AI Governance and Implementation in Radiography Practice
Malamateniou, C., Kumar, A., Lip, G., Pierce, R. L., & Kyratsis, Y. (2026) · Springer eBooks · doi:10.1007/978-3-032-05080-9_4
Han, Y. H., Ouyang, W., Du, H., Ma, M., & Wang, G. (2026) · Journal of Eye Movement Research · doi:10.3390/jemr19030055
Human-Centered Design of LLM-Powered Educational Chatbots: A Study with Secondary Teachers
Reichert, H., Briceno, D., Tabarsi, B. T., & Barnes, T. M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31048-4_32
Suzuki, K., Hiruma, K., Miyamoto, K., Saeki, M., Kanai, T., Tomita, Y., Kikuchi, K., Inoue, G., Kaki, Y., Yagi, M., et al. (2026) · Acute Medicine & Surgery · doi:10.1002/ams2.70149
AI for Accessible and Inclusive Education
Mulia, M. V. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7225-9.ch012
Solim, L. A., Atasoy, D., Yüzkan, S., Koch, V., Bachir, A. A., Yel, İ., & Vogl, T. J. (2026) · iRadiology · doi:10.1002/ird3.70085
Artificial Intelligence: Promises, Pitfalls, and Ethical Considerations
Aung, N., Roney, C. H., Leslie, D., Slabaugh, G., & Petersen, S. E. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-12497-6.00009-9
Human-AI interaction: themes, applications, and challenges
Zhou, L., Li, Y., Zhang, Z., & Chen, X. (2026) · Data Science and Management · doi:10.1016/j.dsm.2026.100214
Guo, Y., & Dong, P. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1998976
- Authors
- Li, M. D., & Little, B. P.
- Affiliations
- University of Alberta (CA)
- Mayo Clinic (US)
- Corresponding author
- Li, Matthew D.
- Venue
- Journal of the American College of Radiology
- Publisher
- Elsevier BV
- Published
- 2023-11
- Volume
- 20
- Issue
- 11
- Pages
- 1126-1130
- ISSN
- 1546-1440 (ISSN-L)
- OpenAlex
- W4382562052
- Cited by
- 52
- References
- 31
- Citations by year
- 2026: 17 · 2025: 26 · 2024: 7 · 2023: 2
- FWCI
- 2.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American College of Radiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Keywords
- radiology education, radiology residents, human-AI teaming, human-AI interaction, perceptual skills
- MeSH terms
- Radiologists, Artificial Intelligence, Forecasting, Humans, Radiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
