CoChat
Sign Up
Cited by 30Artificial Intelligence in Healthcare and EducationRadiology practices and educationCOVID-19 diagnosis using AI

Upskilling or deskilling? Measurable role of an AI-supported training for radiology residents: a lesson from the pandemic

Open access
Savardi, Mattia, Signoroni, Alberto, Benini, Sergio, Vaccher, Filippo, Alberti, Matteo, Ciolli, Pietro, Di Meo, Nunzia, Falcone, Teresa +4 more·2025·Insights into Imaging
DOI 10.1186/s13244-024-01893-4PMC PMC11780016
Open this paper in CoChat — free account

Abstract

Objectives: This article aims to evaluate the use and effects of an artificial intelligence system supporting a critical diagnostic task during radiology resident training, addressing a research gap in this field. Materials and methods: We involved eight residents evaluating 150 CXRs in three scenarios: no AI, on-demand AI, and integrated-AI. The considered task was the assessment of a multi-regional severity score of lung compromise in patients affected by COVID-19. The chosen artificial intelligence tool, fully integrated in the RIS/PACS, demonstrated superior performance in scoring compared to the average radiologist. Using quantitative metrics and questionnaires, we measured the ‘upskilling’ effects of using AI support and residents’ resilience to ‘deskilling,’ i.e., their ability to overcome AI errors. Results: Residents required AI in 70% of cases when left free to choose. AI support significantly reduced severity score errors and increased inter-rater agreement by 22%. Residents were resilient to AI errors above an acceptability threshold. Questionnaires indicated high tool usefulness, reliability, and explainability, with a preference for collaborative AI scenarios. Conclusion: With this work, we gathered quantitative and qualitative evidence of the beneficial use of a high-performance AI tool that is well integrated into the diagnostic workflow as a training aid for radiology residents. Critical relevance statement: Balancing educational benefits and deskilling risks is essential to exploit AI systems as effective learning tools in radiology residency programs. Our work highlights metrics for evaluating these aspects. Key Points: Insights into AI tools’ effects in radiology resident training are lacking. Metrics were defined to observe residents using an AI tool in different settings. This approach is advisable for evaluating AI tools in radiology training. Graphical Abstract:

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13244-024-01893-4

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Automation bias may creep into practice, potentially leading to deskilling within the profession with long-term AI usage, an already recognised phenomenon with AI use in radiology.

    “Whilst this aspect was not fully explored in our work, it highlights how 'automation bias' may creep into practice, potentially leading to deskilling within the profession with long term AI usage, an already recognised phenomenon with AI use in radiology 11 .”

    From Radiology staff perspectives after decommissioning an artificial intelligence (AI) tool for expedited lung cancer triage (2026) · cites this paper as [11]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 38

  1. [1]Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56 30617339 · DOI · Paper page
  2. [2]Montagnon E, Cerny M, Cadrin-Chênevert A et al (2020) Deep learning workflow in radiology: a primer. Insights Imaging 11:22 32040647 PMC7010882 · DOI
  3. [3]Juluru K, Shih H-H, Keshava Murthy KN et al (2021) Integrating AI algorithms into the clinical workflow. Radiol Artif Intell 3:210013 PMC8637237 34870216 · DOI
  4. [4]Choy G, Khalilzadeh O, Michalski M et al (2018) Current applications and future impact of machine learning in radiology. Radiology 288:318–328 29944078 PMC6542626 · DOI · Paper page
  5. [5]Becker CD, Kotter E, Fournier L, Martí-Bonmatí L (2022) Current practical experience with artificial intelligence in clinical radiology: a survey of the European Society of Radiology. Insights Imaging 13:107 35727355 PMC9213582 · DOI
  6. [6]Wu JT, Wong KC, Gur Y et al (2020) Comparison of chest radiograph interpretations by artificial intelligence algorithm vs radiology residents. JAMA Netw Open 3:2022779 PMC7547369 33034642 · DOI
  7. [7]Guermazi A, Tannoury C, Kompel AJ et al (2022) Improving radiographic fracture recognition performance and efficiency using artificial intelligence. Radiology 302:627–636 34931859 · DOI · Paper page
  8. [9]Mello-Thoms C, Mello CAB (2023) Clinical applications of artificial intelligence in radiology. Br J Radiol 96:20221031 37099398 PMC10546456 · DOI
  9. [11]Spencer B (2023) Decoupled learning among medical trainees and artificial intelligence. Acad Manag Proc 2023:16424 · DOI
  10. [12]Ling Kuo RY, Freethy A, Smith J et al (2024) Stakeholder perspectives towards diagnostic artificial intelligence: a co-produced qualitative evidence synthesis. EClinicalMedicine 71:102555 38549586 PMC10973718 · DOI
  11. [14]Hoff T (2011) Deskilling and adaptation among primary care physicians using two work innovations. Health Care Manag Rev 36:338–348 21685794 · DOI
  12. [15]Wiggins WF, Caton MT, Magudia K et al (2020) Preparing radiologists to lead in the era of artificial intelligence: designing and implementing a focused data science pathway for senior radiology residents. Radiol Artif Intell 2:200057 PMC8082300 33937848 · DOI
  13. [16]Keane PA, Topol EJ (2021) AI-facilitated health care requires education of clinicians. Lancet 397:1254 33812482 · DOI
  14. [18]Recht MP, Dewey M, Dreyer K et al (2020) Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. Eur Radiol 30:3576–3584 32064565 · DOI · Paper page
  15. [19]Fogliato R, Chappidi S, Lungren M et al (2022) Who goes first? Influences of human-AI workflow on decision making in clinical imaging. In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 1362–1374 (2022)
  16. [20]Cabitza F, Campagner A, Angius R, Natali C, Reverberi C (2023) AI shall have no dominion: on how to measure technology dominance in AI-supported human decision-making. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–20 (2023)
  17. [21]Rafner J, Dellermann D, Hjorth A et al (2022) Deskilling, upskilling, and reskilling: a case for hybrid intelligence. Morals Mach 1:24–39 · DOI
  18. [24]Simpson SA, Cook TS (2020) Artificial intelligence and the trainee experience in radiology. J Am Coll Radiol 17:1388–1393 33010211 · DOI
  19. [25]Tejani AS, Elhalawani H, Moy L, Kohli M, Kahn CE (2023) Artificial intelligence and radiology education. Radiol Artif Intell 5:220084 PMC9885376 36721409 · DOI
  20. [26]Santomartino SM, Yi PH (2022) Systematic review of radiologist and medical student attitudes on the role and impact of AI in radiology. Acad Radiol 29:1748–1756 35105524 · DOI
  21. [27]Huisman M, Ranschaert E, Parker W et al (2021) An international survey on AI in radiology in 1041 radiologists and radiology residents part 2: expectations, hurdles to implementation, and education. Eur Radiol 31:8797–8806 33974148 PMC8111651 · DOI
  22. [28]van Kooten MJ, Tan CO, Hofmeijer EIS et al (2024) A framework to integrate artificial intelligence training into radiology residency programs: preparing the future radiologist. Insights Imaging 15:15 38228800 PMC10792147 · DOI
  23. [29]Signoroni A, Savardi M, Benini S et al (2021) BS-net: learning Covid-19 pneumonia severity on a large chest X-ray dataset. Med Image Anal 71:102046 33862337 PMC8010334 · DOI
  24. [30]Borghesi A, Maroldi R (2020) COVID-19 outbreak in Italy: experimental chest X-ray scoring system for quantifying and monitoring disease progression. Radiol Med 125:509–513 PMC7194501 32358689
  25. [31]Allahabadi H, Amann J, Balot I et al (2022) Assessing trustworthy AI in times of COVID-19: deep learning for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients. IEEE Trans Technol Soc 3:272–289 36573115 PMC9762021 · DOI
  26. [33]Borghesi A, Zigliani A, Golemi S et al (2020) Chest x-ray severity index as a predictor of in-hospital mortality in coronavirus disease 2019: a study of 302 patients from Italy. Int J Infect Dis 96:291–293 32437939 PMC7207134 · DOI
  27. [35]Cabitza F, Campagner A, Ronzio L et al (2023) Rams, hounds and white boxes: investigating human–AI collaboration protocols in medical diagnosis. Artif Intell Med 138:102506 36990586 · DOI · Paper page
  28. [36]Tejani AS, Fielding JR, Peshock RM (2021) What should radiology residency and fellowship training in artificial intelligence include? A trainee’s perspective—radiology in training. Radiology 299:243–245 33687289 · DOI
  29. [37]Hu R, Rizwan A, Hu Z, Li T, Chung AD, Kwan BYM (2023) An artificial intelligence training workshop for diagnostic radiology residents. Radiol Artif Intell 5:220170 PMC10077074 37035436 · DOI
  30. [38]Yu F, Moehring A, Banerjee O, Salz T, Agarwal N, Rajpurkar P (2024) Heterogeneity and predictors of the effects of AI assistance on radiologists. Nat Med 30:837–849 (2024) PMC10957478 38504016
  31. [39]Jacobs M, He J, Pradier F et al (2021) Designing AI for trust and collaboration in time-constrained medical decisions: a sociotechnical lens. In: Proceedings of the 2021 Chi Conference on Human Factors in Computing Systems, pp. 1–14
  32. [40]Alowais SA, Alghamdi SS, Alsuhebany N et al (2023) Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Med Educ 23:689 37740191 PMC10517477 · DOI · Paper page
  33. [41]Ranschaert E, Topff L, Pianykh O (2021) Optimization of radiology workflow with artificial intelligence. Radiol Clin 59:955–966 34689880 · DOI
  34. [42]Liu Y, Li X, Yan C et al (2022) Comparison of diagnostic accuracy and utility of artificial intelligence–optimized ACR TI-RADS and original ACR TI-RADS: a multi-center validation study based on 2061 thyroid nodules. Eur Radiol 32:7733–7742 35505119 · DOI
  35. [43]Iqbal J, Jaimes DCC, Makineni P et al (2023) Reimagining healthcare: unleashing the power of artificial intelligence in medicine. Cureus 15:e44658 37799217 PMC10549955
  36. [44]de Vries BM, Zwezerijnen GJ, Burchell GL, van Velden FH, Menke-van der Houven van Oordt CW, Boellaard R (2023) Explainable artificial intelligence (XAI) in radiology and nuclear medicine: a literature review. Front Med 10:1180773 PMC10213317 37250654 · DOI · Paper page
  37. [45]Liu C-F, Chen Z-C, Kuo S-C, Lin T-C (2022) Does AI explainability affect physicians’ intention to use AI? Int J Med Inform 168:104884 36228415 · DOI
  38. [46]Fazlollahi AM, Bakhaidar M, Alsayegh A et al (2022) Effect of artificial intelligence tutoring vs expert instruction on learning simulated surgical skills among medical students: a randomized clinical trial. JAMA Netw Open 5:2149008 PMC8864513 35191972 · DOI · Paper page

Cited by

Sorted by citations

Authors
Savardi, M., Signoroni, A., Benini, S., Vaccher, F., Alberti, M., Ciolli, P., Di Meo, N., Falcone, T., Ramanzin, M., Romano, B., Sozzi, F., & Farina, D.
Affiliations
  • University of Brescia (IT)
  • Azienda Socio Sanitaria Territoriale degli Spedali Civili di Brescia (IT)
Corresponding author
Signoroni, Alberto
ORCID
Venue
Insights into Imaging
Publisher
Springer Science and Business Media LLC
Published
2025-01-29
Volume
16
Issue
1
Pages
23-23
ISSN
1869-4101 (ISSN-L)
DOI
10.1186/s13244-024-01893-4
PMC
PMC11780016
OpenAlex
W4406960076
Cited by
30
References
8
Citations by year
2026: 21 · 2025: 8 · 2024: 1
FWCI
5.94
Type
journal article
Language
en
Open access
Licence
Locations
  • Insights into Imaging (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Institutional Research Information System (Università degli Studi di Brescia) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Ministero dell'Università e della Ricerca (FISR2020IP-02278)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
Keywords
artificial intelligence in medical education, radiology residency, COVID-19 pandemic, pulmonary involvement, AI-assisted diagnosis, deskilling, upskilling, interobserver agreement, explainability, collaborative AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review