A holistic approach to implementing artificial intelligence in radiology
Open accessAbstract
Objective: Despite the widespread recognition of the importance of artificial intelligence (AI) in healthcare, its implementation is often limited. This article aims to address this implementation gap by presenting insights from an in-depth case study of an organisation that approached AI implementation with a holistic approach. Materials and methods: We conducted a longitudinal, qualitative case study of the implementation of AI in radiology at a large academic medical centre in the Netherlands for three years. Collected data consists of 43 days of work observations, 30 meeting observations, 18 interviews and 41 relevant documents. Abductive reasoning was used for systematic data analysis, which revealed three change initiative themes responding to specific AI implementation challenges. Results: This study identifies challenges of implementing AI in radiology at different levels and proposes a holistic approach to tackle those challenges. At the technology level, there is the issue of multiple narrow AI applications with no standard use interface; at the workflow level, AI results allow limited interaction with radiologists; at the people and organisational level, there are divergent expectations and limited experience with AI. The case of Southern illustrates that organisations can reap more benefits from AI implementation by investing in long-term initiatives that holistically align both social and technological aspects of clinical practice. Conclusion: This study highlights the importance of a holistic approach to AI implementation that addresses challenges spanning technology, workflow, and organisational levels. Aligning change initiatives between these different levels has proven to be important to facilitate wide-scale implementation of AI in clinical practice. Critical relevance statement: Adoption of artificial intelligence is crucial for future-ready radiological care. This case study highlights the importance of a holistic approach that addresses technological, workflow, and organisational aspects, offering practical insights and solutions to facilitate successful AI adoption in clinical practice. Key points: 1. Practical and actionable insights into successful AI implementation in radiology are lacking. 2. Aligning technology, workflow, organisational aspects is crucial for a successful AI implementation 3. Holistic approach aids organisations to create sustainable value through AI implementation. Graphical Abstract:
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These findings underscore the value of engaging stakeholders throughout the process.
“These findings underscore the value of engaging stakeholders (e.g., national leadership, network teams, Trust clinical, managerial, and technical staff, and AI suppliers) throughout the process, reflecting previous published research on implementing AI in radiology services.25,26”
From Procurement and early deployment of artificial intelligence tools for chest diagnostics in NHS services in England: a rapid, mixed method evaluation (2025) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Strohm, L., Hehakaya, C., Ranschaert, E. R., Boon, W. P. C., & Moors, E. H. M. (2020) · European Radiology · doi:10.1007/s00330-020-06946-y · Cited by 329 · see citing works
Flagged first, then by citations
Research works printed but not linked: 20
- [1]Choy G Khalilzadeh O Michalski M Current Applications and future impact of machine learning in radiology Radiology 2018 288 318 328 10.1148/radiol.2018171820 29944078 PMC6542626 · DOI · Paper page
- [2]Muehlematter UJ Daniore P Vokinger KN Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015–20): a comparative analysis Lancet Digital Health 2021 3 e195 e203 10.1016/S2589-7500(20)30292-2 33478929 · DOI · Paper page
- [3]Kotter E Ranschaert E Challenges and solutions for introducing artificial intelligence (AI) in daily clinical workflow Eur Radiol 2021 31 5 7 10.1007/s00330-020-07148-2 32797308 PMC7755626 · DOI
- [4]Allen B Seltzer SE Langlotz CP A road map for translational research on artificial intelligence in medical imaging: from the 2018 National Institutes of Health/RSNA/ACR/The Academy Workshop J Am Coll Radiol 2019 16 1179 1189 10.1016/j.jacr.2019.04.014 31151893 · DOI
- [5]Gauriau R Bridge C Chen L Using DICOM metadata for radiological image series categorization: a feasibility study on large clinical brain MRI datasets J Digit Imaging 2020 33 747 762 10.1007/s10278-019-00308-x 31950302 PMC7256138 · DOI
- [6]Sohn JH Chillakuru YR Lee S An open-source, vender agnostic hardware and software pipeline for integration of artificial intelligence in radiology workflow J Digit Imaging 2020 33 1041 1046 10.1007/s10278-020-00348-8 32468486 PMC7522128 · DOI
- [7]Rezazade Mehrizi MH van Ooijen P Homan M Applications of artificial intelligence (AI) in diagnostic radiology: a technography study Eur Radiol 2020 31 1805 1811 10.1007/s00330-020-07230-9 32945967 PMC7979626 · DOI
- [8]Leiner T Bennink E Mol CP Bringing AI to the clinic: blueprint for a vendor-neutral AI deployment infrastructure Insights Imaging 2021 12 11 10.1186/s13244-020-00931-1 33528677 PMC7855120 · DOI
- [9]Huisman M Ranschaert E Parker W An international survey on AI in radiology in 1041 radiologists and radiology residents part 2: expectations, hurdles to implementation, and education Eur Radiol 2021 10.1007/s00330-021-07782-4 33974148 PMC8111651 · DOI
- [10]Recht MP Dewey M Dreyer K Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations Eur Radiol 2020 30 3576 3584 10.1007/s00330-020-06672-5 32064565 · DOI · Paper page
- [11]Yin RK Case study research: design and methods 2002 Newbury Park SAGE Publications
- [12]Small ML ‘How many cases do I need?’: on science and the logic of case selection in field-based research Ethnography 2009 10 5 38 10.1177/1466138108099586 · DOI · Paper page
- [13]Lyytinen K Newman M Explaining information systems change: a punctuated socio-technical change model Eur J Inf Syst 2008 17 589 613 10.1057/ejis.2008.50 · DOI
- [14]Pettigrew AM Longitudinal field research on change: theory and practice Organ Sci 1990 1 267 292 10.1287/orsc.1.3.267 · DOI · Paper page
- [15]Carter N Bryant-Lukosius D DiCenso A The use of triangulation in qualitative research Oncol Nurs Forum 2014 41 545 547 10.1188/14.ONF.545-547 25158659 · DOI · Paper page
- [16]Johnson TR Krems JF Use of current explanations in multicausal abductive reasoning Cogn Sci 2001 25 903 939 10.1207/s15516709cog2506_2 · DOI
- [17]Filice RW Mongan J Kohli MD Evaluating artificial intelligence systems to guide purchasing decisions J Am Coll Radiol 2020 17 1405 1409 10.1016/j.jacr.2020.09.045 33035503 · DOI
- [19]Kim B Koopmanschap I Mehrizi MHR How does the radiology community discuss the benefits and limitations of artificial intelligence for their work? A systematic discourse analysis Eur J Radiol 2021 136 109566 10.1016/j.ejrad.2021.109566 33556686 · DOI
- [20]Omoumi P Ducarouge A Tournier A To buy or not to buy—evaluating commercial AI solutions in radiology (the ECLAIR guidelines) Eur Radiol 2021 31 3786 3796 10.1007/s00330-020-07684-x 33666696 PMC8128726 · DOI
- [21]Lund C Of what is this a case?: Analytical movements in qualitative social science research Hum Organ 2014 73 224 234 10.17730/humo.73.3.e35q482014x033l4 · DOI
Cited by
Sorted by citations
Artificial intelligence for diagnostics in radiology practice: a rapid systematic scoping review
Lawrence, R., Dodsworth, E., Massou, E., Sherlaw-Johnson, C., Ramsay, A. I. G., Walton, H., O'Regan, T., Gleeson, F., Crellin, N., Herbert, K., et al. (2025) · eClinicalMedicine · doi:10.1016/j.eclinm.2025.103228 · Cited by 69 · see citing works
Ramsay, A. I. G., Crellin, N., Lawrence, R., Walton, H., Bagri, S., Dodsworth, E., Elphinstone, H., Gleeson, F., Halliday, A., Herbert, K., et al. (2025) · eClinicalMedicine · doi:10.1016/j.eclinm.2025.103481 · Cited by 13 · see citing works
“25,26 Many factors identified in our analysis—including context (at system, organization, and service levels), leadership and engagement approaches, technological considerations, use of data, and local capacity and resourcing (e.g., finance, people)–reflect established findings on implementation of…”
- Authors
- Kim, B., Romeijn, S., van Buchem, M., Mehrizi, M. H. R., & Grootjans, W.
- Affiliations
- Stockholm School of Economics (SE)
- Leiden University Medical Center (NL)
- Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Romeijn, Stephan
- Venue
- Insights into Imaging
- Published
- 2024-01-25
- Volume
- 15
- Issue
- 1
- Pages
- 22-22
- ISSN
- 1869-4101 (ISSN-L)
- PMC
- PMC10811299
- OpenAlex
- W4391226753
- Cited by
- 55
- References
- 1
- Citations by year
- 2026: 15 · 2025: 27 · 2024: 12 · 2020: 1
- FWCI
- 5.05
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Insights into Imaging (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- VU Research Portal (submittedVersion, other-oa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiology practices and education
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- AI implementation, radiology, holistic approach, implementation barriers, clinical workflow, organizational change, artificial intelligence adoption, academic medical centers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
