Black box no more: a scoping review of AI governance frameworks to guide procurement and adoption of AI in medical imaging and radiotherapy in the UK
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Technological advancements in computer science have started to bring artificial intelligence (AI) from the bench closer to the bedside. While there is still lots to do and improve, AI models in medical imaging and radiotherapy are rapidly being developed and increasingly deployed in clinical practice. At the same time, AI governance frameworks are still under development. Clinical practitioners involved with procuring, deploying, and adopting AI tools in the UK should be well-informed about these AI governance frameworks. This scoping review aimed to map out available literature on AI governance in the UK, focusing on medical imaging and radiotherapy. Searches were performed on Google Scholar, Pubmed, and the Cochrane Library, between June and July 2022. Of 4225 initially identified sources, 35 were finally included in this review. A comprehensive conceptual AI governance framework was proposed, guided by the need for rigorous AI validation and evaluation procedures, the accreditation rules and standards, and the fundamental ethical principles of AI. Fairness, transparency, trustworthiness, and explainability should be drivers of all AI models deployed in clinical practice. Appropriate staff education is also mandatory to ensure AI’s safe and responsible use. Multidisciplinary teams under robust leadership will facilitate AI adoption, and it is crucial to involve patients, the public, and practitioners in decision-making. Collaborative research should be encouraged to enhance and promote innovation, while caution should be paid to the ongoing auditing of AI tools to ensure safety and clinical effectiveness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565
Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0611-x · Cited by 11,385
Updated methodological guidance for the conduct of scoping reviews
Peters, M. D. J., Marnie, C., Tricco, A. C., Pollock, D., Munn, Z., Alexander, L., McInerney, P., Godfrey, C. M., & Khalil, H. (2020) · JBI Evidence Synthesis · doi:10.11124/jbies-20-00167 · Cited by 9,046
Artificial intelligence in radiology
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. W. L. (2018) · Nature Reviews Cancer · doi:10.1038/s41568-018-0016-5 · Cited by 3,803
A hands-on guide to doing content analysis
Erlingsson, C., & Brysiewicz, P. (2017) · African Journal of Emergency Medicine · doi:10.1016/j.afjem.2017.08.001 · Cited by 2,358
The rise of artificial intelligence in healthcare applications
Bohr, A., & Memarzadeh, K. (2020) · Artificial Intelligence in Healthcare · doi:10.1016/b978-0-12-818438-7.00002-2 · Cited by 1,636
Privacy in the age of medical big data
Price, W. N., & Cohen, I. G. (2019) · Nature Medicine · doi:10.1038/s41591-018-0272-7 · Cited by 1,522
Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers
Mongan, J., Moy, L., & Kahn, C. E. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2020200029 · Cited by 1,443
Do no harm: a roadmap for responsible machine learning for health care
Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V. X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0548-6 · Cited by 1,266
Kleinheksel, A. J., Rockich-Winston, N., Tawfik, H., & Wyatt, T. R. (2020) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe7113 · Cited by 1,054
A governance model for the application of AI in health care
Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz192 · Cited by 979
Addressing bias in big data and AI for health care: A call for open science
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100347 · Cited by 903
Scoping reviews: reinforcing and advancing the methodology and application
Peters, M. D. J., Marnie, C., Colquhoun, H., Garritty, C. M., Hempel, S., Horsley, T., Langlois, E. V., Lillie, E., O’Brien, K. K., Tunçalp, Ӧ., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01821-3 · Cited by 887 · cited 1×
“26 All searches were performed using explicit, pre-defined keywords related to the topic under exploration.”
MethodThe Clinician and Dataset Shift in Artificial Intelligence
Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., & Saria, S. (2021) · New England Journal of Medicine · doi:10.1056/nejmc2104626 · Cited by 855
Artificial Intelligence in Medicine: Today and Tomorrow
Briganti, G., & Le Moine, O. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.00027 · Cited by 743
Kuziemski, M., & Misuraca, G. (2020) · Telecommunications Policy · doi:10.1016/j.telpol.2020.101976 · Cited by 610 · cited 1×
“72 For the seamless adoption of AI, training is essential.”
BackgroundAung, Y. Y. M., Wong, D. C. S., & Ting, D. S. W. (2021) · British Medical Bulletin · doi:10.1093/bmb/ldab016 · Cited by 511
Identifying Ethical Considerations for Machine Learning Healthcare Applications
Char, D. S., Abràmoff, M. D., & Feudtner, C. (2020) · The American Journal of Bioethics · doi:10.1080/15265161.2020.1819469 · Cited by 415
Wirtz, B. W., Weyerer, J. C., & Sturm, B. J. (2020) · International Journal of Public Administration · doi:10.1080/01900692.2020.1749851 · Cited by 398
Poon, Y.-S. R., Lin, Y. P., Griffiths, P., Yong, K. K., Seah, B., & Liaw, S. Y. (2022) · Human Resources for Health · doi:10.1186/s12960-022-00764-7 · Cited by 394 · cited 1×
“84 Before procuring AI models, a detailed cost/benefit analysis 85 should aim to reduce costs and improve services to benefit the patients and healthcare staff.”
BackgroundGeis, J. R., Brady, A. P., Wu, C. C., Spencer, J., Ranschaert, E., Jaremko, J. L., Langer, S. G., Borondy Kitts, A., Birch, J., Shields, W. F., et al. (2019) · Radiology · doi:10.1148/radiol.2019191586 · Cited by 388
Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications
Schönberger, D. (2019) · International Journal of Law and Information Technology · doi:10.1093/ijlit/eaz004 · Cited by 230
Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance
Raji, I. D., Xu, P., Honigsberg, C., & Ho, D. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534181 · Cited by 209
Larson, D. B., Harvey, H., Rubin, D. L., Irani, N., Tse, J. R., & Langlotz, C. P. (2021) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2020.09.060 · Cited by 198 · cited 1×
“34,42 Hence, a 3-phase validation process has been proposed, consisting of validation against the test set of the data set (internal validation), validation against unseen data (external validation), and validation against diverse datasets from multiple centres.”
MethodSelf-citationThe DECIDE-AI Steering Group (2021) · Nature Medicine · doi:10.1038/s41591-021-01229-5 · Cited by 184
Artificial intelligence in healthcare and medicine: promises, ethical challenges and governance
Guan, J. (2019) · Chinese Medical Sciences Journal · doi:10.24920/003611 · Cited by 170
Artificial Intelligence in Radiology—Ethical Considerations
Brady, A. P., & Neri, E. (2020) · Diagnostics · doi:10.3390/diagnostics10040231 · Cited by 150 · cited 1×
“47,61 Also, transparent AI solutions will help build trust between patients and healthcare providers and between developers and clinical practitioners as end users.”
Backgroundvan de Sande, D., Van Genderen, M. E., Smit, J. M., Huiskens, J., Visser, J. J., Veen, R. E. R., van Unen, E., BA, O. H., Gommers, D., & Bommel, J. V. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100495 · Cited by 135
A reimbursement framework for artificial intelligence in healthcare
Abràmoff, M. D., Roehrenbeck, C., Trujillo, S., Goldstein, J., Graves, A. S., Repka, M. X., & Silva III, E. “. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00621-w · Cited by 117 · cited 1×
“89,90 However, it remains to be seen how relevant this would be in the UK context.”
BackgroundEthical considerations for artificial intelligence: an overview of the current radiology landscape.
Department of Radiology and Nuclear Medicine, U. H. B., & Akinci D'Antonoli, T. (2020) · Diagnostic and Interventional Radiology · doi:10.5152/dir.2020.19279 · Cited by 80 · cited 4×
“51 Safety might also be compromised by the ‘algorithm update problem’ or ‘concept drift’, which can impact the algorithm’s performance over time as new data comes in. 58 Algorithms could be locked to remain static, limiting usability; a solution to this challenge could be to assess the algorithms…”
BackgroundInfluentialCabitza, F., & Zeitoun, J.-D. (2019) · Annals of Translational Medicine · doi:10.21037/atm.2019.04.07 · Cited by 72 · cited 2×
“38 Evidence suggests the need for internal and external validation using unseen data.”
BackgroundHörnle, J. (2019) · International Journal of Law and Information Technology · doi:10.1093/ijlit/eaz002 · Cited by 43
Artificial Intelligence in Cardiovascular Imaging: “Unexplainable” Legal and Ethical Challenges?
Lang, M., Bernier, A., & Knoppers, B. M. (2022) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2021.10.009 · Cited by 37
McKay, F., Williams, B. J., Prestwich, G., Treanor, D., & Hallowell, N. (2022) · Research Involvement and Engagement · doi:10.1186/s40900-022-00357-7 · Cited by 16
Cited by
AI Governance: A Systematic Literature Review
Batool, A., Zowghi, D., & Bano, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00653-w · Cited by 314
Shelmerdine, S. C., Togher, D., Rickaby, S., & Dean, G. (2024) · Clinical Radiology · doi:10.1016/j.crad.2024.05.018 · Cited by 17
Alami, H., Pozelli Sabio, R., Pérez, E. J., Gagnon, M.-P., Langlois, L., Denis, J.-L., Malas, K., Rivard, L., Savoldelli, M., Ag Ahmed, M. A., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/87448 · Cited by 9
“%) article was excluded, as it was a scoping review [37], resulting in the inclusion of 11 (0.1”
Influential
Metadata
- Authors
- Stogiannos, N., Malik, R., Kumar, A., Barnes, A., Pogose, M., Harvey, H., McEntee, M. F., & Malamateniou, C.
- Venue
- The British Journal of Radiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-10-03
- Volume
- 96
- Issue
- 1152
- Pages
- 20221157-20221157
- ISSN
- 0007-1285 (ISSN-L)
- 1748-880X
- OpenAlex
- W4387002104
- Cited by
- 58
- References
- 34
- FWCI
- 2.01
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.