The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Regulatory Frameworks for Development and Evaluation of Artificial Intelligence–Based Diagnostic Imaging Algorithms: Summary and Recommendations
Open accessAbstract
Although artificial intelligence (AI)-based algorithms for diagnosis hold promise for improving care, their safety and effectiveness must be ensured to facilitate wide adoption. Several recently proposed regulatory frameworks provide a solid foundation but do not address a number of issues that may prevent algorithms from being fully trusted. In this article, we review the major regulatory frameworks for software as a medical device applications, identify major gaps, and propose additional strategies to improve the development and evaluation of diagnostic AI algorithms. We identify the following major shortcomings of the current regulatory frameworks: (1) conflation of the diagnostic task with the diagnostic algorithm, (2) superficial treatment of the diagnostic task definition, (3) no mechanism to directly compare similar algorithms, (4) insufficient characterization of safety and performance elements, (5) lack of resources to assess performance at each installed site, and (6) inherent conflicts of interest. We recommend the following additional measures: (1) separate the diagnostic task from the algorithm, (2) define performance elements beyond accuracy, (3) divide the evaluation process into discrete steps, (4) encourage assessment by a third-party evaluator, (5) incorporate these elements into the manufacturers' development process. Specifically, we recommend four phases of development and evaluation, analogous to those that have been applied to pharmaceuticals and proposed for software applications, to help ensure world-class performance of all algorithms at all installed sites. In the coming years, we anticipate the emergence of a substantial body of research dedicated to ensuring the accuracy, reliability, and safety of the algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Given the high stakes nature of medical AI, human oversight is essential and increasingly required by policy makers and regulators.17 26”
From FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare (2025) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“When new software vulnerabilities come to light, they should be documented and reported32,61, and so should any changes made to the AIPM in response to an attack after thorough testing10,23,35,61.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
Experimental — accuracy not yet measuredIn the citing paper
“External validation is ideally performed by independent researchers from other institutions or settings3,8,10,16,27,68,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“To ensure the generalizability of the AIPM to the intended healthcare setting, developers are advised to extensively validate the model for representative data from that setting6–8,10,13–15,24,27,56,64,66,68,72,77,79,87,88 (see phase 1, Representativeness).”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“The literature recommends to assess generalizability on external data from a different time period, place, or healthcare setting3,7,8,10,16,17,27,56,68,72,79.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“The AIPM should be evaluated according to how it interfaces with the end user, and how well the AIPM and the user perform together in a typical environment10,100,102,103.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Designing a good user interface and interaction requires careful consideration of the cognitive load of the end users10,18,68,99,100, by showing only relevant information in the right context, and by allowing adjustment of its behavior by end users99.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“It was proposed that such evaluations can, for example, be done via reader and user studies10,102,103.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“From a user interaction design perspective, it has been recommended that decisions are deterministic (consistently giving the same output for a certain input)10, and that updates of or adaptations to the AIPM should happen cautiously99.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“These mechanisms should detect situations beyond the capabilities of the AIPM10,99, and share the confidence in the predictions with the user10,22,26,99.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Finally, as monitoring and auditing of AIPMs in practice are widely recommended (covered in more detail in phase 6), the developed software should facilitate this10,22,26,32,58,62,104.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Deliberate stress tests like load testing, penetration testing, integration testing and unit testing are important for the verification of the AIPM from a software perspective10,26,35,46,66,79.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“This means adequate logging and traceability of predictions and decisions is required and the AIPM interface should provide sharing of performance data with end users to enable ongoing monitoring of both individual and aggregated cases, quickly highlighting any significant deviations in performance10,26,61,66.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“A feasibility study or implementation pilot is recommended preceding an impact study to ensure correct and safe use in healthcare practice10,18,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Comparable to the feasibility study of phase 5, pilot studies are recommended to examine the potential pitfalls during implementation, considering both software and hardware issues10,18,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Second, the monitoring of dataset shift is also repeatedly mentioned in the literature5,10,22,72,79,91.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Developers are recommended to regularly update their AIPMs over time to improve the AIPM’s predictive performance as new improvements become available and to mitigate dataset shift10,19,23.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Additionally, monitoring the algorithm’s performance helps identify any potential biases or unintended consequences that may arise during its use [43].”
From Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education (2024) · cites this paper as [43]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ai, T., Yang, Z., Hou, H., Zhan, C., Chen, C., Lv, W., Tao, Q., Sun, Z., & Xia, L. (2020) · Radiology · doi:10.1148/radiol.2020200642 · Cited by 5,793
Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR
Fang, Y., Zhang, H., Xie, J., Lin, M., Ying, L., Pang, P., & Ji, W. (2020) · Radiology · doi:10.1148/radiol.2020200432 · Cited by 3,340
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719 · cited 2×
“an image that is not salient to the diagnostic task [31-33].”
Enhancing The Reliability of Out-of-distribution Image Detection in Neural NetworksPreprint
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 1,463
Adversarial attacks on medical machine learning
Finlayson, S. G., Bowers, J. D., Ito, J., Zittrain, J. L., Beam, A. L., & Kohane, I. S. (2019) · Science · doi:10.1126/science.aaw4399 · Cited by 1,088
The current radiological approach to renal cysts.
Bosniak, M. A. (1986) · Radiology · doi:10.1148/radiology.158.1.3510019 · Cited by 1,051
Temporal Changes of CT Findings in 90 Patients with COVID-19 Pneumonia: A Longitudinal Study
Wang, Y., Dong, C., Hu, Y., Li, C., Ren, Q., Zhang, X., Shi, H., & Zhou, M. (2020) · Radiology · doi:10.1148/radiol.2020200843 · Cited by 959
Prokop, M., van Everdingen, W. M., Vellinga, T. V. R., van Ufford, H. M. E. Q., Stöger, J. L., Beenen, L. F. M., Geurts, B. H. J., Gietema, H. A., Krdzalic, J., Schaefer‐Prokop, C. M., et al. (2020) · Radiology · doi:10.1148/radiol.2020201473 · Cited by 922
Pesapane, F., Codari, M., & Sardanelli, F. (2018) · European Radiology Experimental · doi:10.1186/s41747-018-0061-6 · Cited by 902
On instabilities of deep learning in image reconstruction and the potential costs of AI
Antun, V., Renna, F., Poon, C., Adcock, B., & Hansen, A. C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907377117 · Cited by 845
Simpson, S. A., Kay, F. U., Abbara, S., Bhalla, S., Chung, J. H., Chung, M., Henry, T. S., Kanne, J. P., Kligerman, S. J., Ko, J. P., et al. (2020) · Journal of Thoracic Imaging · doi:10.1097/rti.0000000000000524 · Cited by 774
Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution SamplesPreprint
Lee, K., Lee, H., Lee, K., & Shin, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.09325 · Cited by 650
Chest CT Features of COVID-19 in Rome, Italy
Caruso, D., Zerunian, M., Polici, M., Pucciarelli, F., Polidori, T., Rucci, C., Guido, G., Bracci, B., Dominicis, C. D., & Laghi, A. (2020) · Radiology · doi:10.1148/radiol.2020201237 · Cited by 614
Learning Confidence for Out-of-Distribution Detection in Neural NetworksPreprint
DeVries, T., & Taylor, G. W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04865 · Cited by 567
Kazerooni, E. A., Martínez, F. J., Flint, A., Jamadar, D. A., Gross, B. H., Spizarny, D. L., Cascade, P. N., Whyte, R. I., Lynch, J. P., & Toews, G. (1997) · American Journal of Roentgenology · doi:10.2214/ajr.169.4.9308447 · Cited by 468
Key Concepts of Clinical Trials: A Narrative Review
Umscheid, C. A., Margolis, D. J., & Grossman, C. E. (2011) · Postgraduate Medicine · doi:10.3810/pgm.2011.09.2475 · Cited by 414
Chang, Y., Yu, C., Chang, S., Galvin, J. R., Liu, H., Hsiao, C.-H., Kuo, P.-H., Chen, K., Franks, T. J., Huang, K.-M., et al. (2005) · Radiology · doi:10.1148/radiol.2363040958 · Cited by 370
Winkler, J. K., Fink, C., Toberer, F., Enk, A., Deinlein, T., Hofmann-Wellenhof, R., Thomas, L., Lallas, A., Blum, A., Stolz, W., et al. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1735 · Cited by 365
Severe Acute Respiratory Syndrome: Temporal Lung Changes at Thin-Section CT in 30 Patients
OOI, G. C., Khong, P. L., Müller, N. L., Yiu, W. C., Zhou, L. J., Ho, J., LAM, B. K., Nicolaou, S., & TSANG, K. W. T. (2004) · Radiology · doi:10.1148/radiol.2303030853 · Cited by 329
From development to deployment: dataset shift, causality, and shift-stable models in health AI
Subbaswamy, A., & Saria, S. (2019) · Biostatistics · doi:10.1093/biostatistics/kxz041 · Cited by 276
Inconsistent Performance of Deep Learning Models on Mammogram Classification
Wang, X., Liang, G., Zhang, Y., Blanton, H., Bessinger, Z., & Jacobs, N. (2020) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2020.01.006 · Cited by 181
Designing Medical Informatics Research and Library--Resource Projects to Increase What Is Learned
Stead, W. W., Haynes, R. B., Fuller, S. S., Friedman, C. P., Travis, L. E., Beck, J. R., Fenichel, C. H., Chandrasekaran, B., Buchanan, B. G., Abola, E. E., et al. (1994) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1994.95236134 · Cited by 127
Huang, G.-Q., Gong, T., Wang, G., Wang, J., Guo, X., Erpeng, C., Li, S., Li, X., Yu, Y., & Lin, L. (2020) · American Journal of Roentgenology · doi:10.2214/ajr.20.23078 · Cited by 108
Preventing Failures Due to Dataset Shift: Learning Predictive Models That TransportPreprint
Subbaswamy, A., Schulam, P., & Saria, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.04597 · Cited by 93
Is digital medicine different?
Lancet, T. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31562-9 · Cited by 73
Beckett, M. K., Quiter, E. S., Ryan, G. W., Berrebi, C., Taylor, S. L., Cho, M., Pincus, H. A., & Kahn, K. L. (2011) · Implementation Science · doi:10.1186/1748-5908-6-35 · Cited by 60
Evaluation of clinical decision‐aids, with an application to a system for dyspepsia
Spiegelhalter, D. J. (1983) · Statistics in Medicine · doi:10.1002/sim.4780020215 · Cited by 48
Radiology in 2018: Are You Working with AI or Being Replaced by AI?
Bluemke, D. A. (2018) · Radiology · doi:10.1148/radiol.2018184007 · Cited by 45
Imaging Quality Control in the Era of Artificial Intelligence
Larson, D. B., & Boland, G. W. (2019) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2019.05.048 · Cited by 44
Self-citationHecker, S. J., Preston, C., & Foote, M. (2003) · Biotechnology annual review · doi:10.1016/s1387-2656(03)09006-9 · Cited by 4
Research works printed but not linked: 1
- Pan, 2020, Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia [E-pub ahead of print, 2020 Feb 13], Radiology, 200370
Cited by
Sorted by citations
The Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century
Varnosfaderani, S. M., & Forouzanfar, M. (2024) · Bioengineering · doi:10.3390/bioengineering11040337 · Cited by 1,141
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
“17 26 Human-in-the-loop mechanisms should be designed and implemented to perform specific quality checks (eg, to flag biases, errors, or implausible explanations), and to overrule the AI predictions when necessary.”
Self-citationde Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499
Deep learning for diabetic retinopathy detection and classification based on fundus images: A review
Tsiknakis, N., Theodoropoulos, D., Manikis, G. C., Ktistakis, E., Boutsora, O., Bertó, A., Scarpa, F., Scarpa, A., Fotiadis, D. I., & Kostas Marias, A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104599 · Cited by 359
Qin, Z. Z., Ahmed, S., Sarker, M. S., Paul, K., Adel, A. S. S., Naheyan, T., Barrett, R., Banu, S., & Creswell, J. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00116-3 · Cited by 290
Metadata
- Authors
- Larson, D. B., Harvey, H., Rubin, D. L., Irani, N., Tse, J. R., & Langlotz, C. P.
- Affiliations
- Stanford Medicine (US)
- University College London (GB)
- University of Kansas Medical Center (US)
- University of California, Los Angeles (US)
- Corresponding author
- Larson, David B.
- Venue
- Journal of the American College of Radiology
- Publisher
- Elsevier BV
- Published
- 2021-03
- Volume
- 18
- Issue
- 3
- Pages
- 413-424
- ISSN
- 1546-1440 (ISSN-L)
- PMC
- PMC7574690
- OpenAlex
- W3093602247
- Cited by
- 198
- References
- 28
- Citations by year
- 2026: 38 · 2025: 51 · 2024: 36 · 2023: 24 · 2022: 29 · 2021: 20
- FWCI
- 1.39
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American College of Radiology (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- International Business Machines Corporation
- GE Healthcare
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Healthcare Regulation
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- artificial intelligence, diagnostic imaging algorithm, regulatory framework, software as a medical device, algorithm evaluation, performance characterization, third-party evaluation, conflicts of interest, medical device regulation
- MeSH terms
- Algorithms, Artificial Intelligence, Diagnostic Imaging, Software, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
