The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Regulatory Frameworks for Development and Evaluation of Artificial Intelligence–Based Diagnostic Imaging Algorithms: Summary and Recommendations
Open accessAbstract
Although artificial intelligence (AI)-based algorithms for diagnosis hold promise for improving care, their safety and effectiveness must be ensured to facilitate wide adoption. Several recently proposed regulatory frameworks provide a solid foundation but do not address a number of issues that may prevent algorithms from being fully trusted. In this article, we review the major regulatory frameworks for software as a medical device applications, identify major gaps, and propose additional strategies to improve the development and evaluation of diagnostic AI algorithms. We identify the following major shortcomings of the current regulatory frameworks: (1) conflation of the diagnostic task with the diagnostic algorithm, (2) superficial treatment of the diagnostic task definition, (3) no mechanism to directly compare similar algorithms, (4) insufficient characterization of safety and performance elements, (5) lack of resources to assess performance at each installed site, and (6) inherent conflicts of interest. We recommend the following additional measures: (1) separate the diagnostic task from the algorithm, (2) define performance elements beyond accuracy, (3) divide the evaluation process into discrete steps, (4) encourage assessment by a third-party evaluator, (5) incorporate these elements into the manufacturers' development process. Specifically, we recommend four phases of development and evaluation, analogous to those that have been applied to pharmaceuticals and proposed for software applications, to help ensure world-class performance of all algorithms at all installed sites. In the coming years, we anticipate the emergence of a substantial body of research dedicated to ensuring the accuracy, reliability, and safety of the algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Given the high stakes nature of medical AI, human oversight is essential and increasingly required by policy makers and regulators.17 26”
From FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare (2025) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“When new software vulnerabilities come to light, they should be documented and reported32,61, and so should any changes made to the AIPM in response to an attack after thorough testing10,23,35,61.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
Experimental — accuracy not yet measuredIn the citing paper
“External validation is ideally performed by independent researchers from other institutions or settings3,8,10,16,27,68,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“To ensure the generalizability of the AIPM to the intended healthcare setting, developers are advised to extensively validate the model for representative data from that setting6–8,10,13–15,24,27,56,64,66,68,72,77,79,87,88 (see phase 1, Representativeness).”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“The literature recommends to assess generalizability on external data from a different time period, place, or healthcare setting3,7,8,10,16,17,27,56,68,72,79.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“The AIPM should be evaluated according to how it interfaces with the end user, and how well the AIPM and the user perform together in a typical environment10,100,102,103.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Designing a good user interface and interaction requires careful consideration of the cognitive load of the end users10,18,68,99,100, by showing only relevant information in the right context, and by allowing adjustment of its behavior by end users99.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“It was proposed that such evaluations can, for example, be done via reader and user studies10,102,103.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“From a user interaction design perspective, it has been recommended that decisions are deterministic (consistently giving the same output for a certain input)10, and that updates of or adaptations to the AIPM should happen cautiously99.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“These mechanisms should detect situations beyond the capabilities of the AIPM10,99, and share the confidence in the predictions with the user10,22,26,99.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Finally, as monitoring and auditing of AIPMs in practice are widely recommended (covered in more detail in phase 6), the developed software should facilitate this10,22,26,32,58,62,104.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Deliberate stress tests like load testing, penetration testing, integration testing and unit testing are important for the verification of the AIPM from a software perspective10,26,35,46,66,79.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“This means adequate logging and traceability of predictions and decisions is required and the AIPM interface should provide sharing of performance data with end users to enable ongoing monitoring of both individual and aggregated cases, quickly highlighting any significant deviations in performance10,26,61,66.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“A feasibility study or implementation pilot is recommended preceding an impact study to ensure correct and safe use in healthcare practice10,18,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Comparable to the feasibility study of phase 5, pilot studies are recommended to examine the potential pitfalls during implementation, considering both software and hardware issues10,18,72.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Second, the monitoring of dataset shift is also repeatedly mentioned in the literature5,10,22,72,79,91.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Developers are recommended to regularly update their AIPMs over time to improve the AIPM’s predictive performance as new improvements become available and to mitigate dataset shift10,19,23.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [10]
In the citing paper
“Additionally, monitoring the algorithm’s performance helps identify any potential biases or unintended consequences that may arise during its use [43].”
From Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education (2024) · cites this paper as [43]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ai, T., Yang, Z., Hou, H., Zhan, C., Chen, C., Lv, W., Tao, Q., Sun, Z., & Xia, L. (2020) · Radiology · doi:10.1148/radiol.2020200642 · Cited by 5,793
Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR
Fang, Y., Zhang, H., Xie, J., Lin, M., Ying, L., Pang, P., & Ji, W. (2020) · Radiology · doi:10.1148/radiol.2020200432 · Cited by 3,340
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719 · cited 2×
“an image that is not salient to the diagnostic task [31-33].”
Enhancing The Reliability of Out-of-distribution Image Detection in Neural NetworksPreprint
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 1,463
Adversarial attacks on medical machine learning
Finlayson, S. G., Bowers, J. D., Ito, J., Zittrain, J. L., Beam, A. L., & Kohane, I. S. (2019) · Science · doi:10.1126/science.aaw4399 · Cited by 1,088
The current radiological approach to renal cysts.
Bosniak, M. A. (1986) · Radiology · doi:10.1148/radiology.158.1.3510019 · Cited by 1,051
Temporal Changes of CT Findings in 90 Patients with COVID-19 Pneumonia: A Longitudinal Study
Wang, Y., Dong, C., Hu, Y., Li, C., Ren, Q., Zhang, X., Shi, H., & Zhou, M. (2020) · Radiology · doi:10.1148/radiol.2020200843 · Cited by 959
Prokop, M., van Everdingen, W. M., Vellinga, T. V. R., van Ufford, H. M. E. Q., Stöger, J. L., Beenen, L. F. M., Geurts, B. H. J., Gietema, H. A., Krdzalic, J., Schaefer‐Prokop, C. M., et al. (2020) · Radiology · doi:10.1148/radiol.2020201473 · Cited by 922
Pesapane, F., Codari, M., & Sardanelli, F. (2018) · European Radiology Experimental · doi:10.1186/s41747-018-0061-6 · Cited by 902
Research works printed but not linked: 1
- Pan, 2020, Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia [E-pub ahead of print, 2020 Feb 13], Radiology, 200370
Cited by
Sorted by citations
The Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century
Varnosfaderani, S. M., & Forouzanfar, M. (2024) · Bioengineering · doi:10.3390/bioengineering11040337 · Cited by 1,141
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
“17 26 Human-in-the-loop mechanisms should be designed and implemented to perform specific quality checks (eg, to flag biases, errors, or implausible explanations), and to overrule the AI predictions when necessary.”
Self-citationde Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499
Deep learning for diabetic retinopathy detection and classification based on fundus images: A review
Tsiknakis, N., Theodoropoulos, D., Manikis, G. C., Ktistakis, E., Boutsora, O., Bertó, A., Scarpa, F., Scarpa, A., Fotiadis, D. I., & Kostas Marias, A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104599 · Cited by 359
Qin, Z. Z., Ahmed, S., Sarker, M. S., Paul, K., Adel, A. S. S., Naheyan, T., Barrett, R., Banu, S., & Creswell, J. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00116-3 · Cited by 290
How does artificial intelligence in radiology improve efficiency and health outcomes?
van Leeuwen, K. G., de Rooij, M., Schalekamp, S., van Ginneken, B., & Rutten, M. J. C. M. (2022) · Pediatric Radiology · doi:10.1007/s00247-021-05114-8 · Cited by 236
Technology readiness levels for machine learning systems
Lavin, A., Lee, C. M., Visnjic, A., Ganju, S., Newman, D. J., Ganguly, S., Lange, D., Baydin, A. G., Sharma, A., Gibson, A., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33128-9 · Cited by 247
Yelne, S., Chaudhary, M., Dod, K., Sayyad, A., & Sharma, R. (2023) · Cureus · doi:10.7759/cureus.49252 · Cited by 190
The digital scribe in clinical practice: a scoping review and research agenda
van Buchem, M. M., Boosman, H., Bauer, M. P., Kant, I. M. J., Cammel, S. A., & Steyerberg, E. W. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00432-5 · Cited by 152
Franco D’Souza, R., Mathew, M., Mishra, V., & Surapaneni, K. M. (2024) · Medical Education Online · doi:10.1080/10872981.2024.2330250 · Cited by 147
Morley, J., Murphy, L., Mishra, A. B., Joshi, I., & Karpathakis, K. (2022) · JMIR Formative Research · doi:10.2196/31623 · Cited by 182
Parvin, N., Joo, S., Jung, J. H., & Mandal, T. K. (2025) · Nanomaterials · doi:10.3390/nano15120895 · Cited by 131
Jha, A. K., Bradshaw, T., Buvat, I., Hatt, M., Prabhat, K. C., Liu, C., Obuchowski, N., Saboury, B., Slomka, P. J., Sunderland, J. J., et al. (2022) · Journal of Nuclear Medicine · doi:10.2967/jnumed.121.263239 · Cited by 117
Optimization of Radiology Workflow with Artificial Intelligence
Ranschaert, E. R., Topff, L., & Pianykh, O. S. (2021) · Radiologic Clinics of North America · doi:10.1016/j.rcl.2021.06.006 · Cited by 110
Fraser, A. G., Biasin, E., Bijnens, B. H., Bruining, N., Caiani, E. G., Cobbaert, K., Davies, R. H., Gilbert, S., Hovestadt, L., Kamenjašević, E., et al. (2023) · Expert Review of Medical Devices · doi:10.1080/17434440.2023.2184685 · Cited by 116
Jacob, C., Brasier, N., Laurenzi, E., Heuss, S., Mougiakakou, S. G., Çöltekin, A., & Peter, M. K. (2024) · Journal of Medical Internet Research · doi:10.2196/67485 · Cited by 125
Voter, A. F., Meram, E., Garrett, J. W., & Yu, J. J. (2021) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2021.03.005 · Cited by 93
Big data and predictive analytics in healthcare in Bangladesh: regulatory challenges
Hassan, S., Dhali, M., Zaman, F., & Tanveer, M. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e07179 · Cited by 96
Allen, B., Dreyer, K. J., Stibolt, R., Agarwal, S., Coombs, L. P., Treml, C., Elkholy, M., Brink, L., & Wald, C. (2021) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2021.08.022 · Cited by 80
Petersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 92
Olawade, D. B., Plabon, S. B., Ojo, A., Ogunbona, M. A., Makanjuola, B. D., & Olasilola, R. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106362 · Cited by 93
Areas of research to support the system of radiological protection
Laurier, D., Rühm, W., Paquet, F., Applegate, K. E., Cool, D. A., Clement, C. H., & the International Commission on Radiological Protection (ICRP) (2021) · Radiation and Environmental Biophysics · doi:10.1007/s00411-021-00947-1 · Cited by 77
Lambin, P., Woodruff, H. C., Mali, S. A., Zhong, X., Kuang, S., Lavrova, E., Khan, H., Lekadir, K., Zwanenburg, A., Deasy, J. O., et al. (2025) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-025-01067-1 · Cited by 72
Artificial Intelligence in Clinical Oncology: From Data to Digital Pathology and Treatment
Kumar, K. S., Miskovic, V., Blasiak, A., Sundar, R., Pedrocchi, A., Pearson, A. T., Prelaj, A., & Ho, D. (2023) · American Society of Clinical Oncology Educational Book · doi:10.1200/edbk_390084 · Cited by 68
Data Governance in AI - Enabled Healthcare Systems: A Case of the Project Nightingale
Arigbabu, A. T., Olaniyi, O. O., Adigwe, C. S., Adebiyi, O. O., & Ajayi, S. A. (2024) · Asian Journal of Research in Computer Science · doi:10.9734/ajrcos/2024/v17i5441 · Cited by 64
Stogiannos, N., Malik, R., Kumar, A., Barnes, A., Pogose, M., Harvey, H., McEntee, M. F., & Malamateniou, C. (2023) · British Journal of Radiology · doi:10.1259/bjr.20221157 · Cited by 66
Self-citationArtificial intelligence in smart drug delivery systems: a step toward personalized medicine
Panchpuri, M., Painuli, R., & Kumar, C. (2025) · RSC Pharmaceutics · doi:10.1039/d5pm00089k · Cited by 66
Lekadir, K., Osuala, R., Gallin, C., Lazrak, N., Kushibar, K., Tsakou, G., Aussó, S., Alberich, L. C., Marias, K., Tsiknakis, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.09658 · Cited by 58
Expectations for Artificial Intelligence (AI) in Psychiatry
Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., Achtyes, E. D., & Bauer, M. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01378-5 · Cited by 61
Hassan, N., Slight, R. D., Morgan, G., Bates, D. W., Gallier, S., Sapey, E., & Slight, S. P. (2023) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2023-100784 · Cited by 56
Implementing Machine Learning in Interventional Cardiology: The Benefits Are Worth the Trouble
Ali, W. B., Pesaranghader, A., Avram, R., Overtchouk, P., Perrin, N., Laffite, S., Cartier, R., Ibrahim, R., Modine, T., & Hussin, J. (2021) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.711401 · Cited by 56
MAIC–10 brief quality checklist for publications using artificial intelligence and medical images
Cerdá-Alberich, L., Solana, J., Mallol, P., Ribas, G., García-Junco, M., Alberich‐Bayarri, Á., & Martí‐Bonmatí, L. (2023) · Insights into Imaging · doi:10.1186/s13244-022-01355-9 · Cited by 56
Artificial intelligence in stroke rehabilitation: From acute care to long-term recovery
Kopalli, S. R., Shukla, M., Jayaprakash, B., Kundlas, M., Srivastava, A., Jagtap, J., Gulati, M., Chigurupati, S., Ibrahim, E., Khandige, P. S., et al. (2025) · Neuroscience · doi:10.1016/j.neuroscience.2025.03.017 · Cited by 58
Adversarial attacks in radiology – A systematic review
Sorin, V., Soffer, S., Glicksberg, B. S., Barash, Y., Konen, E., & Klang, E. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.111085 · Cited by 54
Machine Learning Algorithms to Detect Subclinical Keratoconus: Systematic Review
Maile, H. P., Li, J.-P. O., Gore, D. M., Leucci, M. T., Mulholland, P. J., Hau, S., Szabó, A., Moghul, I., Balaskas, K., Fujinami, K., et al. (2021) · JMIR Medical Informatics · doi:10.2196/27363 · Cited by 46
Uwimana, A., Gnecco, G., & Riccaboni, M. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109391 · Cited by 51
Real-World and Regulatory Perspectives of Artificial Intelligence in Cardiovascular Imaging
Wellnhofer, E. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.890809 · Cited by 48
Morley, J., Morton, C. E., Karpathakis, K., Taddeo, M., & Floridi, L. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3826358 · Cited by 49
van Leeuwen, K. G., Schalekamp, S., Rutten, M. J. C. M., Huisman, M., Schaefer‐Prokop, C. M., de Rooij, M., van Ginneken, B., Maresch, B., Geurts, B. H. J., van Dijke, C. F., et al. (2024) · Radiology · doi:10.1148/radiol.230981 · Cited by 38
Artificial intelligence in cancer research: learning at different levels of data granularity
Cirillo, D., Núñez‐Carpintero, I., & Valencia, A. (2021) · Molecular Oncology · doi:10.1002/1878-0261.12920 · Cited by 45
Martí‐Bonmatí, L., Koh, D., Riklund, K., Bobowicz, M., Roussakis, Y., Vilanova, J. C., Fütterer, J. J., Rimola, J., Mallol, P., Ribas, G., et al. (2022) · Insights into Imaging · doi:10.1186/s13244-022-01220-9 · Cited by 36
Macrae, C. (2024) · Risk Analysis · doi:10.1111/risa.14273 · Cited by 46
Farah, L., Borget, I., Martelli, N., & Vallée, A. (2024) · Journal of Medical Internet Research · doi:10.2196/51514 · Cited by 40
Artificial intelligence (AI) and ethical concerns: a review and research agenda
Ghosh, M. (2025) · Cogent Business & Management · doi:10.1080/23311975.2025.2551809 · Cited by 46
Kukhareva, P., Weir, C. R., Fiol, G. D., Aarons, G. A., Taft, T., Schlechter, C. R., Reese, T. J., Curran, R. L., Nanjo, C. J., Borbolla, D. A., et al. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104014 · Cited by 42
Reliable and Safe Use of Machine Translation in Medical Settings
Mehandru, N., Robertson, S., & Salehi, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533244 · Cited by 48
External Validation of e‐ASPECTS Software for Interpreting Brain CT in Stroke
Mair, G., White, P., Bath, P. M., Muir, K. W., Salman, R. A., Martin, C., Dye, D., Chappell, F. M., Vacek, A., von Kummer, R., et al. (2022) · Annals of Neurology · doi:10.1002/ana.26495 · Cited by 36
Changes in software as a medical device based on artificial intelligence technologies
Zinchenko, V. V., Четвериков, С. Ф., Akhmad, E. S., Arzamasov, K. M., Vladzymyrskyy, A. V., Андрейченко, А. Е., & Morozov, S. P. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02669-1 · Cited by 30
Vasiliev, Y. A., Vlazimirsky, A. V., Omelyanskaya, O. V., Arzamasov, K. M., Четвериков, С. Ф., Андреевич, Р. Д., & Zelenova, M. A. (2023) · Digital Diagnostics · doi:10.17816/dd321971 · Cited by 28
Alsadoun, L., Ali, H., Mushtaq, M. M., Mushtaq, M., Burhanuddin, M., Anwar, R., Liaqat, M., Bokhari, S. F. H., Hasan, A. H., & Ahmed, F. (2024) · Cureus · doi:10.7759/cureus.67844 · Cited by 31
Lessons learned in transitioning to AI in the medical imaging of COVID-19
Naqa, I. M. E., Hu, Q., Chen, W., Li, H., Fuhrman, J. D., Gorre, N., & Giger, M. L. (2021) · Journal of medical imaging · doi:10.1117/1.jmi.8.s1.010902 · Cited by 30
Zhelev, Z., Peters, J., Rogers, M., Allen, M. J., Kijauskaite, G., Seedat, F., Wilkinson, E. C., & Hyde, C. (2023) · Journal of Medical Screening · doi:10.1177/09691413221144382 · Cited by 31
Chauhan, A. S., Singh, R., Priyadarshi, N., Twala, B., Suthar, S., & Swami, S. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-024-00161-0 · Cited by 28
RetractedAkintuyi, O. B. (2024) · Open Access Research Journal of Engineering and Technology · doi:10.53022/oarjet.2024.6.2.0017 · Cited by 24
Hayat, H., Nukala, A., Nyamira, A., Fan, J., & Wang, P. (2021) · Biomedical Materials · doi:10.1088/1748-605x/ac15b2 · Cited by 26
van der Werff, S. D., van Rooden, S. M., Henriksson, A., Behnke, M., Aghdassi, S. J. S., van Mourik, M. S. M., & Nauclér, P. (2025) · Journal of Internal Medicine · doi:10.1111/joim.20100 · Cited by 28
Tozzi, A. E., Fabozzi, F., Eckley, M., Croci, I., Dell’Anna, V. A., Colantonio, E., & Mastronuzzi, A. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.905770 · Cited by 22
AI-Based CXR First Reading: Current Limitations to Ensure Practical Value
Vasilev, Y. A., Vladzymyrskyy, A. V., Omelyanskaya, O. V., Blokhin, I. A., Kirpichev, Y., & Arzamasov, K. M. (2023) · Diagnostics · doi:10.3390/diagnostics13081430 · Cited by 22
Zippel, C., & Bohnet-Joschko, S. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18105072 · Cited by 22
NAVIGATOR: an Italian regional imaging biobank to promote precision medicine for oncologic patients
Borgheresi, R., Barucci, A., Colantonio, S., Aghakhanyan, G., Assante, M., Bertelli, E., Carlini, E., Carpi, R., Caudai, C., Cavallero, D., et al. (2022) · European Radiology Experimental · doi:10.1186/s41747-022-00306-9 · Cited by 23
Młynarska, E., Bojdo, K., Frankenstein, H., Kustosik, N., Mstowska, W., Przybylak, A., Rysz, J., & Franczyk, B. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14030887 · Cited by 20
Federated Learning in Healthcare: From Research to Real-World Deployment
Bakas, S., Li, X., Shah, P., & Roth, H. R. (2026) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-080125-041414 · Cited by 28
The ABC recommendations for validation of supervised machine learning results in biomedical sciences
Chicco, D., & Jurman, G. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2022.979465 · Cited by 22
Regulatory challenges in ai-based diagnostics: Legal implications of ai use in medical diagnostics
Naili, Y. T., Mangkunegara, I. S., Purwono, P., & Baballe, M. A. (2025) · BIO Web of Conferences · doi:10.1051/bioconf/202515201034 · Cited by 17
The Road Map for ACR Practice Accreditation for Radiology Artificial Intelligence
Larson, D. B., Bhargavan, M., Tilkin, M., Coombs, L. P., & Wald, C. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.02.008 · Cited by 15
Self-citationResponsible adoption of multimodal artificial intelligence in health care: promises and challenges
Azarfar, G., Naimimohasses, S., Rambhatla, S., Komorowski, M., Ferro, D., Lewis, P. R., Gates, D., Shara, N. M., Gascon, G. M., Chang, A. C., et al. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100917 · Cited by 22
Larson, D. B., Doo, F. X., Allen, B., Mongan, J., Flanders, A. E., & Wald, C. (2024) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2024.01.024 · Cited by 14
Self-citationA nested model for AI design and validation
Dubey, A., Yang, Z., & Hattab, G. (2024) · iScience · doi:10.1016/j.isci.2024.110603 · Cited by 18
Human Dignity and Artificial Intelligence in Healthcare: A Basis for a Catholic Ethics on AI
Gozum, I. E. A., & Flake, C. C. (2024) · Journal of Religion and Health · doi:10.1007/s10943-024-02206-1 · Cited by 19
Hao, X., Wang, Y., Li, K., Zhu, T., & Herasevich, V. (2025) · Anesthesiology and Perioperative Science · doi:10.1007/s44254-025-00136-z · Cited by 14
Challenges of Machine Learning and AI (What Is Next?), Responsible and Ethical AI
Gkontra, P., Quaglio, G., Garmendia, A. T., & Lekadir, K. (2023) · Springer eBooks · doi:10.1007/978-3-031-36678-9_17 · Cited by 17
Sindhu, A., Jadhav, U. S., Ghewade, B., Bhanushali, J. D., & Yadav, P. (2024) · Cureus · doi:10.7759/cureus.57657 · Cited by 17
Exploring bias risks in artificial intelligence and targeted medicines manufacturing
Nwebonyi, N., & McKay, F. (2024) · BMC Medical Ethics · doi:10.1186/s12910-024-01112-1 · Cited by 15
Bhimani, M., Miller, A., Agnew, J. D., Ausin, M. S., Raglow-Defranco, M., Mangat, H. S., Voisard, M., Taylor, M., Bierman-Lytle, S., Parikh, V., et al. (2025) · medRxiv · doi:10.1101/2025.03.17.25324157 · Cited by 13
Ammann, C., Hadler, T., Gröschel, J., Kolbitsch, C., & Schulz‐Menger, J. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1118499 · Cited by 13
Mehrtabar, S., Marey, A., Desai, A., Saad, A. M., Desai, V. S., Goñi, J., Pal, B., & Umair, M. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01656-7 · Cited by 14
A review of big data technology and its application in cancer care
Xiao, T., Kong, S., Zhang, Z., Hua, D., & Liu, F. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108577 · Cited by 12
Artificial intelligence methods available for cancer research
Murmu, A., & Győrffy, B. (2024) · Frontiers of Medicine · doi:10.1007/s11684-024-1085-3 · Cited by 15
Regulating AI Adaptation: An Analysis of AI Medical Device UpdatesPreprint
Wu, K., Wu, E., Rodolfa, K. T., Ho, D. E., & Zou, J. (2024) · medRxiv · doi:10.1101/2024.06.26.24309506 · Cited by 19
Federated multimodal AI for precision-equitable diabetes care
Bai, B., Liu, X., & Li, H. Q. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1678047 · Cited by 11
Risk Assessment and Classification of Medical Device Software for the Internet of Medical Things
Brass, I., & Mkwashi, A. (2022) · International Conference on the Internet of Things (IoT) · doi:10.1145/3567445.3571104 · Cited by 11
Antaki, F., Hammana, I., Tessier, M.-C., Boucher, A., Jetté, M. L. D., Beauchemin, C., Hammamji, K., Ong, A. Y., Rhéaume, M., Gauthier, D., et al. (2024) · JMIR Diabetes · doi:10.2196/59867 · Cited by 12
NESTANets: stable, accurate and efficient neural networks for analysis-sparse inverse problems
Neyra-Nesterenko, M., & Adcock, B. (2022) · Sampling Theory Signal Processing and Data Analysis · doi:10.1007/s43670-022-00043-5 · Cited by 10
Ethical and societal aspects of radiological protection for offspring and next generations
Zölzer, F., Schneider, T., Ainsbury, E. A., Goto, A., Liutsko, L., O’Reilly, G., & Lochard, J. (2023) · International Journal of Radiation Biology · doi:10.1080/09553002.2023.2281523 · Cited by 9
Advancing UK Regulatory Science Strategy in the Context of Global Regulation: a Stakeholder Survey
Rivera, S. C., Torlińska, B., Marston, E., Denniston, A. K., Oliver, K. R., Hoare, S., & Calvert, M. (2021) · Therapeutic Innovation & Regulatory Science · doi:10.1007/s43441-021-00263-2 · Cited by 9
Artificial intelligence optimizes the standardized diagnosis and treatment of chronic sinusitis
Liu, Y.-Y., Jiang, S.-P., & Wang, Y. (2025) · Frontiers in Physiology · doi:10.3389/fphys.2025.1522090 · Cited by 10
Sorin, V., Korfiatis, P. D., Bratt, A. K., Leiner, T., Wald, C., Butler, C. S., Cook, C. J., Kline, T. L., & Collins, J. D. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.06.036 · Cited by 9
Generative spaces: collaboration, learning, and innovation in a regulatory sandbox
Macrae, C., & Ansell, C. (2026) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muag024 · Cited by 11
Skepu, A., Phakathi, B., Makgoka, M., Mbita, Z., Damane, B. P., Demetriou, D., & Dlamini, Z. (2023) · Springer eBooks · doi:10.1007/978-3-031-21506-3_9 · Cited by 8
Fogleman, B. M., Goldman, M., Holland, A. B., Dyess, G. A., & Patel, A. (2024) · Cureus · doi:10.7759/cureus.58032 · Cited by 12
You, J., Hu, J., & Jiang, B. (2025) · Sustainable Cities and Society · doi:10.1016/j.scs.2025.106746 · Cited by 8
Yilmaz, R., Alsayegh, A., Bakhaidar, M., Fazlollahi, A. M., Hamdan, N. A., Tee, T., Shalmiev, A., Laroche, D., Santaguida, C., Donoho, D. A., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00032-8 · Cited by 10
Diagnostic and Prognostic Electrocardiogram-Based Models for Rapid Clinical Applications
Islam, M. S., Kalmady, S. V., Hindle, A., Sandhu, R. K., Sun, W., Sepehrvand, N., Greiner, R., & Kaul, P. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.07.003 · Cited by 9
Impact of random outliers in auto-segmented targets on radiotherapy treatment plans for glioblastoma
Poel, R., Rüfenacht, E., Ermiş, E., Müller, M., Fix, M. K., Aebersold, D. M., Manser, P., & Reyes, M. (2022) · Radiation Oncology · doi:10.1186/s13014-022-02137-9 · Cited by 7
Regulatory and Implementation Considerations for Artificial Intelligence
Chen, S., & Lobo, B. C. (2024) · Otolaryngologic Clinics of North America · doi:10.1016/j.otc.2024.04.007 · Cited by 6
Doria, A. S. (2023) · Expert Review of Hematology · doi:10.1080/17474086.2023.2192474 · Cited by 8
Wang, Y., Fu, W., Zhang, Y., Wang, D., Gu, Y., Wang, W., Xu, H. M., Ge, X., Ye, C., Fang, J., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-64893-w · Cited by 6
Openness and Transparency in the Evaluation of Bias in Artificial Intelligence
Larson, D. B. (2022) · Radiology · doi:10.1148/radiol.222263 · Cited by 6
Self-citationBuilding Purpose-Driven Radiology Departments: Distributive and Servant Leadership in Action
Kamran, R., Patlas, M. N., Cecere, M., & Doria, A. S. (2026) · Canadian Association of Radiologists Journal · doi:10.1177/08465371261437071 · Cited by 5
Controversies in Artificial Intelligence in Neurosurgery
Yilmaz, R., Browd, S. R., & Donoho, D. A. (2024) · Neurosurgery Clinics of North America · doi:10.1016/j.nec.2024.08.008 · Cited by 6
Metadata
- Authors
- Larson, D. B., Harvey, H., Rubin, D. L., Irani, N., Tse, J. R., & Langlotz, C. P.
- Affiliations
- Stanford Medicine (US)
- University College London (GB)
- University of Kansas Medical Center (US)
- University of California, Los Angeles (US)
- Corresponding author
- Larson, David B.
- Venue
- Journal of the American College of Radiology
- Publisher
- Elsevier BV
- Published
- 2021-03
- Volume
- 18
- Issue
- 3
- Pages
- 413-424
- ISSN
- 1546-1440 (ISSN-L)
- PMC
- PMC7574690
- OpenAlex
- W3093602247
- Cited by
- 198
- References
- 28
- Citations by year
- 2026: 38 · 2025: 51 · 2024: 36 · 2023: 24 · 2022: 29 · 2021: 20
- FWCI
- 1.39
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American College of Radiology (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- International Business Machines Corporation
- GE Healthcare
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Healthcare Regulation
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- artificial intelligence, diagnostic imaging algorithm, regulatory framework, software as a medical device, algorithm evaluation, performance characterization, third-party evaluation, conflicts of interest, medical device regulation
- MeSH terms
- Algorithms, Artificial Intelligence, Diagnostic Imaging, Software, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
