Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review
Open accessAbstract
The evidence of the impact of traditional statistical (TS) and artificial intelligence (AI) tool interventions in clinical practice was limited. This study aimed to investigate the clinical impact and quality of randomized controlled trials (RCTs) involving interventions evaluating TS, machine learning (ML), and deep learning (DL) prediction tools. A systematic review on PubMed was conducted to identify RCTs involving TS/ML/DL tool interventions in the past decade. A total of 65 RCTs from 26,082 records were included. A majority of them had model development studies and generally good performance was achieved. The function of TS and ML tools in the RCTs mainly included assistive treatment decisions, assistive diagnosis, and risk stratification, but DL trials were only conducted for assistive diagnosis. Nearly two-fifths of the trial interventions showed no clinical benefit compared to standard care. Though DL and ML interventions achieved higher rates of positive results than TS in the RCTs, in trials with low risk of bias (17/65) the advantage of DL to TS was reduced while the advantage of ML to TS disappeared. The current applications of DL were not yet fully spread performed in medicine. It is predictable that DL will integrate more complex clinical problems than ML and TS tools in the future. Therefore, rigorous studies are required before the clinical application of these tools.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A survey of 95 randomized controlled trials of AI software found mixed results of their clinical impact.
“a survey of 95 randomized controlled trials (RCTs) of AI software found mixed results of their clinical impact4.”
From The algorithm journey map: a tangible approach to implementing AI solutions in healthcare (2024) · cites this paper as [4]
The Results (General characteristics; page not available) report 65 RCTs, not 95, and include traditional statistical prediction tools as well as AI tools. The Results (AI tool intervention and its performance; page not available) support the characterization of mixed clinical impact: 61.5% of trials had positive primary outcomes, while 38.5% showed no statistically significant benefit. (LLM, conf=0.90)
Experimental — accuracy not yet measuredImplementation of ML and AI models may not yield net positive outcomes for patients despite their methodological efficacy.
“Despite the demonstrated methodological efficacy of ML and AI models, it is imperative to acknowledge that their implementation may not necessarily yield net positive outcomes for patients.”
From Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions (2024) · cites this paper as [68]
A systematic review of 65 randomized controlled trials found that approximately 40% of AI-based prediction tools that had previously shown satisfactory performance in observational studies failed to demonstrate statistically significant clinical benefits for patients compared to standard clinical care.
“A systematic review of 65 randomized controlled trials evaluating AI-based prediction tools revealed that approximately 40% of the tools, which had previously exhibited satisfactory performance in observational model development or validation studies, failed to demonstrate statistically significant clinical benefits for patients when compared to standard clinical care protocols [68].”
From Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions (2024) · cites this paper as [68]
Even the most accurate AI model will fail if oncologists cannot use it effectively.
“Even the most accurate AI model will fail if oncologists cannot use it effectively [139,140,141].”
From Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery (2025) · cites this paper as [139]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,577 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648 · see citing works
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349 · see citing works
Prediction of coronary heart disease using risk factor categories
Wilson, P. W. F., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998) · Circulation · doi:10.1161/01.cir.97.18.1837 · Cited by 9,581 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388 · see citing works
Collins, P. G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,312 · see citing works
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936 · see citing works
Personalized nutrition by prediction of glycemic responses
Zeevi, D., Korem, T., Zmora, N., Israeli, D., Rothschild, D., Weinberger, A., Ben-Yacov, O., Lador, D., Avnit-Sagi, T., Lotan-Pompan, M., et al. (2015) · Cell · doi:10.1016/j.cell.2015.11.001 · Cited by 2,929 · see citing works
Real-World Evidence — What Is It and What Can It Tell Us?
Sherman, R. E., Anderson, S. A., Pan, G. J. D., Gray, G. W., Gross, T. P., Hunter, N. L., LaVange, L. M., Marinac‐Dabic, D., Marks, P. W., Robb, M. A., et al. (2016) · New England Journal of Medicine · doi:10.1056/nejmsb1609216 · Cited by 2,134 · see citing works
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · see citing works
Blundell, J. E., Baker, J. L., Boyland, E. J., Blaak, E. E., Charzewska, J., Henauw, S. D., Frühbeck, G., González‐Gross, M., Hebebrand, J., Holm, L., et al. (2017) · Obesity Facts · doi:10.1159/000455952 · Cited by 1,980 · see citing works
A Validated Prediction Model for All Forms of Acute Coronary Syndrome
Eagle, K. A., Lim, M. J., Dabbous, O. H., Pieper, K. S., Goldberg, R. J., de Werf, F. V., Goodman, S. G., Granger, C. B., Steg, P. G., Gore, J. M., et al. (2004) · JAMA · doi:10.1001/jama.291.22.2727 · Cited by 1,590 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421 · see citing works
Hill, J. C., Whitehurst, D. G., Lewis, M., Bryan, S., Dunn, K. M., Foster, N. E., Konstantinou, K., Main, C. J., Mason, E., Somerville, S., et al. (2011) · The Lancet · doi:10.1016/s0140-6736(11)60937-9 · Cited by 1,306 · see citing works
Prediction models need appropriate internal, internal-external, and external validation
Steyerberg, E. W., & Harrell, F. E. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.04.005 · Cited by 1,304 · see citing works
Design and analysis of stepped wedge cluster randomized trials.
Hussey, M. A., & Hughes, J. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.05.007 · Cited by 1,297 · see citing works
Risk prediction models: II. External validation, model updating, and impact assessment
Moons, K. G. M., Kengne, A. P., Grobbee, D. E., Royston, P., Vergouwe, Y., Altman, D. G., & Woodward, M. (2012) · Heart · doi:10.1136/heartjnl-2011-301247 · Cited by 1,195 · see citing works
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020) · BMJ · doi:10.1136/bmj.m689 · Cited by 1,152 · see citing works
CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials.
Schulz, K. F., Altman, D. G., & Moher, D. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.005 · Cited by 1,022 · see citing works
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934 · see citing works
Xu, X., Zhang, H.-L., Liu, Q., Sun, S., Zhang, J., Zhu, F.-P., Yang, G., Yan, X., Zhang, Y., & Liu, X.-S. (2019) · Journal of Hepatology · doi:10.1016/j.jhep.2019.02.023 · Cited by 710 · see citing works
Assessing the Gold Standard — Lessons from the History of RCTs
Bothwell, L. E., Greene, J. A., Podolsky, S. H., & Jones, D. S. (2016) · New England Journal of Medicine · doi:10.1056/nejmms1604593 · Cited by 664 · see citing works
Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial
Repici, A., Badalamenti, M., Maselli, R., Correale, L., Radaelli, F., Rondonotti, E., Ferrara, E., Spadaccini, M., Alkandari, A., Fugazza, A., et al. (2020) · Gastroenterology · doi:10.1053/j.gastro.2020.04.062 · Cited by 646 · see citing works
Validation, updating and impact of clinical prediction rules: A review
Toll, D. B., Janssen, K. J. M., Vergouwe, Y., & Moons, K. G. M. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.04.008 · Cited by 584 · see citing works
Vollmer, S. J., Mateen, B. A., Bohner, G., Király, F. J., Ghani, R., Jonsson, P. F., Cumbers, S., Jonas, A., McAllister, K., Myles, P. R., et al. (2020) · BMJ · doi:10.1136/bmj.l6927 · Cited by 554 · see citing works
Application of Artificial Intelligence to Gastroenterology and Hepatology
Berre, C. L., Sandborn, W. J., Aridhi, S., Devignes, M., Fournier, L. S., Smaïl‐Tabbone, M., Danese, S., & Peyrin‐Biroulet, L. (2019) · Gastroenterology · doi:10.1053/j.gastro.2019.08.058 · Cited by 528 · see citing works
Hopewell, S., Dutton, S., Yu, L.-M., Chan, A.-W., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c723 · Cited by 504 · see citing works
Artificial intelligence and machine learning in clinical development: a translational perspective
Shah, P. P., Kendall, F., Khozin, S., Goosen, R., Hu, J., Laramie, J. M., Ringel, M. S., & Schork, N. J. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0148-3 · Cited by 498 · see citing works
Epidemiology and reporting of randomised trials published in PubMed journals
Chan, A.-W., & Altman, D. G. (2005) · The Lancet · doi:10.1016/s0140-6736(05)71879-1 · Cited by 474 · see citing works
Wang, P., Liu, X., Berzin, T. M., Glissen Brown, J. R., Liu, P., Zhou, C., Lei, L., Li, L., Guo, Z., Lei, S., et al. (2020) · The Lancet Gastroenterology & Hepatology · doi:10.1016/s2468-1253(19)30411-x · Cited by 415 · see citing works
Geersing, G.-J., Bouwmeester, W., Zuithoff, P., Spijker, R., Leeflang, M., & Moons, K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0032844 · Cited by 350 · see citing works
Collins, P. G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.010 · Cited by 333 · see citing works
Schulz, M.-A., Yeo, B. T. T., Vogelstein, J. T., Mourao-Miranada, J., Kather, J. N., Kording, K., Richards, B., & Bzdok, D. (2020) · Nature Communications · doi:10.1038/s41467-020-18037-z · Cited by 315 · see citing works
Stafford, I. S., Kellermann, M., Mossotto, E., Beattie, R. M., MacArthur, B. D., & Ennis, S. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0229-3 · Cited by 275 · see citing works
Applications of Machine Learning in Real-Life Digital Health Interventions: Review of the Literature
Triantafyllidis, A. K., & Tsanas, A. (2019) · Journal of Medical Internet Research · doi:10.2196/12286 · Cited by 243 · see citing works
Sollini, M., Antunovic, L., Chiti, A., & Kirienko, M. (2019) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-019-04372-x · Cited by 227 · see citing works
Shung, D. L., Au, B., Taylor, R. A., Tay, J. K., Laursen, S. B., Stanley, A. J., Dalton, H. R., Ngu, J., Schultz, M., & Laine, L. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.09.009 · Cited by 227 · see citing works
Stockman, J. A. (2006) · Yearbook of Pediatrics · doi:10.1016/s0084-3954(07)70181-1 · Cited by 225 · see citing works
Abrol, A., Fu, Z., Salman, M. S., Silva, R. F., Du, Y., Plis, S., & Calhoun, V. D. (2021) · Nature Communications · doi:10.1038/s41467-020-20655-6 · Cited by 221 · see citing works
Evaluating the impact of prediction models: lessons learned, challenges, and recommendations
Kappen, T. H., van Klei, W. A., van Wolfswinkel, L., Kalkman, C. J., Vergouwe, Y., & Moons, K. G. M. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0033-6 · Cited by 215 · see citing works
Brocklehurst, P., Field, D., Greene, K., Juszczak, E., Keith, R., Kenyon, S., Linsell, L., Mabey, C., Newburn, M., Plachcinski, R., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30568-8 · Cited by 207 · see citing works
Shin, S., Austin, P. C., Ross, H. J., Abdel-Qadir, H. M., Freitas, C., TOMLINSON, G. A., Chicco, D., Mahendiran, M., Lawler, P. R., Billia, F., et al. (2020) · ESC Heart Failure · doi:10.1002/ehf2.13073 · Cited by 184 · see citing works
Manz, C. R., Parikh, R. B., Small, D. S., Evans, C. N., Chivers, C., Regli, S. H., Hanson, C. W., Bekelman, J. E., Rareshide, C., O'Connor, N. R., et al. (2020) · JAMA Oncology · doi:10.1001/jamaoncol.2020.4759 · Cited by 146 · see citing works
Blomberg, S. N. F., Christensen, H. C., Lippert, F. K., Ersbøll, A. K., Torp-Petersen, C., Sayre, M. R., Kudenchuk, P. J., & Folke, F. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.32320 · Cited by 144 · see citing works
Reporting guidelines for clinical trials evaluating artificial intelligence interventions are needed
The CONSORT-AI and SPIRIT-AI Steering Group (2019) · Nature Medicine · doi:10.1038/s41591-019-0603-3 · Cited by 143 · see citing works
Benedetto, U., Dimagli, A., Sinha, S., Cocomello, L., Gibbison, B., Caputo, M., Gaunt, T. R., Lyon, M., Holmes, C., & Angelini, G. D. (2020) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2020.07.105 · Cited by 142 · see citing works
A conceptual framework for prognostic research
Kent, P., Cancelliere, C., Boyle, E., Cassidy, J., & Kongsted, A. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01050-7 · Cited by 138 · see citing works
Caballero-Ruiz, E., García-Sáez, G., Rigla, M., Villaplana, M., Pons, B., & Hernando, M. E. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.02.014 · Cited by 136 · see citing works
Machine learning in GI endoscopy: practical guidance in how to interpret a novel field
van der Sommen, F., de Groof, J., Struyvenberg, M. R., van der Putten, J. A., Boers, T. G. W., Fockens, K. N., Schoon, E. J., Curvers, W. L., de With, P. H. N., Mori, Y., et al. (2020) · Gut · doi:10.1136/gutjnl-2019-320466 · Cited by 134 · see citing works
Bailey, T. C., Chen, Y., Mao, Y., Lu, C., Hackmann, G., Micek, S. T., Heard, K. M., Faulkner, K. M., & Kollef, M. H. (2013) · Journal of Hospital Medicine · doi:10.1002/jhm.2009 · Cited by 121 · see citing works
Prediction Models — Development, Evaluation, and Clinical Application
Pencina, M. J., Goldstein, B. A., & D’Agostino, R. B. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2000589 · Cited by 116 · see citing works
Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs)
(Members of the International Diagnostic and Prognosis Prediction (IDAPP) group), Wallace, E., Smith, S. M., Perera, R., Vaucher, P., McCowan, C., Collins, P. G. S., Verbakel, J. Y. J., Lakhanpaul, M., & Fahey, T. P. (2011) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-11-62 · Cited by 113 · see citing works
Cresswell, K., Callaghan, M., Khan, S., Sheikh, Z., Mozaffar, H., & Sheikh, A. (2020) · Health Informatics Journal · doi:10.1177/1460458219900452 · Cited by 87 · see citing works
Smith, A. M., Walsh, J. R., Long, J., Davis, C. B., Henstock, P. V., Hodge, M. R., Maciejewski, M., Mu, X. J., Ra, S., Zhao, S., et al. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-3427-8 · Cited by 78 · see citing works
Global Trend in Artificial Intelligence–Based Publications in Radiology From 2000 to 2018
West, E., Mutasa, S., Zhu, Z., & Ha, R. S. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.19.21346 · Cited by 77 · see citing works
Aziz, M., Fatima, R., Dong, C., Lee‐Smith, W., & Nawras, A. T. (2020) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15070 · Cited by 71 · see citing works
Sadasivam, R. S., Borglund, E. M., Adams, R., Marlin, B. M., & Houston, T. K. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6465 · Cited by 60 · see citing works
Randomized Trial of Model Predictive Control for Improved Anemia Management
Brier, M. E., Gawęda, A. E., Dailey, A., Aronoff, G. R., & Jacobs, A. A. (2010) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.07181009 · Cited by 58 · see citing works
Clemons, M. J., Bouganim, N., Smith, S. R., et al. (2015) · JAMA Oncology · doi:10.1001/jamaoncol.2015.3730 · Cited by 49 · see citing works
Finkelstein, S. M., Lindgren, B. R., Robiner, W. N., et al. (2013) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2013.0049 · Cited by 37 · see citing works
Gosselt, H. R., Verhoeven, M. M. A., Bulatović-Ćalasan, M., Welsing, P. M. J., de Rotte, M. C. F. J., Hazes, J. M. W., Lafeber, F. P. J. G., Hoogendoorn, M., & de Jonge, R. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11010044 · Cited by 37 · see citing works
Cowling, T. E., Cromwell, D. A., Bellot, A., Sharples, L. D., & van der Meulen, J. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.12.018 · Cited by 36 · see citing works
Geersing, G., Hendriksen, J. M. T., Zuithoff, N. P. A., Roes, K. C. B., Oudega, R., Takada, T., Schutgens, R. E. G., & Moons, K. G. M. (2020) · PLoS Medicine · doi:10.1371/journal.pmed.1003142 · Cited by 23 · see citing works
Steinhart, B. D., Levy, P. D., Vandenberghe, H. M., et al. (2016) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2016.08.007 · Cited by 10 · see citing works
Guénancia, C., Stamboul, K., Hachet, O., et al. (2015) · Heart and Vessels · doi:10.1007/s00380-015-0695-8 · Cited by 7 · see citing works
Zhou, Q., Cao, Y.-H., & Chen, Z. (2019) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-019-04493-3 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957 · see citing works
“59”
Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review
Plana, D., Shung, D. L., Grimshaw, A. A., Saraf, A., Sung, J. J. Y., & Kann, B. H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.33946 · Cited by 236 · see citing works
Methods for clinical evaluation of artificial intelligence algorithms for medical diagnosis
Park, S. H., Han, K., Jang, H. Y., Park, J. E., Lee, J.-G., Kim, D. W., & Choi, J. (2023) · Radiology · doi:10.1148/radiol.220182 · Cited by 163 · see citing works
London, A. J. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100622 · Cited by 76 · see citing works
Chen, S., Yu, J., Chamouni, S., Wang, Y., & Li, Y. (2024) · BMC Medicine · doi:10.1186/s12916-024-03566-x · Cited by 73 · see citing works
“…revealed that approximately 40% of the tools, which had previously exhibited satisfactory performance in observational model development or validation studies, failed to demonstrate statistically significant clinical benefits for patients when compared to standard clinical care protocols [68].”
Marwaha, J. S., & Kvedar, J. C. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00572-2 · Cited by 46 · see citing works
Khan, S. D., Hoodbhoy, Z., Raja, M. H. R., Kim, J. Y., Hogg, H. D. J., Manji, A. A. A., Gulamali, F., Hasan, A., Shaikh, A., Tajuddin, S., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000514 · Cited by 36 · see citing works
The algorithm journey map: a tangible approach to implementing AI solutions in healthcare
Boag, W., Hasan, A., Kim, J. Y., Revoir, M., Nichols, M., Ratliff, W., Gao, M., Zilberstein, S., Samad, Z., Hoodbhoy, Z., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01061-4 · Cited by 19 · see citing works
Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery
Kuno, M., Osumi, H., Udagawa, S., Yoshikawa, K., Ooki, A., Shinozaki, E., Ishikawa, T., Oba, J., Yamaguchi, K., & Sakurada, K. (2025) · Current Oncology · doi:10.3390/curroncol32110588 · Cited by 7 · see citing works
A quantitative analysis of global AI medical studies: gaps in randomized controlled trials
Wang, Q., Yang, Y., Qiu, L., Xu, Y., Sun, R., Xue, P., & Qiao, Y. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02698-z · Cited by 4 · see citing works
- Authors
- Zhou, Q., Chen, Z.-H., Cao, Y.-H., & Peng, S.
- Affiliations
- The First Affiliated Hospital, Sun Yat-sen University (CN)
- Sun Yat-sen University (CN)
- Corresponding author
- Zhou, Qian
- Venue
- npj Digital Medicine
- Published
- 2021-10-28
- Volume
- 4
- Issue
- 1
- Pages
- 154-154
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC8553754
- OpenAlex
- W3211034094
- Cited by
- 126
- References
- 71
- Citations by year
- 2026: 19 · 2025: 31 · 2024: 28 · 2023: 19 · 2022: 29
- FWCI
- 1.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Guangdong Medical Research Foundation
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- machine learning, deep learning, clinical impact, trial quality, risk of bias, risk stratification, treatment decision support, clinical implementation of AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
