We need more randomized clinical trials of AI.
Abstract
AbstractIn the first prospective clinical trial of artificial intelligence (AI) assistance in stress echocardiography, there was no difference in diagnostic accuracy between AI assistance and standard-of-care assessment. There is significant value in conducting prospective clinical trials of AI, and there are lessons on implementation to be learned from this study.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This trial adds to the limited evidence evaluating AI interventions against established standards in medicine.
“To the authors’ knowledge, this is the first RCT to compare a fully automated DPP to standard care and adds to the limited evidence evaluating AI interventions against established standards in medicine.38,39”
From An AI-Powered Lifestyle Intervention vs Human Coaching in the Diabetes Prevention Program: A Randomized Clinical Trial (2025) · cites this paper as [39]
This is the first RCT to compare a fully automated DPP to standard care.
“To the authors’ knowledge, this is the first RCT to compare a fully automated DPP to standard care and adds to the limited evidence evaluating AI interventions against established standards in medicine.38,39”
From An AI-Powered Lifestyle Intervention vs Human Coaching in the Diabetes Prevention Program: A Randomized Clinical Trial (2025) · cites this paper as [39]
Cites
Wu, E., Wu, K., Daneshjou, R., Ouyang, D., Ho, D. E., & Zou, J. (2021) · Nature Medicine · doi:10.1038/s41591-021-01312-x · Cited by 494 · see citing works
Self-citationROC and AUC with a Binary Predictor: a Potentially Misleading Metric
Muschelli, J. (2019) · Journal of Classification · doi:10.1007/s00357-019-09345-1 · Cited by 349 · see citing works
Upton, R., Mumith, A., Beqiri, A., Parker, A., Hawkes, W., Gao, S., Porumb, M., Sarwar, R., Marques, P., Markham, D. H., et al. (2021) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2021.10.013 · Cited by 133 · see citing works
Woodward, G., Bajre, M., Bhattacharyya, S., Breen, M. A., Chiocchia, V., Dawes, H., Dehbi, H., Descamps, T., Frangou, E., Fazakarley, C.-A., et al. (2023) · American Heart Journal · doi:10.1016/j.ahj.2023.05.003 · Cited by 27 · see citing works
PROTEUS: A Prospective RCT Evaluating Use of AI in Stress Echocardiography
Upton, R., Akerman, A. P., Marwick, T. H., Johnson, C. L., Piotrowska, H., Bajre, M., Breen, M. A., Dawes, H., Dehbi, H., Descamps, T., et al. (2024) · NEJM AI · doi:10.1056/aioa2400865 · Cited by 23 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Generalizability of FDA-Approved AI-Enabled Medical Devices for Clinical Use
Windecker, D., Baj, G., Shiri, I., Kazaj, P. M., Kaesmacher, J., Gräni, C., & Siontis, G. C. M. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.8052 · Cited by 123 · see citing works
Berkhout, W. E. M., van Wijngaarden, J. J., Workum, J. D., van de Sande, D., Hilling, D. E., Jung, C., Meyfroidt, G., Gommers, D., Buijsman, S. N. R., & van Genderen, M. E. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.22866 · Cited by 75 · see citing works
Rethinking clinical trials for medical AI with dynamic deployments of adaptive systems
Rosenthal, J. T., Beecy, A., & Sabuncu, M. R. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01674-3 · Cited by 59 · see citing works
Mathioudakis, N., Lalani, B., Abusamaan, M. S., Alderfer, M., Alver, D., Dobs, A., Kane, B., McGready, J., Riekert, K., Ringham, B., et al. (2025) · JAMA · doi:10.1001/jama.2025.19563 · Cited by 41 · see citing works
Evaluating interactions of patients with large language models for medical information
Carl, N., Haggenmüller, S., Wies, C., Nguyen, L., Winterstein, J. T., Hetz, M. J., Mangold, M. H., Hartung, F. O., Grüne, B., Holland‐Letz, T., et al. (2025) · British Journal of Urology · doi:10.1111/bju.16676 · Cited by 22 · see citing works
AI-Based EMG Reporting: A Randomized Controlled Trial
Gorenshtein, A., Weisblat, Y., Khateb, M., Kenan, G., Tsirkin, I., Fayn, G., Geller, S., & Shelly, S. (2025) · Journal of Neurology · doi:10.1007/s00415-025-13261-3 · Cited by 22 · see citing works
Randomized Controlled Trials Evaluating Artificial Intelligence in Cardiovascular Care
Barzilai, D. H., Sudri, K., Goshen, G., Klang, E., Zimlichman, E., Barbash, I. M., & Cohen, M. S. (2025) · JACC Advances · doi:10.1016/j.jacadv.2025.102152 · Cited by 20 · see citing works
Reimbursement in the age of generalist radiology artificial intelligence
Dogra, S., Silva, E., & Rajpurkar, P. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01352-w · Cited by 19 · see citing works
Segun-Omosehin, O., Omiye, J. A., Elalfy, A., Sheriff, S. M., Kuti, F., Omole, O., Ogunyankin, K. O., Idemili-Aronu, N., Olawepo, J. O., Ezeanolue, E. E., et al. (2025) · npj Cardiovascular Health · doi:10.1038/s44325-025-00055-9 · Cited by 12 · see citing works
Integrating statistical design and inference: A roadmap for robust and trustworthy medical AI
Wei, Q., Cui, M., Liu, Z., Liu, Z., Feng, G., Li, Y., Christiani, D. C., Li, L., Wang, J., Hao, Y., et al. (2025) · The Innovation Medicine · doi:10.59717/j.xinn-med.2025.100145 · Cited by 6 · see citing works
Artificial intelligence for sepsis management in the emergency department
de Vries, F. M. C., Wiersinga, W. J., & Schinkel, M. (2026) · BMC Infectious Diseases · doi:10.1186/s12879-026-12780-2 · Cited by 5 · see citing works
Pantilimonescu, T. F., Dămian, C., Radu, V. D., Hogea, M., Costachescu, O. A., Onofrei, P., Toma, B., Zelinschi, D. O., Roca, I. C., Ursu, R. G., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14144942 · Cited by 5 · see citing works
Verma, H., Mistry, S., Jayam, K. V., Shrestha, P., Adkins, L., Liang, M., Fares, A. F., Zarrinpar, A., Braithwaite, D., & Karanth, S. D. (2025) · Cancers · doi:10.3390/cancers17213537 · Cited by 4 · see citing works
Petch, J., Bortesi, J. P. T., Sheth, T., Natarajan, M. K., Pinilla‐Echeverri, N., Di, S., Bangdiwala, S. I., Mosleh, K., Ibrahim, O., Bainey, K. R., et al. (2025) · JMIR Research Protocols · doi:10.2196/71726 · Cited by 3 · see citing works
From general to specific: Tailoring large language models for real‐world medical communications
Sun, X., Tang, W., Huang, Z., Long, E., & Wan, P. (2024) · Clinical and Translational Medicine · doi:10.1002/ctm2.70157 · Cited by 2 · see citing works
Siddiqui, N. S., Bouchi, Y., Shah, S. J. H., AlQarni, S., Sood, S., Lee, Y., Park, J. S., & Kang, J. (2025) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00067
Augmented intelligence should be good for medicine, if medicine is to remain good for us
Idan, D., Celi, L. A., Einav, S., & Frenkel, A. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00256-2
van de Sande, D., Grünhagen, D. J., Hilling, D. E., van der Ster, B. J. P., Smit, J. M., van Bommel, J., Verhoef, C. K., Gommers, D., & van Genderen, M. E. (2025) · NEJM AI · doi:10.1056/aics2401132
Sahashi, Y., & Ouyang, D. (2026) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2025.10.081
Self-citationResponse to Spillover Effects in Randomized Evaluations of Translational AI
Leuchter, R. K., Turner, W. B., & Ouyang, D. (2026) · NEJM AI · doi:10.1056/aip2600014
Self-citationNakhare, S., Kelshiker, M. A., Young, H., Re, F., Chhatwal, K., Jaffar-Karballai, M., Petri, C. F., Velmurugan, R., Schmidt, A., Peters, N. S., et al. (2026) · Expert Review of Cardiovascular Therapy · doi:10.1080/14779072.2026.2675469
Zaretsky, J., Kim, C., Major, V. J., Verplanke, B., Sonne, C., Small, W. R., Solanki, P., Fenelon, L., Tursunova, N., Gutjahr, A., et al. (2026) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2026.100395
Chin, A. T., Zhu, N., Vangala, S., Woo, H. E., Wisk, L. E., Kingsley, T., Mafi, J. N., & Lukac, P. J. (2026) · medRxiv · doi:10.64898/2026.08.26.26361496
Zong, H., Wang, J., Cha, J., Wu, R., Hu, S., Zhao, Y., & Shen, B. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001773
- Authors
- Ouyang, D., & Hogan, J.
- Affiliations
- Cedars-Sinai Medical Center (US)
- Brown University (US)
- Venue
- NEJM AI
- Publisher
- Massachusetts Medical Society
- Published
- 2024-10-24
- Volume
- 1
- Issue
- 11
- Pages
- AIe2400881-AIe2400881
- ISSN
- 2836-9386 (ISSN-L)
- OpenAlex
- W4403518164
- Cited by
- 24
- References
- 5
- Citations by year
- 2026: 7 · 2025: 15 · 2024: 2
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- NEJM AI (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- artificial intelligence, randomized controlled trials, AI evaluation, evidence generation, clinical research methodology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
