Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index; 6 of the 16 sources its claims cite were checked for support.

Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review

Open access
Han, Ryan, Acosta, Julián N, Shakeri, Zahra, Ioannidis, John P A, Topol, Eric J, Rajpurkar, Pranav · 2024 · The Lancet Digital Health

Abstract

This scoping review of randomised controlled trials on artificial intelligence (AI) in clinical practice reveals an expanding interest in AI across clinical specialties and locations. The USA and China are leading in the number of trials, with a focus on deep learning systems for medical imaging, particularly in gastroenterology and radiology. A majority of trials (70 [81%] of 86) report positive primary endpoints, primarily related to diagnostic yield or performance; however, the predominance of single-centre trials, little demographic reporting, and varying reports of operational efficiency raise concerns about the generalisability and practicality of these results. Despite the promising outcomes, considering the likelihood of publication bias and the need for more comprehensive research including multicentre trials, diverse outcome measures, and improved reporting standards is crucial. Future AI trials should prioritise patient-relevant outcomes to fully understand AI's true effects and limitations in health care.

Overview

CoChat Summary

TL;DR

This scoping review identified 86 randomised controlled trials of AI integrated into clinical practice published up to Nov 14, 2023, with most trials concentrated in gastroenterology and radiology and in the USA and China. Although 70 (81%) of 86 trials reported a favourable primary endpoint, the authors caution that single-centre designs, limited demographic reporting, variable operational-efficiency results, and likely publication bias limit generalisability.

Key findings

  • Of 86 included RCTs, 70 (81%) reported a favourable result for their primary endpoint; among 81 trials designed to show improvement, 65 (80%) reported significant improvement, while 3 of 5 non-inferiority trials established non-inferiority.
  • Diagnostic yield or performance was the most common primary endpoint category (46 [54%] of 86), followed by care management (18 [21%]), patient behaviour and symptoms (15 [17%]), and clinical decision making (7 [8%]).
  • Trials were predominantly single-centre (54 [63%] of 86) and single-country (79 [92%] of 86), with the USA conducting 27 (31%) and China 26 (30%); median final-analysis sample size was 359 patients (IQR 150–1050).
  • Among 52 trials reporting operational time, 18 (35%) found a significant decrease, 13 (25%) a significant increase, and 21 (40%) no significant change.

Methods

  • Scoping review following PRISMA-ScR, registered with PROSPERO (CRD42022326955), searching PubMed, SCOPUS, CENTRAL, and the International Clinical Trials Registry Platform for studies published Jan 1, 2018–Nov 14, 2023.
  • Inclusion required RCTs with a substantial non-linear AI component integrated into clinical practice and full-text publication in a peer-reviewed English-language journal; linear risk scores, secondary studies, abstracts, and non-integrated interventions were excluded.
  • Screening and extraction were done by independent investigators, with discrepancies resolved by a third reviewer; due to heterogeneity, no formal meta-analysis was conducted and simple descriptive statistics were presented.

Limitations

  • The paper states two limitations: the search was English-only, potentially excluding relevant non-English trials, and the review does not address updated trends in trial risk of bias.
  • Evident from the text: the authors emphasise likely publication bias, noting that a 2023 systematic review identified 627 AI-enabled technology trials registered on ClinicalTrials.gov but only nine (1%) readily identified as published.
  • Evident from the text: the predominance of single-centre trials, limited race or ethnicity reporting (22 trials, of which 18 were from the USA), and heterogeneous operational-time results raise concerns about generalisability and practicality.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/s2589-7500(24)00047-5

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1: Study selection
Figure 1: Study selectionCC BY-NC-ND · PubMed Central
Figure 2: Randomised controlled trials of artificial intelligence in clinical practice across countries and specialties Norway, France, Sweden, Denmark, Germany, and Slovenia each comprise 2% of the distribution. Canada, Belgium, Ireland, Greece, Latvia, Switzerland, Romania, Bangladesh, Rwanda, Malawi, Viet Nam, and Singapore each comprise 1% of the distribution.
Figure 2: Randomised controlled trials of artificial intelligence in clinical practice across countries and specialties Norway, France, Sweden, Denmark, Germany, and Slovenia each comprise 2% of the distribution. Canada, Belgium, Ireland, Greece, Latvia, Switzerland, Romania, Bangladesh, Rwanda, Malawi, Viet Nam, and Singapore each comprise 1% of the distribution.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

5 Valid · 0 Not valid · 4 Not checked · 39 Own results and interpretations

21 more claims

Create a free account to see all 48 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 14
  • Probably not a substantial claim. 10

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 51 references

Cited by

See all 336

Metadata

Authors
Han, R., Acosta, J. N., Shakeri, Z., Ioannidis, J. P. A., Topol, E. J., & Rajpurkar, P.
Affiliations
  • Harvard University (US)
  • Yale University (US)
  • University of Toronto (CA)
  • Stanford Medicine (US)
  • Scripps Research Institute (US)
Corresponding author
Topol, Eric J
Venue
The Lancet Digital Health
Publisher
Elsevier BV
Published
2024-05
Volume
6
Issue
5
Pages
e367-e373
ISSN
2589-7500 (ISSN-L)
OpenAlex
W4395067202
Cited by
336
References
51
Citations by year
2026: 152 · 2025: 151 · 2024: 33
FWCI
7.66
Type
journal article
Language
en
Open access
Licence
Locations
  • The Lancet Digital Health (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion, cc-by-nc-nd)
Funders
  • NIH
  • NIGMS
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
clinical implementation of AI, randomized controlled trials, deep learning, diagnostic imaging, diagnostic performance, gastroenterology, radiology, generalizability, publication bias, reporting guidelines, patient-relevant outcomes
MeSH terms
Deep Learning, Artificial Intelligence, Humans, Randomized Controlled Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review. https://app.cochat.ai/papers/10.1016/s2589-7500(24)00047-5
The paper
Han, R., Acosta, J. N., Shakeri, Z., Ioannidis, J. P. A., Topol, E. J., & Rajpurkar, P. (2024). Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review. The Lancet Digital Health. https://doi.org/10.1016/s2589-7500(24)00047-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.