CoChat

Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review

Open access
Yin, Jiamin, Ngiam, Kee Yuan, Teo, Hock Hai · 2021 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.

Abstract

Background: Artificial intelligence (AI) applications are growing at an unprecedented pace in health care, including disease diagnosis, triage or screening, risk analysis, surgical operations, and so forth. Despite a great deal of research in the development and validation of health care AI, only few applications have been actually implemented at the frontlines of clinical practice. Objective: The objective of this study was to systematically review AI applications that have been implemented in real-life clinical practice. Methods: We conducted a literature search in PubMed, Embase, Cochrane Central, and CINAHL to identify relevant articles published between January 2010 and May 2020. We also hand searched premier computer science journals and conferences as well as registered clinical trials. Studies were included if they reported AI applications that had been implemented in real-world clinical settings. Results: We identified 51 relevant studies that reported the implementation and evaluation of AI applications in clinical practice, of which 13 adopted a randomized controlled trial design and eight adopted an experimental design. The AI applications targeted various clinical tasks, such as screening or triage (n=16), disease diagnosis (n=16), risk analysis (n=14), and treatment (n=7). The most commonly addressed diseases and conditions were sepsis (n=6), breast cancer (n=5), diabetic retinopathy (n=4), and polyp and adenoma (n=4). Regarding the evaluation outcomes, we found that 26 studies examined the performance of AI applications in clinical settings, 33 studies examined the effect of AI applications on clinician outcomes, 14 studies examined the effect on patient outcomes, and one study examined the economic impact associated with AI implementation. Conclusions: This review indicates that research on the clinical implementation of AI applications is still at an early stage despite the great potential. More research needs to assess the benefits and challenges associated with clinical AI applications through a more rigorous methodology.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/25759

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Flow diagram of the literature search based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.
Figure 1 Flow diagram of the literature search based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.CC BY · PubMed Central
Figure 2 Distribution of the included articles from 2010 to 2020.
Figure 2 Distribution of the included articles from 2010 to 2020.CC BY · PubMed Central
Figure 3 Country distribution of the involved hospitals.
Figure 3 Country distribution of the involved hospitals.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 86 references

Research works printed but not linked: 1

  1. 10.1136/bjophthalmol-2019-314729 · DOI

Cited by

Metadata

Authors
Yin, J., Ngiam, K. Y., & Teo, H. H.
Affiliations
  • National University of Singapore (SG)
  • National University Hospital (SG)
Corresponding author
Teo, Hock Hai
Venue
Journal of Medical Internet Research
Published
2021-04-22
Volume
23
Issue
4
Pages
e25759
ISSN
1438-8871 (ISSN-L)
OpenAlex
W3134138028
Cited by
419
References
86
Citations by year
2026: 75 · 2025: 126 · 2024: 106 · 2023: 69 · 2022: 34 · 2021: 6
FWCI
4.50
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • National University of Singapore (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National University of Singapore
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
artificial intelligence in healthcare, clinical implementation of AI, clinical practice, disease diagnosis, risk analysis, sepsis, breast cancer, diabetic retinopathy, patient outcomes, economic impact of AI
MeSH terms
Artificial Intelligence, Humans, Randomized Controlled Trials as Topic, Risk Assessment, Sepsis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.