The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Open accessPublished version of a preprint
Abstract
Background: Artificial intelligence (AI) applications are growing at an unprecedented pace in health care, including disease diagnosis, triage or screening, risk analysis, surgical operations, and so forth. Despite a great deal of research in the development and validation of health care AI, only few applications have been actually implemented at the frontlines of clinical practice. Objective: The objective of this study was to systematically review AI applications that have been implemented in real-life clinical practice. Methods: We conducted a literature search in PubMed, Embase, Cochrane Central, and CINAHL to identify relevant articles published between January 2010 and May 2020. We also hand searched premier computer science journals and conferences as well as registered clinical trials. Studies were included if they reported AI applications that had been implemented in real-world clinical settings. Results: We identified 51 relevant studies that reported the implementation and evaluation of AI applications in clinical practice, of which 13 adopted a randomized controlled trial design and eight adopted an experimental design. The AI applications targeted various clinical tasks, such as screening or triage (n=16), disease diagnosis (n=16), risk analysis (n=14), and treatment (n=7). The most commonly addressed diseases and conditions were sepsis (n=6), breast cancer (n=5), diabetic retinopathy (n=4), and polyp and adenoma (n=4). Regarding the evaluation outcomes, we found that 26 studies examined the performance of AI applications in clinical settings, 33 studies examined the effect of AI applications on clinician outcomes, 14 studies examined the effect on patient outcomes, and one study examined the economic impact associated with AI implementation. Conclusions: This review indicates that research on the clinical implementation of AI applications is still at an early stage despite the great potential. More research needs to assess the benefits and challenges associated with clinical AI applications through a more rigorous methodology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A recent systematic review examined all studies of AI application in clinical practice, but was limited by restriction to English language and only searching full manuscripts published between January 2010 and May 2020 [13].”
From Randomized Controlled Trials of Artificial Intelligence in Clinical Practice: Systematic Review (2022) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Of these, 6 were related to diabetes.”
From An Assessment of How Clinicians and Staff Members Use a Diabetes Artificial Intelligence Prediction Tool: Mixed Methods Study (2023) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These applications used computer vision to diagnose diabetic retinopathy from retinal images and EHR data to predict those at risk for hyperglycemia.”
From An Assessment of How Clinicians and Staff Members Use a Diabetes Artificial Intelligence Prediction Tool: Mixed Methods Study (2023) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Unfortunately, the implementation of AI tools for diabetes has lagged, and few tools are used in practice, limiting their impact.”
From An Assessment of How Clinicians and Staff Members Use a Diabetes Artificial Intelligence Prediction Tool: Mixed Methods Study (2023) · cites this paper as [31]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Despite the ongoing development and testing of AI applications in healthcare, the implementation of medical AI systems in clinical care remains in its early stages [2, 3].”
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Cochrane Handbook for Systematic Reviews of Interventions
(2019) · doi:10.1002/9781119536604 · Cited by 14,828
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Cumpston, M., Li, T., Page, M. J., Chandler, J., Welch, V. A., Higgins, J. P., & Thomas, J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.ed000142 · Cited by 7,218
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Artificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817
The practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246
A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955
McCarthy, J., Minsky, M., Rochester, N., et al. (2006) · The AI Magazine · doi:10.1609/aimag.v27i4.1904 · Cited by 2,029
Research works printed but not linked: 1
- 10.1136/bjophthalmol-2019-314729 · DOI
Cited by
Petersson, L., Larsson, I., Nygren, J. M., Nilsen, P., Neher, M., Reed, J. E., Tyskbo, D., & Svedberg, P. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08215-8 · Cited by 434
Esmaeilzadeh, P. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102861 · Cited by 377
Artificial Intelligence Applications in Health Care Practice: Scoping Review
Sharma, M., Savage, C., Nair, M., Larsson, I., Svedberg, P., & Nygren, J. M. (2022) · Journal of Medical Internet Research · doi:10.2196/40238 · Cited by 198
Next-generation agentic AI for transforming healthcare
Karunanayake, N. (2025) · Informatics and Health · doi:10.1016/j.infoh.2025.03.001 · Cited by 165
Hogg, H. D. J., Al-Zubaidy, M., Technology Enhanced Macular Services Study Reference Group, Talks, J., Denniston, A. K., Kelly, C. J., Malawana, J., Papoutsi, C., Teare, M. D., Keane, P. A., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/39742 · Cited by 163
Metadata
- Authors
- Yin, J., Ngiam, K. Y., & Teo, H. H.
- Affiliations
- National University of Singapore (SG)
- National University Hospital (SG)
- Corresponding author
- Teo, Hock Hai
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2021-04-22
- Volume
- 23
- Issue
- 4
- Pages
- e25759
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC8103304
- OpenAlex
- W3134138028
- Cited by
- 419
- References
- 86
- Citations by year
- 2026: 75 · 2025: 126 · 2024: 106 · 2023: 69 · 2022: 34 · 2021: 6
- FWCI
- 4.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- National University of Singapore (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National University of Singapore
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- artificial intelligence in healthcare, clinical implementation of AI, clinical practice, disease diagnosis, risk analysis, sepsis, breast cancer, diabetic retinopathy, patient outcomes, economic impact of AI
- MeSH terms
- Artificial Intelligence, Humans, Randomized Controlled Trials as Topic, Risk Assessment, Sepsis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review. https://app.cochat.ai/papers/10.2196/25759
- The paper
- Yin, J., Ngiam, K. Y., & Teo, H. H. (2021). Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review. Journal of Medical Internet Research. https://doi.org/10.2196/25759
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.