Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 158 of the 158 references it cites were found in a registry or an index; 12 of the 40 sources its claims cite were checked for support.

Current applications and challenges in large language models for patient care: a systematic review

Open access

Published version of a preprint

Abstract

Background: The introduction of large language models (LLMs) into clinical practice promises to improve patient education and empowerment, thereby personalizing medical care and broadening access to medical knowledge. Despite the popularity of LLMs, there is a significant gap in systematized information on their use in patient care. Therefore, this systematic review aims to synthesize current applications and limitations of LLMs in patient care. Methods: We systematically searched 5 databases for qualitative, quantitative, and mixed methods articles on LLMs in patient care published between 2022 and 2023. From 4349 initial records, 89 studies across 29 medical specialties were included. Quality assessment was performed using the Mixed Methods Appraisal Tool 2018. A data-driven convergent synthesis approach was applied for thematic syntheses of LLM applications and limitations using free line-by-line coding in Dedoose. Results: We show that most studies investigate Generative Pre-trained Transformers (GPT)-3.5 (53.2%, n = 66 of 124 different LLMs examined) and GPT-4 (26.6%, n = 33/124) in answering medical questions, followed by patient information generation, including medical text summarization or translation, and clinical documentation. Our analysis delineates two primary domains of LLM limitations: design and output. Design limitations include 6 second-order and 12 third-order codes, such as lack of medical domain optimization, data transparency, and accessibility issues, while output limitations include 9 second-order and 32 third-order codes, for example, non-reproducibility, non-comprehensiveness, incorrectness, unsafety, and bias. Conclusions: This review systematically maps LLM applications and limitations in patient care, providing a foundational framework and taxonomy for their implementation and evaluation in healthcare settings.

Overview

CoChat Summary

TL;DR

This systematic review synthesized 89 studies across 29 medical specialties on large language models (LLMs) in patient care published 2022–2023, finding most examined GPT-3.5/GPT-4 for medical question answering and patient information tasks. It maps limitations into design and output domains, with non-comprehensiveness and incorrectness each reported in 87.6% of studies.

Key findings

  • Most studies investigated GPT-3.5 (53.2%, n = 66/124 LLMs examined) and GPT-4 (26.6%, n = 33/124); 94.4% (n = 84/89) examined LLMs as medical chatbots, and 20.2% (n = 18/89) focused on patient information generation.
  • Thematic synthesis identified design limitations (6 second-order and 12 third-order codes) and output limitations (9 second-order and 32 third-order codes).
  • Common output limitations included non-comprehensiveness (87.6%, n = 78/89), incorrectness (87.6%, n = 78/89), unsafeness (43.8%, n = 39/89), non-reproducibility (42.7%, n = 38/89), and bias (6.7%, n = 6/89).
  • Incorrectness was classified as hallucination (42.7%, n = 38/89), delirium (38.2%, n = 34/89), confabulation (20.2%, n = 18/89), delusion (15.7%, n = 14/89), illusion (13.5%, n = 12/89), and extrapolation (12.4%, n = 11/89).

Methods

  • Systematic review pre-registered in PROSPERO (CRD42024504542) and conducted per PRISMA; searched Web of Science, PubMed, Embase/Embase Classic, ACM Digital Library, and IEEE Xplore as of January 25, 2024, for qualitative, quantitative, and mixed methods articles published January 1, 2022–December 31, 2023. From 4349 records, 89 studies across 29 medical specialties were included.
  • Quality assessment used the Mixed Methods Appraisal Tool 2018; a data-driven convergent synthesis approach was applied with free line-by-line coding in Dedoose. No meta-analysis was conducted due to diversity of outcomes and study designs.
  • Most included studies were quantitative non-randomized (94.4%, n = 84/89), 4 were qualitative (4.5%), 1 was quantitative randomized (1.1%), and zero mixed methods studies were included.

Limitations

  • Stated: the review focused on patient care and excluded research directed at clinicians only; no meta-analysis of LLM performance was possible; eligible studies may have been missed if not in the 5 databases or not available in English; the January 2022 cut-off risks missing predecessor LLMs such as GPT-3; rapid LLM development may make the review and taxonomy outdated.
  • Stated: patient acceptance was not assessed in any of the included studies, and whether ongoing human oversight and validation is feasible remains subject to further research.
  • Evident from the text: most studies were quantitative non-randomized (94.4%) and zero mixed methods studies were included due to classification; conflicts of interest/funding were not explicitly stated in 25.8% (n = 23/89) of studies; IRB approval was not reported in 61.8% (n = 55/89) and deemed unnecessary in 31.5% (n = 28/89).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s43856-024-00717-2

Cites

158 of 158 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Preferred reporting items for systematic reviews and meta-analyzes (PRISMA) flow diagram. A total of 4349 reports were identified from Web of Science, PubMed, Embase/Embase Classic, ACM Digital Library, and IEEE Xplore. After excluding 1358 duplicates, 2991 underwent initial screening and 126 were deemed suitable for potential inclusion and underwent full-text screening. Two articles could not be retrieved because the authors or the corresponding title and abstract could not be identified online. After full text screening, 35 articles were excluded and 89 articles were included in the final review.
Fig. 1 Preferred reporting items for systematic reviews and meta-analyzes (PRISMA) flow diagram. A total of 4349 reports were identified from Web of Science, PubMed, Embase/Embase Classic, ACM Digital Library, and IEEE Xplore. After excluding 1358 duplicates, 2991 underwent initial screening and 126 were deemed suitable for potential inclusion and underwent full-text screening. Two articles could not be retrieved because the authors or the corresponding title and abstract could not be identified online. After full text screening, 35 articles were excluded and 89 articles were included in the final review.CC BY · PubMed Central
Fig. 2 Schematic illustration of the identified disciplines, languages, and clinical concepts of large language models (LLMs) applications in patient care. A Column plot showing the distribution of medical specialties in which LLMs have been tested for patient care. B Pie chart illustrating the distribution of languages in which LLMs have been tested. C Schematic representation of the concepts identified for the application of LLMs in patient care.
Fig. 2 Schematic illustration of the identified disciplines, languages, and clinical concepts of large language models (LLMs) applications in patient care. A Column plot showing the distribution of medical specialties in which LLMs have been tested for patient care. B Pie chart illustrating the distribution of languages in which LLMs have been tested. C Schematic representation of the concepts identified for the application of LLMs in patient care.CC BY · PubMed Central
Fig. 3 Illustration of the hierarchical tree structure for the thematic synthesis of large language model (LLM) limitations in patient care, including the presence of codes for each concept. The font size of each concept is shown in proportion to its frequency in the studies analyzed. Our analysis delineates two primary domains of LLM limitations: design and output. Design limitations included 6 second-order and 12 third-order codes, while output limitations included 9 second-order and 32 third-order codes.
Fig. 3 Illustration of the hierarchical tree structure for the thematic synthesis of large language model (LLM) limitations in patient care, including the presence of codes for each concept. The font size of each concept is shown in proportion to its frequency in the studies analyzed. Our analysis delineates two primary domains of LLM limitations: design and output. Design limitations included 6 second-order and 12 third-order codes, while output limitations included 9 second-order and 32 third-order codes.CC BY · PubMed Central

Verification

Claims the paper makes

11 Valid · 1 Not valid · 20 Not checked · 106 Own results and interpretations

67 more claims · 4 with their cited sources checked

Create a free account to see all 138 claims and to verify the 28 references still open.

See all claims — free account
  • Probably not a substantial claim. 40
  • It may state two separate claims as one. 38
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

1 retracted

Show all 158 references

Cited by

See all 358

Metadata

Authors
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., Truhn, D., Cuocolo, R., Adams, L. C., & Bressem, K. K.
Affiliations
  • TUM Klinikum (DE)
  • Humboldt-Universität zu Berlin (DE)
  • Leiden University Medical Center (NL)
  • University College London (GB)
  • Azienda Ospedaliero-Universitaria Cagliari (IT)
  • Deutsches Herzzentrum München (DE)
  • Heidelberg University (DE)
  • Universitätsklinikum Aachen (DE)
  • University of Salerno (IT)
Corresponding author
Busch, Felix
Venue
Communications Medicine
Published
2025-01-21
Volume
5
Issue
1
Pages
26-26
ISSN
2730-664X (ISSN-L)
OpenAlex
W4406658975
Cited by
358
References
158
Citations by year
2026: 226 · 2025: 123
FWCI
20.62
Type
journal article
Language
en
Open access
Licence
Locations
  • Communications Medicine (publishedVersion, cc-by)
  • medRxiv (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Leiden Repository (Leiden University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UNICA IRIS Institutional Research Information System (University of Cagliari) (submittedVersion, other-oa)
  • mediaTUM (Technical University of Munich) (submittedVersion)
  • RWTH Publications (RWTH Aachen) (submittedVersion, other-oa)
Funders
European Commission (101079894)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
large language models, patient care, GPT-3.5, GPT-4, patient education, medical question answering, medical text summarization, medical translation, clinical documentation, LLM limitations, algorithmic bias, reproducibility, data transparency, patient empowerment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Current applications and challenges in large language models for patient care: a systematic review. https://app.cochat.ai/papers/10.1038/s43856-024-00717-2
The paper
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., Truhn, D., Cuocolo, R., Adams, L. C., & Bressem, K. K. (2025). Current applications and challenges in large language models for patient care: a systematic review. Communications Medicine. https://doi.org/10.1038/s43856-024-00717-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.