Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index; 35 of the 78 sources its claims cite were checked for support.

The future landscape of large language models in medicine

Open access

Abstract

Large language models (LLMs) are artificial intelligence (AI) tools specifically trained to process and generate text. LLMs attracted substantial public attention after OpenAI’s ChatGPT was made publicly available in November 2022. LLMs can often answer questions, summarize, paraphrase and translate text on a level that is nearly indistinguishable from human capabilities. The possibility to actively interact with models like ChatGPT makes LLMs attractive tools in various fields, including medicine. While these models have the potential to democratize medical knowledge and facilitate access to healthcare, they could equally distribute misinformation and exacerbate scientific misconduct due to a lack of accountability and transparency. In this article, we provide a systematic and comprehensive overview of the potentials and limitations of LLMs in clinical practice, medical research and medical education.

Overview

CoChat Summary

TL;DR

This review surveys how large language models (LLMs) may affect patient care, medical research, and medical education, emphasizing both their capabilities and their risks. It argues that LLMs could democratize medical knowledge and assist communication, documentation, research, and teaching, but current limitations in factuality, bias, accountability, transparency, and reproducibility must be addressed before clinical use.

Key findings

  • LLMs can augment patient care through medical knowledge and reasoning, translations, summaries, and documentation; GPT-4 “now exceeds the passing score on every step of the US-medical licensing exam (USMLE),” and documentation/administrative requirements consume around 25% of clinicians’ workdays.
  • In medical research, LLMs can summarize evidence, generate scientific abstracts that humans struggle to differentiate from human-written ones, and assist with coding, but outputs currently require significant human revision and LLMs do not always provide appropriate detailed summaries or critical appraisals of up-to-date, high-quality, peer-reviewed evidence.
  • In medical education, LLMs can serve as personalized teaching assistants, simplify complex concepts, and simulate patient conversations, but they risk decreasing critical thinking and creativity and may lead to “externalization of medical reasoning.”
  • Ethical and safety concerns include misinformation, data privacy and leakage, prompt injection attacks, lack of public training datasets and source code, commercial monopolies, and reproducibility problems due to model version changes; the authors observed substantial differences between GPT-3.5 and GPT-4 prompted queries and minor changes across GPT-3.5 versions.

Methods

  • Narrative review/overview; the abstract describes it as a “systematic and comprehensive overview of the potentials and limitations of LLMs in clinical practice, medical research and medical education.”
  • No primary data, sample, or formal quantitative analysis are reported; illustrative model outputs were generated using ChatGPT (GPT-4—May 3, 2023; GPT-3.5—May 3, 2023) and additional examples are in Supplementary Data.
  • The main analysis is a qualitative synthesis across patient care, medical research, and medical education, with example prompts and outputs used to illustrate capabilities and limitations.

Limitations

  • Stated by the paper: LLMs reproduce existing biases, hallucinate false information, spread misinformation, lack accountability, and currently have no mechanisms to ensure output correctness; their knowledge is static, reproducibility is compromised by model updates or access restrictions, training datasets and source code are often not public, and development is driven primarily by commercial companies.
  • Stated by the paper: safety guardrails can themselves pose limitations, for example if bias prevention leads to different symptoms in men and women being overlooked.
  • Evident from the text: despite being described as “systematic,” no formal systematic review methodology, search strategy, inclusion criteria, or data extraction process is reported, and the illustrative examples are limited to selected ChatGPT versions rather than a comprehensive empirical evaluation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s43856-023-00370-1

Cites

70 of 70 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Large language models (LLMs) in medicine. a Simplified design of the architecture behind ChatGPT, including training, iterations of reinforcement learning by human feedback, choice of available model and implementation of guardrails to improve safety. b Overview of potential applications for LLMs in medicine, including patient care, research, and education. c Limitations of LLMs in their current state.
Fig. 1 Large language models (LLMs) in medicine. a Simplified design of the architecture behind ChatGPT, including training, iterations of reinforcement learning by human feedback, choice of available model and implementation of guardrails to improve safety. b Overview of potential applications for LLMs in medicine, including patient care, research, and education. c Limitations of LLMs in their current state.CC BY · PubMed Central

Verification

Claims the paper makes

16 Valid · 3 Not valid · 21 Not checked · 36 Own results and interpretations

23 more claims

Create a free account to see all 76 claims and to verify the 43 references still open.

See all claims — free account
  • It may state two separate claims as one. 31
  • Probably not a substantial claim. 29

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

5 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 63 references

Research works printed but not linked: 3

  1. Gao, L., Schulman, J. & Hilton, J. Scaling laws for reward model overoptimization. PMLR 202, 10835–10866 (2023).
  2. Xiu, Y. & Thompson, P. Flipped university class: a study of motivation and learning. Int. J. Inf. Commun. Technol. Educ. 19, 41–63 (2020).
  3. Sha, L., Li, Y., Gasevic, D. & Chen, G. Bigger data or fairer data? Augmenting BERT via active sampling for educational text classification. In Proc. 29th International Conference on Computational Linguistics, 1275–1285 (International Committee on Computational Linguistics, 2022).

Cited by

See all 1,045

Metadata

Authors
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., Wagner, S. J., & Kather, J. N.
Affiliations
  • Universitätsklinikum Aachen (DE)
  • University Hospital Carl Gustav Carus (DE)
  • Else Kröner Fresenius Center for Digital Health (DE)
  • Helmholtz Munich (DE)
  • Heidelberg University (DE)
Corresponding author
Kather, Jakob Nikolas
Venue
Communications Medicine
Published
2023-10-10
Volume
3
Issue
1
Pages
141-141
ISSN
2730-664X (ISSN-L)
OpenAlex
W4387500346
Cited by
1,045
References
63
Citations by year
2026: 304 · 2025: 435 · 2024: 293 · 2023: 13
FWCI
12.29
Type
journal article
Language
en
Open access
Licence
Locations
  • Communications Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Qucosa (Saxon State and University Library Dresden) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
Funders
European Union’s Horizon Europe and innovation programme
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
large language models, ChatGPT, artificial intelligence in healthcare, clinical practice, medical research, medical education, misinformation, research misconduct, transparency and accountability, health care access

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The future landscape of large language models in medicine. https://app.cochat.ai/papers/10.1038/s43856-023-00370-1
The paper
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., Wagner, S. J., & Kather, J. N. (2023). The future landscape of large language models in medicine. Communications Medicine. https://doi.org/10.1038/s43856-023-00370-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.