Cited by 66Artificial Intelligence in Healthcare and EducationElectronic Health Records SystemsPatient Safety and Medication Errors

Large language model as clinical decision support system augments medication safety in 16 clinical specialties

Open access
Ong, Jasmine Chiat Ling, Jin, Liyuan, Elangovan, Kabilan, Lim, Gilbert Yong San, Lim, Daniel Yan Zheng, Sng, Gerald Gui Ren, Ke, Yu He, Tung, Joshua Yi Min +11 more·2025·Cell Reports Medicine

Abstract

Large language models (LLMs) have emerged as tools to support healthcare delivery, from automating tasks to aiding clinical decision-making. This study evaluated LLMs as alternative to rule-based alert systems, focusing on their ability to identify prescribing errors. This was designed as a prospective, cross-over, open-label study involving 91 error scenarios based on 40 clinical vignettes across 16 medical and surgical specialties. We developed and validated five LLM models using a retrieval-augmented generation framework. The best-performing model evaluated three different implementation strategies: LLM-based clinical decision support system (CDSS) alone, pharmacist plus LLM-based CDSS (co-pilot), and pharmacist alone. The co-pilot arm demonstrated the best performance with an accuracy of 61% (precision 0.57, recall 0.61, and F1 0.59). In detecting errors posing serious harm, the co-pilot mode increased accuracy by 1.5-fold over the pharmacist alone. Effective LLM integration for complex tasks like medication chart reviews can enhance healthcare professional performance, improving patient safety.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.xcrm.2025.102323

Cites

42 of 44 references verified2 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • These AI technologies have improved diagnostic accuracy, medication safety, workflow efficiency, and access to specialist care.

    “They have improved diagnostic accuracy, medication safety, workflow efficiency, and access to specialist care 5,6 .”

    From AI-induced never-skilling in medical education (2026) · cites this paper as [5]

    The Results (Comparative performance of different modes of delivery; page unavailable) report better identification of drug-related problems with pharmacist–LLM collaboration, supporting the medication-safety part of the claim, although the overall accuracy difference was not statistically significant. The paper does not establish improved diagnostic accuracy, workflow efficiency, or access to specialist care; its Discussion instead describes possible workflow disruption and envisages, rather than demonstrates, filling service gaps in low-resource settings. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Studies highlighted the potential risks of data privacy breaches associated with the use of proprietary LLMs in clinical practice.

    “Studies highlighted the potential risks of data privacy breaches associated with the use of proprietary LLMs in clinical practice.74,79,83”

    From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [74]

    The STAR★Methods section says no identifiable patient data were used, and the Discussion addresses limitations such as bias, prompting, and generalizability. Neither passage discusses a risk of privacy breaches from proprietary LLMs in clinical practice. Searches for privacy, proprietary models, and patient-data breaches found no passage addressing that claim. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

1 retracted

Flagged first, then by citations

Research works printed but not linked: 1

  1. [6]Osheroff J.A. Teich J.M. Levick D. Saldana L. Velasco F. Sittig D.F. Rogers K. Jenders R. Improving Outcomes with Clinical Decision Support: An Implementer's Guide Second Edition 2023 HIMSS

Cited by

Sorted by citations

Authors
Ong, J. C. L., Jin, L., Elangovan, K., Lim, G. Y. S., Lim, D. Y. Z., Sng, G. G. R., Ke, Y. H., Tung, J. Y. M., Zhong, R. J., Koh, C. M. Y., Lee, K. Z. H., Chen, X., Ch’ng, J. K., Than, A., Goh, K. J., Lim, C. P., Ng, T. M., Liu, N., & Ting, D. S. W.
Affiliations
  • Singapore General Hospital (SG)
  • SingHealth (SG)
  • Tan Tock Seng Hospital (SG)
  • Duke-NUS Medical School (SG)
Corresponding author
Ting, Daniel Shu Wei
Venue
Cell Reports Medicine
Publisher
Elsevier BV
Published
2025-10
Volume
6
Issue
10
Pages
102323
ISSN
2666-3791 (ISSN-L)
OpenAlex
W4414564500
Cited by
58
References
33
Citations by year
2026: 52 · 2025: 5 · 2024: 1
FWCI
13.72
Type
journal article
Language
en
Open access
Locations
  • Cell Reports Medicine (publishedVersion, cc-by)
  • SSRN Electronic Journal (acceptedVersion)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Duke-NUS Medical School (05/FY2022/EX/66-A128, Duke-NUS/RSF/2021/0018, 05/FY2020/EX/15-A58)
  • National Medical Research Council (MOH-000655-00, MOH-001014-00, MOH-001689-00)
  • Agency for Science, Technology and Research (A20H4g2141, H20C6a0032)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
Keywords
large language models, clinical decision support systems, drug safety, prescribing errors, medication error detection, retrieval-augmented generation, patient safety, medical specialties
MeSH terms
Large Language Models, Humans, Language, Medication Errors, Prospective Studies, Cross-Over Studies, Decision Support Systems, Clinical, Patient Safety

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review