Cited by 4Artificial Intelligence in Healthcare and EducationPharmaceutical Practices and Patient OutcomesMachine Learning in Healthcare

ChatGPT-5 for drug–drug interaction detection in the intensive care unit: a real-world cohort study on large language model advances and implications for clinical pharmacists

Azmakan, Helia, Hashemian, Farshad·2026·Journal of the American Pharmacists Association

Abstract

Background Large language models (LLMs) such as ChatGPT are increasingly applied in healthcare, but their performance in detecting clinically relevant potential drug-drug interactions (pDDIs) in intensive care units (ICUs) remains underexplored.Objectives This study evaluated the performance of ChatGPT-4.1 and ChatGPT-5 in identifying pDDIs among ICU patients compared with standard interaction checkers, benchmarking its outputs against ICU mortality.Methods This retrospective cohort included 311 ICU patients. Medications were entered into ChatGPT-4.1 and ChatGPT-5 using a uniform prompt. The pDDIs identified by ChatGPT were compared with outputs from 3 reference databases. Agreement metrics were calculated. Regression models (LASSO with refitted logistic regression) adjusted for confounders and compared mortality associations between ChatGPT- and reference-derived pDDIs. Reproducibility was assessed by repeating ChatGPT queries on a random sample of 50 patients.Results ChatGPT-4.1 and -5 identified fewer interactions than standard databases (8 and 10 vs. 22 total; 1 and 2 vs. 4 significant per patient). Agreement with references was moderate for total pDDIs and lower for significant ones. ChatGPT-5 identified pDDIs were independently associated with ICU mortality (odds ratio: 2.372, 95% CI: 1.236-4.696, P: 0.011), whereas ChatGPT-4.1 pDDIs were not. None of the versions outperformed standard tools in discrimination or clinical utility. Reproducibility was also limited, with ChatGPT-5 achieving only 68% consistency across repeated runs.Conclusion ChatGPT-4.1 and -5 showed limited sensitivity, inconsistent reproducibility, and no added clinical benefit. Therefore, pharmacist oversight remains essential for safe integration of LLMs in ICU practice. However, ChatGPT-5 demonstrated higher agreement with references and an independent association with ICU mortality.

Integrity signals

Registry

Crossref · 10.1016/j.japh.2026.103019

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Azmakan, H., & Hashemian, F.
Corresponding author
Hashemian, Farshad
Venue
Journal of the American Pharmacists Association
Publisher
Elsevier BV
Published
2026-07
Volume
66
Issue
4
Pages
103019
ISSN
1544-3191 (ISSN-L)
OpenAlex
W7118903111
Cited by
4
References
35
Citations by year
2026: 4
FWCI
8.05
Type
journal article
Language
en
Open access
closed
Locations
  • Journal of the American Pharmacists Association (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
ChatGPT, large language models, drug-drug interactions, potential drug-drug interactions, intensive care unit, ICU mortality, clinical pharmacists, reproducibility

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review