ChatGPT-5 for drug–drug interaction detection in the intensive care unit: a real-world cohort study on large language model advances and implications for clinical pharmacists
Abstract
Background Large language models (LLMs) such as ChatGPT are increasingly applied in healthcare, but their performance in detecting clinically relevant potential drug-drug interactions (pDDIs) in intensive care units (ICUs) remains underexplored.Objectives This study evaluated the performance of ChatGPT-4.1 and ChatGPT-5 in identifying pDDIs among ICU patients compared with standard interaction checkers, benchmarking its outputs against ICU mortality.Methods This retrospective cohort included 311 ICU patients. Medications were entered into ChatGPT-4.1 and ChatGPT-5 using a uniform prompt. The pDDIs identified by ChatGPT were compared with outputs from 3 reference databases. Agreement metrics were calculated. Regression models (LASSO with refitted logistic regression) adjusted for confounders and compared mortality associations between ChatGPT- and reference-derived pDDIs. Reproducibility was assessed by repeating ChatGPT queries on a random sample of 50 patients.Results ChatGPT-4.1 and -5 identified fewer interactions than standard databases (8 and 10 vs. 22 total; 1 and 2 vs. 4 significant per patient). Agreement with references was moderate for total pDDIs and lower for significant ones. ChatGPT-5 identified pDDIs were independently associated with ICU mortality (odds ratio: 2.372, 95% CI: 1.236-4.696, P: 0.011), whereas ChatGPT-4.1 pDDIs were not. None of the versions outperformed standard tools in discrimination or clinical utility. Reproducibility was also limited, with ChatGPT-5 achieving only 68% consistency across repeated runs.Conclusion ChatGPT-4.1 and -5 showed limited sensitivity, inconsistent reproducibility, and no added clinical benefit. Therefore, pharmacist oversight remains essential for safe integration of LLMs in ICU practice. However, ChatGPT-5 demonstrated higher agreement with references and an independent association with ICU mortality.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The quality, scope, and currency of data accessible to LLMs are relevant to their clinical performance.
“The quality, scope, and currency of data accessible to LLMs are relevant to their clinical performance.86,101,106”
From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [86]
Cites
Multicollinearity and misleading statistical results
Kim, J. (2019) · Korean Journal of Anesthesiology · doi:10.4097/kja.19087 · Cited by 3,032 · see citing works
Evaluation and mitigation of the limitations of large language models in clinical decision-making
Hager, P., Jungmann, F., Holland, R., Bhagat, K., Hubrecht, I., Knauer, M., Vielhauer, J., Makowski, M., Braren, R., Kaissis, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03097-1 · Cited by 798 · see citing works
ChatGPT and the future of medical writing
Biswas, S. (2023) · Radiology · doi:10.1148/radiol.223312 · Cited by 571 · see citing works
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358 · see citing works
Sociodemographic biases in medical decision making by large language models
Omar, M., Soffer, S., Agbareia, R., Bragazzi, N. L., Apakama, D. U., Horowitz, C. R., Charney, A. W., Freeman, R., Kummer, B., Glicksberg, B. S., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03626-6 · Cited by 173 · see citing works
Zheng, W. Y., Richardson, L. C., Li, L., Day, R. O., Westbrook, J. I., & Baysari, M. T. (2017) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-017-2357-5 · Cited by 170 · see citing works
Kernberg, A., Gold, J. A., & Mohan, V. (2024) · Journal of Medical Internet Research · doi:10.2196/54419 · Cited by 120 · see citing works
Ntinopoulos, V., Biefer, H. R. C., Tudorache, I., Papadopoulos, N., Odavic, D., Risteski, P. S., Haeussler, A., & Dzemali, O. (2025) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2024-101139 · Cited by 93 · see citing works
A scoping review on generative AI and large language models in mitigating medication related harm
Ong, J. C. L., Chen, M. H., Ng, N., Elangovan, K., Tan, N. Y. T., Jin, L., Xie, Q., Ting, D. S. W., Rodriguez-Monguio, R., Bates, D. W., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01565-7 · Cited by 64 · see citing works
Automated generation of discharge summaries: leveraging large language models with clinical data
Ganzinger, M., Kunz, N., Fuchs, P., Lyu, C. K., Loos, M., Dugas, M., & Pausch, T. M. (2025) · Scientific Reports · doi:10.1038/s41598-025-01618-7 · Cited by 56 · see citing works
Van De Sijpe, G., Quintens, C., Walgraeve, K., Van Laer, E., Penny, J., De Vlieger, G., Schrijvers, R., De Munter, P., Foulon, V., Casteels, M., et al. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01783-z · Cited by 54 · see citing works
Moëll, B., & Aronsson, F. S. (2025) · Journal of Medical Internet Research · doi:10.2196/75849 · Cited by 50 · see citing works
Dsouza, V. S., Leyens, L., Kurian, J. R., Brand, A. M., & Brand, H. (2025) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2025.02.008 · Cited by 48 · see citing works
Bakker, T., Abu–Hanna, A., Dongelmans, D. A., Vermeijden, W., Bosman, R. J., de Lange, D. W., Klopotowska, J. E., de Keizer, N. F., Hendriks, S. A., Cate, J. T. M. H. T., et al. (2020) · Journal of Critical Care · doi:10.1016/j.jcrc.2020.11.020 · Cited by 46 · see citing works
Large language models in critical care
Biesheuvel, L. A., Workum, J. D., Reuland, M., van Genderen, M. E., Thoral, P., Dongelmans, D., & Elbers, P. (2025) · Journal of Intensive Medicine · doi:10.1016/j.jointm.2024.12.001 · Cited by 36 · see citing works
Role of artificial intelligence in pharmacy practice: A narrative review
Wong, A., Wentz, E. E., Palisano, N., Dirani, M., Elsamadisi, P., Qashou, F., Celi, L. A., Badawi, O., & Nazer, L. H. (2023) · JACCP JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · doi:10.1002/jac5.1856 · Cited by 35 · see citing works
Franc, J. M., Hertelendy, A. J., Cheng, L. T. W., Hata, R., & Verde, M. (2024) · Journal of Medical Internet Research · doi:10.2196/55648 · Cited by 30 · see citing works
Radha Krishnan, R. P., Hung, E. H., Ashford, M., Edillo, C. E., Gardner, C., Hatrick, H. B., Kim, B., Lai, A. W. Y., Li, X., Zhao, Y. X., et al. (2024) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.16275 · Cited by 29 · see citing works
Can large language models detect drug–drug interactions leading to adverse drug reactions?
Sicard, J., Montastruc, F., Achalme, C., Jonville-Bera, A. P., Songue, P., Babin, M., Soeiro, T., Schiro, P., de Canecaude, C., & Barus, R. (2025) · Therapeutic Advances in Drug Safety · doi:10.1177/20420986251339358 · Cited by 29 · see citing works
Demass, T. B., Guadie, A. G., Mengistu, T. B., Belay, Z. A., Melese, A. A., Berneh, A. A., Gedefaw, L., Wagaye, F. E., & Bantie, G. M. (2023) · Scientific Reports · doi:10.1038/s41598-023-39190-7 · Cited by 27 · see citing works
Elnaem, M. H., Okuyan, B., Mubarak, N., Thabit, A. K., AbouKhatwa, M. M., Ramatillah, D. L., Isah, A., Al‐Jumaili, A. A., & Nazar, N. I. M. (2025) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-025-01936-w · Cited by 27 · see citing works
Xu, H., Wang, Y., Xun, Y., Shao, R., & Jiao, Y. (2025) · QJM · doi:10.1093/qjmed/hcaf114 · Cited by 21 · see citing works
Li, D., Wu, L., Zhang, M., Shpyleva, S., Lin, Y.-C., Huang, H., Li, T., & Xu, J. (2024) · Frontiers in Drug Safety and Regulation · doi:10.3389/fdsfr.2024.1379260 · Cited by 19 · see citing works
Busser, B. D., Roth, L., & Loof, H. D. (2024) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-024-01839-2 · Cited by 19 · see citing works
Urquhart, E., Ryan, J., Hartigan, S., Nita, C., Hanley, C., Moran, P., Bates, J., Jooste, R., Judge, C., Laffey, J. G., et al. (2024) · Intensive Care Medicine Experimental · doi:10.1186/s40635-024-00656-1 · Cited by 18 · see citing works
Accuracy and reproducibility of ChatGPT responses to real‐world drug information questions
Khatri, S., Sengul, A., Moon, J., & Jackevicius, C. A. (2025) · JACCP JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · doi:10.1002/jac5.70038 · Cited by 18 · see citing works
Kim, J., Kincaid, J. W. R., Rao, A. S., Lie, W., Fuh, L., Landman, A., & Succi, M. D. (2024) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2024.102304 · Cited by 14 · see citing works
Bottacin, W. E., Luquetta, A., Gomes-Jr, L., de Souza, T. T., Reis, W. C. T., & Melchiors, A. C. (2024) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-024-01803-0 · Cited by 11 · see citing works
Impact of artificial intelligence on future clinical pharmacy research and scholarship
Chan, A., Baker, W. L., Abazia, D. T., Bauman, J. L., DeVane, C. L., Goodlet, K. J., Hall, N., Hicks, J. K., Jones, E., Lu, C., et al. (2025) · JACCP JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · doi:10.1002/jac5.70003 · Cited by 11 · see citing works
Cheng, H. Y. (2024) · JMIR Formative Research · doi:10.2196/63494 · Cited by 9 · see citing works
Wu, X., Huang, Y., & He, Q. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106088 · Cited by 8 · see citing works
Tecen-Yücel, K., Ozdemir-Ayduran, N., Kara, E., et al. (2025) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-025-01868-5 · Cited by 7 · see citing works
Oscar, M. M., Boland, M., & Cadogan, C. A. (2025) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2025.06.107 · Cited by 4 · see citing works
Harnessing ChatGPT for digital tools in pharmacy practice
Yakob, R. A., Bamgboje‐Ayodele, A., Collins, J. C., & Aslani, P. (2025) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2025.06.106 · Cited by 3 · see citing works
Artificial intelligence is here, but are we ready?
Nguyen, K. A., & Zillich, A. J. (2025) · JACCP JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · doi:10.1002/jac5.70021 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations
Yakob, R. A., Bamgboje-Ayodele, A., Collins, J. C., & Aslani, P. (2026) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2026.09.007
- Authors
- Azmakan, H., & Hashemian, F.
- Corresponding author
- Hashemian, Farshad
- Venue
- Journal of the American Pharmacists Association
- Publisher
- Elsevier BV
- Published
- 2026-07
- Volume
- 66
- Issue
- 4
- Pages
- 103019
- ISSN
- 1544-3191 (ISSN-L)
- OpenAlex
- W7118903111
- Cited by
- 4
- References
- 35
- Citations by year
- 2026: 4
- FWCI
- 8.05
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Pharmacists Association (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- ChatGPT, large language models, drug-drug interactions, potential drug-drug interactions, intensive care unit, ICU mortality, clinical pharmacists, reproducibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review