Poor performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal dysfunction
Open accessAbstract
Purpose Clinical decision support systems (CDSS) are used to identify drugs with potential need for dose modification in patients with renal impairment. ChatGPT holds the potential to be integrated in the electronic health record (EHR) system to give such dosing advices. In this study, we aim to evaluate the performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal impairment.Methods This cross-sectional study was performed at Tergooi Medical Center, the Netherlands. CDSS alerts regarding renal dysfunction were collected from the electronic health record (EHR) during a 2-week period and were presented to ChatGPT and an expert panel. Alerts were presented with and without patient variables. To evaluate the performance, suggested medication interventions were compared.Results In total, 172 CDDS alerts were generated for 80 patients. Indecisive responses by ChatGPT to alerts were excluded. For alerts presented without patient variables, ChatGPT provided "correct and identical" responses to 19.9%, "correct and different" responses to 26.7%, and "incorrect responses to 53.4% of the alerts. For alerts including patient variables, ChatGPT provided "correct and identical" responses to 16.7%, "correct and different" responses to 16.0%, and "incorrect responses to 67.3% of the alerts. Accuracy was better for newer drugs such as direct oral anticoagulants.Conclusion The performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal dysfunction was poor. Based on these results, we conclude that ChatGPT, in its current state, is not appropriate for automatic integration into our EHR to handle CDSS alerts related to renal dysfunction.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some studies in this review used de-identified data to evaluate the LLMs.
“Some studies in this review also used de-identified data to evaluate the LLMs, 30,61,65”
From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [61]
Cites
Toprak, Ö., Kurt, H., Sarı, Y., Şarkış, C., Us, H., & Kırık, A. (2017) · Kidney & Blood Pressure Research · doi:10.1159/000468530 · Cited by 29,769 · see citing works
A New Equation to Estimate Glomerular Filtration Rate
Levey, A. S., Stevens, L. A., Schmid, C. H., Zhang, Y. (., Castro, A. F., Feldman, H. I., Kusek, J. W., Eggers, P., Van Lente, F., Greene, T., et al. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-9-200905050-00006 · Cited by 25,603 · see citing works
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · cited 1× · see citing works
“Ayers et al. showed in a cross-sectional study that ChatGPT may be able to aid in responses to patient questions [21].”
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738 · see citing works
Dave, T., Athaluri, S. A., & Singh, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1169595 · Cited by 1,204 · see citing works
Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review
Plana, D., Shung, D. L., Grimshaw, A. A., Saraf, A., Sung, J. J. Y., & Kann, B. H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.33946 · Cited by 236 · see citing works
Clinical Decision Support Systems
Wasylewicz, A. T. M., & Scheepers-Hoeks, A. A.-M. (2018) · Springer eBooks · doi:10.1007/978-3-319-99713-1_11 · Cited by 132 · see citing works
Roosan, D., Padua, P., Khan, R. K. R., Khan, H., Verzosa, C., & Wu, Y. (2023) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2023.11.023 · Cited by 124 · see citing works
Huang, X., Estau, D., Liu, X., Yu, Y., Qin, J., & Li, Z. (2023) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15896 · Cited by 117 · see citing works
Performance and risks of ChatGPT used in drug information: an exploratory real-world analysis
Morath, B., Chiriac, U., Jaszkowski, E., Deiß, C., Nürnberg, H., Hörth, K., Hoppe‐Tichy, T., & Green, K. (2023) · European Journal of Hospital Pharmacy · doi:10.1136/ejhpharm-2023-003750 · Cited by 79 · see citing works
How ChatGPT works: a mini review
Briganti, G. (2023) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-023-08337-7 · Cited by 67 · see citing works
Al-Dujaili, Z., Omari, S., Pillai, J., & Al Faraj, A. (2023) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2023.08.012 · Cited by 56 · see citing works
Adherence with Dosing Guideline in Patients with Impaired Renal Function at Hospital Discharge
Maanen, A. C. D., van Marum, R. J., Jansen, P. A. F., Rijkom, J. E. F. Z., van Solinge, W. W., & Egberts, T. C. G. (2015) · PLoS ONE · doi:10.1371/journal.pone.0128237 · Cited by 54 · see citing works
Self-citationRole of ChatGPT-4 for Medical Researchers
Datt, M., Sharma, H., Aggarwal, N., & Sharma, S. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03336-5 · Cited by 29 · see citing works
Assessing the applicability and appropriateness of ChatGPT in answering clinical pharmacy questions
Fournier, A., Fallet, C., Sadeghipour, F., & Perrottet, N. (2023) · Annales Pharmaceutiques Françaises · doi:10.1016/j.pharma.2023.11.001 · Cited by 29 · see citing works
Machine-Made Empathy? Why Medicine Still Needs Humans-Reply.
Ayers, J., Dredze, M., & Smith, D. M. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.4392 · Cited by 2 · see citing works
Effect of isotretinoin on CYP2D6 and CYP3A activity in patients with severe acne
Zhao, Y., Vary, J. C., Yadav, A. S., Czuba, L. C., Shum, S., LaFrance, J., Huang, W., Isoherranen, N., & Hébert, M. F. (2023) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15938 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Du, X.-S., Zhou, Z., Wang, Y.-F., Chuang, Y.-W., Li, Y.-M., Yang, R., Zhang, W., Wang, X., Chen, X.-Y., Guan, H., et al. (2024) · medRxiv · doi:10.1101/2024.08.11.24311828 · Cited by 36 · see citing works
Aydin, S., Karabacak, M., Vlachos, V., & Margetis, K. D. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1527864 · Cited by 25 · see citing works
Applications of Generative Artificial Intelligence in Electronic Medical Records: A Scoping Review
Morjaria, L., Gandhi, B., Haider, N., Mellon, M., & Sibbald, M. (2025) · Information · doi:10.3390/info16040284 · Cited by 11 · see citing works
Hahne, J., & Carpenter, B. D. (2026) · Patient Education and Counseling · doi:10.1016/j.pec.2026.109492 · Cited by 12 · see citing works
ChatGPT in pharmacy practice: disruptive or destructive innovation? A scoping review
Lima, T. D. M., Bonafé, M., Baby, A. R., & Visacri, M. B. (2024) · Scientia Pharmaceutica · doi:10.3390/scipharm92040058 · Cited by 6 · see citing works
Suppan, M., Fubini, P. E., Stefani, A., Gisselbaek, M., Samer, C. F., & Savoldelli, G. L. (2025) · JMIR AI · doi:10.2196/66796 · Cited by 3 · see citing works
Solar radiation estimation using an AI-driven empirical model: Calibration and evaluation
Ekici, C. (2025) · Case Studies in Thermal Engineering · doi:10.1016/j.csite.2025.107080 · Cited by 2 · see citing works
Conducting Medication Reviews: A Comparative Study Between ChatGPT ‐4 and Healthcare Professionals
ten Hoope, S., Marongiu, S., Siegert, C. E. H., Heerdink, E. R., Janssen, M. J. A., & Karapinar‐Çarkit, F. (2026) · Journal of the American Geriatrics Society · doi:10.1111/jgs.70415 · Cited by 1 · see citing works
Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations
Yakob, R. A., Bamgboje-Ayodele, A., Collins, J. C., & Aslani, P. (2026) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2026.09.007
ChatGPT does make up scientific references: should it be currently renamed CheatGPT?
Mikus, G. (2024) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-024-03758-7
Suppan, M., Fubini, P. E., Stefani, A., Gisselbaek, M., Samer, C. F., & Savoldelli, G. L. (2024) · doi:10.2196/preprints.66796
Thapa, R. B., Khadka, R., Dahal, P., Karki, S., & Shrestha, S. (2025) · Exploratory Research in Clinical and Social Pharmacy · doi:10.1016/j.rcsop.2025.100698
Chuang, K.-C., Lin, H., & Lin, H.-M. (2026) · medRxiv · doi:10.64898/2026.05.23.26353939
Barbonus, C., Sultzer, R., & Bertsche, T. (2026) · JMIR AI · doi:10.2196/87803
Abejez-Arrizabalaga, Á., Pellejero-Sagastizábal, G., Aznar-Gimeno, R., Franco, B., Letona-Giménez, S., Morte-Romea, E., Origüen, J., Pérez-Rodríguez, M. T., Retamar, P., Ruizendaal, E., et al. (2026) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2026.08.025
- Authors
- van Nuland, M., Snoep, J. D., Egberts, T., Erdogan, A., Wassink, R., & van der Linden, P. D.
- Affiliations
- Tergooi (NL)
- Utrecht University (NL)
- Venue
- European Journal of Clinical Pharmacology
- Published
- 2024-04-09
- Volume
- 80
- Issue
- 8
- Pages
- 1133-1140
- ISSN
- 0031-6970 (ISSN-L)
- 1432-1041
- OpenAlex
- W4394620742
- Cited by
- 15
- References
- 17
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 4
- FWCI
- 1.42
- Type
- journal article
- Language
- en
- Open access
- green
- dspace.library.uu.nl
- Licence
- Locations
- European Journal of Clinical Pharmacology (publishedVersion)
- PubMed (publishedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- ChatGPT, clinical decision support systems, renal dysfunction, dose adjustment, hospitalized patients, electronic health records, large language models, medication safety alerts, direct oral anticoagulants
- MeSH terms
- Adult, Aged, Aged, 80 and over, Cross-Sectional Studies, Female, Hospitalization, Humans, Male, Middle Aged, Netherlands, Decision Support Systems, Clinical, Medical Order Entry Systems, Renal Insufficiency, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
