ChatGPT as a tool for medical education and clinical decision-making on the wards: Case study
Open accessPublished version of a preprint
Abstract
Background: Large language models (LLMs) are computational artificial intelligence systems with advanced natural language processing capabilities that have recently been popularized among health care students and educators due to their ability to provide real-time access to a vast amount of medical knowledge. The adoption of LLM technology into medical education and training has varied, and little empirical evidence exists to support its use in clinical teaching environments. Objective: The aim of the study is to identify and qualitatively evaluate potential use cases and limitations of LLM technology for real-time ward-based educational contexts. Methods: A brief, single-site exploratory evaluation of the publicly available ChatGPT-3.5 (OpenAI) was conducted by implementing the tool into the daily attending rounds of a general internal medicine inpatient service at a large urban academic medical center. ChatGPT was integrated into rounds via both structured and organic use, using the web-based “chatbot” style interface to interact with the LLM through conversational free-text and discrete queries. A qualitative approach using phenomenological inquiry was used to identify key insights related to the use of ChatGPT through analysis of ChatGPT conversation logs and associated shorthand notes from the clinical sessions. Results: Identified use cases for ChatGPT integration included addressing medical knowledge gaps through discrete medical knowledge inquiries, building differential diagnoses and engaging dual-process thinking, challenging medical axioms, using cognitive aids to support acute care decision-making, and improving complex care management by facilitating conversations with subspecialties. Potential additional uses included engaging in difficult conversations with patients, exploring ethical challenges and general medical ethics teaching, personal continuing medical education resources, developing ward-based teaching tools, supporting and automating clinical documentation, and supporting productivity and task management. LLM biases, misinformation, ethics, and health equity were identified as areas of concern and potential limitations to clinical and training use. A code of conduct on ethical and appropriate use was also developed to guide team usage on the wards. Conclusions: Overall, ChatGPT offers a novel tool to enhance ward-based learning through rapid information querying, second-order content exploration, and engaged team discussion regarding generated responses. More research is needed to fully understand contexts for educational use, particularly regarding the risks and limitations of the tool in clinical settings and its impacts on trainee development.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Face-to-face simulations are effective but resource-intensive, requiring extensive time, personnel, and coordination, which limits their frequency and scalability.
“These face-to-face simulations are effective but resource-intensive, requiring extensive time, personnel, and coordination, which inherently limits their frequency and scalability (Skryd & Lawrence, 2024).”
From DIALOGUE: A Generative AI-Based Pre–Post Simulation Study to Enhance Diagnostic Communication in Medical Students Through Virtual Type 2 Diabetes Scenarios (2025) · cites this paper as Skryd & Lawrence, 2024
The Results section on cognitive aids in acute care scenarios discusses an RRT debrief and the effectiveness of cognitive aids, not the effectiveness or resource demands of face-to-face simulations. Searches for simulation staffing, time, frequency, and scalability found no passage addressing those claims; page numbers were not available in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Addressing bias in big data and AI for health care: A call for open science
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100347 · Cited by 903 · see citing works
The rise of <scp>ChatGPT</scp> : Exploring its potential in medical education
Lee, H. (2024) · Anatomical Sciences Education · doi:10.1002/ase.2270 · Cited by 596 · cited 1× · see citing works
“This dialogue can be personalized to the level of the learner and leveraged to answer clinical questions, provide differential diagnoses, generate resources, and facilitate the assimilation of complex medical concepts [3].”
BackgroundHow Is ChatGPT’s Behavior Changing Over Time?
Chen, L., Zaharia, M., & Zou, J. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.5317da47 · Cited by 507 · see citing works
Preiksaitis, C., & Rose, C. (2023) · JMIR Medical Education · doi:10.2196/48785 · Cited by 468 · cited 1× · see citing works
“…medical notes and academic abstracts and completing the United States Medical Licensing Examination (USMLE); a surfeit of opportunities to explore the impact of generative AI tools in medical education has also been identified at the undergraduate, graduate, and professional levels [5,6,19-22].”
BackgroundLarge language models propagate race-based medicine
Omiye, J. A., Lester, J. C., Spichak, S., Rotemberg, V., & Daneshjou, R. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00939-z · Cited by 417 · see citing works
Lewinski, A. A., Crowley, M. J., Miller, C., Bosworth, H. B., Jackson, G. L., Steinhauser, K., White-Clark, C., McCant, F., & Zullig, L. L. (2021) · Medical Care · doi:10.1097/mlr.0000000000001553 · Cited by 296 · see citing works
An exploratory survey about using ChatGPT in education, healthcare, and research
Hosseini, M., Gao, C. A., Liebovitz, D. M., Carvalho, A. M., Ahmad, F. S., Luo, Y., MacDonald, N., Holmes, K. L., & Kho, A. (2023) · PLOS ONE · doi:10.1371/journal.pone.0292216 · Cited by 238 · see citing works
Flagged first, then by citations
Research works printed but not linked: 21
- [1]Thirunavukarasu AJ Ting DSJ Elangovan K Gutierrez L Tan TF Ting DSW Large language models in medicine Nat Med 2023 08 29 8 1930 1940 10.1038/s41591-023-02448-8 37460753 10.1038/s41591-023-02448-8 37460753 · DOI · Paper page
- [2]Singhal K Azizi S Tu T Mahdavi SS Wei J Chung HW Scales N Tanwani A Cole-Lewis H Pfohl S Payne P Seneviratne M Gamble P Kelly C Babiker A Schärli Nathanael Chowdhery A Mansfield P Demner-Fushman D Agüera Y Arcas Blaise Webster D Corrado GS Matias Y Chou K Gottweis J Tomasev N Liu Y Rajkomar A Barral J Semturs C Karthikesalingam A Natarajan V Large language models encode clinical knowledge Nature 2023 08 620 7972 172 180 10.1038/s41586-023-06291-2 37438534 10.1038/s41586-023-06291-2 37438534 PMC10396962 · DOI · Paper page
- [4]Gilson A Safranek CW Huang T Socrates V Chi L Taylor RA Chartash D How does ChatGPT perform on the United States Medical Licensing Examination (USMLE)? The implications of large language models for medical education and knowledge assessment JMIR Med Educ 2023 02 08 9 e45312 10.2196/45312 36753318 v9i1e45312 36753318 PMC9947764 · DOI · Paper page
- [5]Kung TH Cheatham M Medenilla A Sillos C De Leon L Elepaño C Madriaga M Aggabao R Diaz-Candido G Maningo J Tseng V Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models PLOS Digit Health 2023 02 2 2 e0000198 10.1371/journal.pdig.0000198 36812645 PDIG-D-22-00371 36812645 PMC9931230 · DOI · Paper page
- [6]Gao CA Howard FM Markov NS Dyer EC Ramesh S Luo Y Pearson AT Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers NPJ Digit Med 2023 6 1 75 10.1038/s41746-023-00819-6 10.1038/s41746-023-00819-6 37100871 10.1038/s41746-023-00819-6 37100871 PMC10133283 · DOI · Paper page
- [7]Alkaissi H McFarlane SI Artificial hallucinations in ChatGPT: implications in scientific writing Cureus 2023 02 15 2 e35179 10.7759/cureus.35179 36811129 36811129 PMC9939079 · DOI · Paper page
- [8]Arif TB Munaf U Ul-Haque I The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online 2023 12 28 1 2181052 10.1080/10872981.2023.2181052 36809073 36809073 PMC9946299 · DOI · Paper page
- [9]Abd-Alrazaq A AlSaad R Alhuwail D Ahmed A Healy PM Latifi S Aziz S Damseh R Alabed Alrazak S Sheikh J Large language models in medical education: opportunities, challenges, and future directions JMIR Med Educ 2023 06 01 9 e48291 10.2196/48291 37261894 v9i1e48291 37261894 PMC10273039 · DOI · Paper page
- [10]Ramanadhan S Revette AC Lee RM Aveling EL Pragmatic approaches to analyzing qualitative data for implementation science: an introduction Implement Sci Commun 2021 06 29 2 1 70 10.1186/s43058-021-00174-1 34187595 10.1186/s43058-021-00174-1 34187595 PMC8243847 · DOI
- [12]Cook CE Décary S Higher order thinking about differential diagnosis Braz J Phys Ther 2020 24 1 1 7 10.1016/j.bjpt.2019.01.010 30723033 S1413-3555(18)31066-9 30723033 PMC6994315 · DOI
- [14]Tay SW Ryan P Ryan CA Systems 1 and 2 thinking processes and cognitive reflection testing in medical students Can Med Educ J 2016 10 7 2 e97 e103 28344696 28344696 PMC5344059
- [16]Stedman TL Stedman's Medical Dictionary for the Health Professions and Nursing 2012 Philadelphia Wolters Kluwer Health/Lippincott Williams & Wilkins
- [17]Sellmann T Alchab S Wetzchewald D Meyer J Rassaf T Thal SC Burisch C Marsch S Breuckmann F Simulation-based randomized trial of medical emergency cognitive aids Scand J Trauma Resusc Emerg Med 2022 07 11 30 1 45 10.1186/s13049-022-01028-y 35820939 10.1186/s13049-022-01028-y 35820939 PMC9277856 · DOI
- [18]Yokoe M Hata J Takada T Strasberg SM Asbun HJ Wakabayashi G Kozaka K Endo I Deziel DJ Miura F Okamoto K Hwang T Huang WSH Ker CG Chen MF Han HS Yoon YS Choi IS Yoon DS Noguchi Y Shikata S Ukai T Higuchi R Gabata T Mori Y Iwashita Y Hibi T Jagannath P Jonas E Liau KH Dervenis C Gouma DJ Cherqui D Belli G Garden OJ Giménez ME de Santibañes E Suzuki K Umezawa A Supe AN Pitt HA Singh H Chan ACW Lau WY Teoh AYB Honda G Sugioka A Asai K Gomi H Itoi T Kiriyama S Yoshida M Mayumi T Matsumura N Tokumura H Kitano S Hirata K Inui K Sumiyama Y Yamamoto M Tokyo Guidelines 2018: diagnostic criteria and severity grading of acute cholecystitis (with videos) J Hepatobiliary Pancreat Sci 2018 01 25 1 41 54 10.1002/jhbp.515 29032636 29032636 · DOI
- [19]Tsang R Practical applications of ChatGPT in undergraduate medical education J Med Educ Curric Dev 2023 10 23821205231178449 10.1177/23821205231178449 37255525 10.1177_23821205231178449 37255525 PMC10226299 · DOI
- [20]Khan RA Jawaid M Khan AR Sajjad M ChatGPT—reshaping medical education and clinical management Pak J Med Sci 2023 39 2 605 607 10.12669/pjms.39.2.7653 36950398 PJMS-39-605 36950398 PMC10025693 · DOI · Paper page
- [21]Benoit JRA ChatGPT for clinical vignette generation, revision, and evaluation medRxiv Preprint posted online on February 8, 2023. 10.1101/2023.02.04.23285478 · DOI · Paper page
- [23]Cross J Robinson R Devaraju S Vaughans A Hood R Kayalackakom T Honnavar P Naik S Sebastian R Transforming medical education: assessing the integration of ChatGPT into faculty workflows at a Caribbean Medical School Cureus 2023 07 15 7 e41399 10.7759/cureus.41399 37426402 37426402 PMC10328790 · DOI
- [28]Lawrence K Linwood SL Digital Health Equity Digital Health [Internet] 2022 04 29 Brisbane (AU) Exon Publications 35605078
- [29]Singh N Lawrence K Richardson S Mann DM Centering health equity in large language model deployment PLOS Digit Health 2023 2 10 e0000367 10.1371/journal.pdig.0000367 37874780 PDIG-D-23-00181 37874780 PMC10597518 · DOI
- [30]Rezaei M Shahidi M Zero-shot learning and its applications from autonomous vehicles to COVID-19 diagnosis: a review Intell Based Med 2020 12 3 100005 10.1016/j.ibmed.2020.100005 33043311 S2666-5212(20)30005-3 33043311 PMC7531283 · DOI
Cited by
Sorted by citations
Mapping the use of artificial intelligence in medical education: a scoping review
Rincón, E. H. H., Jimenez, D., Aguilar, L. A. C., Flórez, J. M. P., Tapia, Á. E. R., & Peñuela, C. L. J. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07089-8 · Cited by 145 · see citing works
Integrating ChatGPT in Orthopedic Education for Medical Undergraduates: Randomized Controlled Trial
Gan, W., Ouyang, J., Li, H., Xue, Z., Zhang, Y., Dong, Q., Huang, J., Zheng, X., & Zhang, Y. (2024) · Journal of Medical Internet Research · doi:10.2196/57037 · Cited by 73 · see citing works
“Designing a specific learning application based on the network can increase the output of knowledge points, but it may not be able to respond promptly to the personalized user query input [16].”
Suárez-García, R. X., Chavez-Castañeda, Q., Orrico-Pérez, R., Valencia-Marin, S., Castañeda-Ramírez, A. E., Quiñones-Lara, E., Ramos-Cortés, C. A., Gaytán-Gómez, A. M., Cortés-Rodríguez, J., Jarquín-Ramírez, J., et al. (2025) · European Journal of Investigation in Health, Psychology and Education · doi:10.3390/ejihpe15080152 · Cited by 18 · see citing works
“These face-to-face simulations are effective but resource-intensive, requiring extensive time, personnel, and coordination, which inherently limits their frequency and scalability (Skryd & Lawrence, 2024).”
Scherr, R., Spina, A., Dao, A., Andalib, S., Halaseh, F. F., Blair, S., Wiechmann, W., & Rivera, R. (2025) · JMIR Formative Research · doi:10.2196/66478 · Cited by 10 · see citing works
Quantum leap in medical mentorship: exploring ChatGPT’s transition from textbooks to terabytes
Chokkakula, S., Chong, S., Yang, B., Jiang, H., Yu, J., Han, R., Attitalla, I. H., Yin, C., & Zhang, S. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1517981 · Cited by 12 · see citing works
Fairness in AI-driven oncology: investigating racial and gender biases in large language models
Agrawal, A. (2024) · Cureus · doi:10.7759/cureus.69541 · Cited by 10 · see citing works
- Authors
- Skryd, A., & Lawrence, K.
- Affiliations
- NYU Langone Health (US)
- New York University (US)
- Corresponding author
- Skryd, Anthony
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-05-08
- Volume
- 8
- Pages
- e51346
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC11112466
- OpenAlex
- W4393061716
- Cited by
- 63
- References
- 7
- Citations by year
- 2026: 22 · 2025: 32 · 2024: 9
- FWCI
- 5.89
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Keywords
- ChatGPT, large language models, medical education, clinical decision-making, ward-based teaching, general internal medicine, differential diagnosis, misinformation, ethics of AI in healthcare, codes of conduct, clinical documentation, continuing medical education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
