Large Language Models and Medical Education: Preparing for a Rapid Transformation in How Trainees Will Learn to be Doctors
Open accessAbstract
Artificial intelligence has the potential to revolutionize health care but has yet to be widely implemented. In part, this may be because, to date, we have focused on easily predicted rather than easily actionable problems. Large language models (LLMs) represent a paradigm shift in our approach to artificial intelligence because they are easily accessible and already being tested by frontline clinicians, who are rapidly identifying possible use cases. LLMs in health care have the potential to reduce clerical work, bridge gaps in patient education, and more. As we enter this era of healthcare delivery, LLMs will present both opportunities and challenges in medical education. Future models should be developed to support trainees to develop skills in clinical reasoning, encourage evidence-based medicine, and offer case-based training opportunities. LLMs may also change what we continue teaching trainees with regard to clinical documentation. Finally, trainees can help us train and develop the LLMs of the future as we consider the best ways to incorporate LLMs into medical education. Ready or not, LLMs will soon be integrated into various aspects of clinical practice, and we must work closely with students and educators to make sure these models are also built with trainees in mind to responsibly chaperone medical education into the next era.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The importance of equipping future doctors with key AI skills has increased.
“Moreover, the importance of equipping future doctors with key AI skills has increased, leading to efforts to incorporate AI literacy into undergraduate medical education (UGME) [2–4].”
From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · see citing works
Guyatt, G. H. (1992) · JAMA · doi:10.1001/jama.1992.03490170092032 · Cited by 4,447 · see citing works
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · see citing works
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Shanafelt, T. D., Boone, S., Tan, L., Dyrbye, L. N., Sotile, W., Satele, D., West, C. P., Sloan, J., & Oreskovich, M. R. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2012.3199 · Cited by 3,439 · see citing works
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136 · see citing works
Physician burnout: contributors, consequences and solutions
West, C. P., Dyrbye, L. N., & Shanafelt, T. D. (2018) · Journal of Internal Medicine · doi:10.1111/joim.12752 · Cited by 2,768 · see citing works
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738 · see citing works
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022) · doi:10.1101/2022.12.23.521610 · Cited by 1,099 · see citing works
Progress in evidence-based medicine: a quarter century on
Djulbegovic, B., & Guyatt, G. H. (2017) · The Lancet · doi:10.1016/s0140-6736(16)31592-6 · Cited by 1,022 · see citing works
ChatGPT and other large language models are double-edged swords
Shen, Y., Heacock, L., Elias, J., Hentel, K. D., Reig, B., Shih, G., & Moy, L. (2023) · Radiology · doi:10.1148/radiol.230163 · Cited by 1,008 · see citing works
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
U.S. Department of Health and Human (2015) · Journal of Pain & Palliative Care Pharmacotherapy · doi:10.3109/15360288.2015.1037530 · Cited by 891 · see citing works
A large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847 · see citing works
Tools such as ChatGPT threaten transparent science; here are our ground rules for their use
(2023) · Nature · doi:10.1038/d41586-023-00191-1 · Cited by 605 · see citing works
Shifting machine learning for healthcare from development to deployment and from models to data
Zhang, A., Xing, L., Zou, J., & Wu, J. C. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00898-y · Cited by 420 · see citing works
Schoen, C., Osborn, R., Huynh, P. T., Doty, M. M., Zapert, K., Peugh, J., & Davis, K. C. (2005) · Health Affairs · doi:10.1377/hlthaff.w5.509 · Cited by 375 · see citing works
Ryan, R. E., Santesso, N., Lowe, D., Hill, S., Grimshaw, J. M., Prictor, M., Kaufman, C., Cowie, G., & Taylor, M. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007768.pub3 · Cited by 273 · see citing works
Large language models associate Muslims with violence
Abid, A., Farooqi, M., & Zou, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00359-2 · Cited by 193 · see citing works
Bridging the implementation gap of machine learning in healthcare
Seneviratne, M. G., Shah, N. H., & Chu, L. (2020) · BMJ Innovations · doi:10.1136/bmjinnov-2019-000359 · Cited by 173 · see citing works
Electronic health records and clinician burnout: A story of three eras
Johnson, K. B., Neuss, M. J., & Detmer, D. E. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa274 · Cited by 97 · see citing works
Preparing Medical Students for Clerkships: A Descriptive Analysis of Transition Courses
Poncelet, A. N., & OʼBrien, B. C. (2008) · Academic Medicine · doi:10.1097/acm.0b013e31816be675 · Cited by 90 · see citing works
Gabrielson, A. T., Odisho, A. Y., & Canes, D. (2023) · Journal of Urology · doi:10.1097/ju.0000000000003383 · Cited by 76 · see citing works
Predicting 30-day hospital readmissions using artificial neural networks with medical code embedding
Liu, W., Singh, K., Ryan, A. M., Sukul, D., Mahmoudi, E., Waljee, A. K., Stansbury, C. M., Zhu, J., & Nallamothu, B. K. (2020) · PLoS ONE · doi:10.1371/journal.pone.0221606 · Cited by 62 · see citing works
Predictive model-based interventions to reduce outpatient no-shows: a rapid systematic review
Oikonomidi, T., Norman, G., McGarrigle, L., Stokes, J., van der Veer, S. N., & Dowding, D. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac242 · Cited by 45 · see citing works
Assessing the Quality of the After-Visit Summary (AVS) in a Primary-Care Clinic
Mir, T. H., Osayande, A. S., Kone, K., Bridges, K., & Day, P. G. (2019) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2019.01.180055 · Cited by 11 · see citing works
Post-Visit Patient Understanding About Newly Prescribed Medications
Ho, T., Campos, B. S., & Tarn, D. M. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-020-06540-4 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [6]Chui M Hall B Mayhew H Singla A Sukharevsky A McKinsey and Co 2022 https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2022-and-a-half-decade-in-review
- [9]Murray SG Wachter RM Cucina RJ Discrimination by artificial intelligence in a commercial electronic health record—a case study Health Aff Forefr https://www.healthaffairs.org/content/forefront/discrimination-artificial-intelligence-commercial-electronic-health-record-case-study
- [27]Lucy L Bamman D Association for Computational Linguistics 2021 48 55 https://aclanthology.org/2021.nuse-1.5
- [32]International Conference on Machine Learning (ICML) 2023 https://icml.cc/Conferences/2023/llm-policy
Cited by
Sorted by citations
Benefits and Harms of Large Language Models in Digital Mental Health
De Choudhury, M., Pendse, S. R., & Kumar, N. (2023) · doi:10.31234/osf.io/y8ax9 · Cited by 78 · see citing works
Medical Ethics of Large Language Models in Medicine
Ong, J. C. L., Chang, Y., Wasswa, W., Butte, A. J., Shah, N. H., Chew, L. S. T., Liu, N., Doshi‐Velez, F., Lu, W., Savulescu, J., et al. (2024) · NEJM AI · doi:10.1056/aira2400038 · Cited by 59 · see citing works
Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis
Yu, H., Fan, L., Li, L., Zhou, J., Ma, Z., Xian, L., Hua, W., He, S., Jin, M., Zhang, Y., et al. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-024-00171-8 · Cited by 69 · see citing works
Artificial intelligence in undergraduate medical education: an updated scoping review
Simoni, J., Urtubia-Fernandez, J., Mengual, E., Simoni, D. A., Royo, M., Egaña-Yin, D., Hertog, O. L. A., López-Ortiz, L., Muñoz-Tomás, A., Santiago-Martínez, P., et al. (2025) · BMC Medical Education · doi:10.1186/s12909-025-08188-2 · Cited by 62 · see citing works
“Moreover, the importance of equipping future doctors with key AI skills has increased, leading to efforts to incorporate AI literacy into undergraduate medical education (UGME) [2–4].”
Generative Artificial Intelligence in Anatomic Pathology
Brodsky, V., Ullah, E., Bychkov, A., Song, A. H., Walk, E. E., Louis, P., Rasool, G., Singh, R. S., Mahmood, F., Bui, M. M., et al. (2025) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2024-0215-ra · Cited by 48 · see citing works
Liu, F., Chang, X., Zhu, Q., Huang, Y., Li, Y., & Wang, H. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06232-1 · Cited by 35 · see citing works
Simulacrum of Stories: Examining Large Language Models as Qualitative Research Participants
Kapania, S., Agnew, W. S., Eslami, M., Heidari, H., & Fox, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713220 · Cited by 37 · see citing works
Giacobbe, D. R., Marelli, C., Manna, B. L., Padua, D., Malva, A., Guastavino, S., Signori, A., Mora, S., Rosso, N., Campi, C., et al. (2025) · npj Antimicrobials and Resistance · doi:10.1038/s44259-025-00084-5 · Cited by 29 · see citing works
Assessing ChatGPT’s Responses to Otolaryngology Patient Questions
Carnino, J. M., Pellegrini, W. R., Willis, M., Cohen, M. B., Paz‐Lansberg, M., Davis, E. M., Grillone, G. A., & Levi, J. R. (2024) · Annals of Otology Rhinology & Laryngology · doi:10.1177/00034894241249621 · Cited by 16 · see citing works
Rasouli, S., Alkurdi, D., & Jia, B. (2024) · medRxiv · doi:10.1101/2024.07.25.24311022 · Cited by 15 · see citing works
Salehin, I., Sajib, M. T. A., Badhon, N. H., Rifat, M. S. H., Amin, N., & Moon, N. N. (2025) · Engineering Reports · doi:10.1002/eng2.70365 · Cited by 9 · see citing works
Thaker, N. G., Redjal, N., Loaiza‐Bonilla, A., Penberthy, D. R., Showalter, T., Choudhri, A., Williamson, S. K., Thaker, G. H., Shah, C., Ward, M. C., et al. (2024) · AI in Precision Oncology · doi:10.1089/aipo.2023.0007 · Cited by 8 · see citing works
Chen, C. J., Bilolikar, V. K., VanNest, D., Raphael, J. S., & Shaffer, G. (2024) · Medicine Advances · doi:10.1002/med4.77 · Cited by 7 · see citing works
Bridging LLMs, Education, and Sustainability: Guiding Students in Local Community Initiatives
Jurišević, N., Nikolić, N., Nemś, A., Gordić, D., Rakić, N., Končalović, D., & Kocsis, D. L. (2025) · Sustainability · doi:10.3390/su172210148 · Cited by 7 · see citing works
The Role of Natural Language Processing in Graduate Medical Education: A Scoping Review
Janumpally, R. (2025) · Cureus · doi:10.7759/cureus.81078 · Cited by 6 · see citing works
Stawicki, S. P., Martinez-Baladejo, M. T., & Ng-Pellegrino, A. (2023) · International Journal of Academic Medicine · doi:10.4103/ijam.ijam_63_23 · Cited by 6 · see citing works
Ratnagandhi, A., Godavarthy, P., Gnaneswaran, M., Lim, B., & Vittalraj, R. (2025) · Anesthesia Research · doi:10.3390/anesthres2010004 · Cited by 5 · see citing works
Weerathunge, T., Jayalal, S., & Wijayasiriwardhane, K. (2025) · International Conference on Advanced Research in Computing (ICARC) · doi:10.1109/icarc64760.2025.10963313 · Cited by 4 · see citing works
Etli, D. (2024) · Journal of Quality in Health Care & Economics · doi:10.23880/jqhe-16000361 · Cited by 4 · see citing works
Assessing risk of bias of cohort studies with large language models
Xia, D., Lai, H., Zhao, W., Huang, J., Liu, J., Ye, Z., Liu, J., Sun, M., Hou, L., Pan, B., et al. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10028 · Cited by 2 · see citing works
Implications of Large Language Models in Medical EducationPreprint
Fan, K. S. (2024) · doi:10.62594/brmo4385 · Cited by 2 · see citing works
(2024) · International Journal of Educational Curriculum Management and Research · doi:10.38007/ijecmr.2024.050117 · Cited by 1 · see citing works
In Reply to Daungsupawong and Wiwanitkit
Dennstädt, F., Hastings, J., Putora, P. M., Vu, E., Fischer, G. F., Süveg, K., Glatzer, M., Riggenbach, E., Hà, H.-L., & Čihorić, N. (2024) · Advances in Radiation Oncology · doi:10.1016/j.adro.2024.101511
Procedural Guideline Execution Training Improves LLM Performance in Rule-Based Clinical TasksPreprint
Boonmee, P. (2025) · doi:10.31224/4544
Enseñando bajo presión: los retos de educar en cuidado intensivo
Yepes, A., Moreno-Araque, J. L., & Castaño, M. (2025) · Acta Colombiana de Cuidado Intensivo · doi:10.1016/j.acci.2025.04.004
McMaster, B., Yanamala, S. R., Alshaibi, R. R., Okoroha, K. R., Hevesi, M., Saris, D. B. F., Krych, A. J., & Tagliero, A. J. (2026) · Journal of ISAKOS Joint Disorders & Orthopaedic Sports Medicine · doi:10.1016/j.jisako.2026.101085
García-López, I. M., Espinosa, J. M. M., & Soledad, M. (2026) · IEEE Revista Iberoamericana de Tecnologias del Aprendizaje · doi:10.1109/rita.2026.3701046
Zhang, J., Liu, Y., & Wang, B. (2026) · Medicine · doi:10.1097/md.0000000000049849
Andrei, S., Giet, T., Belouard, A., Ştefan, M. G., Popescu, M., Tanaka, S., Montravers, P., & Gouél‐Cheron, A. (2026) · JMIR Formative Research · doi:10.2196/95859
Generative artificial intelligence technologies in pathology
Ullah, E. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40383-5.00013-3
- Authors
- Ravi, A., Neinstein, A., & Murray, S. G.
- Affiliations
- Department of Medicine
- University of California, San Francisco (US)
- Corresponding author
- Ravi, Akshay
- Venue
- ATS Scholar
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-06-14
- Volume
- 4
- Issue
- 3
- Pages
- 282-292
- ISSN
- 2690-7097 (ISSN-L)
- PMC
- PMC10547030
- OpenAlex
- W4380627488
- Cited by
- 31
- References
- 26
- Citations by year
- 2026: 5 · 2025: 12 · 2024: 12 · 2023: 2
- FWCI
- 1.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- ATS Scholar (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Keywords
- large language models, medical education, artificial intelligence in healthcare, clinical reasoning, evidence-based medicine, case-based training, clinical documentation, trainee engagement, LLM training data, patient education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
