The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Harnessing the potential of large language models in medical education: promise and pitfalls
Open accessAbstract
Objectives: To provide balanced consideration of the opportunities and challenges associated with integrating Large Language Models (LLMs) throughout the medical school continuum. Process: Narrative review of published literature contextualized by current reports of LLM application in medical education. Conclusions: LLMs like OpenAI’s ChatGPT can potentially revolutionize traditional teaching methodologies. LLMs offer several potential advantages to students, including direct access to vast information, facilitation of personalized learning experiences, and enhancement of clinical skills development. For faculty and instructors, LLMs can facilitate innovative approaches to teaching complex medical concepts and fostering student engagement. Notable challenges of LLMs integration include the risk of fostering academic misconduct, inadvertent overreliance on AI, potential dilution of critical thinking skills, concerns regarding the accuracy and reliability of LLM-generated content, and the possible implications on teaching staff.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Lastly, the quality and difficulty of the questions often do not meet expectations (Artsi et al., 2024; Benítez et al., 2024) (Limit3).”
From MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedback (2025) · cites this paper as Benítez et al., 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Foundation models for generalist medical artificial intelligence
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023) · Nature · doi:10.1038/s41586-023-05881-4 · Cited by 2,028
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Lee, P., Bubeck, S., & Petro, J. (2023) · New England Journal of Medicine · doi:10.1056/nejmsr2214184 · Cited by 1,738
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540
Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty, M., & Blike, G. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0961 · Cited by 1,494
Artificial Intelligence and Machine Learning in Clinical Medicine, 2023
Haug, C. J., & Drazen, J. M. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2302038 · Cited by 1,215
Using Technology to Maintain the Education of Residents During the COVID-19 Pandemic
Chick, R. C., Clifton, G. T., Peace, K. M., Propper, B. W., Hale, D. F., Alseidi, A. A., & Vreeland, T. J. (2020) · Journal of surgical education · doi:10.1016/j.jsurg.2020.03.018 · Cited by 1,086
Outliers: The Story of Success
Jackson, F. R. (2009) · Multicultural Perspectives · doi:10.1080/15210960903028818 · Cited by 1,052
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045
Summary of ChatGPT-Related research and perspective towards the future of large language models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100017 · Cited by 855
Eysenbach, G. (2023) · JMIR Medical Education · doi:10.2196/46885 · Cited by 840
Introducing artificial intelligence training in medical education
Paranjape, K., Schinkel, M., Nannan Panday, R., Car, J., & Nanayakkara, P. (2019) · JMIR Medical Education · doi:10.2196/16048 · Cited by 634
ChatGPT - Reshaping medical education and clinical management
Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.39.2.7653 · Cited by 628
Using ChatGPT to write patient clinic letters
Ali, S. R., Dobbs, T. D., Hutchings, H. A., & Whitaker, I. S. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00048-1 · Cited by 375
Using large language models in psychology
Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., et al. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00241-5 · Cited by 369
AI-Generated Medical Advice—GPT and Beyond
Haupt, C. E., & Marks, M. (2023) · JAMA · doi:10.1001/jama.2023.5321 · Cited by 346
The Inevitable Reimagining of Medical Education
Emanuel, E. (2020) · JAMA · doi:10.1001/jama.2020.1227 · Cited by 324
Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis
Macdonald, C., Adeloye, D., Sheikh, A., & Rudan, I. (2023) · Journal of Global Health · doi:10.7189/jogh.13.01003 · Cited by 243
Could AI help you to write your next paper?
Hutson, M. (2022) · Nature · doi:10.1038/d41586-022-03479-w · Cited by 236
The Role of Large Language Models in Medical Education: Applications and Implications
Safranek, C. W., Sidamon-Eristoff, A. E., Gilson, A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/50945 · Cited by 233
McGaghie, W. C., Cohen, E. R., & Wayne, D. B. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181ffacdb · Cited by 201
ChatGPT: promise and challenges for deployment in low- and middle-income countries
Wang, X., Sanders, H. M., Liu, Y., Seang, K., Tran, B. X., Atanasov, A. G., Qiu, Y., Tang, S., Car, J., Wang, Y. X., et al. (2023) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2023.100905 · Cited by 151
Self-citationTime Spent on Clinical Documentation
Oxentenko, A. S. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.534 · Cited by 139
The self-regulated learning of medical students in the clinical environment – a scoping review
Cho, K. K., Marjadi, B., Langendyk, V., & Hu, W. (2017) · BMC Medical Education · doi:10.1186/s12909-017-0956-6 · Cited by 133
Where did the day go? A time-motion study of hospitalists
Tipping, M. D., Forth, V. E., O'Leary, K. J., Malkenson, D. M., Magill, D. B., Englert, K., & Williams, M. V. (2010) · Journal of Hospital Medicine · doi:10.1002/jhm.790 · Cited by 127
Medical student use of digital learning resources
Scott, K. M., Morris, A., & Marais, B. J. (2017) · The Clinical Teacher · doi:10.1111/tct.12630 · Cited by 115
Ng, F. Y. C., Thirunavukarasu, A. J., Cheng, H., Tan, T. F., Gutierrez, L., Lan, Y., Ong, J. C. L., Chong, Y. S., Ngiam, K. Y., Ho, D., et al. (2023) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101230 · Cited by 113
Self-citationRevisiting the Core Entrustable Professional Activities for Entering Residency
Amiel, J. M., Andriole, D. A., Biskobing, D. M., Brown, D. R., Cutrer, W. B., Emery, M. T., Mejicano, G. C., Ryan, M. S., Swails, J. L., & Wagner, D. P. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004088 · Cited by 107
Ten tips for overcoming language barriers in science
Amano, T., Rojas, C. J. R., Boum, Y., Calvo, M., & Misra, B. B. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01137-1 · Cited by 104
Medical student preferences for self‐directed study resources in gross anatomy
Choi-Lundberg, D. L., Low, T. F., Patman, P., Turner, P. A., & Sinha, S. N. (2015) · Anatomical Sciences Education · doi:10.1002/ase.1549 · Cited by 86
Research in Medical School: A Survey Evaluating Why Medical Students Take Research Years
Pathipati, A. S., & Taleghani, N. (2016) · Cureus · doi:10.7759/cureus.741 · Cited by 80
Proof of Concept: Using ChatGPT to Teach Emergency Physicians How to Break Bad News
Webb, J. J. (2023) · Cureus · doi:10.7759/cureus.38755 · Cited by 80
Varas, J., Coronel, B. V., Villagrán, I., Escalona, G., HERNANDEZ, R., Schuit, G., Espinoza, V. D., Lagos, A., Jarry, C., Neyem, A., et al. (2023) · Revista do Colégio Brasileiro de Cirurgiões · doi:10.1590/0100-6991e-20233605-en · Cited by 63
Beware of references when using ChatGPT as a source of information to write scientific articles
Sanchez-Ramos, L., Lin, L., & Romero, R. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2023.04.004 · Cited by 60
Emerging Roles of Virtual Patients in the Age of AI
Combs, C. D., & Combs, P. F. (2019) · The AMA Journal of Ethic · doi:10.1001/amajethics.2019.153 · Cited by 50
Ikonne, U. S., Campbell, A. M., Whelihan, K., Bay, R. C., & Lewis, J. H. (2018) · Journal of Osteopathic Medicine · doi:10.7556/jaoa.2018.174 · Cited by 50
Chang, C., Colón-Berlingeri, M., Mavis, B., Laird-Fick, H. S., Parker, C., & Solomon, D. (2021) · Academic Medicine · doi:10.1097/acm.0000000000003766 · Cited by 49
Working with Standardized Patients: A Primer
Bosek, M. S. D., Li, S., & Hicks, F. D. (2007) · International Journal of Nursing Education Scholarship · doi:10.2202/1548-923x.1437 · Cited by 46
Cost-Effectiveness of Using Standardized Patients to Assess Student-Pharmacist Communication Skills
Gillette, C., Stanton, R. B., Rockich‐Winston, N., Rudolph, M. J., & Anderson, H. G. (2017) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe6120 · Cited by 45
Evaluation of a Theory-Based Virtual Counseling Application in Nursing Education
Shorey, S., Ang, E. N. K., Ng, E. D., Yap, J. Y. G., Lau, L. S. T., Chui, C. K., & Chan, Y. H. (2023) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000999 · Cited by 39
The Cost of Board Examination and Preparation: An Overlooked Factor in Medical Student Debt
Bhatnagar, V., Díaz, S., & Bucur, P. A. (2019) · Cureus · doi:10.7759/cureus.4168 · Cited by 34
Desai, A. P., Hegde, A. M., & Das, D. G. (2020) · JAMA · doi:10.1001/jama.2020.2956 · Cited by 30
Communication training for pharmacy students with standard patients using artificial intelligence
Nakagawa, N., Odanaka, K., Ohara, H., & Kisara, S. (2022) · Currents in Pharmacy Teaching and Learning · doi:10.1016/j.cptl.2022.06.021 · Cited by 30
Kelly, M., & Murphy, A. W. (2004) · Medical Teacher · doi:10.1080/01421590400005475 · Cited by 28
Documentation as composing: how medical students and residents use writing to think and learn
Bowker, D., Torti, J. M. I., & Goldszmidt, M. A. (2022) · Advances in Health Sciences Education · doi:10.1007/s10459-022-10167-x · Cited by 28
Large Language Models and Their Implications on Medical Education
Bair, H., & Norden, J. G. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005265 · Cited by 25
Prober, C. G., & Desai, S. V. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005262 · Cited by 12
Pivoting during a Pandemic: Reimagining Intern Orientation EPA-Based OSCE
Suneja, M., Thoma, K. D., Franklin, E., & Rowat, J. A. (2023) · Southern Medical Journal · doi:10.14423/smj.0000000000001525 · Cited by 4
Nelson, G., Swenson, J., Kozmenko, V., & Thompson, P. (2022) · Aesculapius · doi:10.70657/ahsm.v3.2022.01 · Cited by 3
Flagged first, then by citations
Research works printed but not linked: 2
- Murphy, 2021, Medical Schools Ponder Move to Shorter Pre-Clinical Clerkship Curriculum
- Association of American Medical Colleges, 2014, Core Entrustable Professional Activities for Entering Residency Curriculum Developers’ Guide
Cited by
Sorted by citations
Large Language Models in Healthcare and Medical Applications: A Review
Maity, S., & Saikia, M. J. (2025) · Bioengineering · doi:10.3390/bioengineering12060631 · Cited by 249
Vrdoljak, J., Boban, Z., Vilović, M., Kumrić, M., & Božić, J. (2025) · Healthcare · doi:10.3390/healthcare13060603 · Cited by 202
Kıyak, Y. S., & Emekli, E. (2024) · Postgraduate Medical Journal · doi:10.1093/postmj/qgae065 · Cited by 169
Large language models in biomedicine and health: current research landscape and future directions
Lu, Z., Peng, Y., Cohen, T., Ghassemi, M., Weng, C., & Tian, S. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae202 · Cited by 92
Mess, S. A., Mackey, A., & Yarowsky, D. (2025) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000006450 · Cited by 84
Metadata
- Authors
- Benítez, T. M., Xu, Y., Boudreau, J. D., Kow, A. W. C., Bello, F., Van Phuoc, L., Wang, X., Sun, X., Leung, G. K.-K., Lan, Y., Wang, Y., Cheng, D., Tham, Y.-C., Wong, T. Y., & Chung, K. C.
- Affiliations
- University of Michigan (US)
- Tsinghua University (CN)
- McGill University (CA)
- National University of Singapore (SG)
- College of Health Sciences, VinUniversity , Hanoi, 100000, Vietnam
- Beihang University (CN)
- Shanghai Jiao Tong University (CN)
- Department of Surgery, School of Clinical Medicine, LKS Faculty of Medicine, The University of Hong Kong, Queen Mary Hospital , Hong Kong, 999077, China
- Beijing Institute of Ophthalmology, Beijing Tongren Hospital, Beijing Ophthalmology and Visual Sciences Key Laboratory, Capital University of Medical Science , Beijing, 100730, China
- Chinese University of Hong Kong, Shenzhen (CN)
- Singapore National Eye Center (SG)
- Corresponding author
- Tham, Yih-Chung
- Chung, Kevin C
- ORCID
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-01-24
- Volume
- 31
- Issue
- 3
- Pages
- 776-783
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC10873781
- OpenAlex
- W4391225179
- Cited by
- 100
- References
- 51
- Citations by year
- 2026: 25 · 2025: 55 · 2024: 19
- FWCI
- 2.26
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion, other-oa)
- PubMed (publishedVersion)
- The HKU Scholars Hub (University of Hong Kong) (submittedVersion, cc-by-nc-nd)
- Funders
- National Key Research and Development Program of China (2022YFC2502802)
- National Natural Science Foundation of China (8238810007)
- National Institutes of Health
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › Innovations in Medical Education
- Keywords
- large language models, ChatGPT, medical education, AI in teaching, personalized learning, clinical skills development, academic misconduct, overreliance on AI, critical thinking, medical curriculum
- MeSH terms
- Clinical Competence, Education, Medical, Humans, Language, Learning, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
