The DOI matches its Crossref record; 303 of the 308 references it cites were found in a registry or an index; 12 of the 157 sources its claims cite were checked for support.
A scoping review of artificial intelligence in medical education: BEME Guide No. 84
Open accessAbstract
BACKGROUND: Artificial Intelligence (AI) is rapidly transforming healthcare, and there is a critical need for a nuanced understanding of how AI is reshaping teaching, learning, and educational practice in medical education. This review aimed to map the literature regarding AI applications in medical education, core areas of findings, potential candidates for formal systematic review and gaps for future research. METHODS: This rapid scoping review, conducted over 16 weeks, employed Arksey and O'Malley's framework and adhered to STORIES and BEME guidelines. A systematic and comprehensive search across PubMed/MEDLINE, EMBASE, and MedEdPublish was conducted without date or language restrictions. Publications included in the review spanned undergraduate, graduate, and continuing medical education, encompassing both original studies and perspective pieces. Data were charted by multiple author pairs and synthesized into various thematic maps and charts, ensuring a broad and detailed representation of the current landscape. RESULTS: The review synthesized 278 publications, with a majority (68%) from North American and European regions. The studies covered diverse AI applications in medical education, such as AI for admissions, teaching, assessment, and clinical reasoning. The review highlighted AI's varied roles, from augmenting traditional educational methods to introducing innovative practices, and underscores the urgent need for ethical guidelines in AI's application in medical education. CONCLUSION: The current literature has been charted. The findings underscore the need for ongoing research to explore uncharted areas and address potential risks associated with AI use in medical education. This work serves as a foundational resource for educators, policymakers, and researchers in navigating AI's evolving role in medical education. A framework to support future high utility reporting is proposed, the FACETS framework.
Overview
CoChat Summary
TL;DR
This BEME scoping review maps 278 publications on artificial intelligence in medical education, finding a publication surge since 2018 and diverse applications across admissions, teaching, assessment, clinical reasoning, and large language model exam performance. It proposes the FACETS framework and calls for ethical guidelines and further research.
Key findings
- The review included 278 publications: 191 (68.7%) articles/innovations and 87 (31.3%) perspectives. Geographic distribution was predominantly Anglo-North American (n = 138, 49.6%) and European (n = 51, 18.4%); no studies were identified from Africa. The earliest publication was in 1992, with a surge from 2018 onward and 114 publications as of August 2023.
- AI applications included assessment (n = 50, 26.2%), knowledge/attitudes about AI (n = 51, 26.7%), teaching about AI or AI-augmented instruction (n = 34, 17.8%), LLM performance on medical exams (n = 32, 16.8%), clinical reasoning (n = 13, 6.8%), admissions/selection (n = 7, 3.6%), and automation of case/procedure logs (n = 4, 2.1%).
- Technical use varied: AI was treated as a general concept without computation in 65 studies (34%); traditional ML in 41 (21.5%), deep learning in 81 (42.4%), NLP in 78 (40.8%), and data mining in 4 (2.1%). Under SAMR, AI was substitution in 36 (18.8%), augmentation in 34 (17.8%), modification in 10 (5.2%), redefinition in 11 (5.8%), and not applicable in 100 (52.4%).
- LLM performance was variable: models scored at or above passing thresholds on some local, national, and specialty licensure exams but performed modestly or poorly on others; newer models generally outperformed older ones, though not universally, and performance was often better on English or translated questions (e.g., 22% accuracy in French vs. 73% in Italian). The review proposes the FACETS framework for future reporting.
Methods
- Rapid scoping review conducted over 16 weeks, guided by Arksey and O’Malley’s framework refined by Levac et al., and reported per STORIES, BEME guidance, and PRISMA-ScR.
- Searched PubMed/MEDLINE, EMBASE, and MedEdPublish without date or language restrictions. The database search yielded 2,123 publications; after duplicate removal, 2,019 remained; title/abstract screening narrowed to 310; full-text review excluded 70; 241 were included; hand-searching added 37, yielding 278 total. Screening was by author pairs with discrepancies resolved by consensus or third author; inter-rater reliability was j = 0.86.
- Data were charted by multiple author pairs using a piloted extraction form. Articles/innovations and perspectives were charted separately and synthesized into thematic maps, charts, and narrative accounts; perspectives were categorized into five groups.
Limitations
- The paper states that the scoping tradition does not synthesize evidence; there was subjectivity in categorizing studies; terminology was inconsistent; the search was iterated and studies may have been missed; inclusion of perspective articles may be perceived as diluting empirical strength; and the sixth stage of the Arksey and O’Malley framework (external expert consultation) was not performed, though author-team experts fulfilled this role.
- The text also makes evident that the rapid 16-week timeframe and evolving AI literature may have limited comprehensiveness, and no studies were identified from Africa, indicating a geographic evidence gap.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 7 Not valid · 81 Not checked · 98 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Other regions produced only a handful of studies and no studies were identified from Africa.”
§ A scoping review of artificial intelligence in medical education: BEME Guide No. 84 > Overview
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A substantial portion (n ¼ 51, 26.7%) investigated knowledge or attitudes about AI in medicine (A109-A159), and the performance of large language models (LLMs) on medical questions or exams was evaluated in 32 (16.8%) of the studies (A160-A191.”
§ A scoping review of artificial intelligence in medical education: BEME Guide No. 84 > Articles and innovations included in the review
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most included publications fall under the category of articles or innovations (n ¼ 191, 68.7%), as defined above, with the remainder being perspectives (n ¼ 87, 31.3%).”
§ A scoping review of artificial intelligence in medical education: BEME Guide No. 84 > Overview
plainn = 191- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Another study (A3) compared faculty scoring of medical student performance evaluations (MSPEs) to a machine learning (ML) model's scores.The ML model was unable to successfully rank MSPEs using commercially available sentiment analysis programs, largely because all MSPEs were highly positive.”
§ A scoping review of artificial intelligence in medical education: BEME Guide No. 84 > AI use in admissions and selection for medical school and residency
What the sources say
- Not matchedcites A3
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“A total of thirty-three articles addressed teaching about or instruction with AI (A8-41).”
§ A scoping review of artificial intelligence in medical education: BEME Guide No. 84 > Teaching about AI and AI-augmented instruction
What the sources say
- Not matchedcites A8-41
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
55 more claims
Create a free account to see all 195 claims and every source verdict.
- Probably not a substantial claim. 86
- It may state two separate claims as one. 84
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“In fact, 161 reports on the use of AI in UGME were published from 2023 until April 2024, none of which were part of Gordon et al.’s analysis.”
From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [8]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concurrently, the scholarly literature concerning its application in medical education has expanded substantially, thereby fueling the demand for its integration into medical curricula [8, 9].”
From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [8]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There is an increasing recognition that AI-related skills—such as computational thinking, data ethics, and interdisciplinary collaboration—cannot be taught as optional extras. They must be integrated into the fundamental structure of medical training [8–10].”
From Institutionalizing convergence education for medical artificial intelligence (2025) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reviews and guidance documents on AI implementation in medical education consistently point to fragmented efforts and limited evidence.”
From Principles for Responsible AI in Health Professions Education, Research, and Care: Health CARE-AI (Contextual, Accountable, Responsible, and Equitable Artificial Intelligence) Framework Delphi Consensus Study (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Despite enthusiasm about integrating AI in HPE and the increased amount of research on this topic, there is a lack of basic literacy and understanding of AI technology among HPE students [6, 17–20].”
From Shaping the future: perspectives on the Integration of Artificial Intelligence in health profession education: a multi-country survey (2024) · cites this paper as [18]
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“East Asian (n ¼ 22, 7.9%), Middle Eastern (n ¼ 22, 7.6%), and South Asian (n ¼ 17, 6.1%)”
§ A scoping review of artificial intelligence in medical education: BEME Guide No. 84 > Overview
Reported 7.6 · recomputed 7.913669
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A scoping review of artificial intelligence in medical education: BEME Guide No. 84 > Articles and innovations included in the review
“34 (17.8%)focused on teaching about AI or AI-augmented instruction (A8-A41)”
§ A scoping review of artificial intelligence in medical education: BEME Guide No. 84 > Teaching about AI and AI-augmented instruction
“A total of thirty-three articles addressed teaching about or instruction with AI (A8-41).Of these, 12 focused on AI as a learning tool, 11 on curriculum development, three on AI use for personalized education, five on the use of AI in medical simulation, and two on AI educational competencies.”
Citations
Cites
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565 · cited 1×
“We also conformed to the PRISMA-ScR reporting standards (Tricco et al. 2018), drawing on support from Peters et al. (2020, 2022) to offset the absence of specialized reporting standards for scoping reviews in health professions education.”
MethodScoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742 · cited 2×
“Our methodology was guided by Arksey and O ’ Malley ’ s (2005) scoping review framework, which was refined by Levac et al. (2010).”
BackgroundMethodScoping studies: advancing the methodology
Levac, D., Colquhoun, H., & O'Brien, K. K. (2010) · Implementation Science · doi:10.1186/1748-5908-5-69 · Cited by 15,268 · cited 3×
“Our methodology was guided by Arksey and O ’ Malley ’ s (2005) scoping review framework, which was refined by Levac et al. (2010).”
BackgroundMethodUpdated methodological guidance for the conduct of scoping reviews
Peters, M. D. J., Marnie, C., Tricco, A. C., Pollock, D., Munn, Z., Alexander, L., McInerney, P., Godfrey, C. M., & Khalil, H. (2020) · JBI Evidence Synthesis · doi:10.11124/jbies-20-00167 · Cited by 9,046 · cited 1×
“We also conformed to the PRISMA-ScR reporting standards (Tricco et al. 2018), drawing on support from Peters et al. (2020, 2022) to offset the absence of specialized reporting standards for scoping reviews in health professions education.”
MethodKung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning
Clarke, V., & Braun, V. (2013) · UWE Research Repository (UWE Bristol) · Cited by 2,786
Cultural Shock: Adjustment to New Cultural Environments
Oberg, K. (1960) · Practical Anthropology · doi:10.1177/009182966000700405 · Cited by 2,706
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2020) · Applied Sciences · doi:10.48550/arxiv.2009.13081 · Cited by 2,339
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
Research works printed but not linked: 3
- 10.1016/s1532-0464.(02)00004-7 · doi:10.1016/s1532-0464.(02)00004-7
- 10.1016/s0933-3657.(97)00054-7 · doi:10.1016/s0933-3657.(97)00054-7
- 10.1016/s2589-7500.(20)30023-6 · doi:10.1016/s2589-7500.(20)30023-6
Cited by
Feigerlova, E., Hani, H., & Hothersall-Davies, E. (2025) · BMC Medical Education · doi:10.1186/s12909-025-06719-5 · Cited by 166
Mapping the use of artificial intelligence in medical education: a scoping review
Rincón, E. H. H., Jimenez, D., Aguilar, L. A. C., Flórez, J. M. P., Tapia, Á. E. R., & Peñuela, C. L. J. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07089-8 · Cited by 145
“…and Natural Language Processing to grade and provide feedback on clinical cases, and AI-powered Chatbots that help students review medical literature, prepare for exams like the United States Medical Licensing Examination (USMLE), and develop critical thinking and communication skills [14].”
Ahsan, Z. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07744-0 · Cited by 141
“Techniques like machine learning, natural language processing (NLP), neural networks, and large language models (LLMs) are being utilized to create virtual patient simulations, automate assessments, provide personalized learning feedback, and predict learner outcomes [2].”
Pham, T. D., Karunaratne, N., Exintaris, B., Liu, D., Lay, T., Yuriev, E., & Lim, A. (2025) · Medical Education · doi:10.1111/medu.15746 · Cited by 100
“2,18,19 22 GenAI has disrupted traditional educational approaches, necessitating their reassessment to ensure effective and responsible use of GenAI.”
Duan, S., Liu, C., Rong, T., Zhao, Y., & Liu, B. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07177-9 · Cited by 85
Metadata
- Authors
- Gordon, M., Daniel, M., Ajiboye, A., Uraiby, H., Xu, N. Y., Bartlett, R., Hanson, J., Haas, M., Spadafore, M., Grafton-Clarke, C., Gasiea, R. Y., Michie, C., Corral, J., Kwan, B., Dolmans, D., & Thammasitboon, S.
- Affiliations
- University of Lancashire (GB)
- University of California San Diego (US)
- University Hospitals of Leicester NHS Trust (GB)
- Washington University in St. Louis (US)
- University of Michigan (US)
- University of East Anglia
- Blackpool Teaching Hospitals NHS Foundation Trust (GB)
- University of Nevada, Reno (US)
- Maastricht University (NL)
- Baylor College of Medicine (US)
- Corresponding author
- Gordon, Morris
- ORCID
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2024-02-29
- Volume
- 46
- Issue
- 4
- Pages
- 446-470
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W4392769984
- Semantic Scholar
- a222625d335b1ff3beda12ff8d029df8702c9861
- Cited by
- 679
- References
- 306
- Citations by year
- 2026: 395 · 2025: 238 · 2024: 43
- FWCI
- 12.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Medical Teacher (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- CLOK (University of Central Lancashire) (submittedVersion)
- Digital Commons-TMC (Texas Medical Center) (submittedVersion)
- Digital Commons@Becker (Washington University School of Medicine) (submittedVersion, cc-by-nc-nd)
- Research Publications (Maastricht University) (submittedVersion, cc-by-nc-nd)
- eScholarship (California Digital Library) (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- artificial intelligence, medical education, AI-assisted assessment, clinical reasoning, AI ethics, reporting framework, continuing medical education, undergraduate medical education, graduate medical education
- MeSH terms
- Artificial Intelligence, Education, Medical, Humans, Learning, Teaching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A scoping review of artificial intelligence in medical education: BEME Guide No. 84. https://app.cochat.ai/papers/10.1080/0142159x.2024.2314198
- The paper
- Gordon, M., Daniel, M., Ajiboye, A., Uraiby, H., Xu, N. Y., Bartlett, R., Hanson, J., Haas, M., Spadafore, M., Grafton-Clarke, C., Gasiea, R. Y., Michie, C., Corral, J., Kwan, B., Dolmans, D., & Thammasitboon, S. (2024). A scoping review of artificial intelligence in medical education: BEME Guide No. 84. Medical Teacher. https://doi.org/10.1080/0142159x.2024.2314198
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.