Artificial Intelligence Education and Tools for Medical and Health Informatics Students: Systematic Review
Open accessPublished version of a preprint
The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index.
Abstract
Background: The use of artificial intelligence (AI) in medicine will generate numerous application possibilities to improve patient care, provide real-time data analytics, and enable continuous patient monitoring. Clinicians and health informaticians should become familiar with machine learning and deep learning. Additionally, they should have a strong background in data analytics and data visualization to use, evaluate, and develop AI applications in clinical practice. Objective: The main objective of this study was to evaluate the current state of AI training and the use of AI tools to enhance the learning experience. Methods: A comprehensive systematic review was conducted to analyze the use of AI in medical and health informatics education, and to evaluate existing AI training practices. PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols) guidelines were followed. The studies that focused on the use of AI tools to enhance medical education and the studies that investigated teaching AI as a new competency were categorized separately to evaluate recent developments. Results: This systematic review revealed that recent publications recommend the integration of AI training into medical and health informatics curricula. Conclusions: To the best of our knowledge, this is the first systematic review exploring the current state of AI education in both medicine and health informatics. Since AI curricula have not been standardized and competencies have not been determined, a framework for specialized AI training in medical and health informatics education is proposed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“While numerous authors have called for structured and intentional learning programs, to date, there has been no published framework to guide teaching, learning, and assessing health care students and practitioners in this emerging and transformative domain [7,9-12].”
From Considering Clinician Competencies for the Implementation of Artificial Intelligence–Based Tools in Health Care: Findings From a Scoping Review (2022) · cites this paper as [11]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Overall, this includes learning the AI terminology [13,37,45], strengths and limitations of AI [12,13,16,37,42], risks of AI [12,13,17,37,39,42], controlling for bias [42], critical appraisal AI literature and tools [12,13,34,37,39,45,46], and shared decision-making [12,13,37,41,45].”
From An evidence-based approach to artificial intelligence education for medical students: A systematic review (2023) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Eleven studies suggest that physicians should understand the role of data science in AI [12,13,16,37,39,41,42,45].”
From An evidence-based approach to artificial intelligence education for medical students: A systematic review (2023) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In particular, knowledge related to data stewardship [12,13,37,41], data preprocessing and acquisition [13], standardization [13], data analytics [13,17,39], health data infrastructure [45], programming [39,41], and big data [13,45], may help physicians appreciate the obstacles faced by data scientists and improve the multidisciplinary collaboration [13,39].”
From An evidence-based approach to artificial intelligence education for medical students: A systematic review (2023) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Nine studies discuss the need for a fundamental understanding of AI ethics to improve the adoption of AI in medicine and ensure its safe use [12,13,16,37,39,45].”
From An evidence-based approach to artificial intelligence education for medical students: A systematic review (2023) · cites this paper as [39]
Experimental — accuracy not yet measured
20 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Exploring the impact of artificial intelligence on teaching and learning in higher education
Popenici, S. A. D., & Kerr, S. (2017) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-017-0062-8 · Cited by 2,019 · cited 1×
“Even though there are several studies that explored how AI algorithms were helping enhance education [8-12], the number of peer-reviewed publications that focused on artificial intelligence education in medicine is limited (Figure 1).”
BackgroundImplementing Machine Learning in Health Care — Addressing Ethical Challenges
Char, D. S., Shah, N. H., & Magnus, D. (2018) · New England Journal of Medicine · doi:10.1056/nejmp1714229 · Cited by 1,857 · cited 1×
“The implementation of ML in health care could result in unintended challenges and biased decisions, depending on the algorithms, data sources, and methodologies used [7].”
MethodUnintended consequences of machine learning in medicine
Cabitza, F., Rasoini, R., & Gensini, G. F. (2017) · JAMA · doi:10.1001/jama.2017.7797 · Cited by 1,066
Chan, K. S., & Zary, N. (2019) · JMIR Medical Education · doi:10.2196/13930 · Cited by 877 · cited 4×
“For example, Chan and Zary [15] evaluated existing AI applications in medicine, determined that the primary reason to use AI in medical education was to provide feedback, and identified that significant challenges included the assessment of effectiveness and management of technical difficulties.”
BackgroundInfluentialMedical students' attitude towards artificial intelligence: a multicentre survey
Pinto dos Santos, D., Giese, D., Brodehl, S., Chon, S. H., Staab, W., Kleinert, R., Maintz, D., & Baeßler, B. (2019) · European Radiology · doi:10.1007/s00330-018-5601-1 · Cited by 754
Computational intelligence: a logical approach
(1998) · Choice Reviews Online · doi:10.5860/choice.35-5701 · Cited by 723 · cited 1×
“In other words, computers can mimic human intelligence using AI techniques [4].”
BackgroundIntroducing artificial intelligence training in medical education
Paranjape, K., Schinkel, M., Nannan Panday, R., Car, J., & Nanayakkara, P. (2019) · JMIR Medical Education · doi:10.2196/16048 · Cited by 634 · cited 4×
“IV 2 Introducing Artificial Intelligence Training in Medical Education Netherlands Paranjape et al, 2019 [39]”
BackgroundMethodInfluentialMedical Education Must Move From the Information Age to the Age of Artificial Intelligence
Wartman, S. A., & Combs, C. D. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002044 · Cited by 585 · cited 1×
“V2Medical Education Must Move From the Information Age to the Age of Artificial Intelligence United States Wartman and Combs, 2018 [32] JMIR Med Educ 2020 | vol. 6 | iss.”
BackgroundCanadian Association of Radiologists White Paper on Artificial Intelligence in Radiology
Tang, A., Tam, R., Cadrin-Chênevert, A., Guest, W., Chong, J., Barfett, J., Chepelev, L., Cairns, R., Mitchell, J. R., Cicero, M. D., et al. (2018) · Canadian Association of Radiologists Journal · doi:10.1016/j.carj.2018.02.002 · Cited by 522
Artificial intelligence in medical education
Masters, K. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1595557 · Cited by 469 · cited 2×
“V 2 Artificial intelligence in medical education Oman Masters, 2019 [36]”
BackgroundQuestions for artificial intelligence in health care
Maddox, T. M., Rumsfeld, J. S., & Payne, P. R. O. (2019) · JAMA · doi:10.1001/jama.2018.18932 · Cited by 385 · cited 1×
“Consequently, there is a need to determine the most appropriate application areas for AI in health care [46].”
BackgroundJust Imagine: New Paradigms for Medical Education
Mehta, N. B., Hull, A. L., Young, J. B., & Stoller, J. K. (2013) · Academic Medicine · doi:10.1097/acm.0b013e3182a36a07 · Cited by 323
Machine learning and medical education
Kolachalama, V. B., & Garg, P. S. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0061-1 · Cited by 320 · cited 11×
“The authors investigated the integration of NLP to medical This study reviewed the appli- cation of NLPd to medical IV1A Review of Natural Language Processing in Medical Education United States Chary et al, 2018 [19] education resources using published manuscripts and stated education and identified concepts from NLP…”
BackgroundInfluentialAmmenwerth, E., Demiris, G., Hasman, A., Haux, R., Hersh, W., Hovenga, E., Lun, K. C., Marin, H., Martin-Sanchez, F., Wright, G., et al. (2010) · Methods of Information in Medicine · doi:10.3414/me5119 · Cited by 319 · cited 1×
“Even though several organizations, agencies, and work groups such as the International Medical Informatics Association [47], the Commission on Accreditation for Health Informatics and Information Management Education [48,49], the Health Informatics Society of Australia [50], the TIGER (Technology Informatics Guiding E…”
BackgroundWinkler-Schwartz, A., Bissonnette, V., Mirchi, N., Ponnudurai, N., Yilmaz, R., Ledwos, N., Siyar, S., Azarnoush, H., Karlik, B., & Del Maestro, R. F. (2019) · Journal of Surgical Education · doi:10.1016/j.jsurg.2019.05.015 · Cited by 295 · cited 2×
“The authors The authors developed a checklist to assess surgical IV 1 Artificial Intelligence in Medical Education: Best Canada WinklerSchwartz et al, 2019 [14] emphasized the need to add further elements.”
MethodReimagining Medical Education in the Age of AI
(2019) · AMA Journal of Ethics · doi:10.1001/amajethics.2019.146 · Cited by 263
Nye, B. D. (2015) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0028-6 · Cited by 259 · cited 1×
“Even though there are several studies that explored how AI algorithms were helping enhance education [8-12], the number of peer-reviewed publications that focused on artificial intelligence education in medicine is limited (Figure 1).”
BackgroundTeacherbot: interventions in automated teaching
Bayne, S. (2015) · Teaching in Higher Education · doi:10.1080/13562517.2015.1020783 · Cited by 202 · cited 1×
“Even though there are several studies that explored how AI algorithms were helping enhance education [8-12], the number of peer-reviewed publications that focused on artificial intelligence education in medicine is limited (Figure 1).”
BackgroundWhat should medical students know about artificial intelligence in medicine?
Park, S. H., Do, K.-H., Kim, S., Park, J. H., & Lim, Y.-S. (2019) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2019.16.18 · Cited by 196
The Roles Of Artificial Intelligence In Education: Current Progress And Future Prospects
Rand 1700 Main St, S. M., McArthur, D., Lewis, M., Rand 1700 Main St, S. M., Bishary, M., & Rand 1700 Main St, S. M. (2005) · i-manager's Journal of Educational Technology · doi:10.26634/jet.1.4.972 · Cited by 187 · cited 1×
“Even though there are several studies that explored how AI algorithms were helping enhance education [8-12], the number of peer-reviewed publications that focused on artificial intelligence education in medicine is limited (Figure 1).”
BackgroundGods and Robots: Myths, Machines, and Ancient Dreams of Technology
Mayor, A. (2018) · Princeton University Press eBooks · doi:10.2307/j.ctvc779xn · Cited by 164
Mantas, J., Ammenwerth, E., Demiris, G., Hasman, A., Haux, R., Hersh, W., Hovenga, E. J. S., Lun, K. C., Marin, H. F., Martin-Sanchez, F. J., et al. (2011) · European Journal for Biomedical Informatics · doi:10.24105/ejbi.2011.07.2.2 · Cited by 142
Developing the Serious Games potential in nursing education
Dariel, O. P. D., Raby, T., Ravaut, F., & Rothan‐Tondeur, M. (2013) · Nurse Education Today · doi:10.1016/j.nedt.2012.12.014 · Cited by 119
A Review of Natural Language Processing in Medical Education
Chary, M. A., Parikh, S., Manini, A. F., Boyer, E. W., & Radeous, M. (2018) · Western Journal of Emergency Medicine · doi:10.5811/westjem.2018.11.39725 · Cited by 119 · cited 4×
“Another systematic review conducted by Chary et al [19] identified 30 articles that assessed the application of natural language processing (NLP) to medical education.”
BackgroundMethodInfluentialArtificial intelligence systems for complex decision-making in acute care medicine: a review
Lynn, L. A. (2019) · Patient Safety in Surgery · doi:10.1186/s13037-019-0188-2 · Cited by 104 · cited 2×
“Maturation of AI technologies changed the roles of clinicians, and novel decision-making processes in medical settings and innovative AI-based protocols have the potential to provide diagnostic and treatment decisions by analyzing complex data sets [41].”
BackgroundArtificial intelligence and medical imaging 2018: French Radiology Community white paper
(2018) · Diagnostic and Interventional Imaging · doi:10.1016/j.diii.2018.10.003 · Cited by 94
Rethinking Anatomy: How to Overcome Challenges of Medical Education’s Evolution
Guimarães, B., Dourado, L., Tsisar, S., Diniz, J. M., Madeira, M. D., & Ferreira, M. A. (2017) · Acta Médica Portuguesa · doi:10.20344/amp.8404 · Cited by 87 · cited 3×
“In their 2017 article, Guimarães et al [21] reviewed current education models for anatomy education, and recommended the use of AI analytic tools to personalize the learning process.”
BackgroundInfluentialWhat we measure … and what we should measure in medical education
Boulet, J. R., & Durning, S. J. (2018) · Medical Education · doi:10.1111/medu.13652 · Cited by 80 · cited 3×
“and what we should measure in medical education United States Boulet and Durning, 2019 [24]”
BackgroundAI in medical education—another grand challenge for medical informatics
Lillehaug, S.-I., & Lajoie, S. P. (1998) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(97)00054-7 · Cited by 78 · cited 3×
“Lillehaug and Lajoie [16] proposed greater integration of AI in their 1998 paper, and they were among the first researchers who advocated for intelligent decision support systems and AI-based applications for medical education.”
BackgroundInfluentialAlonso-Silverio, G. A., Pérez-Escamirosa, F., Bruno-Sanchez, R., Ortiz-Simon, J. L., Muñoz-Guerrero, R., Minor-Martinez, A., & Alarcón-Paredes, A. (2018) · Surgical Innovation · doi:10.1177/1553350618777045 · Cited by 68 · cited 2×
“Another study that used augmented reality and AI algorithms evaluated a case study about the development of a laparoscopic box trainer to assess surgical psychomotor skills, and concluded the proposed system had potential benefits [29].”
BackgroundYang, Y.-Y., & Shulruf, B. (2019) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2019.16.7 · Cited by 65
Virtual patient simulator for distributed collaborative medical education
Caudell, T. P., Summers, K. L., Holten, J., Hakamata, T., Mowafi, M., Jacobs, J., Lozanoff, B. K., Lozanoff, S., Wilks, D., Keep, M. F., et al. (2003) · The Anatomical Record Part B: The New Anatomist · doi:10.1002/ar.b.10007 · Cited by 61 · cited 12×
“The authors investigated the integration of NLP to medical This study reviewed the appli- cation of NLPd to medical IV1A Review of Natural Language Processing in Medical Education United States Chary et al, 2018 [19] education resources using published manuscripts and stated education and identified concepts from NLP…”
BackgroundInfluentialTelehealth Innovations in Health Education and Training
Conde, J. G., De, S., Hall, R. W., Johansen, E., Meglan, D., & Peng, G. C. Y. (2009) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2009.0152 · Cited by 60 · cited 2×
“V 1 Telehealth Innovations in Health Education and Training United States Conde et al, 2009 [25]”
BackgroundClinical judgement in the era of big data and predictive analytics
Chin‐Yee, B., & Upshur, R. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12852 · Cited by 59 · cited 3×
“IV 2 Clinical judgement in the era of big data and predictive analytics Canada Chin-Yee and Upshur, 2017 [37]”
BackgroundMethodInfluentialSapci, A. H., & Sapci, H. A. (2020) · JMIR Medical Informatics · doi:10.2196/15748 · Cited by 47 · cited 1×
“Furthermore, a recent study evaluated health informatics students' skills in developing AI apps and emphasized the need to develop new competencies [53].”
BackgroundSelf-citationUsing Artificial Intelligence and Gaming to Improve New Nurse Transition
Sitterding, M. C., Raab, D. L., Saupe, J. L., & Israel, K. J. (2019) · Nurse Leader · doi:10.1016/j.mnl.2018.12.013 · Cited by 39 · cited 4×
“…This research discussed the preliminary pilot study data IV1Using Artificial Intelligence and Gaming to Improve New Nurse Transition United States Sitterding et al, 2019 [23] stated as the limitations of the preliminary findings. from a virtual reality simulation education intervention that…”
BackgroundResultInfluentialImmersive Virtual Environments for Medical Training
Bowyer, M. W., Streete, K. A., Muniz, G., & Liu, A. V. (2008) · Seminars in Colon and Rectal Surgery · doi:10.1053/j.scrs.2008.02.005 · Cited by 36 · cited 4×
“The paper described various virtual reality environments This study highlighted the role of advanced virtual enviIV1Immersive Virtual Environments for Medical Training United States Bowyer et al, 2008 [22] where students can interact with AI-based simulators. ronments and surgical simulators as a training platform for…”
BackgroundInfluentialComputer-assisted teaching and learning in medicine
Klar, R., & Bayer, U. (1990) · International Journal of Bio-Medical Computing · doi:10.1016/0020-7101(90)90016-n · Cited by 33 · cited 2×
“Klar and Bayer's paper [27], published in 1990, was one of the first publications about the application of AI, and they discussed the integration of expert knowledge into computer-assisted teaching in medicine.”
BackgroundMethod(2000) · PubMed · Cited by 28
Research on Application of Artificial Intelligence in Medical Education
Zhao, H., Li, G., & Feng, W. (2018) · 2018 International Conference on Engineering Simulation and Intelligent Control (ESAIC) · doi:10.1109/esaic.2018.00085 · Cited by 22
Technology will improve doctors’ relationships with patients, says Topol review
Rimmer, A. (2019) · BMJ · doi:10.1136/bmj.l661 · Cited by 17 · cited 1×
“The report emphasized the skills gap in the workforce, and made some recommendations about the integration of digital health care technologies, AI, and data analytics in undergraduate curricula [42,43].”
BackgroundGauthier, G., VanLehn, K. A., & Frasson, C. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45108-0 · Cited by 15
Kabassi, K., Virvou, M. K., Tsihrintzis, G. A., Vlachos, Y., & Perrea, D. N. (2008) · Intelligent Decision Technologies · doi:10.3233/idt-2008-2304 · Cited by 8
Research works printed but not linked: 6
- Mayor, A, 2018, Princeton: Princeton University Press
- 10.1609/aimag.v27i4.1911 · doi:10.1609/aimag.v27i4.1911
- Artificial intelligenceOxford Reference2020-06-22https://www.oxfordreference.com/view/10.1093/oi/authority.20110803095426960 · doi:10.1093/oi/authority.20110803095426960
- Clinical Informatics Becomes a Board-certified Medical Subspecialty Following ABMS Vote20112020-03-01https://www.amia.org/news-and-publications/press-release/ci-is-subspecialty
- HowickJChalmersIGlasziouPGreenhalghTHeneghanCLiberatiAMoschettiIPhillipsBThorntonHGoddardOHodgkinsonMOCEBM Levels of Evidence Working GroupThe Oxford 2011 Levels of EvidenceOxford Centre of Evidence-Based Medicine20142020-03-15https://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf
- Souze, A, 2018, ATD Public Policy Advisory Group, 1
Cited by
Bond, M., Khosravi, H., De Laat, M., Bergdahl, N., Negrea, V., Oxley, E., Pham, P., Chong, S. W., & Siemens, G. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00436-z · Cited by 852
Karaca, O., Çalışkan, S. A., & Demir, K. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02546-6 · Cited by 377
Artificial intelligence in medical education: a cross-sectional needs assessment
Civaner, M. M., Uncu, Y., Bulut, F., Chalil, E. G., & Tatli, A. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03852-3 · Cited by 363
“As Sapci and Sapci revealed in their systematic review, the integration of AI training into medical and health informatics curricula is an important need for future physicians [32].”
BackgroundHow the nursing profession should adapt for a digital future
Booth, R. G., Strudwick, G., McBride, S., O’Connor, S., & Solano López, A. L. (2021) · BMJ · doi:10.1136/bmj.n1190 · Cited by 330
Artificial Intelligence in Undergraduate Medical Education: A Scoping Review
Lee, J., Wu, A. S., Li, D., & Kulasegaram, K. (. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004291 · Cited by 324
Metadata
- Authors
- Sapci, A. H., & Sapci, H. A.
- Affiliations
- Adelphi University (US)
- Corresponding author
- Sapci, A Hasan
- Sapci, H Aylin
- Venue
- JMIR Medical Education
- Publisher
- JMIR Publications Inc.
- Published
- 2020-06-30
- Volume
- 6
- Issue
- 1
- Pages
- e19285
- ISSN
- 2369-3762 (ISSN-L)
- PMC
- PMC7367541
- OpenAlex
- W3039746272
- Cited by
- 377
- References
- 46
- Citations by year
- 2026: 68 · 2025: 103 · 2024: 85 · 2023: 56 · 2022: 30 · 2021: 34 · 2020: 1
- FWCI
- 2.30
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Education (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- artificial intelligence in education, medical education, health informatics education, AI training, AI tools in learning, machine learning, deep learning, data analytics, data visualization, AI competencies, curriculum integration, training framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.