CoChat
Sign Up
Cited by 79Artificial Intelligence in Healthcare and EducationInnovations in Medical EducationArtificial Intelligence in Education

An evidence-based approach to artificial intelligence education for medical students: A systematic review

Open access
Pupic, Nikola, Ghaffari-zadeh, Aryan, Hu, Ricky, Singla, Rohit, Darras, Kathryn, Karwowska, Anna, Forster, Bruce B.·2023·PLOS Digital Health
DOI 10.1371/journal.pdig.0000255PMC PMC10681314
Open this paper in CoChat — free accountOpen PDF

Abstract

The exponential growth of artificial intelligence (AI) in the last two decades has been recognized by many as an opportunity to improve the quality of patient care. However, medical education systems have been slow to adapt to the age of AI, resulting in a paucity of AI-specific education in medical schools. The purpose of this systematic review is to evaluate the current evidence-based recommendations for the inclusion of an AI education curriculum in undergraduate medicine. Six databases were searched from inception to April 23, 2022 for cross sectional and cohort studies of fair quality or higher on the Newcastle-Ottawa scale, systematic, scoping, and integrative reviews, randomized controlled trials, and Delphi studies about AI education in undergraduate medical programs. The search yielded 991 results, of which 27 met all the criteria and seven more were included using reference mining. Despite the limitations of a high degree of heterogeneity among the study types and a lack of follow-up studies evaluating the impacts of current AI strategies, a thematic analysis of the key AI principles identified six themes needed for a successful implementation of AI in medical school curricula. These themes include ethics, theory and application, communication, collaboration, quality improvement, and perception and attitude. The themes of ethics, theory and application, and communication were further divided into subthemes, including patient-centric and data-centric ethics; knowledge for practice and knowledge for communication; and communication for clinical decision-making, communication for implementation, and communication for knowledge dissemination. Based on the survey studies, medical professionals and students, who generally have a low baseline knowledge of AI, have been strong supporters of adding formal AI education into medical curricula, suggesting more research needs to be done to push this agenda forward.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review synthesizes evidence on artificial intelligence (AI) education in undergraduate medical curricula, identifying six recurring themes and strong support for formal AI teaching despite low baseline knowledge. It concludes that a standardized, evidence-based AI curriculum is warranted, but current evidence is heterogeneous and lacks follow-up studies.

Key findings

  • From 991 records, 34 studies were included. Thematic analysis identified six themes: ethics, theory and application, communication, collaboration/teamwork, quality improvement, and perception and attitude. Ethics was divided into patient-centric and data-centric ethics; theory and application into knowledge for practice and knowledge for development; communication into clinical decision-making, implementation, and knowledge dissemination.
  • Survey studies had a mean sample size of 578 (range 62–1459), with a mean respondent sex ratio of 54% female and 46% male. Support for AI education was high: 81% of physicians supported more formal AI training (Banerjee et al.), 63.36% of medical students believed basic AI literacy should be part of their curriculum (Teng et al.), and 92% said AI-related teaching should be incorporated into the core medical curriculum, with 86% interested in interdisciplinary learning (Ejaz et al.).
  • Baseline knowledge was generally low: 74.4% of participants were not knowledgeable about AI (Kansal et al.), 62.4% of students reported 0 hours of AI training (Blease et al.), and only 5.5% of physicians rated their AI knowledge as excellent (Scheetz et al., cited in the introduction). Attitudes included some hesitancy about unemployment and radiology job opportunities, but overall showed support for AI education.

Methods

  • PRISMA-guided systematic review searching MEDLINE, EMBASE, CINAHL, ERIC, NCBI, and Web of Science from inception to April 23, 2022. Inclusion criteria included undergraduate medical students, cross-sectional/cohort studies rated fair or higher on the Newcastle-Ottawa Scale (NOS), systematic/scoping/integrative reviews, randomized controlled trials, and Delphi studies; exclusions included non-English studies, postgraduate education, allied health, AI applications rather than AI education, and poor-quality studies.
  • Screening and data extraction were performed independently and in duplicate, with conflicts resolved by a third reviewer. Quality was assessed with NOS for cross-sectional/cohort studies and ROBIS for reviews. Thematic analysis used a MindMap to group themes; no meta-analysis was performed due to significant heterogeneity.
  • Of 34 included studies, 23 were assessed with NOS and 11 with ROBIS. One NOS study was good and 22 were fair; seven ROBIS reviews were low risk, one unclear, and three high risk. No RCTs were identified.

Limitations

  • The paper states that heterogeneity among included studies made it difficult to draw strongly supported conclusions; a lack of follow-up studies evaluating current AI education strategies made effectiveness difficult to assess; no meta-analysis could be conducted; and the review focused only on undergraduate medicine, not the continuum of medical education.
  • Evident from the text: no RCTs were identified, and three included reviews were rated high risk of bias. The text also reports 23 survey and 12 non-survey studies despite stating 34 total included studies, and the abstract says seven additional articles were included by reference mining while the Methods says eight.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pdig.0000255

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 157 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    63.36% of medical students support fundamental AI literacy training.

    “Banerjee et al. and Teng et al., reporting that 81% of physicians and 63·36% of medical students, respectively, support fundamental AI literacy training [7,8].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [7] M Banerjee, 2021, The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers, BMC Med Educ source

      Not checked in the free run.

    • cites [8] M Teng, 2022, Health Care Students’ Perspectives on Artificial Intelligence: Countrywide Survey in Canada, JMIR Med Educ source

      Not checked in the free run.

  • Findingnone given

    Based on the survey studies, medical professionals and students generally have a low baseline knowledge of AI.

    “Based on the survey studies, medical professionals and students, who generally have a low baseline knowledge of AI, have been strong supporters of adding formal AI education into medical curricula, suggesting more research needs to be done to push this agenda forward.”
    Quote in the paperSubstantial claim
  • Findingcitation

    Eleven studies suggest that physicians should understand the role of data science in AI.

    “Eleven studies suggest that physicians should understand the role of data science in AI [12,13,16,37,39,41,42,45].”
    Quote in the paperNot a substantial claim0 of 8 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [16] P Whiting, 2016, ROBIS: a new tool to assess risk of bias in systematic reviews was developed, J Clin Epi source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

    • cites [39] AH Sapci, 2020, Artificial intelligence education and tools for medical and health informatics students: systematic review, JMIR Med Educ source

      Not checked in the free run.

    • cites [41] LT Car, 2021, Digital health training programs for medical students: scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [42] J Grunhut, 2021, Educating future physicians in artificial intelligence (AI): An integrative review and proposed changes, J Med Educ Curric Dev source

      Not checked in the free run.

    • cites [45] MP Khurana, 2022, Digital health competencies in medical school education: a scoping review and Delphi method study, BMC Med Educ source

      Not checked in the free run.

  • Findingown data

    No RCTs were published within the search criteria.

    “No RCTs were published within our search criteria.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The search yielded 27 results that met all the criteria.

    “The search yielded 991 results, of which 27 met all the criteria and seven more were included using reference mining.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 62
  • May combine two claims. 31

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    81% of physicians support fundamental AI literacy training.

    “Banerjee et al. and Teng et al., reporting that 81% of physicians and 63·36% of medical students, respectively, support fundamental AI literacy training [7,8].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [7] M Banerjee, 2021, The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers, BMC Med Educ source

      Not checked in the free run.

    • cites [8] M Teng, 2022, Health Care Students’ Perspectives on Artificial Intelligence: Countrywide Survey in Canada, JMIR Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The original search yielded 991 studies after duplicates were removed.

    “The original search yielded 991 studies after duplicates were removed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Of the 34 total included studies, 23 were evaluated with the NOS criteria and 11 with the ROBIS criteria.

    “Of the 34 total included studies, 23 were evaluated with the NOS criteria and 11 with the ROBIS criteria.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Seven studies met the ROBIS criteria for low risk, one study was of unclear risk, and three studies were of high risk.

    “Seven studies met the ROBIS criteria for low risk [12,13,38,39,40,41,42], one study was of unclear risk [43], and three studies were of high risk [44–46].”
    Quote in the paperNot a substantial claim0 of 10 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [38] L Yang, 2022, Stakeholders’ perspectives on the future of artificial intelligence in radiology: a scoping review, Eur Radiol source

      Not checked in the free run.

    • cites [39] AH Sapci, 2020, Artificial intelligence education and tools for medical and health informatics students: systematic review, JMIR Med Educ source

      Not checked in the free run.

    • cites [40] ER Han, 2019, Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review, BMC Med Educ source

      Not checked in the free run.

    • cites [41] LT Car, 2021, Digital health training programs for medical students: scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [42] J Grunhut, 2021, Educating future physicians in artificial intelligence (AI): An integrative review and proposed changes, J Med Educ Curric Dev source

      Not checked in the free run.

    • cites [43] KS Chan, 2019, Applications and challenges of implementing artificial intelligence in medical education: integrative review, JMIR Med Educ source

      Not checked in the free run.

    • cites [44] M Senbekov, 2020, The recent progress and applications of digital technologies in healthcare: a review, Int J Telemed Appl

      Not checked in the free run.

    • cites [45] MP Khurana, 2022, Digital health competencies in medical school education: a scoping review and Delphi method study, BMC Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The mean sample size of the survey studies was 578 (range 62-1459).

    “The mean sample size of the survey studies was 578 (range 62–1459).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingcitation

    Knowledge for practice includes learning AI terminology, strengths and limitations of AI, risks of AI, controlling for bias, critical appraisal of AI literature and tools, and shared decision-making.

    “Overall, this includes learning the AI terminology [13,37,45], strengths and limitations of AI [12,13,16,37,42], risks of AI [12,13,17,37,39,42], controlling for bias [42], critical appraisal AI literature and tools [12,13,34,37,39,45,46], and shared decision-making [12,13,37,41,45].”
    Quote in the paperSubstantial claim0 of 8 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

    • cites [45] MP Khurana, 2022, Digital health competencies in medical school education: a scoping review and Delphi method study, BMC Med Educ source

      Not checked in the free run.

    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [16] P Whiting, 2016, ROBIS: a new tool to assess risk of bias in systematic reviews was developed, J Clin Epi source

      Not checked in the free run.

    • cites [42] J Grunhut, 2021, Educating future physicians in artificial intelligence (AI): An integrative review and proposed changes, J Med Educ Curric Dev source

      Not checked in the free run.

    • cites [17] O Karaca, 2021, Medical artificial intelligence readiness scale for medical students (MAIRS-MS)–development, validity and reliability study, BMC Med Educ source

      Not checked in the free run.

    • cites [39] AH Sapci, 2020, Artificial intelligence education and tools for medical and health informatics students: systematic review, JMIR Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Seven more articles were included using reference mining.

    “The search yielded 991 results, of which 27 met all the criteria and seven more were included using reference mining.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Two sub-themes were identified within theory and application: knowledge for practice and knowledge for development.

    “Two sub-themes were identified within theory and application: knowledge for practice and knowledge for development.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The mean male to female ratio of respondents was 54% female and 46% male.

    “The mean male to female ratio of respondents was 54% female and 46% male.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Eight additional articles were included from reference lists.

    “Eight additional articles were included.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The mean respondent years of experience working with AI was only reported in three studies.

    “The mean respondent years of experience working with AI was only reported in three studies [7,21,28].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [7] M Banerjee, 2021, The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers, BMC Med Educ source

      Not checked in the free run.

    • cites [21] CJ Park, 2021, Medical student perspectives on the impact of artificial intelligence on the practice of medicine, Curr Probl Diagn Radiol source

      Not checked in the free run.

    • cites [28] C Blease, 2022, Machine learning in medical education: a survey of the experiences and opinions of medical students in Ireland, BMJ Health Care Inform source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    One study met the NOS criteria for good and 22 studies met the NOS criteria for fair.

    “One study met the NOS criteria for good [17] and 22 studies met the NOS criteria for fair [7,8,18–36,37].”
    Quote in the paperNot a substantial claim0 of 10 sources checked
    • cites [17] O Karaca, 2021, Medical artificial intelligence readiness scale for medical students (MAIRS-MS)–development, validity and reliability study, BMC Med Educ source

      Not checked in the free run.

    • cites [7] M Banerjee, 2021, The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers, BMC Med Educ source

      Not checked in the free run.

    • cites [8] M Teng, 2022, Health Care Students’ Perspectives on Artificial Intelligence: Countrywide Survey in Canada, JMIR Med Educ source

      Not checked in the free run.

    • cites [18] B Gong, 2019, Influence of artificial intelligence on Canadian medical students’ preference for radiology specialty: A National survey study, Acad Radiol source

      Not checked in the free run.

    • cites [19] D Pinto Dos Santos, 2019, Medical students’ attitude towards artificial intelligence: a multicentre survey, Eur Radiol source

      Not checked in the free run.

    • cites [20] C Sit, 2020, Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: a multicentre survey, Insights Imaging

      Not checked in the free run.

    • cites [21] CJ Park, 2021, Medical student perspectives on the impact of artificial intelligence on the practice of medicine, Curr Probl Diagn Radiol source

      Not checked in the free run.

    • cites [22] K Reeder, 2022, Impact of artificial intelligence on US medical students’ choice of radiology, Clin Imaging source

      Not checked in the free run.

    • cites [23] M Alelyani, 2021, Radiology community attitude in Saudi Arabia about the applications of artificial intelligence in radiology, Healthcare source

      Not checked in the free run.

    • cites [24] P Auloge, 2020, Interventional radiology and artificial intelligence in radiology: Is it time to enhance the vision of our medical students?, Insights Imaging source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The studies were separated into two categories: survey (n = 23) and non-survey (n = 12).

    “The studies were separated into two categories: survey (n = 23) and non-survey (n = 12).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Medical AI ethics divides into patient-centric and data-centric ethics.

    “Medical AI ethics divides into patient-centric and data-centric ethics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Findingcitation

    Quality improvement is reflected by the need for evaluating and iterating on any AI curriculum designed for medical schools on a regular basis to improve the learning experience.

    “Quality improvement was also reflected by the need for evaluating and iterating on any AI curriculum designed for medical schools on a regular basis to further improve the learning experience [12].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Four key relationships for communication were identified: data scientist to physician, physician to physician/allied healthcare professional, physician to patient, and physician to payer.

    “Furthermore, four key relationships for communication were identified: data scientist to physician [13], physician to physician/allied healthcare professional [12,13,42], physician to patient [12,13,37,41,42], and physician to payer [13,37].”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [42] J Grunhut, 2021, Educating future physicians in artificial intelligence (AI): An integrative review and proposed changes, J Med Educ Curric Dev source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

    • cites [41] LT Car, 2021, Digital health training programs for medical students: scoping review, JMIR Med Educ source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Three key types of communication were identified: communication for clinical decision-making, communication for implementation, and communication for knowledge dissemination.

    “Three key types of communication were identified: communication for clinical decision-making [13], communication for implementation [13,37], and communication for knowledge dissemination [12,13,37,41,42].”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [41] LT Car, 2021, Digital health training programs for medical students: scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [42] J Grunhut, 2021, Educating future physicians in artificial intelligence (AI): An integrative review and proposed changes, J Med Educ Curric Dev source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Quality improvement in AI education is reflected through critical appraisal of AI literature and tools, physician involvement in AI tool design/development, ability to identify and control for bias, communication with data scientists and engineers, and communication with patients.

    “Quality improvement was reflected through the need for critical appraisal of AI literature and tools [12,13,34,37,39], physician involvement in AI tool design/development [13,39], the ability to identify and control for bias [45], communication with data scientists and engineers [13], and communication with patients [12,13,37,41,42].”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [34] C Blacketer, 2021, Medical student knowledge and critical appraisal of machine learning: a multicentre international cross-sectional study, Intern Med J source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

    • cites [39] AH Sapci, 2020, Artificial intelligence education and tools for medical and health informatics students: systematic review, JMIR Med Educ source

      Not checked in the free run.

    • cites [45] MP Khurana, 2022, Digital health competencies in medical school education: a scoping review and Delphi method study, BMC Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Communication for implementation focuses on understanding how physician roles and workflows will change when AI gets implemented.

    “Communication for implementation focuses on understanding how physician roles and workflows will change when AI gets implemented [13,33].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [33] C Blease, 2020, US primary care in 2029: A Delphi survey on the impact of machine learning, PLoS One source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Communication for knowledge dissemination includes the need to show empathy and patience when communicating with and educating others about AI.

    “It also includes the need to show empathy and patience when communicating with and educating others about AI [13,33,40].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [33] C Blease, 2020, US primary care in 2029: A Delphi survey on the impact of machine learning, PLoS One source

      Not checked in the free run.

    • cites [40] ER Han, 2019, Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review, BMC Med Educ source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    The key relationships involved in communication for implementation are physician to physician/allied healthcare professional and physician to payer.

    “The key relationships involved will be physician to physician/allied healthcare professional and physician to payer.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Communication for implementation also focuses on economic effects of implementing AI tools and how to communicate with stakeholders.

    “It also focuses on the economic effects of implementing AI tools and how to communicate with stakeholders [13,37].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Regulatory challenges for AI adoption include integrating AI into clinical settings and workflow, data sharing between institutions, and translation and interpretation of AI models in a healthcare setting.

    “Regulatory challenges include integrating AI into clinical settings and workflow, data sharing between institutions, and the translation and interpretation of AI models in a health- care setting [3,13,43].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [3] K Paranjape, 2019, Introducing artificial intelligence training in medical education, JMIR Med Educ source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [43] KS Chan, 2019, Applications and challenges of implementing artificial intelligence in medical education: integrative review, JMIR Med Educ source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Findingcitation

    Other important skills for physicians include knowledge related to machine learning, deep learning, natural language processing, model development, and AI tool design and development.

    “Other important skills include knowledge related to machine learning (ML) [12,13], deep learning (DL) [13], natural language processing (NLP) [13], model development [13], and AI tool design and development [13,39].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [39] AH Sapci, 2020, Artificial intelligence education and tools for medical and health informatics students: systematic review, JMIR Med Educ source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    The key relationships for communication for practice are data scientist to physician and physician to physician/allied healthcare professional.

    “The key relationships for communication for practice are data scientist to physician and physician to physician/allied healthcare professional.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Communication for knowledge dissemination focuses on sharing knowledge to improve understanding of AI tools with respect to their role in care, strengths, limitations, and risks.

    “Communication for knowledge dissemination focuses on sharing knowledge to improve the understanding of AI tools with respect to their role in care, as well as the strengths, limitations, and risks of AI [12,13,17,37,39,42].”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [17] O Karaca, 2021, Medical artificial intelligence readiness scale for medical students (MAIRS-MS)–development, validity and reliability study, BMC Med Educ source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

    • cites [39] AH Sapci, 2020, Artificial intelligence education and tools for medical and health informatics students: systematic review, JMIR Med Educ source

      Not checked in the free run.

    • cites [42] J Grunhut, 2021, Educating future physicians in artificial intelligence (AI): An integrative review and proposed changes, J Med Educ Curric Dev source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Perceptions and attitudes of medical students and professionals reflected some hesitancy about AI's impacts on medicine, such as unemployment, but ultimately showed overwhelming support in favour of AI.

    “The perceptions and attitudes of medical students and professionals reflected some hesitancy surrounding the impacts of AI on medicine, such as unemployment [25], but ultimately showed overwhelming support in favour of AI [7–9,26].”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [25] A. Doganer, 2021, The approaches and expectations of the health sciences students towards artificial intelligence, Karya J Health Sci

      Not checked in the free run.

    • cites [7] M Banerjee, 2021, The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers, BMC Med Educ source

      Not checked in the free run.

    • cites [8] M Teng, 2022, Health Care Students’ Perspectives on Artificial Intelligence: Countrywide Survey in Canada, JMIR Med Educ source

      Not checked in the free run.

    • cites [9] J Scheetz, 2021, A survey of clinicians on the use of artificial intelligence in ophthalmology, dermatology, radiology and radiation oncology, Sci Rep source

      Not checked in the free run.

    • cites [26] H Ejaz, 2022, Artificial intelligence and medical education: A global mixed-methods study of medical students’ perspectives, Digit Health

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Studies did not always clarify specific issues within each area.

    “Unfortunately, studies did not always clarify specific issues within each area.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Our study shows that although there is a high degree of heterogeneity among programs and research, the themes of ethics, theory and application, communication, collaboration, quality improvement, and perception and attitudes are recurring.

    “Our study shows that although there is a high degree of heterogeneity among programs and research, the themes of ethics, theory and application, communication, collaboration, quality improvement, and perception and attitudes are recurring.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Many studies advocated for fundamental statistics knowledge but did not define what is entailed.

    “Many studies advocated for fundamental statistics knowledge but did not define what is entailed.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Although physicians are aware of AI as a concept, only 5.5% report their knowledge as excellent.

    “Scheetz et al. reported that although physicians are aware of AI as a concept, only 5·5% report their knowledge as excellent, further suggesting the need for a curricular reform [9].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] J Scheetz, 2021, A survey of clinicians on the use of artificial intelligence in ophthalmology, dermatology, radiology and radiation oncology, Sci Rep source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    These programs, and others, are mostly extracurricular and vary significantly in what is included in their proposed curricula.

    “These programs, and others, are mostly extracurricular and vary significantly in what is included in their proposed curricula.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    The shift from the information age to the AI age promotes abstraction, which would require physicians to employ AI to manage the wealth of information without memorising it, allowing them to prioritize important tasks.

    “This shift promotes abstraction, which would require physicians to employ AI to manage the wealth of information without memorising it, allowing them to prioritize important tasks.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Graduating physicians do not need to understand the complexities of AI algorithms, but they must know when they are applicable and the strengths and weaknesses of the data output.

    “McCoy et al. state that graduating physicians do not need to understand the complexities of AI algorithms, but they must know when they are applicable and the strengths and weaknesses of the data output.6”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6] LG McCoy, 2020, What do medical students actually need to know about artificial intelligence?, NPJ Digit Med source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Financial barriers, resistance of educators to change curricula, lack of staff with teaching experience in AI, and limited digitalization are some of the main barriers for implementing AI into the medical curriculum.

    “Despite the support for including AI education, financial barriers, resistance of educators to change curricula, lack of staff with teaching experience in AI, and limited digitalization are some of the main barriers for implementing AI into the medical curriculum [3,10,12,13].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [3] K Paranjape, 2019, Introducing artificial intelligence training in medical education, JMIR Med Educ source

      Not checked in the free run.

    • cites [10] R Hu, 2022, Insights from teaching artificial intelligence to medical students in Canada, Commun Med source

      Not checked in the free run.

    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Numerous experts have advocated for the incorporation of AI training and literacy into medical curricula.

    “Numerous experts have advocated for the incorporation of AI training and literacy into medical curricula [3–6].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [3] K Paranjape, 2019, Introducing artificial intelligence training in medical education, JMIR Med Educ source

      Not checked in the free run.

    • cites [4] SA Wartman, 2018, Medical education must move from the information age to the age of artificial intelligence, Acad Med source

      Not checked in the free run.

    • cites [5] K. Masters, 2019, Artificial intelligence in medical education, Med Teach source

      Not checked in the free run.

    • cites [6] LG McCoy, 2020, What do medical students actually need to know about artificial intelligence?, NPJ Digit Med source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Clinical practice is changing from the information age to the AI age.

    “Paranjape et al. and Wartman and Combs believe that clinical practice is changing from the information age to the AI age [3,4].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [3] K Paranjape, 2019, Introducing artificial intelligence training in medical education, JMIR Med Educ source

      Not checked in the free run.

    • cites [4] SA Wartman, 2018, Medical education must move from the information age to the age of artificial intelligence, Acad Med source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation

    Curricular reform should address the knowledge all physicians need for everyday practice, while extracurricular programs should address what some physicians need to drive innovation.

    “These authors also differentiate between the knowledge that all physicians need for everyday practice versus what some physicians need to drive innovation, highlighting that a curricular reform should address the former and extracurricular programs the latter [6].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] LG McCoy, 2020, What do medical students actually need to know about artificial intelligence?, NPJ Digit Med source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    AI applications in medicine have undergone exponential growth in the past two decades.

    “Artificial intelligence (AI) applications in medicine have undergone exponential growth in the past two decades [1].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] G Briganti, 2020, Artificial intelligence in medicine: today and tomorrow, Front Med source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    This systematic review followed PRISMA guidelines.

    “This systematic review followed PRISMA (Preferred Reporting Items for Systematic Review and Meta-analysis) guidelines [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] L Shamseer, 2015, Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation, BMJ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    This systematic review used the Newcastle-Ottawa scale and the Risk of Bias in Systematic Reviews tools to assess bias.

    “This systematic review also used the Newcastle-Ottawa scale (NOS) and the Risk of Bias in Systematic Reviews (ROBIS) tools to assess bias [15,16].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [15] Wells GA, Shea B, O’Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2019. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp (accessed Nov, 1, 2022).

      Not checked in the free run.

    • cites [16] P Whiting, 2016, ROBIS: a new tool to assess risk of bias in systematic reviews was developed, J Clin Epi source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Conflicts during screening were resolved by a third reviewer when consensus was not reached.

    “Any conflicts that arose when consensus was not reached during the screening steps were resolved by a third reviewer (RS).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Quality assessment was performed by two reviewers using the NOS criteria for cross-sectional and cohort studies and the ROBIS criteria for reviews.

    “Quality assessment was performed by two reviewers (NP and AG) using the NOS criteria for cross sectional and cohort studies and the ROBIS criteria for reviews [15,16].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [15] Wells GA, Shea B, O’Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2019. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp (accessed Nov, 1, 2022).

      Not checked in the free run.

    • cites [16] P Whiting, 2016, ROBIS: a new tool to assess risk of bias in systematic reviews was developed, J Clin Epi source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationcitation

    Developing ability to visualize how physician role and workflow may change with AI during UGME, and learning implementation techniques, may improve future integration of AI in healthcare.

    “In terms of implementation, developing the ability to visualize how the role and workflow of a physician may change with AI during UGME [13,33], and learning implementation techniques that would help with the AI transition, may improve the future integration of AI in healthcare.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [33] C Blease, 2020, US primary care in 2029: A Delphi survey on the impact of machine learning, PLoS One source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Intellectual property over AI algorithms is protected by copyright.

    “the intellectual property over the algorithms is protected by copyright [13,46].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [46] SM Santomartino, 2022, Systematic Review of Radiologist and Medical Student Attitudes on the Role and Impact of AI in Radiology, Acad Radiol source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    A meta-analysis of the included articles was not performed due to the significant heterogeneity in the study types included.

    “A meta-analysis of the included articles was not performed due to the significant heterogeneity in the study types included.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Abstract and full-text screening were performed independently and in duplicate by two reviewers.

    “Abstract and full-text screening were performed independently and in duplicate by two reviewers (NP and AG) using the Covidence platform (Veritas Health Innovation Ltd., Melbourne, Australia).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Knowledge related to data stewardship, data preprocessing and acquisition, standardization, data analytics, health data infrastructure, programming, and big data may help physicians appreciate obstacles faced by data scientists and improve multidisciplinary collaboration.

    “In particular, knowledge related to data stewardship [12,13,37,41], data preprocessing and acquisition [13], standardization [13], data analytics [13,17,39], health data infrastructure [45], programming [39,41], and big data [13,45], may help physicians appreciate the obstacles faced by data scientists and improve the multidisciplinary collaboration [13,39].”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

    • cites [41] LT Car, 2021, Digital health training programs for medical students: scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [17] O Karaca, 2021, Medical artificial intelligence readiness scale for medical students (MAIRS-MS)–development, validity and reliability study, BMC Med Educ source

      Not checked in the free run.

    • cites [39] AH Sapci, 2020, Artificial intelligence education and tools for medical and health informatics students: systematic review, JMIR Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Data extraction was checked for accuracy and completion by a third reviewer.

    “Data extraction was checked for accuracy and completion by a third reviewer (RS).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The inclusion and exclusion criteria for this study were determined a priori.

    “The inclusion and exclusion criteria for this study were determined a priori.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The goal of communication for clinical decision-making is to equip physicians with tools needed to successfully use AI in a clinical setting.

    “The goal of this communication is to equip physicians with the tools needed to successfully use AI in a clinical setting [12,13,16,40,44,45].”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [16] P Whiting, 2016, ROBIS: a new tool to assess risk of bias in systematic reviews was developed, J Clin Epi source

      Not checked in the free run.

    • cites [40] ER Han, 2019, Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review, BMC Med Educ source

      Not checked in the free run.

    • cites [44] M Senbekov, 2020, The recent progress and applications of digital technologies in healthcare: a review, Int J Telemed Appl

      Not checked in the free run.

    • cites [45] MP Khurana, 2022, Digital health competencies in medical school education: a scoping review and Delphi method study, BMC Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Learning about economic considerations with respect to AI is crucial to understanding business and clinical aspects of AI implementation.

    “From an economic perspective, learning about the various economic considerations with respect to AI is crucial to having an understanding of the business and clinical aspects of AI implementation [13].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    This role's knowledge may vary by country and healthcare system (i.e., public vs private payer healthcare system).

    “This role’s knowledge may vary by country and healthcare system (i.e., public vs private payer healthcare system).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Charow et al. suggest that the knowledge gap in AI ethics can be bridged by establishing a regulatory body tasked with creating systematic guidelines for AI ethics.

    “Charow et al. also suggest that this knowledge gap can be bridged by establishing a regulatory body that is tasked with creating systematic guidelines for AI ethics [13].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Communication for clinical decision-making and communication for knowledge dissemination are both patient-facing skills that would be used on a daily basis in clinics.

    “Communication for clinical decision-making and communication for knowledge dissemination are both patient-facing skills that would be used on a daily basis in clinics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Knowledge for development is likely more suited for select individuals who are more passionate for AI and would like to play a larger role in its advocacy and implementation.

    “Although important, knowledge for development is likely more suited for select individuals who are more passionate for AI and would like to play a larger role in its advocacy and implementation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Post-graduate medical education can then develop these talents further.

    “Post-graduate medical education can then develop these talents further.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationcitation

    Providing medical students with opportunity to learn about AI ethics before clinical experiences could better prepare them to work with AI and better communicate to patients about AI.

    “Furthermore, by providing medical students with an opportunity to learn about AI ethics before they enter their clinical experiences, it could not only better prepare them to work with AI, but also better communicate to patients about AI [12,40].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [40] ER Han, 2019, Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review, BMC Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Medical students could consider AI part of their biostatistics coursework.

    “AI is advanced predictive statistics, therefore medical students could consider it part of their biostatistics coursework.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: AI is advanced predictive statistics.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Predicting workflow changes, economic implications, and stakeholder communication would be more logistical.

    “Predicting workflow changes, economic implications, and stakeholder communication would be more logistical.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationcitation

    A strong theoretical base of AI for practice would equip learners with skills needed to critically appraise AI literature and tools before implementing them, interpret model output, and communicate AI findings to others.

    “This would equip them with the skills needed to critically appraise AI literature and tools before implementing them [12,13,34,37,39], interpret model output [13,37,42], and communicate AI findings to others [12,13,42].”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [34] C Blacketer, 2021, Medical student knowledge and critical appraisal of machine learning: a multicentre international cross-sectional study, Intern Med J source

      Not checked in the free run.

    • cites [37] H Liyanage, 2019, Artificial intelligence in primary health care: perceptions, issues, and challenges, Yearb Med Inform source

      Not checked in the free run.

    • cites [39] AH Sapci, 2020, Artificial intelligence education and tools for medical and health informatics students: systematic review, JMIR Med Educ source

      Not checked in the free run.

    • cites [42] J Grunhut, 2021, Educating future physicians in artificial intelligence (AI): An integrative review and proposed changes, J Med Educ Curric Dev source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Charow et al. support the need for AI-specific ethics to be taught in medical schools to address ethical, legal, and data governance issues.

    “Charow et al. support the need for AI-specific ethics to be taught in medical schools to address ethical, legal, and data governance issues [13].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    This study focused specifically on AI education in undergraduate medicine, rather than on the continuum of medical education.

    “Lastly, this study focused specifically on AI education in undergraduate medicine, rather than on the continuum of medical education.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    This is the first study that examines evidence-based recommendations for implementation of AI educational programs in global undergraduate medical curriculum.

    “This is the first study that examines the evidence-based recommendations for implementation of AI educational pro- grams in the global undergraduate medical curriculum.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation

    There needs to be capacity-building to increase the pool of qualified instructors as those already teaching are overwhelmed.

    “In addition, there needs to be capacity-building to increase the pool of qualified instructors as those already teaching is overwhelmed [48].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [48] Dandar V, Grigsby RK, Bunton S. Burnout among US medical school faculty. AAMC Anal Brief 2019; 19: 1–3. https://www.aamc.org/media/9921/download?attachment (Accessed Dec 15, 2022).

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Ejaz et al. reported that 43% of participants expressed worry regarding impact of AI on medicine, while 92% expressed that AI-related teaching needs to be incorporated into core medical curriculum.

    “For example, Ejaz et al. report that despite 43% of participants expressing worry regarding the impact of AI on medicine, 92% expressed that AI-related teaching needs to be incorporated into the core medical curriculum [26].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] H Ejaz, 2022, Artificial intelligence and medical education: A global mixed-methods study of medical students’ perspectives, Digit Health

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Although there is hesitancy from students surrounding AI implementation, most can see benefits outweighing risks, leading to a positive attitude toward learning about AI.

    “Although there is hesitancy from students surrounding AI implementation, most can see the benefits outweighing the risks, leading to a positive attitude toward learning about AI.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Establishing a strong theoretical base of AI for practice, including its strengths and limitations, is what should be strived for when educating future learners.

    “We also found that establishing a strong theoretical base of AI for practice, including its strengths and limitations, is what we should strive for when educating future learners.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Lee et al. suggest the need for development of a standardized list of core AI competencies and implementation of an AI curriculum in medical schools.

    “Lee et al. support this sentiment, suggesting the need for the development of a standardized list of core AI competencies and the implementation of an AI curriculum to medical schools [12].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Due to inability for contextualization of AI algorithms, physicians will still need to regularly conduct contextual analyses when handling care of patients to supplement AI.

    “Due to the inability for contextualization of AI algorithms, physicians will still need to regularly conduct contextual analyses when handling the care of patients to supplement the AI.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: AI algorithms cannot be contextualized.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    With a regulatory body and standardized curriculum, medical schools could ensure future practitioners are able to recognize and identify ethical and legal issues such as patient confidentiality, equity, inclusion, and patient rights.

    “With a regulatory body and a standardized curriculum, medical schools could ensure that future practitioners are able to recognize and identify ethical and legal issues that may arise such as patient confidentiality [13], equity [13,45], inclusion [13], and patient rights [13].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

    • cites [45] MP Khurana, 2022, Digital health competencies in medical school education: a scoping review and Delphi method study, BMC Med Educ source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Studies suggest that ML, DL, and NLP should be included in early years of training in undergraduate medical education to increase student exposure to AI.

    “Studies suggest that ML, DL, and NLP should be included in the early years of training in undergraduate medical education in order to increase student exposure to AI [12,13].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [12] J Lee, 2021, Artificial intelligence in undergraduate medical education: a scoping review, Acad Med source

      Not checked in the free run.

    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    This regulatory body would monitor data processing, regulate data sharing between institutions, and oversee general implementation of AI in healthcare.

    “This regulatory body would also monitor data processing, regulate data sharing between institutions, and oversee the general implementation of AI in healthcare [13].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] R Charow, 2021, Artificial intelligence education programs for health care professionals: Scoping review, JMIR Med Educ source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Interdisciplinary engagement with data scientists would improve AI infrastructure by solving regulatory, economic, and organisational obstacles and applying quality improvement techniques for constant monitoring.

    “Interdisciplinary engagement with data scientists would improve AI infrastructure by solving regulatory, economic, and organisational obstacles and applying quality improvement techniques for constant monitoring.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The feedback and attitude of medical trainees towards AI implementation in medical curriculum makes it evident that a systematic approach towards AI education in medicine is warranted.

    “The feedback and attitude of medical trainees towards AI implementation in medical curriculum makes it evident that a systematic approach towards AI education in medicine is warranted.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation

    As medical school curricula are already saturated with content, it may be difficult to move forward without gaining support of stakeholders.

    “As medical school curricula are already saturated with content [47], it may be difficult to move forward without gaining the support of these stakeholders.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47] W IsHak, 2013, Burnout in medical students: a systematic review, Clin Teach source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    This would allow physicians to explain strengths and limitations of AI in a way patients can understand.

    “This would allow physicians to explain the strengths and limitations of AI in a way that patients can understand.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Ejaz et al. reported that 86% of students were interested in more interdisciplinary learning within computer science and biomedical engineering fields.

    “The same authors also report that 86% of students were interested in more interdisciplinary learning within the computer science and biomedical engineering fields [26].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] H Ejaz, 2022, Artificial intelligence and medical education: A global mixed-methods study of medical students’ perspectives, Digit Health

      Not checked in the free run.

78 of 78

  • The authors state this as an assumption.

    Graduating medical students should have a basic understanding of AI and how it can improve patient care.

    “It is now reasonable to assume that graduating medical students should have a basic understanding of AI and how it can improve patient care.”

  • The paper relies on this without support.

    AI is advanced predictive statistics.

    “AI is advanced predictive statistics, therefore medical students could consider it part of their biostatistics coursework.”

  • The paper relies on this without support.

    AI algorithms cannot be contextualized.

    “Due to the inability for contextualization of AI algorithms, physicians will still need to regularly conduct contextual analyses when handling the care of patients to supplement the AI.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Few structured programs thoroughly address both the practical and ethical dimensions of AI in medicine.

    “Few structured programs thoroughly address both the practical and ethical dimensions of AI in medicine3–10.”

    From Developing a Canadian artificial intelligence medical curriculum using a Delphi study (2024) · cites this paper as [8]

    The Introduction says existing AI training programs are mostly extracurricular, vary substantially in content, and lack a unified medical curriculum. The Results and Discussion identify both practical application and ethics as important curricular topics, but do not establish how many existing structured programs thoroughly cover both. The review also focuses on undergraduate medical education, narrower than AI education in medicine generally. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Contributions elaborated mainly or solely by medical students strongly call for the inclusion of AI education in medical curricula; students were enthusiastic about learning AI and recognized its crucial role in the future of medicine, but reported a lack of formal AI training and insufficient AI-related learning opportunities; researchers and educators also demanded inclusion of AI in medical Cur

    “Finally, contributions elaborated mainly or solely by medical students strongly call for the inclusion of AI education in medical curricula [50–54]. Students were enthusiastic about learning AI and recognized its crucial role in the future of medicine [45, 55]. However, they reported a lack of formal AI training and insufficient AI-related learning opportunities in their medical education [46, 56,”

    From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [56]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • There will likely be a need for certain physicians to take on a larger role with respect to AI innovation and integration, but the vast majority will be using AI in their everyday practice.

    “There will likely be a need for certain physicians to take on a larger role with respect to AI innovation and integration, but the vast majority will be using AI in their everyday practice8,18.”

    From Developing a Canadian artificial intelligence medical curriculum using a Delphi study (2024) · cites this paper as [8]

  • Challenges related to faculty members included lack of AI knowledge and expertise, resistance to adopting new technologies, and struggles with staff shortages.

    “Challenges related to faculty members, such as the lack of AI knowledge and expertise [56, 180], resistance to adopting new technologies [5, 77, 180], and struggles with staff shortages, were also acknowledged in several studies [174].”

    From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [56]

  • Students support incorporating AI education into healthcare curricula.

    “Studies show that students support this idea [40, 41].”

    From Medical, dental, and nursing students’ attitudes and knowledge towards artificial intelligence: a systematic review and meta-analysis (2024) · cites this paper as [41]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Kansal et al. publication year differs

    Minor

    Table 1 gives 2019 for Kansal et al., while the reference gives 2022.

  • Reeder and Lee publication year differs

    Minor

    Table 1 gives 2021 for Reeder and Lee, while the reference gives 2022.

  • Park et al. publication year differs

    Minor

    Table 1 gives 2020 for Park et al., while the reference gives 2021.

  • Survey categories total 35, not 34

    Minor

    The reported survey and non-survey counts add to 35, although the results state that 34 studies were included.

    Reported 34 · recomputed 35

  • Reference-mined article count differs

    Minor

    The abstract says seven additional articles were included through reference mining, while the methods say eight.

  • Gong et al. publication year differs

    Minor

    Table 1 gives 2018 for Gong et al., while the reference gives 2019.

  • Theory-and-application subtheme has two names

    Minor

    The abstract names the second subtheme "knowledge for communication," whereas the results name it "knowledge for development."

  • Survey sample-size range excludes an included survey

    Minor

    The stated maximum survey sample size is 1459, but Table 1 lists Teng et al. as a survey study with a sample size of 2167.

8 of 8

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Dandar V, Grigsby RK, Bunton S. Burnout among US medical school faculty. AAMC Anal Brief 2019; 19: 1–3. https://www.aamc.org/media/9921/download?attachment (Accessed Dec 15, 2022).

Cited by

Sorted by citations

Related

Authors
Pupic, N., Ghaffari-zadeh, A., Hu, R., Singla, R., Darras, K., Karwowska, A., & Forster, B. B.
Affiliations
  • University of British Columbia (CA)
  • Queen's University (CA)
  • University of Ottawa (CA)
Corresponding author
Ghaffari-zadeh, Aryan
ORCID
Venue
PLOS Digital Health
Publisher
Public Library of Science (PLoS)
Published
2023-11-27
Volume
2
Issue
11
Pages
e0000255
ISSN
2767-3170 (ISSN-L)
DOI
10.1371/journal.pdig.0000255
PMC
PMC10681314
OpenAlex
W4389042814
Semantic Scholar
53fb5f7eb768582f52555c5a90701052e8a0b17c
Cited by
79
References
47
Citations by year
2026: 39 · 2025: 23 · 2024: 16 · 2023: 1
FWCI
0.92
Type
journal article
Language
en
Open access
Licence
Locations
  • PLOS Digital Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
Keywords
artificial intelligence in education, medical education, undergraduate medical curriculum, AI ethics, clinical communication, clinical decision-making, evidence-based recommendations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review