CoChat
Sign Up
Cited by 81Artificial Intelligence in Healthcare and EducationInnovations in Medical EducationArtificial Intelligence in Education

Developing Medical Education Curriculum Reform Strategies to Address the Impact of Generative AI: Qualitative Study

Open access
Shimizu, Ikuo, Kasai, Hajime, Shikino, Kiyoshi, Araki, Nobuyuki, Takahashi, Zaiya, Onodera, Misaki, Kimura, Yasuhiko, Tsukamoto, Tomoko +4 more·2023·JMIR Medical Education
DOI 10.2196/53466PMC PMC10722362

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Background: Generative artificial intelligence (GAI), represented by large language models, have the potential to transform health care and medical education. In particular, GAI’s impact on higher education has the potential to change students’ learning experience as well as faculty’s teaching. However, concerns have been raised about ethical consideration and decreased reliability of the existing examinations. Furthermore, in medical education, curriculum reform is required to adapt to the revolutionary changes brought about by the integration of GAI into medical practice and research. Objective: This study analyzes the impact of GAI on medical education curricula and explores strategies for adaptation. Methods: The study was conducted in the context of faculty development at a medical school in Japan. A workshop involving faculty and students was organized, and participants were divided into groups to address two research questions: (1) How does GAI affect undergraduate medical education curricula? and (2) How should medical school curricula be reformed to address the impact of GAI? The strength, weakness, opportunity, and threat (SWOT) framework was used, and cross-SWOT matrix analysis was used to devise strategies. Further, 4 researchers conducted content analysis on the data generated during the workshop discussions. Results: The data were collected from 8 groups comprising 55 participants. Further, 5 themes about the impact of GAI on medical education curricula emerged: improvement of teaching and learning, improved access to information, inhibition of existing learning processes, problems in GAI, and changes in physicians’ professionality. Positive impacts included enhanced teaching and learning efficiency and improved access to information, whereas negative impacts included concerns about reduced independent thinking and the adaptability of existing assessment methods. Further, GAI was perceived to change the nature of physicians’ expertise. Three themes emerged from the cross-SWOT analysis for curriculum reform: (1) learning about GAI, (2) learning with GAI, and (3) learning aside from GAI. Participants recommended incorporating GAI literacy, ethical considerations, and compliance into the curriculum. Learning with GAI involved improving learning efficiency, supporting information gathering and dissemination, and facilitating patient involvement. Learning aside from GAI emphasized maintaining GAI-free learning processes, fostering higher cognitive domains of learning, and introducing more communication exercises. Conclusions: This study highlights the profound impact of GAI on medical education curricula and provides insights into curriculum reform strategies. Participants recognized the need for GAI literacy, ethical education, and adaptive learning. Further, GAI was recognized as a tool that can enhance efficiency and involve patients in education. The study also suggests that medical education should focus on competencies that GAI hardly replaces, such as clinical experience and communication. Notably, involving both faculty and students in curriculum reform discussions fosters a sense of ownership and ensures broader perspectives are encompassed.

AI snapshot

Written from the paper’s full text.

TL;DR

This qualitative workshop study at a Japanese medical school used SWOT and cross-SWOT analysis with faculty and students to examine how generative AI affects undergraduate medical education curricula and how curricula should respond. It identified five impact themes and three reform strategy themes: learning about GAI, learning with GAI, and learning aside from GAI.

Key findings

  • The workshop involved 55 participants (49 faculty and 6 students) in 8 groups; content analysis of SWOT discussions produced 169 items and 5 themes: improvement of teaching and learning, improved access to information, inhibition of existing learning processes, problems in GAI, and changes in physicians’ professionality.
  • Positive impacts included improvement of teaching and learning (48 items; 37 strengths, 11 opportunities) and improved information access (20 items; 14 strengths, 6 opportunities). Negative impacts included inhibition of existing learning processes (36 items; 29 weaknesses, 7 threats) and potential problems in GAI (28 items; 11 weaknesses, 17 threats). Changes in physicians’ professionality was both positive and negative (37 items; 7 strengths, 13 opportunities, 4 weaknesses, 13 threats).
  • Cross-SWOT analysis produced 104 items and 3 curriculum reform themes: learning about GAI (22 items), learning with GAI (57 items), and learning aside from GAI (25 items).
  • Recommendations included GAI literacy, ethics, and compliance; using GAI to improve learning efficiency, generate materials, support information gathering, adaptive learning, group learning, and virtual patients; and maintaining GAI-free learning, fostering higher cognitive domains, increasing communication exercises, and promoting workplace participation. The strategies were incorporated into the university’s Guidelines on the Use of Generative AI in Teaching and Learning in October 2023.

Methods

  • The study was conducted during faculty development at Chiba University School of Medicine, Japan, in August 2023. A lecture on GAI was followed by a workshop with faculty and students divided into groups of 7-8 to address two research questions on GAI’s curricular impact and needed reform.
  • The workshop used the SWOT framework and cross-SWOT (TOWS) matrix to develop strategies. Faculty were invited from all 58 departments, with 47 agreeing; 55 participants took part, and all but 1 had no experience with credited AI or GAI courses.
  • Qualitative content analysis was conducted on the group work comments. Four researchers independently coded comments to identify themes, followed by checking by another researcher; disagreements were discussed by all authors until consensus.

Limitations

  • Stated: the study was conducted at a single medical school in Japan, and the authors note that GAI implementation in medicine may vary by cultural and curricular context.
  • Stated: the number of students was smaller than that of faculty, and although facilitators encouraged attention to hierarchy, no structural devices eliminated hierarchy between students and faculty.
  • Stated: the study did not include opinions from health professionals other than physicians; the authors recommend future interprofessional workshops. Evident from the text: the student sample was only 6 and students were selected from those who had attended formal meetings, which may limit representativeness.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/53466

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 37 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The qualitative analysis identified three curricular response principles: learning about GAI, learning with GAI, and learning aside from GAI.

    “Curricular response principles were broadly classified into “learning about GAI,” “learning with GAI,” and “learning aside from GAI.””
    Quote in the paperSubstantial claim
  • Findingown data

    A third major, both positive and negative, impact was identified as the change in physicians' professionality.

    “Further, a third major, both positive and negative, impact was identified as the change in physicians’ professionality.”
    Quote in the paperSubstantial claim
  • Findingown data

    One positive impact was the improvement of teaching and learning.

    “In total, 1 positive impact was the improvement of teaching and learning.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Content analysis of the cross-SWOT results established 3 themes from 104 items for medical education curriculum reform strategies to address the impact of GAI: learning about GAI, learning with GAI, and learning aside from GAI.

    “In terms of medical education curriculum reform strategies to address the impact of GAI, content analysis of results of the cross-SWOT analysis established 3 themes from 104 items (learning about GAI, learning with GAI, and learning aside from GAI; Table 5).”
    Quote in the paperSubstantial claim
  • Findingown data

    The study found a new approach that can promote the use of simulated patients in education and adaptive learning as part of "learning with GAI."

    “Interestingly, we found a new approach that can promote the use of simulated patients in education and adaptive learning as part of “learning with GAI.””
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 63
  • May combine two claims. 42

  • Classified as probably not a substantial claim.

    Findingown data

    Continuing with existing learning strategies would result in lower orders in the cognitive domain.

    “It was also noted that existing learner assessment methods would be less applicable and that continuing with existing learning strategies would result in lower orders in the cognitive domain.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Participants suggested that supporting adaptive and group learning and promoting the use of digital patients more diverse than real patients would introduce clinical case-based learning.

    “Moreover, participants pointed out that supporting adaptive and group learning as well as promoting the use of digital patients who are more diverse than real patients into education would introduce clinical case-based learning.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    GAI could be useful as a new emergent tool because it allows students to suggest ideas.

    “Further, GAI could be useful as a new emergent tool because it allows students to suggest ideas.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Participants expected that facilitating literacy in second languages, such as English for Japanese students, would improve the curriculum.

    “Hence, participants expected that facilitating literacy in second languages, such as English for Japanese students, would improve the curriculum.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: In cultures where English is not the native language, such as Japan, literacy in English is a major issue.

  • Classified as probably not a substantial claim.

    Findingown data

    Participants recommended incorporating GAI literacy, ethical considerations, and compliance into the curriculum.

    “Participants recommended incorporating GAI literacy, ethical considerations, and compliance into the curriculum.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants proposed learning with GAI for using GAI in existing curricula.

    “Learning with GAI was proposed for the use of GAI in existing curricula.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Learning with GAI involved improving learning efficiency, supporting information gathering and dissemination, and facilitating patient involvement.

    “Learning with GAI involved improving learning efficiency, supporting information gathering and dissemination, and facilitating patient involvement.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Teachers were concerned that the use of GAI would reduce learners' ability to think independently.

    “First, teachers were concerned that the use of GAI would reduce learners’ ability to think independently.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants expected that the educational significance of digital patients could be amplified through the use of GAI, which can easily generate a wide variety of problems.

    “While generally useful for understanding standardized clinical conditions [38], participants expected that the educational significance of digital patients could be amplified through the use of GAI, which can easily generate a wide variety of problems.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Content analysis of the workshop discussions using the SWOT framework resulted in 169 items, from which 5 themes about the impact of GAI on medical education curricula were established: improvement of teaching and learning, improved access to information, inhibition of the existing learning processes, problems in GAI, and changes in physicians' professionality.

    “In terms of the impact of GAI, content analysis of the discussions using the SWOT framework resulted in 169 items, from which 5 themes were established (improvement of teaching and learning, improved access to information, inhibition of the existing learning processes, problems in GAI, and changes in physicians’ professionality).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Participants suggested that teachers could use GAI to generate educational materials.

    “Further, it was suggested that teachers could also use GAI to generate educational materials.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Subthemes included those related to the positioning of medical expertise, such as the declining value of specialized knowledge and the ability to gather information from patients.

    “Subthemes included those related to the positioning of medical expertise, such as the declining value of specialized knowledge and the ability to gather information from patients, and those that were summative, such as the changing nature of work due to labor-saving clinical and research work, the resulting reevaluation of face-to-face contact, and the increased volatility of the roles of future”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Faculty members believed that GAI would assist them in creating better instructional content and materials and help them become more efficient.

    “The faculty members believed that GAI would assist them in creating better instructional content and materials and help them become more efficient.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participants recognized the need for GAI literacy, ethical education, and adaptive learning.

    “Participants recognized the need for GAI literacy, ethical education, and adaptive learning.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Participants’ perceptions and workshop discussions accurately reflect the actual impact of GAI on medical education curricula.

  • May combine two claims.

    Findingown data

    GAI was recognized as a tool that can enhance efficiency and involve patients in education.

    “Further, GAI was recognized as a tool that can enhance efficiency and involve patients in education.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Participants’ perceptions and workshop discussions accurately reflect the actual impact of GAI on medical education curricula.

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Learning aside from GAI emphasized maintaining GAI-free learning processes, fostering higher cognitive domains of learning, and introducing more communication exercises.

    “Learning aside from GAI emphasized maintaining GAI-free learning processes, fostering higher cognitive domains of learning, and introducing more communication exercises.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Participants suggested that medical education curricula include learning about GAI, specifically its characteristics, proper use in medicine, and ethics and compliance to mitigate its impact.

    “As for learning about GAI, participants suggested that in addition to learning the characteristics of GAI, they should learn about the proper use of GAI in medicine as well as ethics and compliance to mitigate its impact.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Subthemes included summative ones such as the changing nature of work due to labor-saving clinical and research work, the resulting reevaluation of face-to-face contact, and the increased volatility of the roles of future physicians.

    “Subthemes included those related to the positioning of medical expertise, such as the declining value of specialized knowledge and the ability to gather information from patients, and those that were summative, such as the changing nature of work due to labor-saving clinical and research work, the resulting reevaluation of face-to-face contact, and the increased volatility of the roles of future”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    It was noted that existing learner assessment methods would be less applicable.

    “It was also noted that existing learner assessment methods would be less applicable and that continuing with existing learning strategies would result in lower orders in the cognitive domain.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Another positive impact was improved access to information: participants believed that GAI would make it easier for students to gather information, given that the use of GAI is simpler than traditional search functions.

    “Another positive impact was improved access to information: the participants believed that the GAI would make it easier for students to gather information, given that the use of GAI is simpler than traditional search functions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Participants noted that clinical education requires resources such as case scenarios and images, and the ability to generate them would be useful for education.

    “The participants also noted that clinical education requires resources such as case scenarios and images, and the ability to generate them would be useful for education.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The five themes were categorized into positive, negative, and both positive and negative impacts based on the bias of the SWOT analysis.

    “These themes were categorized into positive, negative, and both positive and negative impacts based on the bias of the SWOT analysis (Table 4).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Potential problems in GAI such as doubt of authenticity and ethical issues were concerned.

    “Second, potential problems in GAI such as doubt of authenticity and ethical issues were concerned.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The study found perspectives that have not been recommended by AI experts in the past.

    “Furthermore, we found perspectives in this report that have not been recommended by AI experts in the past.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants were concerned that existing assessments using students' output may be less reliable.

    “In terms of learners’ assessment, participants were concerned that existing assessments using students’ output may be less reliable.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The negative impacts of GAI and the reform strategies to overcome them had much in common with public proposals, but it is significant that the faculty members themselves were able to outline their own proposals.

    “Among the items mentioned in the results of analysis, the negative impacts of GAI and the reform strategies to overcome them had much in common with public proposals, but it is significant that the faculty members themselves were able to outline their own proposals.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    A distinctive example is the recommendation that curricula should focus on "changes in physicians' competencies" as the impact of GAI and "learning aside from GAI" to respond to these impacts.

    “A distinctive example is the recommendation that curricula should focus on “changes in physicians’ competencies” as the impact of GAI and “learning aside from GAI” to respond to these impacts.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Concerns have been raised about ethical considerations and decreased reliability of existing examinations in relation to GAI.

    “However, concerns have been raised about ethical consideration and decreased reliability of the existing examinations.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [7] source

      Not checked in the free run.

    • cites [8] source

      Not checked in the free run.

    • cites [9] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    GAI has the potential to assist education by providing an adaptive and personalized environment.

    “On the one hand, GAI has the potential to assist education in terms of providing an adaptive and personalized environment [5].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Generative artificial intelligence (GAI), represented by large language models, has the potential to transform health care and medical education.

    “Generative artificial intelligence (GAI), represented by large language models, have the potential to transform health care and medical education.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [1] source

      Not checked in the free run.

    • cites [2] source

      Not checked in the free run.

    • cites [3] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    Although current GAI algorithms may have factual errors and biases, many nuanced responses point to its ability to enhance student learning.

    “Although current GAI algorithms may have factual errors and biases, many nuanced responses point to its ability to enhance student learning.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    Each group was assisted in discussions and work by 1 facilitator trained in SWOT analysis, in addition to the participants, to promote common discussions and minimize differences between groups.

    “To promote common discussions and minimize differences between groups, each group was assisted in discussions and work by 1 facilitator trained in SWOT analysis, in addition to the participants.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Education and health care are both major areas where SWOT has been frequently used.

    “In particular, education and health care are both major areas where SWOT has been frequently used [27].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The impact of GAI on education is disputed, with studies raising concerns about ethical considerations, evaluation practices, scientific integrity, and potential negative effects on students' higher order thinking skills.

    “On the other hand, the impact of GAI on education is disputed [6], with studies raising concerns about the ethical considerations of ChatGPT [7], evaluation practices [8], scientific integrity [9], and potential negative effects on students’ higher order thinking skills [10].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [6] source

      Not checked in the free run.

    • cites [7] source

      Not checked in the free run.

    • cites [8] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    GAI is disrupting traditional educational practices and forcing teachers to adapt to its potential positive and negative impacts.

    “As with past introductions of new technologies into education, GAI is disrupting traditional practices and forcing teachers to adapt to its potential positive and negative impacts [5].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [5] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Several examples of SWOT implementation in the medical education domain have been reported in academic journals and used for strategic planning in chaotic situations such as the COVID-19 pandemic in 2019.

    “Several examples of implementation in the medical education domain have been reported in academic journals and used for strategic planning in chaotic situations such as the COVID-19 pandemic in 2019 [28].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    There is a movement in higher education to limit learners' use of GAI.

    “Thus, there is a movement in higher education to limit learners’ use of GAI.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisereasoning

    Academic views on GAI do not uniformly declare that AI tools pose a serious threat to higher education.

    “These academic views on GAI do not uniformly declare that AI tools pose a serious threat to higher education.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The SWOT framework was first described academically by Learned et al and has been used as an important tool for dealing with complex strategic situations by reducing information to improve decision-making.

    “The SWOT framework was first described academically by Learned et al [25] and has been used as an important tool for dealing with complex strategic situations by reducing the amount of information to improve decision-making.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] Learned, EP, 1969, Business Policy: Text and Cases

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Premisecitation

    Many researchers expect that academia will adapt its teaching and assessment practices to accommodate a world where AI is freely available.

    “Further, many researchers expect that academia will adapt its teaching and assessment practices to accommodate the new reality of living, working, and learning in a world where AI is freely available [15].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] LiuABridgemanDMillerBAs uni goes back, here's how teachers and students can use ChatGPT to save time and improve learningThe Conversation20232023-11-23https://theconversation.com/as-uni-goes-back-heres-how-teachers-and-students-can-use-chatgpt-to-save-time-and-improve-learning-199884

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The achievement objectives included in the MCC define approximately two-thirds of each university's curriculum, with the remainder allowed to be unique to each university.

    “The latest MCC (revised in 2022) lists the ability to recognize and use information technology, including AI, as 1 of the 10 core competencies; the achievement objectives included in the MCC define approximately two-thirds of each university’s curriculum, with the remainder allowed to be unique to each university [20].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] The model core curriculum for medical education in JapanMedical Education Model Core Curriculum Expert Research Committee20222023-09-30https://www.mext.go.jp/content/20230315-mxt_igaku-000026049_00003.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The latest MCC (revised in 2022) lists the ability to recognize and use information technology, including AI, as 1 of the 10 core competencies.

    “The latest MCC (revised in 2022) lists the ability to recognize and use information technology, including AI, as 1 of the 10 core competencies; the achievement objectives included in the MCC define approximately two-thirds of each university’s curriculum, with the remainder allowed to be unique to each university [20].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] The model core curriculum for medical education in JapanMedical Education Model Core Curriculum Expert Research Committee20222023-09-30https://www.mext.go.jp/content/20230315-mxt_igaku-000026049_00003.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Points of disagreement on the data were discussed by all authors, and consensus was reached.

    “Points of disagreement on the data were discussed by all authors, and consensus was reached.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    Four researchers read the comments at the discussion and coded them to identify themes independently, followed by SI checking the analysis.

    “Further, 4 researchers (IS, HK, KS, and NA) read the comments at the discussion and coded them to identify themes that emerged from the qualitative data independently, followed by SI checking the analysis.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisereasoning

    The SWOT and cross-SWOT methods are suitable for the study's research questions because brainstorming questions can be used to reach a consensus.

    “We chose the SWOT and cross-SWOT methods for this study because, first, they are suitable for our research questions since brainstorming questions can be used to reach a consensus.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    SWOT has been used to find gaps and matches between competencies and resources and the business environment because it can assess alternatives and complex decision-making situations.

    “Specifically, it has been used to find gaps and matches between competencies and resources and the business environment [26] because it can assess alternatives and complex decision-making situations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] Wheelan, TL, 1998, Strategic Management and Business Policy. 5th Edition

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisereasoning

    The SWOT and cross-SWOT methods can establish strategies based on external as well as internal factors and barriers as well as facilitators.

    “Further, these methods can establish strategies based on external as well as internal factors and barriers as well as facilitators.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Premisenone given

    Medical education is not only a type of higher education but also a place to acquire the professional competencies required for postgraduate work.

    “This is because medical education is not only a type of higher education but also a place to acquire the professional competencies required for postgraduate work.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    All medical schools in Japan offer a 6-year curriculum to students entering after high school and share a model core curriculum (MCC) as the nationally uniform exit competency for certifying medical degrees.

    “All medical schools in Japan offer a 6-year curriculum to students entering after high school [19] and share a model core curriculum (MCC) as the nationally uniform exit competency for certifying medical degrees.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    The combined use of GAI in the clinical workplace is expected to be part of a physician's skill set and, through training, will assist physicians in practice.

    “Hence, the combined use of GAI in the clinical workplace is expected to be part of a physician’s skill set and, through training, will be expected to assist physicians in the practice [33].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The inclusion of students in addition to medical school faculty from all areas of medical education helped to strengthen the conclusions about GAI.

    “Similarly, in this study, the inclusion of students in addition to medical school faculty from all areas of medical education in this study of GAI curriculum reform helped to strengthen the conclusions about GAI.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationown data

    This study highlights the profound impact of GAI on medical education curricula and provides insights into curriculum reform strategies.

    “This study highlights the profound impact of GAI on medical education curricula and provides insights into curriculum reform strategies.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Participants’ perceptions and workshop discussions accurately reflect the actual impact of GAI on medical education curricula.
    • Rests on an assumption: The workshop participants (faculty and students at one Japanese medical school) are sufficiently representative that the identified themes and strategies apply to medical education curricula more broadly.
    • Rests on an assumption: Qualitative content analysis of workshop comments can validly identify themes about GAI’s impact and curriculum reform.

  • May combine two claims.

    Interpretationcitation

    The need to learn about the shortcomings of GAI and specific ideas for incorporating GAI into existing educational strategies were consistent with the recommendations of experts.

    “Among the study’s findings, the need to learn about the shortcomings of GAI and some of the specific ideas for incorporating GAI into existing educational strategies were consistent with the recommendations of experts [34].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    GAI can extend views on a problem in clinical reasoning and add differential diagnoses.

    “In clinical reasoning [36], for example, GAI can extend our views on a problem. Ultimately, it can add several differential diagnoses that we may not have thought of.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [36] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    Involving both faculty and students in curriculum reform discussions fosters a sense of ownership and ensures broader perspectives are encompassed.

    “Notably, involving both faculty and students in curriculum reform discussions fosters a sense of ownership and ensures broader perspectives are encompassed.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Participants’ perceptions and workshop discussions accurately reflect the actual impact of GAI on medical education curricula.
    • Rests on an assumption: The workshop participants (faculty and students at one Japanese medical school) are sufficiently representative that the identified themes and strategies apply to medical education curricula more broadly.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    A total of 55 participants (49 faculty and 6 students) discussed the group work, and all agreed to participate in this study.

    “A total of 55 participants (49 faculty and 6 students) discussed the group work, and all agreed to participate in this study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The study was performed following the Declaration of Helsinki and approved by the ethics committee or institutional review board at Chiba University Graduate School of Medicine (3425).

    “This study was performed following the Declaration of Helsinki and approved by the ethics committee or institutional review board at Chiba University Graduate School of Medicine (3425).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    All participants were informed in advance via a written document that their opinions would be recorded anonymously and analyzed collectively and that they could not withdraw after participation, and they communicated their consent on paper.

    “All participants were informed in advance via a written document that their opinions would be recorded anonymously and analyzed collectively and that they could not withdraw after participation. The participants then communicated their consent on paper.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Competencies that cannot be replaced by GAI will continue to be required, and as GAI becomes more prevalent in the clinical workplace, the role of physicians is expected to be focused on tasks that cannot be replaced by AI.

    “In areas such as medicine, where competencies of health professions are indispensable human resources, competencies that cannot be replaced by GAI, as listed from the cross-SWOT analysis in our study, will continue to be required, and as GAI becomes more prevalent in the clinical workplace, the role of physicians is expected to be focused on tasks that cannot be replaced by AI.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    When such an innovative technology is quickly introduced, it is essential that the entire organization is ready to embrace change.

    “When such an innovative technology is quickly introduced, it is essential that the entire organization is ready to embrace change [46].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [46] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Experience in the workplace cannot be replaced by GAI, nor can evaluation be faked, and hence will be emphasized more in future medical education curricula.

    “For example, experience in the workplace cannot be replaced by GAI, nor can evaluation be faked, and hence will be emphasized more in future medical education curricula.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Experience in the workplace cannot be replaced by GAI, nor can evaluation be faked.

  • Classified as probably not a substantial claim.

    Interpretationnone given

    The importance of assessing skills and attitudes will increase because they will account for a greater proportion of a physician's competence.

    “Simultaneously, the importance of assessing skills and attitudes will increase because they will account for a greater proportion of a physician’s competence.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    A novel assessment system has been proposed, by which the utility of assessment can complement the former approach by integrating the results of various types of feedback opportunities programmatically.

    “Instead, a novel assessment system has been proposed, by which the utility of assessment can complement the former approach by integrating the results of various types of feedback opportunities programmatically [42].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [42] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    These principles will be the 3 pillars of medical education curriculum reform in the GAI era.

    “These principles will be the 3 pillars of medical education curriculum reform in the GAI era.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The faculty development program allows faculty members to formulate their own curriculum reform proposals and thus is expected to lead to the introduction of more effective ways of incorporating GAI in the context of each university.

    “Conversely, our faculty development program allows faculty members to formulate their own curriculum reform proposals and thus is expected to lead to the introduction of more effective ways of incorporating GAI in the context of each university.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Steinert et al points out that faculty development is also a place of community of practice.

    “Furthermore, Steinert et al [46] points out that faculty development is also a place of community of practice.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [46] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Certain commonalities in higher education and undergraduate medical education competencies may serve as an example of how GAI may be used.

    “However, certain commonalities in higher education and undergraduate medical education competencies may serve as an example of how GAI may be used.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    The involvement of students, who are stakeholders in the learning process, and the fact that GAI provided an opportunity to incorporate students' opinions on matters that will have a large impact on their future as health professionals will give validity to the reformed curriculum.

    “From this perspective, it is significant that students, who are stakeholders in the learning process, are also involved, and the fact that the GAI has provided an opportunity to incorporate students’ opinions on matters that will have a large impact on their future as health professionals will give validity to the reformed curriculum.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    The issue of GAI has a common impact on faculty and medical students, so hierarchy is not expected to have had a significant impact on the product.

    “However, since the issue of GAI has a common impact on faculty and medical students, we do not expect that hierarchy had a significant impact on the product.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The issue of GAI has a common impact on faculty and medical students.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    GAI is not just an educational device or technique; rather, it has the potential to revolutionize education, and its technological progress is rapid.

    “In this respect, GAI is not just an educational device or technique; rather, it has the potential to revolutionize education, and its technological progress is rapid.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    The number of students was smaller than that of faculty, and the facilitator did not incorporate structural devices to eliminate the issue of hierarchy between students and faculty.

    “Second, the number of students was smaller than that of faculty, and although the facilitator encouraged participants to pay attention to the issue of hierarchy and generate opinions during the workshop, he did not incorporate any structural devices to eliminate the issue of hierarchy between students and faculty.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Adopting a bottom-up approach facilitates faculty consensus on the curriculum and minimizes gaps in the objectives of reform and what needs to be done.

    “Specifically, adopting a bottom-up approach facilitates faculty consensus on the curriculum and minimizes gaps in the objectives of reform and what needs to be done.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Evaluating GAI-generated information could be an effective learning and assessing approach.

    “In this respect, evaluating GAI-generated information could also be an effective learning and assessing approach.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Since medicine is highly context dependent, future implementation of GAI in medicine may vary greatly depending on cultural and curricular characteristics.

    “Since medicine is highly context dependent, future implementation of GAI in medicine may vary greatly depending on cultural and curricular characteristics.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Medicine is highly context dependent.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Learners could expect to reach higher orders of cognitive domain such as "apply" and "analyze," rather than "know" and "understand."

    “Moreover, learners could expect to reach higher orders of cognitive domain such as “apply” and “analyze,” rather than “know” and “understand.””
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    These transitions are consistent with, and in fact promote, the paradigm shift in medical education, noted a decade ago.

    “These transitions are consistent with, and in fact promote, the paradigm shift in medical education, noted a decade ago [43].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] source

      Not checked in the free run.

76 of 76

  • The paper relies on this without support.

    In cultures where English is not the native language, such as Japan, literacy in English is a major issue.

    “Further, in cultures where English is not the native language, such as Japan, literacy in English—the de facto standard academic language of the world—is a major issue.”

  • The paper relies on this without support.

    AI is being used routinely in clinical and medical research workplaces.

    “Hence, as AI is being used routinely in clinical and medical research workplaces, literacy of information technologies, including AI and GAI, should be part of competencies acquired at graduation.”

  • The paper relies on this without support.

    Participants’ perceptions and workshop discussions accurately reflect the actual impact of GAI on medical education curricula.

    “This study highlights the profound impact of GAI on medical education curricula and provides insights into curriculum reform strategies.”

  • The paper relies on this without support.

    The workshop participants (faculty and students at one Japanese medical school) are sufficiently representative that the identified themes and strategies apply to medical education curricula more broadly.

    “This study highlights the profound impact of GAI on medical education curricula and provides insights into curriculum reform strategies.”

  • The paper relies on this without support.

    GAI will continue to be integrated into medical practice and research, requiring curriculum reform.

    “Furthermore, in medical education, curriculum reform is required to adapt to the revolutionary changes brought about by the integration of GAI into medical practice and research.”

  • The paper relies on this without support.

    Qualitative content analysis of workshop comments can validly identify themes about GAI’s impact and curriculum reform.

    “Qualitative content analysis was conducted to analyze the comments in the product of group work consisting of SWOT and cross-SWOT [30].”

  • The paper relies on this without support.

    The general state of medical education curricula has not yet been well defined, and no study has yet investigated and summarized the needs of medical schools and their faculty members who manage the actual curriculum.

    “However, the general state of medical education curricula has not yet been well defined, and to our knowledge, no study has yet investigated and summarized the needs of medical schools and their faculty members who manage the actual curriculum.”

  • The paper relies on this without support.

    Policies for the use of GAI that have been formulated in various countries and several universities have adopted a top-down approach.

    “Similarly, policies for the use of GAI that have been formulated in various countries and several universities have adopted the same top-down approach.”

  • The paper relies on this without support.

    The issue of GAI has a common impact on faculty and medical students.

    “since the issue of GAI has a common impact on faculty and medical students”

  • The paper relies on this without support.

    Medicine is highly context dependent.

    “Since medicine is highly context dependent”

  • The paper relies on this without support.

    Experience in the workplace cannot be replaced by GAI, nor can evaluation be faked.

    “experience in the workplace cannot be replaced by GAI, nor can evaluation be faked”
11 of 11

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Several studies have highlighted the benefits of using GenAI for adaptive curriculum development.

    “Several studies have highlighted the benefits of using GenAI for adaptive curriculum development (Ruiz-Rojas et al., 2023;Shimizu et al., 2023;Singha & Singha, 2024).”

    From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Shimizu et al., 2023

    The Discussion (Principal Findings; page unavailable) says GenAI can help create educational resources with less effort and promote adaptive learning, and it hopes those resources will inform medical curricula. The paper also explores curriculum reform strategies, but it does not show GenAI being used to develop adaptive curricula; its reform proposals were formulated by faculty and students. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Medical students and educators found GAI enhances clinical reasoning.

    “Two previous descriptive interview studies showed mixed attitudes toward GAI; medical students and educators found it enhances clinical reasoning [22], while nursing students believed it improves understanding and efficiency but worried it may impede critical thinking and emotional skills [23].”

    From Generative artificial intelligence in physiotherapy education: great potential amidst challenges- a qualitative interview study (2025) · cites this paper as [22]

    The Discussion (Principal Findings; page unavailable) says GAI can broaden clinical reasoning by suggesting additional differential diagnoses. But the Results describe students’ and faculty members’ views on learning support and clinical case materials, not a finding that GAI enhances clinical reasoning; the citing sentence presents that potential benefit as a participant finding. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Medical students do not resist the use of AI in medical education.

    “As previously reported, medical students do not resist the use of AI in medical education [21], which was evident in this study.”

    From Enhancing Medical Interview Skills Through AI-Simulated Patient Interactions: Nonrandomized Controlled Trial (2024) · cites this paper as [21]

    The Results (page unavailable) say that students thought generative AI could assist their learning, which supports a positive attitude toward its use. The paper does not establish that medical students generally do not resist AI in medical education; that broader claim goes beyond the reported views. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Completely replacing traditional instructor-led training with AI carries risks.

    “Completely replacing traditional instructor-led training with AI carries risks, and further studies thereon are required [13,21].”

    From Enhancing Medical Interview Skills Through AI-Simulated Patient Interactions: Nonrandomized Controlled Trial (2024) · cites this paper as [21]

    The Results (page unavailable) report faculty concerns that GAI could reduce students’ independent thinking and undermine existing assessment methods. This supports the general concern about risks, but the study does not examine completely replacing instructor-led training with AI or establish the risks of that specific change. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Eight groups cannot each have 7-8 of the 55 participants

    Minor

    Eight groups of at least 7 participants would require at least 56 participants, but the paper reports 55.

  • Six students cannot be assigned across all eight groups

    Minor

    The paper reports 6 students and 8 groups, so at least 2 groups could not have had a student assigned.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Wheelan, TL, 1998, Strategic Management and Business Policy. 5th Edition

Cited by

Sorted by citations

Related

Authors
Shimizu, I., Kasai, H., Shikino, K., Araki, N., Takahashi, Z., Onodera, M., Kimura, Y., Tsukamoto, T., Yamauchi, K., Asahina, M., Ito, S., & Kawakami, E.
Affiliations
  • Chiba University (JP)
  • Chiba University Hospital (JP)
Corresponding author
Shimizu, Ikuo
ORCID
Venue
JMIR Medical Education
Publisher
JMIR Publications Inc.
Published
2023-11-30
Volume
9
Pages
e53466
ISSN
2369-3762 (ISSN-L)
DOI
10.2196/53466
PMC
PMC10722362
OpenAlex
W4388861785
Cited by
81
References
44
Citations by year
2026: 34 · 2025: 29 · 2024: 17 · 2023: 1
FWCI
3.23
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Medical Education (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Japan Society for the Promotion of Science (23K21504)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
Keywords
generative AI, medical education, curriculum reform, large language models, AI literacy, AI ethics education, faculty development, undergraduate medical education, assessment methods, independent thinking, physician professionalism, SWOT analysis, communication skills

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review