The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 17 of the 23 sources its claims cite were checked for support.
Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial
Open accessAbstract
BACKGROUNDS: Traditional methods of teaching history-taking in medical education are limited by scalability and resource intensity. This study aims to assess the effectiveness of simulated patient interactions based on a custom-designed Generative Pre-trained Transformer (GPT) model, developed using OpenAI's ChatGPT GPTs platform, in enhancing medical students' history-taking skills compared to traditional role-playing methods. METHODS: A total of 56 medical students were randomly assigned into two groups: an GPT group using GPT-simulated patients and a control group using traditional role-playing. Pre- and post-training assessments were conducted using a structured clinical examination to measure students' abilities in history collection, clinical reasoning, communication skills, and professional behavior. Additionally, students' evaluations of the educational effectiveness, satisfaction, and recommendation likelihood were assessed. RESULTS: The GPT-simulation group showed significantly higher post-training scores in the structured clinical examination compared to the control group (86.79 ± 5.46,73.64 ± 4.76, respectively, P < 0.001). Students in the GPT group exhibited higher enthusiasm for learning, greater self-directed learning motivation, and better communication feedback abilities compared to the control group (P < 0.05). Additionally, the student satisfaction survey revealed that the GPT group rated higher on the diversity of diseases encountered, ease of use, and likelihood of recommending the training compared to the control group (P < 0.05). CONCLUSIONS: GPT-based history-taking training effectively enhances medical students' history-taking skills, providing a solid foundation for the application of artificial intelligence (AI) in medical education. CLINICAL TRIAL NUMBER: NCT06766383.
Overview
CoChat Summary
TL;DR
This single-center randomized clinical trial found that a custom GPT-based simulated-patient system improved fifth-year medical students’ history-taking performance more than traditional instructor-led role-playing. The GPT group scored significantly higher on the post-training structured clinical examination and reported greater motivation, satisfaction, and lower anxiety.
Key findings
- Post-training structured clinical examination scores were significantly higher in the GPT group than the control group: 86.79 ± 5.46 vs. 73.64 ± 4.76, P < 0.001, with a reported Cohen’s d = 2.57.
- Score improvement from pre- to post-training was also greater in the GPT group: 29.39 ± 13.14 vs. 18.96 ± 12.00, P = 0.003.
- GPT-group students reported significantly higher post-class self-directed learning (median 3 [IQR 2–4] vs. 1.5 [1–2], P = 0.004), enthusiasm for learning (3 [2–4] vs. 1.5 [1–2], P < 0.001), communication/feedback skills (4 [2–4] vs. 1.5 [1–2], P = 0.006), and interview logical reasoning (4 [2.25–4] vs. 1.5 [1–2], P = 0.036).
- Satisfaction ratings were significantly higher in the GPT group for diversity of diseases encountered (4 [3–4] vs. 1.5 [1–2], P = 0.004), ease of use (3 [2–4] vs. 1.5 [1–2], P < 0.001), and likelihood of recommending the training (3 [2–4] vs. 1.5 [1–2], P < 0.001); naturalness of interaction did not differ significantly (P = 0.973).
Methods
- Single-center randomized controlled trial with 56 fifth-year medical students from Anhui Medical University, randomly assigned 1:1 to a GPT-simulated patient group (n = 28) or a traditional role-playing control group (n = 28); no dropouts.
- The GPT group used a custom GPT developed on OpenAI’s ChatGPT GPTs platform with uploaded internal medicine/diagnostic references and prompt engineering, no fine-tuning; sessions were 30 minutes, three times per week for four weeks, matched to control sessions with clinical-skills instructors.
- Outcomes included a pre- and post-training structured clinical examination adapted from OSCE, scored by two blinded instructors across history collection, clinical reasoning, communication skills, and professional behavior; secondary outcomes were 5-point Likert questionnaires on educational effectiveness and satisfaction, plus qualitative thematic analysis. Statistical comparisons used independent-samples t-tests, rank-sum tests, chi-square tests, and Cohen’s d.
Limitations
- The paper states the main limitations were a relatively small sample size, single-institution conduct, short follow-up without long-term educational effects, and no formal inter-rater reliability indices (e.g., percentage agreement, ICC, or G-coefficients), so scorer variability cannot be ruled out.
- The paper also notes GPT-specific limitations: the system is highly data-dependent, and it cannot display visual non-verbal behaviors such as facial expression, eye contact, or hand gestures, which may have reduced communicative complexity and may partially inflate the GPT group’s advantage over role-play.
- Evident from the text: the custom GPT used platform-default inference settings, the specific GPT-4 sub-model was not disclosed, and no model fine-tuning was performed, which may limit reproducibility and adaptive learning claims.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
11 Valid · 2 Not valid · 3 Not checked · 79 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Additionally, due to the inherent strain in doctor-patient relationships [3], it is impractical to find a sufficient number of patients for each student to gain adequate interviewing experience.”
§ Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial > Background
What the sources say
- Supportedcites [3] Liang Z, Xu M, Liu G, Zhou Y, Howard P. Patient-centred care and patient autonomy: doctors’ views in Chinese hospitals. BMC Med Ethics. 2022;23(1):38. source
The citation is attached to the statement about strained doctor–patient relationships, not the subsequent claim about finding patients for student interviews. The Background identifies erosion of trust as a major reason for medical disputes, and the Discussion describes rising disputes as indicating eroded trust between patients and doctors. No page numbers are available in our copy. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, most existing research remains conceptual or exploratory, with limited empirical evidence on the practical implementation of GPT-based systems in specific skill domains.”
§ Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial > Background
What the sources say
- Partly supportedcites [4] Stretton B, Kovoor J, Arnold M, Bacchi S. ChatGPT-Based learning: generative artificial intelligence in medical education. Med Sci Educ. 2024;34(1):215–7. sourceExperimental — accuracy not yet measured
The paper’s conclusion (page unavailable in our copy) calls for further research on the effectiveness and implementation of ChatGPT-generated CBL/PBL cases, supporting a gap in practical evidence for that application. It does not establish that most existing research across GPT-based medical education is conceptual or exploratory, or assess evidence across specific skill domains; it also describes existing experimental work on medical proficiency. (LLM, conf=0.83)
- Partly supportedcites [5] Lee H. The rise of chatgpt: exploring its potential in medical education. Anat Sci Educ. 2024;17(5):926–31. sourceExperimental — accuracy not yet measured
The Potential Applications and Benefits section says ChatGPT’s efficacy in medical education “remains to be seen in practice,” and the Conclusions call for robust evaluations of learning outcomes. This supports the concern about limited practical evidence, but the paper does not establish that most existing research is conceptual or exploratory, or assess evidence across specific skill domains. (LLM, conf=0.83)
- Could not verifycites [6] Holderried F, Stegemann-Philipps C, Herrmann-Werner A, Festl-Wietek T, Holderried M, Eickhoff C, et al. A Language Model-Powered simulated patient with automated feedback for history taking: prospective study. JMIR Med Educ. 2024;10:e59213. sourceExperimental — accuracy not yet measured
The Introduction describes the study’s focus on GPT-4 feedback for history taking, and the Discussion’s Comparison With Prior Work cites other empirical studies. Neither section characterizes most existing research as conceptual or exploratory or assesses how much empirical evidence exists for implementation in specific skill domains. Searches for those characterizations found no supporting passage in our copy. (LLM, conf=0.72)
- Partly supportedcites [7] Brügge E, Ricchizzi S, Arenbeck M, Keller MN, Schur L, Stummer W, et al. Large Language models improve clinical decision making of medical students through patient simulation and structured feedback: a randomized controlled trial. BMC Med Educ. 2024;24(1):1391. sourceExperimental — accuracy not yet measured
The “LLM-simulated CDM-training” section says more empirical studies are needed to evaluate LLMs in clinical decision-making training. “Incorporating LLM-generated feedback” says few studies of LLM-generated patient simulations existed and none had implemented feedback or debriefing. These passages support a gap in that specific skill domain, but not the broader assertion that most GPT-based medical-education research is conceptual or exploratory. Page numbers are unavailable in our copy. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“students in the GPT group experienced less anxiety during the interviews (P < 0.001)”
§ Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial > Results > Feedback on educational effectiveness
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Students’ qualitative feedback supported this, with many emphasizing the benefit of being able to “practice anytime and receive useful suggestions right away.””
§ Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial > Discussion > Interpretation of results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thematic analysis of student feedback revealed three key themes: perceived usefulness, convenience, and engagement.”
§ Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial > Results > Qualitative feedback
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
35 more claims
Create a free account to see all 95 claims and every source verdict.
- Probably not a substantial claim. 25
- It may state two separate claims as one. 37
Citations
Cites
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944 · cited 1×
“These improvements likely stem from the GPT system’s flexibility, allowing learners autonomy in pacing and scenario selection, thus promoting intrinsic motivation—a critical factor for sustained educational impact [13, 14].”
Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions
Abd-alrazaq, A., AlSaad, R., Alhuwail, D., Ahmed, A., Healy, P. M., Latifi, S., Aziz, S., Damseh, R., Alabed Alrazak, S., & Sheikh, J. (2023) · JMIR Medical Education · doi:10.2196/48291 · Cited by 722 · cited 1×
“With the rapid development of AI technology in recent years, its application in medical education has been steadily increasing [19].”
Boscardin, C. K., Gin, B., Golde, P. B., & Hauer, K. E. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005439 · Cited by 620 · cited 1×
“Moreover, several reviews have highlighted the broader potential of generative AI in enhancing learner engagement, facilitating self-directed learning, and supporting personalized feedback in medical training [5, 8, 9].”
The rise of ChatGPT : Exploring its potential in medical education
Lee, H. (2024) · Anatomical Sciences Education · doi:10.1002/ase.2270 · Cited by 596 · cited 2×
“Moreover, several reviews have highlighted the broader potential of generative AI in enhancing learner engagement, facilitating self-directed learning, and supporting personalized feedback in medical training [5, 8, 9].”
Preiksaitis, C., & Rose, C. (2023) · JMIR Medical Education · doi:10.2196/48785 · Cited by 552 · cited 1×
“Moreover, several reviews have highlighted the broader potential of generative AI in enhancing learner engagement, facilitating self-directed learning, and supporting personalized feedback in medical training [5, 8, 9].”
Role play with large language models
Shanahan, M., McDonell, K., & Reynolds, L. (2023) · Nature · doi:10.1038/s41586-023-06647-8 · Cited by 491 · cited 1×
“This supports Shanahan et al.‘s view that GPT agents can simulate a wide range of characters through ongoing, adaptive conversation [12].”
Role-Play with Large Language Models
Shanahan, M., McDonell, K., & Reynolds, L. (2023) · arXiv · doi:10.48550/arxiv.2305.16367 · Cited by 490
A systematic review of large language models and their implications in medical education
Lucas, H. C., Upperman, J. S., & Robinson, J. R. (2024) · Medical Education · doi:10.1111/medu.15402 · Cited by 287 · cited 1×
“GPT group showed significantly greater improvement in most primary and secondary outcomes compared to the control group trends in the application of technology-driven educational tools in the field of medicine [11].”
Teaching history taking to medical students: a systematic review
Keifenheim, K. E., Teufel, M., Ip, J., Speiser, N., Leehr, E. J., Zipfel, S., & Herrmann-Werner, A. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0443-x · Cited by 218
Effective virtual patient simulators for medical communication training: A systematic review
Lee, J., Kim, H., Kim, K. H., Jung, D., Jowsey, T., & Webster, C. S. (2020) · Medical Education · doi:10.1111/medu.14152 · Cited by 210 · cited 2×
“Although emerging technologies such as virtual reality and machine learning offer valuable supplements, they do not fully resolve the limitations of conventional approaches, which still require critical reform to meet the demands of modern clinical training [1, 2].”
Cited by
Kıyak, Y. S., İş-Kara, T., & Emekli, E. (2026) · Medical Science Educator · doi:10.1007/s40670-025-02621-3 · Cited by 11
Lai, N. M., Lim, Y. S., Thein, W. M., Bhargava, P., Thomas, P., & Ong, Q. C. (2026) · JMIR Medical Education · doi:10.2196/88933 · Cited by 13
Müller, Y., Lemberg, H., Gellert, P. R., & Zoellick, J. C. (2026) · JMIR Medical Education · doi:10.2196/88092 · Cited by 6
“Finally, one other study used a similar approach to communication training as we did [32].”
Thind, B. S., Javidi, D., & Schwartz, L. M. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1830254 · Cited by 6
“…patients or generative clinical simulations, making this the most mature and widely implemented category to date, as reflected by publication volume, diversity of institutional contexts, and frequency of curricular deployment across the included literature (14, 17 – 19, 41 – 43, 46 – 49, 52).”
Xie, G., Liao, J., Tang, X., Fu, H., Liu, D., Deng, L., Wang, T., & Hong, X. (2025) · BMC Medical Education · doi:10.1186/s12909-026-09458-3 · Cited by 4
“Most studies lacked pre-registered protocols [25, 31], suggesting a risk of selective outcome reporting.”
Influential
Metadata
- Authors
- Wang, Z., Fan, T.-T., Li, M.-L., Zhu, N.-J., & Wang, X.-C.
- Affiliations
- Second Affiliated Hospital of Anhui Medical University (CN)
- Venue
- BMC Medical Education
- Published
- 2025-07-10
- Volume
- 25
- Issue
- 1
- Pages
- 1030-1030
- ISSN
- 1472-6920 (ISSN-L)
- PMC
- PMC12243320
- OpenAlex
- W4412177686
- Cited by
- 32
- References
- 20
- Citations by year
- 2026: 28 · 2025: 2
- FWCI
- 7.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- BMC Medical Education (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- 2023 Anhui Province Quality Engineering Project (2023jyxm1150)
- 2024 Anhui Province Quality Engineering Project (2024jyxm0779)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Social Sciences › Education › Simulation-Based Education in Healthcare
- Keywords
- history-taking skills, generative pre-trained transformer, ChatGPT, medical students, role play, artificial intelligence in medical education, clinical reasoning, communication skills
- MeSH terms
- Adult, Clinical Competence, Education, Medical, Education, Medical, Undergraduate, Educational Measurement, Feasibility Studies, Female
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial. https://app.cochat.ai/papers/10.1186/s12909-025-07614-9
- The paper
- Wang, Z., Fan, T.-T., Li, M.-L., Zhu, N.-J., & Wang, X.-C. (2025). Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial. BMC Medical Education. https://doi.org/10.1186/s12909-025-07614-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.