Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 17 of the 23 sources its claims cite were checked for support.

Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial

Open access
Wang, Zhen, Fan, Ting-Ting, Li, Meng-Li, Zhu, Nin-Jun, Wang, Xiao-Chen·2025·BMC Medical Education

Abstract

BACKGROUNDS: Traditional methods of teaching history-taking in medical education are limited by scalability and resource intensity. This study aims to assess the effectiveness of simulated patient interactions based on a custom-designed Generative Pre-trained Transformer (GPT) model, developed using OpenAI's ChatGPT GPTs platform, in enhancing medical students' history-taking skills compared to traditional role-playing methods. METHODS: A total of 56 medical students were randomly assigned into two groups: an GPT group using GPT-simulated patients and a control group using traditional role-playing. Pre- and post-training assessments were conducted using a structured clinical examination to measure students' abilities in history collection, clinical reasoning, communication skills, and professional behavior. Additionally, students' evaluations of the educational effectiveness, satisfaction, and recommendation likelihood were assessed. RESULTS: The GPT-simulation group showed significantly higher post-training scores in the structured clinical examination compared to the control group (86.79 ± 5.46,73.64 ± 4.76, respectively, P < 0.001). Students in the GPT group exhibited higher enthusiasm for learning, greater self-directed learning motivation, and better communication feedback abilities compared to the control group (P < 0.05). Additionally, the student satisfaction survey revealed that the GPT group rated higher on the diversity of diseases encountered, ease of use, and likelihood of recommending the training compared to the control group (P < 0.05). CONCLUSIONS: GPT-based history-taking training effectively enhances medical students' history-taking skills, providing a solid foundation for the application of artificial intelligence (AI) in medical education. CLINICAL TRIAL NUMBER: NCT06766383.

Overview

CoChat Summary

TL;DR

This single-center randomized clinical trial found that a custom GPT-based simulated-patient system improved fifth-year medical students’ history-taking performance more than traditional instructor-led role-playing. The GPT group scored significantly higher on the post-training structured clinical examination and reported greater motivation, satisfaction, and lower anxiety.

Key findings

  • Post-training structured clinical examination scores were significantly higher in the GPT group than the control group: 86.79 ± 5.46 vs. 73.64 ± 4.76, P < 0.001, with a reported Cohen’s d = 2.57.
  • Score improvement from pre- to post-training was also greater in the GPT group: 29.39 ± 13.14 vs. 18.96 ± 12.00, P = 0.003.
  • GPT-group students reported significantly higher post-class self-directed learning (median 3 [IQR 2–4] vs. 1.5 [1–2], P = 0.004), enthusiasm for learning (3 [2–4] vs. 1.5 [1–2], P < 0.001), communication/feedback skills (4 [2–4] vs. 1.5 [1–2], P = 0.006), and interview logical reasoning (4 [2.25–4] vs. 1.5 [1–2], P = 0.036).
  • Satisfaction ratings were significantly higher in the GPT group for diversity of diseases encountered (4 [3–4] vs. 1.5 [1–2], P = 0.004), ease of use (3 [2–4] vs. 1.5 [1–2], P < 0.001), and likelihood of recommending the training (3 [2–4] vs. 1.5 [1–2], P < 0.001); naturalness of interaction did not differ significantly (P = 0.973).

Methods

  • Single-center randomized controlled trial with 56 fifth-year medical students from Anhui Medical University, randomly assigned 1:1 to a GPT-simulated patient group (n = 28) or a traditional role-playing control group (n = 28); no dropouts.
  • The GPT group used a custom GPT developed on OpenAI’s ChatGPT GPTs platform with uploaded internal medicine/diagnostic references and prompt engineering, no fine-tuning; sessions were 30 minutes, three times per week for four weeks, matched to control sessions with clinical-skills instructors.
  • Outcomes included a pre- and post-training structured clinical examination adapted from OSCE, scored by two blinded instructors across history collection, clinical reasoning, communication skills, and professional behavior; secondary outcomes were 5-point Likert questionnaires on educational effectiveness and satisfaction, plus qualitative thematic analysis. Statistical comparisons used independent-samples t-tests, rank-sum tests, chi-square tests, and Cohen’s d.

Limitations

  • The paper states the main limitations were a relatively small sample size, single-institution conduct, short follow-up without long-term educational effects, and no formal inter-rater reliability indices (e.g., percentage agreement, ICC, or G-coefficients), so scorer variability cannot be ruled out.
  • The paper also notes GPT-specific limitations: the system is highly data-dependent, and it cannot display visual non-verbal behaviors such as facial expression, eye contact, or hand gestures, which may have reduced communicative complexity and may partially inflate the GPT group’s advantage over role-play.
  • Evident from the text: the custom GPT used platform-default inference settings, the specific GPT-4 sub-model was not disclosed, and no model fine-tuning was performed, which may limit reproducibility and adaptive learning claims.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12909-025-07614-9

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Flow chart
Fig. 1 Flow chartCC BY-NC-ND · PubMed Central
Fig. 2 Structured Clinical Examination Scores of both groups before and after the training. Legend: Structured clinical examination scores of the GPT and control groups before and after training. GPT group demonstrated significantly higher post-training scores(P < 0.001)
Fig. 2 Structured Clinical Examination Scores of both groups before and after the training. Legend: Structured clinical examination scores of the GPT and control groups before and after training. GPT group demonstrated significantly higher post-training scores(P < 0.001)CC BY-NC-ND · PubMed Central
Fig. 3 Comparison of Students’ Evaluation of the Educational Effectiveness of the Training Methods. Legend: Students’ evaluation of educational effectiveness on a 5-point Likert scale. GPT group reported significantly higher self-directed learning, enthusiasm, communication skills, and reasoning ability, and lower anxiety(P < 0.05)
Fig. 3 Comparison of Students’ Evaluation of the Educational Effectiveness of the Training Methods. Legend: Students’ evaluation of educational effectiveness on a 5-point Likert scale. GPT group reported significantly higher self-directed learning, enthusiasm, communication skills, and reasoning ability, and lower anxiety(P < 0.05)CC BY-NC-ND · PubMed Central
Fig. 4 Comparison of Students’ Satisfaction with the Training Methods. Legend: Student satisfaction scores with the training method across three dimensions. GPT group rated higher in diversity of cases, ease of use, and likelihood of recommending the training (P < 0.05)
Fig. 4 Comparison of Students’ Satisfaction with the Training Methods. Legend: Student satisfaction scores with the training method across three dimensions. GPT group rated higher in diversity of cases, ease of use, and likelihood of recommending the training (P < 0.05)CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

11 Valid · 2 Not valid · 3 Not checked · 79 Own results and interpretations

35 more claims

Create a free account to see all 95 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 25
  • It may state two separate claims as one. 37

Create a free account to see these claims.

Citations

Cites

Show all 20 references

Cited by

See all 32

Metadata

Authors
Wang, Z., Fan, T.-T., Li, M.-L., Zhu, N.-J., & Wang, X.-C.
Affiliations
Second Affiliated Hospital of Anhui Medical University (CN)
Venue
BMC Medical Education
Published
2025-07-10
Volume
25
Issue
1
Pages
1030-1030
ISSN
1472-6920 (ISSN-L)
OpenAlex
W4412177686
Cited by
32
References
20
Citations by year
2026: 28 · 2025: 2
FWCI
7.93
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Education (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • 2023 Anhui Province Quality Engineering Project (2023jyxm1150)
  • 2024 Anhui Province Quality Engineering Project (2024jyxm0779)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
  • Social Sciences › Education › Simulation-Based Education in Healthcare
Keywords
history-taking skills, generative pre-trained transformer, ChatGPT, medical students, role play, artificial intelligence in medical education, clinical reasoning, communication skills
MeSH terms
Adult, Clinical Competence, Education, Medical, Education, Medical, Undergraduate, Educational Measurement, Feasibility Studies, Female

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial. https://app.cochat.ai/papers/10.1186/s12909-025-07614-9
The paper
Wang, Z., Fan, T.-T., Li, M.-L., Zhu, N.-J., & Wang, X.-C. (2025). Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial. BMC Medical Education. https://doi.org/10.1186/s12909-025-07614-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.