Development of a GPT-4-powered virtual simulated patient and communication training platform for medical students to practice discussing abnormal mammogram results with patients: Multiphase study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Background: Standardized patients (SPs) prepare medical students for difficult conversations with patients. Despite their value, SP-based simulation training is constrained by available resources and competing clinical demands. Researchers are turning to artificial intelligence and large language models, such as generative pretrained transformers, to create communication training that incorporates virtual simulated patients (VSPs). GPT-4 is a large language model advance allowing developers to design virtual simulation scenarios using text-based prompts instead of relying on branching path simulations with prescripted dialogue. These nascent developmental practices have not taken root in the literature to guide other researchers in developing their own simulations. Objective: This study aims to describe our developmental process and lessons learned for creating a GPT-4–driven VSP. We designed the VSP to help medical student learners rehearse discussing abnormal mammography results with a patient as a primary care physician (PCP). We aimed to assess GPT-4’s ability to generate appropriate VSP responses to learners during spoken conversations and provide appropriate feedback on learner performance. Methods: A research team comprised of physicians, a medical student, an educator, an SP program director, a learning experience designer, and a health care researcher conducted the study. A formative phase with in-depth knowledge user interviews informed development, followed by a development phase to create the virtual training module. The team conducted interviews with 5 medical students, 5 PCPs, and 5 breast cancer survivors. They then developed a VSP using simulation authoring software and provided the GPT-4–enabled VSP with an initial prompt consisting of a scenario description, emotional state, and expectations for learner dialogue. It was iteratively refined through an agile design process involving repeated cycles of testing, documenting issues, and revising the prompt. As an exploratory feature, the simulation used GPT-4 to provide written feedback to learners about their performance communicating with the VSP and their adherence to guidelines for difficult conversations. Results: In-depth interviews helped establish the appropriate timing, mode of communication, and protocol for conversations between PCPs and patients during the breast cancer screening process. The scenario simulated a telephone call between a physician and patient to discuss the abnormal results of a diagnostic mammogram that that indicated a need for a biopsy. Preliminary testing was promising. The VSP asked sensible questions about their mammography results and responded to learner inquiries using a voice replete with appropriate emotional inflections. GPT-4 generated performance feedback that successfully identified strengths and areas for improvement using relevant quotes from the learner-VSP conversation, but it occasionally misidentified learner adherence to communication protocols. Conclusions: GPT-4 streamlined development and facilitated more dynamic, humanlike interactions between learners and the VSP compared to branching path simulations. For the next steps, we will pilot-test the VSP with medical students to evaluate its feasibility and acceptability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 First stage of the schematic of the original scenario flow design for a branching path simulation that consists of 6 distinct stages (1 for each step of the Setting, Perception, Invitation, Knowledge, Emotion, and Summarize [SPIKES] protocol), 12 critical decision points, 68 scripted user response options (consisting of ideal, acceptable, and unacceptable examples), and 65 scripted patient responses. Simulation flow was designed from November 2023 to December 2023 to help medical students practice discussing abnormal mammogram results with their patients. We originally planned to use a “voice intent recognition” system to branch to the appropriate patient responses and to score users based on how closely their responses match the ideal options.](https://figures.cochat.ai/paper-figures/6074e1262f4e4c0f944b392626a7879e/development-of-a-gpt-4-powered-virtual-simulated-patient-and-communication-figure-1.jpg)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Unlike traditional systems, LLMs like GPT-4 can simulate dynamic patient behavior, adapt to diverse learner inputs, and provide real-time feedback, potentially transforming how students rehearse complex conversations such as diagnostic delivery (Weisman et al., 2025).”
From DIALOGUE: A Generative AI-Based Pre–Post Simulation Study to Enhance Diagnostic Communication in Medical Students Through Virtual Type 2 Diabetes Scenarios (2025) · cites this paper as Weisman et al., 2025
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Siegel, R. L., Miller, K. D., Wagle, N. S., & Jemal, A. (2023) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21763 · Cited by 16,303
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“Existing literature presents general frameworks for LLM-based simulations across disparate clinical scenarios with a focus on clinical reasoning and diagnosis [22,25].”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
SPIKES—A Six-Step Protocol for Delivering Bad News: Application to the Patient with Cancer
Baile, W. F., Buckman, R., Lenzi, R., Glober, G., Beale, E. A., & Kudelka, A. P. (2000) · The Oncologist · doi:10.1634/theoncologist.5-4-302 · Cited by 3,075
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613
Doctor-patient communication: a review.
Ha, J. F., & Longnecker, N. E. (2010) · PubMed · Cited by 2,014
The TRIPOD-LLM reporting guideline for studies using large language models
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., Demner-Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03425-5 · Cited by 615
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510 · cited 1×
“…of LLMs to deviate from scripted conversations offers more humanlike interactions but can lead to too much improvisation, creating off-topic or biased responses from inappropriate source data and fabricating information when no source data are available, referred to as hallucination [16-18].”
Teaching communication skills to medical students, a challenge in the curriculum?
Deveugele, M., Derese, A., Maesschalck, S. D., Willems, S., Driel, M. V., & Maeseneer, J. D. (2005) · Patient Education and Counseling · doi:10.1016/j.pec.2005.06.004 · Cited by 252
Beyond breaking bad news: how to help patients who suffer.
Rabow, M. W., & McPhee, S. J. (1999) · Europe PMC (PubMed Central) · Cited by 250
Tavakoly Sany, S. B., Behzhad, F., Ferns, G., & Peyman, N. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-4901-8 · Cited by 231
Breaking bad news: the S-P-I-K-E-S strategy
Buckman, R. A. (2005) · Community Oncology · doi:10.1016/s1548-5315(11)70867-1 · Cited by 223
Oh, N., Choi, G.-S., & Lee, W. Y. (2023) · Annals of Surgical Treatment and Research · doi:10.4174/astr.2023.104.5.269 · Cited by 201
Evaluation of the performance of GPT-3.5 and GPT-4 on the Polish Medical Final Examination
Rosoł, M., Gąsior, J. S., Łaba, J., Korzeniewski, K., & Młyńczak, M. (2023) · Scientific Reports · doi:10.1038/s41598-023-46995-z · Cited by 194
Holderried, F., Stegemann–Philipps, C., Herschbach, L., Moldt, J.-A., Nevins, A., Griewatz, J., Holderried, M., Herrmann-Werner, A., Festl-Wietek, T., & Mahling, M. (2024) · JMIR Medical Education · doi:10.2196/53961 · Cited by 156
Holderried, F., Stegemann-Philipps, C., Herrmann-Werner, A., Festl-Wietek, T., Holderried, M., Eickhoff, C., & Mahling, M. (2024) · JMIR Medical Education · doi:10.2196/59213 · Cited by 152
The effect of using a large language model to respond to patient messages
Chen, S., Guevara, M., Moningi, S., Hoebers, F., Elhalawani, H., Kann, B. H., Chipidza, F. E., Leeman, J., Aerts, H. J. W. L., Miller, T., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00060-8 · Cited by 147
Evaluating large language models as agents in the clinic
Mehandru, N., Miao, B. Y., Almaraz, E. R., Sushil, M., Butte, A. J., & Alaa, A. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01083-y · Cited by 143
AI and Medical Education — A 21st-Century Pandora’s Box
Cooper, A. Z., & Rodman, A. M. (2023) · New England Journal of Medicine · doi:10.1056/nejmp2304993 · Cited by 132
Ho, T.-Q. H., Bissell, M. C. S., Kerlikowske, K. M., Hubbard, R. A., Sprague, B. L., Lee, C. I., Tice, J. A., Tosteson, A. N. A., & Miglioretti, D. L. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.2440 · Cited by 85
Proof of Concept: Using ChatGPT to Teach Emergency Physicians How to Break Bad News
Webb, J. J. (2023) · Cureus · doi:10.7759/cureus.38755 · Cited by 80 · cited 2×
“Our work advances the work of Webb [37] and other studies that use text-based communication by simulating verbal communication between users and VSP.”
Beyond ChatGPT: What does GPT-4 add to healthcare? The dawn of a new era
Wójcik, S., Rulkiewicz, A., Pruszczyk, P., Lisik, W., Poboży, M., & Domienik‐Karłowicz, J. (2023) · Cardiology Journal · doi:10.5603/cj.97515 · Cited by 78
Sardesai, N., Russo, P., Martin, J., & Sardesai, A. (2024) · Postgraduate Medical Journal · doi:10.1093/postmj/qgad137 · Cited by 67
Embodied Conversational Agents for Chronic Diseases: Scoping Review
Jiang, Z., Huang, X., Wang, Z., Liu, Y., Huang, L., & Luo, X. (2023) · Journal of Medical Internet Research · doi:10.2196/47134 · Cited by 63
Automatic Interactive Evaluation for Large Language Models with State Aware Patient SimulatorPreprint
Liao, Y., Meng, Y., Wang, Y., Li, L., Liu, H., Wang, Y., & Wang, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4890649 · Cited by 56
Assessing GPT-4’s Performance in Delivering Medical Advice: Comparative Analysis With Human Experts
Jo, E., Song, S., Kim, J.-H., Lim, S., Kim, J. H., Cha, J., Kim, Y.-M., & Joo, H. J. (2024) · JMIR Medical Education · doi:10.2196/51282 · Cited by 52
Liaw, S. Y., Wu, L. T., Soh, S. L.-H., Ringsted, C. V., Lau, T. C., & Lim, W. S. (2020) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2020.03.013 · Cited by 50
ChatGPT: when the artificial intelligence meets standardized patients in clinical training
Liu, X., Wu, C., Lai, R., Lin, H., Xu, Y., Lin, Y., & Zhang, W. (2023) · Journal of Translational Medicine · doi:10.1186/s12967-023-04314-0 · Cited by 37
Potter, L., & Jefferies, C. (2024) · Future Healthcare Journal · doi:10.1016/j.fhj.2024.100043 · Cited by 31
De Mattei, L., Morato, M. Q., Sidhu, V., Gautam, N., Mendonca, C. T., Tsai, A., Hammer, M., Creighton-Wong, L., & Azzam, A. (2024) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2024.101536 · Cited by 28 · cited 1×
“VSPs are gaining acceptance from educators and students alike, marked by the necessities of social distancing during the COVID-19 pandemic and lingering clinical teaching challenges in its wake [36].”
Large Language Models and Their Implications on Medical Education
Bair, H., & Norden, J. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005265 · Cited by 25
GPT-4: the future of artificial intelligence in medical school assessments
Haruna-Cooper, L., & Rashid, M. A. (2023) · Journal of the Royal Society of Medicine · doi:10.1177/01410768231181251 · Cited by 23
Use of Standardized Patients in, Undergraduate Medical Genetics Education
McGovern, M. M., Johnston, M., Brown, K., Zinberg, R. E., & Cohen, D. (2006) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1803_3 · Cited by 23
Beyond breaking bad news: Helping patients who suffer
Rabow, M. W., & McPhee, S. J. (2000) · BMJ · doi:10.1136/sbmj.000365 · Cited by 21
Leveraging Large Language Model as Simulated Patients for Clinical Education
Li, Y., Zeng, C., Zhong, J., Zhang, R., Zhang, M., & Zou, L. (2024) · arXiv · doi:10.48550/arxiv.2404.13066 · Cited by 16 · cited 2×
“This is a reason why VSP-based training scenarios have had limited evaluation capabilities, at best providing point-based evaluations [22].”
Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator
Liao, Y., Meng, Y., Wang, Y., Liu, H., Wang, Y., & Wang, Y. (2024) · arXiv · doi:10.48550/arxiv.2403.08495 · Cited by 15 · cited 1×
“Researchers have proposed evaluation frameworks such as the automated interactive evaluation framework and Artificial Intelligence Structured Clinical Examinations to evaluate the performance of LLMs to carry out clinical tasks that can be applied in this regard [39,40].”
Adnan, A. I. (2022) · Cureus · doi:10.7759/cureus.26717 · Cited by 14
Reidy, J. A., Clark, M. A., Berman, H. A., Chan, S. H., Gawande, A. A., Streid, J. L., Vesel, T., Young, M. E., Zehm, A., & Schaefer, K. G. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03702-2 · Cited by 12
Doctor-Patient Communication: An Overview
Sethuraman, K. (2001) · Jaypee Brothers Medical Publishers (P) Ltd. eBooks · doi:10.5005/jp/books/11607_2 · Cited by 10
Nelson, G., Swenson, J., Kozmenko, V., & Thompson, P. (2022) · Aesculapius · doi:10.70657/ahsm.v3.2022.01 · Cited by 10
Research works printed but not linked: 7
- Ha, JF, 2010, Ochsner J, 10, 38
- Nelson, G, 2022, S D Med, 75, 454
- Li, Y, arXiv. Preprint posted online on April 13, 2024
- Bojic, L, arXiv. Preprint posted online on December 15, 2023
- Nori, H, arXiv. Preprint posted online on March 20, 2023
- Ruan, J, arXiv. Preprint posted online on August 7, 2023
- Rabow, MW, 1999, West J Med, 171, 260
Cited by
Zeng, J., Qi, W., Shen, S., Liu, X., Li, S., Wang, B., Dong, C., Zhu, X., Shi, Y., Lou, X., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/79091 · Cited by 76
A comprehensive survey of AI agents in healthcare
Xu, G., Li, X., Chen, Y., Duan, Y., Wu, S., Yu, H., Chiu, C.-H., Ni, J., Tang, N., Li, T. J.-J., et al. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105045 · Cited by 56
Suárez-García, R. X., Chavez-Castañeda, Q., Orrico-Pérez, R., Valencia-Marin, S., Castañeda-Ramírez, A. E., Quiñones-Lara, E., Ramos-Cortés, C. A., Gaytán-Gómez, A. M., Cortés-Rodríguez, J., Jarquín-Ramírez, J., et al. (2025) · European Journal of Investigation in Health, Psychology and Education · doi:10.3390/ejihpe15080152 · Cited by 18
“Unlike traditional systems, LLMs like GPT-4 can simulate dynamic patient behavior, adapt to diverse learner inputs, and provide real-time feedback, potentially transforming how students rehearse complex conversations such as diagnostic delivery (Weisman et al., 2025).”
Dávidovics, A., Dávidovics, K., Hillebrand, P., Rendeki, S., & Németh, T. (2025) · BMC Medical Education · doi:10.1186/s12909-025-08507-7 · Cited by 12
Kıyak, Y. S., İş-Kara, T., & Emekli, E. (2026) · Medical Science Educator · doi:10.1007/s40670-025-02621-3 · Cited by 13
Metadata
- Authors
- Weisman, D., Sugarman, A., Huang, Y. M., Gelberg, L., Ganz, P. A., & Comulada, W. S.
- Affiliations
- University of California, Los Angeles (US)
- Corresponding author
- Comulada, Warren Scott
- ORCID
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2025-04-17
- Volume
- 9
- Pages
- e65670
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC12046251
- OpenAlex
- W4407545831
- Cited by
- 37
- References
- 37
- Citations by year
- 2026: 28 · 2025: 9
- FWCI
- 23.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Breast Cancer Research Foundation
- University of California, Los Angeles
- National Cancer Institute (P30CA016042)
- Jonsson Comprehensive Cancer Center
- Topics
- Social Sciences › Education › Simulation-Based Education in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Education › Innovations in Medical Education
- Keywords
- GPT-4, large language models, medical communication training, difficult conversations, breast cancer screening, prompt engineering, telephone consultation, medical students, breast cancer survivors, artificial intelligence in medical education, shared decision-making, informed consent, informed refusal
- MeSH terms
- Breast Neoplasms, Communication, Female, Humans, Male, Mammography, Physician-Patient Relations, Students, Medical, Patient Simulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.