Virtual Patient Simulations Using Social Robotics Combined With Large Language Models for Clinical Reasoning Training in Medical Education: Mixed Methods Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
Background: Virtual patients (VPs) are computer-based simulations of clinical scenarios used in health professions education to address various learning outcomes, including clinical reasoning (CR). CR is a crucial skill for health care practitioners, and its inadequacy can compromise patient safety. Recent advancements in large language models (LLMs) and social robots have introduced new possibilities for enhancing VP interactivity and realism. However, their application in VP simulations has been limited, and no studies have investigated the effectiveness of combining LLMs with social robots for CR training. Objective: The aim of the study is to explore the potential added value of a social robotic VP platform combined with an LLM compared to a conventional computer-based VP modality for CR training of medical students. Methods: A Swedish explorative proof-of-concept study was conducted between May and July 2023, combining quantitative and qualitative methodology. In total, 15 medical students from Karolinska Institutet and an international exchange program completed a VP case in a social robotic platform and a computer-based semilinear platform. Students’ self-perceived VP experience focusing on CR training was assessed using a previously developed index, and paired 2-tailed t test was used to compare mean scores (scales from 1 to 5) between the platforms. Moreover, in-depth interviews were conducted with 8 medical students. Results: The social robotic platform was perceived as more authentic (mean 4.5, SD 0.7 vs mean 3.9, SD 0.5; odds ratio [OR] 2.9, 95% CI 0.0-1.0; P=.04) and provided a beneficial overall learning effect (mean 4.4, SD 0.6 versus mean 4.1, SD 0.6; OR 3.7, 95% CI 0.1-0.5; P=.01) compared with the computer-based platform. Qualitative analysis revealed 4 themes, wherein students experienced the social robot as superior to the computer-based platform in training CR, communication, and emotional skills. Limitations related to technical and user-related aspects were identified, and suggestions for improvements included enhanced facial expressions and VP cases simulating multiple personalities. Conclusions: A social robotic platform enhanced by an LLM may provide an authentic and engaging learning experience for medical students in the context of VP simulations for training CR. Beyond its limitations, several aspects of potential improvement were identified for the social robotic platform, lending promise for this technology as a means toward the attainment of learning outcomes within medical education curricula.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Borg et al. combined an LLM with a social robot to create more authentic case encounters50;”
From A large language model digital patient system enhances ophthalmology history taking skills (2025) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Lawrence, R. C., Felson, D. T., Helmick, C. G., Arnold, L. M., Choi, H., Deyo, R. A., Gabriel, S., Hirsch, R., Hochberg, M. C., Hunder, G. G., et al. (2008) · Arthritis & Rheumatism · doi:10.1002/art.23176 · Cited by 5,499
Estimates of the prevalence of arthritis and other rheumatic conditions in the United States: Part I
Helmick, C. G., Felson, D. T., Lawrence, R. C., Gabriel, S., Hirsch, R., Kwoh, C. K., Liang, M. H., Kremers, H. M., Mayes, M. D., Merkel, P. A., et al. (2008) · Arthritis & Rheumatism · doi:10.1002/art.23177 · Cited by 2,058 · cited 1×
“Rheumatic diseases affect a substantial proportion of the population and are considered a significant public health concern [22-24].”
Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions
Abd-alrazaq, A., AlSaad, R., Alhuwail, D., Ahmed, A., Healy, P. M., Latifi, S., Aziz, S., Damseh, R., Alabed Alrazak, S., & Sheikh, J. (2023) · JMIR Medical Education · doi:10.2196/48291 · Cited by 722
A scoping review of artificial intelligence in medical education: BEME Guide No. 84
Gordon, M., Daniel, M., Ajiboye, A., Uraiby, H., Xu, N. Y., Bartlett, R., Hanson, J., Haas, M., Spadafore, M., Grafton-Clarke, C., et al. (2024) · Medical Teacher · doi:10.1080/0142159x.2024.2314198 · Cited by 679 · cited 1×
“The emergence of large language models (LLMs) and their application in dialogue systems and social robots have introduced novel possibilities within this field [16,17].”
Virtual patients: a critical literature review and proposed next steps
Cook, D. A., & Triola, M. M. (2009) · Medical Education · doi:10.1111/j.1365-2923.2008.03286.x · Cited by 624 · cited 1×
“VPs have been proposed as an effective educational tool for practicing clinical reasoning (CR) in undergraduate medical education [4] while also ensuring that medical students encounter representative patient cases, which might not always be possible to encounter during clinical rotations [5-7].”
Kononowicz, A. A., Woodham, L. A., Edelbring, S., Stathakarou, N., Davies, D., Saxena, N., Tudor Car, L., Carlstedt-Duke, J., Car, J., & Zary, N. (2019) · Journal of Medical Internet Research · doi:10.2196/14676 · Cited by 600 · cited 2×
“Interestingly, students have been shown to perform better at examinations after exposure to VP simulations compared to other educational activities with similar learning outcomes, but the positive effect of VPs on CR training is still unclear [2,13].”
Self-citationComputerized Virtual Patients in Health Professions Education: A Systematic Review and Meta-Analysis
Cook, D. A., Erwin, P. J., & Triola, M. M. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181edfe13 · Cited by 570 · cited 1×
“A common feature of VP platforms is that they can facilitate the obtainment of medical history, the conduct of physical examinations, and referral for laboratory tests and the provision of their findings to culminate in a reasonable diagnosis and management plan [11].”
Furhat: A Back-Projected Human-Like Robot Head for Multiparty Human-Machine Interaction
Al Moubayed, S., Beskow, J., Skantze, G., & Granström, B. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-34584-5_9 · Cited by 297 · cited 3×
“A VP case illustrating a common rheumatic disease (ie, polymyalgia rheumatica) was developed for presentation in a social robotic platform from Furhat robotics [27] and a conventional semilinear computer-based platform (ie, the virtual interactive case simulator [VIC]) [28].”
Self-citationVirtual Patient Simulation at U.S. and Canadian Medical Schools
Huang, G., Reynolds, R., & Candler, C. (2007) · Academic Medicine · doi:10.1097/acm.0b013e31803e8a0a · Cited by 233
Research works printed but not linked: 1
- BrownTMannBRyderNSubbiahMKaplanJDDhariwalPNeelakantanAShyamPSastryGAskellAAgarwalSHerbert-VossAKruegerGHenighanTChildRRameshAZieglerDMLanguage models are few-shot learners2020Advances in Neural Information Processing SystemsDecember 6-12, 2020Vancouver, BC18771901
Cited by
Zeng, J., Qi, W., Shen, S., Liu, X., Li, S., Wang, B., Dong, C., Zhu, X., Shi, Y., Lou, X., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/79091 · Cited by 76
“For the measurement tools, 3 validated questionnaires were used to assess usability in the technological characteristics [21,46]; 1 validated questionnaire was used to assess authenticity in the simulation realism [50].”
InfluentialA large language model digital patient system enhances ophthalmology history taking skills
Luo, M.-J., Bi, S., Pang, J., Liu, L., Tsui, C.-K., Lai, Y., Chen, W., Yang, Y., Xu, K., Zhao, L., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01841-6 · Cited by 39
“…41 ; Yao et al. developed a related LLM-powered patient dialog system 42 ; Borg et al. combined an LLM with a social robot to create more authentic case encounters 50 ; and Holderried et al. found that a ChatGPT chatbot acting as a simulated patient produced highly plausible history responses 51 .”
Huang, S., Wen, C., Bai, X., Li, S., Wang, S., Wang, X., & Yang, D. (2025) · Journal of Medical Internet Research · doi:10.2196/68538 · Cited by 33
“Research has shown that ChatGPT can assist students in reviewing key concepts, reinforcing theoretical knowledge [1,15], and simulating clinical scenarios to enhance clinical reasoning [16].”
Li, D., & Lutfi, S. L. (2025) · JMIR Medical Informatics · doi:10.2196/79039 · Cited by 45
“…medical students and practitioners, using mixed methods (top-k accuracy, F 1 -score, SUS, CUQ, and expert ratings) with comparisons across AI models, physicians, or prompt variations; small sample sizes (10-50 students and 3-10 experts) and inconsistent metrics limit generalizability [19,40].”
InfluentialElhilali, A., Ngo, A. S.-H., Reichenpfader, D., & Denecke, K. (2025) · JMIR Medical Education · doi:10.2196/81271 · Cited by 34
“Borg et al [19] combined the LLM-based simulation (GPT-3.”
Metadata
- Authors
- Borg, A., Georg, C., Jobs, B., Huss, V., Waldenlind, K., Ruiz, M., Edelbring, S., Skantze, G., & Parodis, I.
- Affiliations
- Karolinska University Hospital (SE)
- Karolinska Institutet (SE)
- Örebro University (SE)
- KTH Royal Institute of Technology (SE)
- Corresponding author
- Parodis, Ioannis
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2025-03-03
- Volume
- 27
- Pages
- e63312
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11914843
- OpenAlex
- W4408098677
- Cited by
- 51
- References
- 36
- Citations by year
- 2026: 37 · 2025: 14
- FWCI
- 32.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Education › Simulation-Based Education in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- virtual patients, social robots, large language models, clinical reasoning training, medical students, computer simulation, communication skills, emotional competence, patient safety, health professions education, facial expressions
- MeSH terms
- Clinical Reasoning, Large Language Models, Adult, Education, Medical, Female, Humans, Language, Male, Robotics, Students, Medical, Sweden, Patient Simulation, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.