Virtual Patient Simulations Using Social Robotics Combined With Large Language Models for Clinical Reasoning Training in Medical Education: Mixed Methods Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
Background: Virtual patients (VPs) are computer-based simulations of clinical scenarios used in health professions education to address various learning outcomes, including clinical reasoning (CR). CR is a crucial skill for health care practitioners, and its inadequacy can compromise patient safety. Recent advancements in large language models (LLMs) and social robots have introduced new possibilities for enhancing VP interactivity and realism. However, their application in VP simulations has been limited, and no studies have investigated the effectiveness of combining LLMs with social robots for CR training. Objective: The aim of the study is to explore the potential added value of a social robotic VP platform combined with an LLM compared to a conventional computer-based VP modality for CR training of medical students. Methods: A Swedish explorative proof-of-concept study was conducted between May and July 2023, combining quantitative and qualitative methodology. In total, 15 medical students from Karolinska Institutet and an international exchange program completed a VP case in a social robotic platform and a computer-based semilinear platform. Students’ self-perceived VP experience focusing on CR training was assessed using a previously developed index, and paired 2-tailed t test was used to compare mean scores (scales from 1 to 5) between the platforms. Moreover, in-depth interviews were conducted with 8 medical students. Results: The social robotic platform was perceived as more authentic (mean 4.5, SD 0.7 vs mean 3.9, SD 0.5; odds ratio [OR] 2.9, 95% CI 0.0-1.0; P=.04) and provided a beneficial overall learning effect (mean 4.4, SD 0.6 versus mean 4.1, SD 0.6; OR 3.7, 95% CI 0.1-0.5; P=.01) compared with the computer-based platform. Qualitative analysis revealed 4 themes, wherein students experienced the social robot as superior to the computer-based platform in training CR, communication, and emotional skills. Limitations related to technical and user-related aspects were identified, and suggestions for improvements included enhanced facial expressions and VP cases simulating multiple personalities. Conclusions: A social robotic platform enhanced by an LLM may provide an authentic and engaging learning experience for medical students in the context of VP simulations for training CR. Beyond its limitations, several aspects of potential improvement were identified for the social robotic platform, lending promise for this technology as a means toward the attainment of learning outcomes within medical education curricula.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Borg et al. combined an LLM with a social robot to create more authentic case encounters50;”
From A large language model digital patient system enhances ophthalmology history taking skills (2025) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Lawrence, R. C., Felson, D. T., Helmick, C. G., Arnold, L. M., Choi, H., Deyo, R. A., Gabriel, S., Hirsch, R., Hochberg, M. C., Hunder, G. G., et al. (2008) · Arthritis & Rheumatism · doi:10.1002/art.23176 · Cited by 5,499
Estimates of the prevalence of arthritis and other rheumatic conditions in the United States: Part I
Helmick, C. G., Felson, D. T., Lawrence, R. C., Gabriel, S., Hirsch, R., Kwoh, C. K., Liang, M. H., Kremers, H. M., Mayes, M. D., Merkel, P. A., et al. (2008) · Arthritis & Rheumatism · doi:10.1002/art.23177 · Cited by 2,058 · cited 1×
“Rheumatic diseases affect a substantial proportion of the population and are considered a significant public health concern [22-24].”
Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions
Abd-alrazaq, A., AlSaad, R., Alhuwail, D., Ahmed, A., Healy, P. M., Latifi, S., Aziz, S., Damseh, R., Alabed Alrazak, S., & Sheikh, J. (2023) · JMIR Medical Education · doi:10.2196/48291 · Cited by 722
A scoping review of artificial intelligence in medical education: BEME Guide No. 84
Gordon, M., Daniel, M., Ajiboye, A., Uraiby, H., Xu, N. Y., Bartlett, R., Hanson, J., Haas, M., Spadafore, M., Grafton-Clarke, C., et al. (2024) · Medical Teacher · doi:10.1080/0142159x.2024.2314198 · Cited by 679 · cited 1×
“The emergence of large language models (LLMs) and their application in dialogue systems and social robots have introduced novel possibilities within this field [16,17].”
Virtual patients: a critical literature review and proposed next steps
Cook, D. A., & Triola, M. M. (2009) · Medical Education · doi:10.1111/j.1365-2923.2008.03286.x · Cited by 624 · cited 1×
“VPs have been proposed as an effective educational tool for practicing clinical reasoning (CR) in undergraduate medical education [4] while also ensuring that medical students encounter representative patient cases, which might not always be possible to encounter during clinical rotations [5-7].”
Kononowicz, A. A., Woodham, L. A., Edelbring, S., Stathakarou, N., Davies, D., Saxena, N., Tudor Car, L., Carlstedt-Duke, J., Car, J., & Zary, N. (2019) · Journal of Medical Internet Research · doi:10.2196/14676 · Cited by 600 · cited 2×
“Interestingly, students have been shown to perform better at examinations after exposure to VP simulations compared to other educational activities with similar learning outcomes, but the positive effect of VPs on CR training is still unclear [2,13].”
Self-citationComputerized Virtual Patients in Health Professions Education: A Systematic Review and Meta-Analysis
Cook, D. A., Erwin, P. J., & Triola, M. M. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181edfe13 · Cited by 570 · cited 1×
“A common feature of VP platforms is that they can facilitate the obtainment of medical history, the conduct of physical examinations, and referral for laboratory tests and the provision of their findings to culminate in a reasonable diagnosis and management plan [11].”
Furhat: A Back-Projected Human-Like Robot Head for Multiparty Human-Machine Interaction
Al Moubayed, S., Beskow, J., Skantze, G., & Granström, B. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-34584-5_9 · Cited by 297 · cited 3×
“A VP case illustrating a common rheumatic disease (ie, polymyalgia rheumatica) was developed for presentation in a social robotic platform from Furhat robotics [27] and a conventional semilinear computer-based platform (ie, the virtual interactive case simulator [VIC]) [28].”
Self-citationVirtual Patient Simulation at U.S. and Canadian Medical Schools
Huang, G., Reynolds, R., & Candler, C. (2007) · Academic Medicine · doi:10.1097/acm.0b013e31803e8a0a · Cited by 233
Kononowicz, A. A., Zary, N., Edelbring, S., Corral, J., & Hege, I. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0296-3 · Cited by 203 · cited 1×
“Virtual patients (VPs) are computed interactive simulations of clinical scenarios for educational or learning purposes [1].”
Self-citationRencic, J. J., Trowbridge, R. L., Fagan, M. J., Szauter, K. M., & Durning, S. J. (2017) · Journal of General Internal Medicine · doi:10.1007/s11606-017-4159-y · Cited by 165
Plackett, R., Kassianos, A. P., Mylan, S., Kambouri, M., Raine, R., & Sheringham, J. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03410-x · Cited by 143 · cited 1×
“Interestingly, students have been shown to perform better at examinations after exposure to VP simulations compared to other educational activities with similar learning outcomes, but the positive effect of VPs on CR training is still unclear [2,13].”
Towards a typology of virtual patients
Huwendiek, S., De leng, B. A., Zary, N., Fischer, M. R., Ruiz, J. G., & Ellaway, R. (2009) · Medical Teacher · doi:10.1080/01421590903124708 · Cited by 122 · cited 3×
“VP platforms can be designed in a linear fashion, where medical information and treatment options are gathered in a fixed order without consequences, or with a branched design, where interactions also impact the outcome of the case [3].”
Irfan, B., Kuoppamäki, S., Hosseini, A., & Skantze, G. (2025) · Autonomous Robots · doi:10.1007/s10514-025-10190-y · Cited by 108 · cited 3×
“Using LLMs can cause response delays, complicating turn-taking, as users might not realize that the robot is generating a reply [16].”
Self-citationDesign principles for virtual patients: a focus group study among students
Huwendiek, S., Reichert, F., Bosse, H., De Leng, B. A., Van Der Vleuten, C. P. M., Haag, M., Hoffmann, G. F., & Tönshoff, B. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03369.x · Cited by 105
Twelve tips to support the development of clinical reasoning skills using virtual patient cases
Posel, N., Mcgee, J. B., & Fleiszer, D. M. (2015) · Medical Teacher · doi:10.3109/0142159x.2014.993951 · Cited by 99 · cited 1×
“VPs have been proposed as an effective educational tool for practicing clinical reasoning (CR) in undergraduate medical education [4] while also ensuring that medical students encounter representative patient cases, which might not always be possible to encounter during clinical rotations [5-7].”
Kavanaugh, A., Antoni, C., Krueger, G. G., Yan, S., BALA, M. M., Dooley, L. T., Beutler, A., Guzzo, C., & Gladman, D. (2005) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2005.040196 · Cited by 96
Real-time emotion generation in human-robot dialogue using large language models
Mishra, C., Verdonschot, R. G., Peter Hagoort, A., & Skantze, G. (2023) · Frontiers in Robotics and AI · doi:10.3389/frobt.2023.1271610 · Cited by 93 · cited 1×
“The animated face expresses subtle emotions through facial expressions and gaze behavior, crucial for conveying a patient’s internal state [30].”
Self-citationExperiencing virtual patients in clinical learning: a phenomenological study
Edelbring, S., Dastmalchi, M., Hult, H., Lundberg, I. E., & Dahlgren, L. O. (2011) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9265-0 · Cited by 77 · cited 2×
“Previous research shows that medical students perceive VP simulation as a satisfactory educational activity for training CR while they acknowledge the lack of complexity and level of interactivity compared with authentic patient encounters [12].”
Self-citationExploring Turn-taking Cues in Multi-party Human-robot Discussions about Objects
Skantze, G., Johansson, M., & Beskow, J. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2820749 · Cited by 75 · cited 1×
“To address this, various approaches can be adopted, such as the use of a turn-holding cue to indicate that the robot is actively “thinking” about a response [32].”
Self-citationKononowicz, A. A., Hege, I., Edelbring, S., Sobocan, M., Huwendiek, S., & Durning, S. J. (2020) · Medical Teacher · doi:10.1080/0142159x.2019.1708293 · Cited by 67 · cited 1×
“CR is an essential skill for health care practitioners, and its inadequacy may threaten patient safety, making it a key goal in HPE [8-10].”
Self-citation12 Tips: Guidelines for authoring virtual patient cases
Posel, N. H., FLEISZER, D. M., & Shore, B. M. (2009) · Medical Teacher · doi:10.1080/01421590902793867 · Cited by 66 · cited 2×
“The case development in both platforms adhered to distinct principles for VP case development proposed by Posel et al [29].”
Coşkun, Ö., Kıyak, Y. S., & Budakoğlu, I. İ. (2025) · Medical Teacher · doi:10.1080/0142159x.2024.2327477 · Cited by 64
Using artificial intelligence to create diverse and inclusive medical case vignettes for education
Bakkum, M. J., Hartjes, M. G., Piët, J. D., Donker, E. M., Likic, R., Sanz, E., de Ponti, F., Verdonk, P., Richir, M. C., van Agtmael, M. A., et al. (2024) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15977 · Cited by 56
Huwendiek, S., de Leng, B. A., Kononowicz, A. A., Kunzmann, R., Muijtjens, A. M. M., van der Vleuten, C. P. M., Hoffmann, G. F., Tönshoff, B., & Dolmans, D. H. J. M. (2014) · Medical Teacher · doi:10.3109/0142159x.2014.970622 · Cited by 40 · cited 3×
“Its development combined a theoretical framework for CR training along with the involvement of an international team of VP experts, supporting its content validity [26].”
InfluentialParodis, I., Andersson, L., Durning, S. J., Hege, I., Knez, J., Kononowicz, A. A., Lidskog, M., Petreski, T., Szopa, M., & Edelbring, S. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182111202 · Cited by 40 · cited 2×
“International recommendations for integrating CR training into HPE curricula [10] were used as a guideline when coding words and phrases related to CR training.”
Self-citationCampillos-Llanos, L., Thomas, C., Bilinski, É., Zweigenbaum, P., & Rosset, S. (2020) · Natural Language Engineering · doi:10.1017/s1351324919000329 · Cited by 34 · cited 1×
“Recent technological developments have enabled the integration of dialogue systems with natural language processing capabilities, embodied conversational agents [14], and social robots [15] to enhance the interactivity and realism of VP interactions.”
Borg, A., Jobs, B., Huss, V., Gentline, C., Espinosa, F., Ruiz, M., Edelbring, S., Georg, C., Skantze, G., & Parodis, I. (2024) · Rheumatology International · doi:10.1007/s00296-024-05731-0 · Cited by 34 · cited 1×
“However, their use for VPs has been limited to preliminary endeavors [18-21].”
Self-citationDeveloping a European longitudinal and interprofessional curriculum for clinical reasoning
Hege, I., Adler, M., Donath, D., Durning, S. J., Edelbring, S., Elvén, M., Frankowska, A., Georg, C., Huwendiek, S., Körner, M., et al. (2023) · Diagnosis · doi:10.1515/dx-2022-0103 · Cited by 18 · cited 1×
“To ensure that patients with rheumatic diseases receive appropriate care, it is crucial to provide evidence-based training in CR for medical students [25].”
Self-citationIncreasing Reasoning Awareness: Video Analysis of Students’ Two-Party Virtual Patient Interactions
Edelbring, S., Parodis, I., & Lundberg, I. E. (2018) · JMIR Medical Education · doi:10.2196/mededu.9137 · Cited by 16 · cited 1×
“VPs have been proposed as an effective educational tool for practicing clinical reasoning (CR) in undergraduate medical education [4] while also ensuring that medical students encounter representative patient cases, which might not always be possible to encounter during clinical rotations [5-7].”
Self-citationDoes a robot’s gaze aversion affect human gaze aversion?
Mishra, C., Offrede, T., Fuchs, S. I., Mooshammer, C. R., & Skantze, G. (2023) · Frontiers in Robotics and AI · doi:10.3389/frobt.2023.1127626 · Cited by 13 · cited 1×
“We adopted a gaze aversion technique [33] combined with LED lights at the base of the robot to communicate its turn-taking status, indicating whether it is speaking or listening.”
Self-citationJohansson, R., Skantze, G., & Jönsson, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67401-8_20 · Cited by 12 · cited 1×
“Recent technological developments have enabled the integration of dialogue systems with natural language processing capabilities, embodied conversational agents [14], and social robots [15] to enhance the interactivity and realism of VP interactions.”
Self-citationGonçalves, N. P., Lopes, S. A., Sepriano, A. R., Henriques, A. R., Pires, T., Branco, J., Canhão, H. M., & Rodrigues, A. M. (2023) · PubMed · Cited by 4
Real-Time Hybrid Language Model for Virtual Patient Conversations
Ng, H. W., Koh, A. T. Y., Foong, A., & Ong, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36272-9_71 · Cited by 4 · cited 1×
“However, their use for VPs has been limited to preliminary endeavors [18-21].”
Managing Inflammatory Bowel Disease in Spondyloarthritis
Fakih, O., & Service de Rhumatologie, C. D. B. (2023) · touchREVIEWS in RMD · doi:10.17925/rmd.2023.2.1.7 · Cited by 1
Research works printed but not linked: 1
- BrownTMannBRyderNSubbiahMKaplanJDDhariwalPNeelakantanAShyamPSastryGAskellAAgarwalSHerbert-VossAKruegerGHenighanTChildRRameshAZieglerDMLanguage models are few-shot learners2020Advances in Neural Information Processing SystemsDecember 6-12, 2020Vancouver, BC18771901
Cited by
Zeng, J., Qi, W., Shen, S., Liu, X., Li, S., Wang, B., Dong, C., Zhu, X., Shi, Y., Lou, X., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/79091 · Cited by 76
“For the measurement tools, 3 validated questionnaires were used to assess usability in the technological characteristics [21,46]; 1 validated questionnaire was used to assess authenticity in the simulation realism [50].”
InfluentialA large language model digital patient system enhances ophthalmology history taking skills
Luo, M.-J., Bi, S., Pang, J., Liu, L., Tsui, C.-K., Lai, Y., Chen, W., Yang, Y., Xu, K., Zhao, L., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01841-6 · Cited by 39
“…41 ; Yao et al. developed a related LLM-powered patient dialog system 42 ; Borg et al. combined an LLM with a social robot to create more authentic case encounters 50 ; and Holderried et al. found that a ChatGPT chatbot acting as a simulated patient produced highly plausible history responses 51 .”
Huang, S., Wen, C., Bai, X., Li, S., Wang, S., Wang, X., & Yang, D. (2025) · Journal of Medical Internet Research · doi:10.2196/68538 · Cited by 33
“Research has shown that ChatGPT can assist students in reviewing key concepts, reinforcing theoretical knowledge [1,15], and simulating clinical scenarios to enhance clinical reasoning [16].”
Li, D., & Lutfi, S. L. (2025) · JMIR Medical Informatics · doi:10.2196/79039 · Cited by 45
“…medical students and practitioners, using mixed methods (top-k accuracy, F 1 -score, SUS, CUQ, and expert ratings) with comparisons across AI models, physicians, or prompt variations; small sample sizes (10-50 students and 3-10 experts) and inconsistent metrics limit generalizability [19,40].”
InfluentialElhilali, A., Ngo, A. S.-H., Reichenpfader, D., & Denecke, K. (2025) · JMIR Medical Education · doi:10.2196/81271 · Cited by 34
“Borg et al [19] combined the LLM-based simulation (GPT-3.”
Metadata
- Authors
- Borg, A., Georg, C., Jobs, B., Huss, V., Waldenlind, K., Ruiz, M., Edelbring, S., Skantze, G., & Parodis, I.
- Affiliations
- Karolinska University Hospital (SE)
- Karolinska Institutet (SE)
- Örebro University (SE)
- KTH Royal Institute of Technology (SE)
- Corresponding author
- Parodis, Ioannis
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2025-03-03
- Volume
- 27
- Pages
- e63312
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11914843
- OpenAlex
- W4408098677
- Cited by
- 51
- References
- 36
- Citations by year
- 2026: 37 · 2025: 14
- FWCI
- 32.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Education › Simulation-Based Education in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- virtual patients, social robots, large language models, clinical reasoning training, medical students, computer simulation, communication skills, emotional competence, patient safety, health professions education, facial expressions
- MeSH terms
- Clinical Reasoning, Large Language Models, Adult, Education, Medical, Female, Humans, Language, Male, Robotics, Students, Medical, Sweden, Patient Simulation, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.