CoChat

Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials

Open access
Jiang, Nan, Zhang, Yuelun, Liang, Siyu, Lyu, Xiaohong, Chen, Shi, Huang, Xiaoming, Pan, Hui · 2024 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index; 21 of the 41 sources its claims cite were checked for support.

Abstract

Background Virtual simulation (VS) is a developing education approach with the recreation of reality using digital technology. The teaching effectiveness of VSs compared to mannequins and real persons (RPs) has never been investigated in medical and nursing education. Objective This study aims to compare VSs and mannequins or RPs in improving the following clinical competencies: knowledge, procedural skills, clinical reasoning, and communication skills. Methods Following Cochrane methodology, a meta-analysis was conducted on the effectiveness of VSs in pre- and postregistration medical or nursing participants. The Cochrane Library, PubMed, Embase, and Educational Resource Information Centre databases were searched to identify English-written randomized controlled trials up to August 2024. Two authors independently selected studies, extracted data, and assessed the risk of bias. All pooled estimates were based on random-effects models and assessed by trial sequential analyses. Leave-one-out, subgroup, and univariate meta-regression analyses were performed to explore sources of heterogeneity. Results A total of 27 studies with 1480 participants were included. Overall, there were no significant differences between VSs and mannequins or RPs in improving knowledge (standard mean difference [SMD]=0.08; 95% CI –0.30 to 0.47; I2=67%; P=.002), procedural skills (SMD=–0.12; 95% CI –0.47 to 0.23; I2=75%; P<.001), clinical reasoning (SMD=0.29; 95% CI –0.26 to 0.85; I2=88%; P<.001), and communication skills (SMD=–0.02; 95% CI: –0.62 to 0.58; I2=86%; P<.001). Trial sequential analysis for clinical reasoning indicated an insufficient sample size for a definitive judgment. For procedural skills, subgroup analyses showed that VSs were less effective among nursing participants (SMD=–0.55; 95% CI –1.07 to –0.03; I2=69%; P=.04). Univariate meta-regression detected a positive effect of publication year (β=.09; P=.02) on communication skill scores. Conclusions Given favorable cost-utility plus high flexibility regarding time and space, VSs are viable alternatives to traditional face-to-face learning modalities. The comparative effectiveness of VSs deserves to be followed up with the emergence of new technology. In addition, further investigation of VSs with different design features will provide novel insights to drive education reform. Trial Registration PROSPERO CRD42023466622; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=466622

Overview

CoChat Summary

TL;DR

This meta-analysis of 27 randomized controlled trials with 1480 participants found no significant differences between virtual simulations and mannequins or real persons for knowledge, procedural skills, clinical reasoning, or communication skills in medical and nursing education. Virtual simulations were less effective for procedural skills among nursing participants, while communication skill effects increased with later publication year.

Key findings

  • Across 27 studies and 1480 participants, pooled effects showed no significant differences between virtual simulations and mannequins or real persons for knowledge (SMD=0.08, 95% CI –0.30 to 0.47; I²=67%; P=.002), procedural skills (SMD=–0.12, 95% CI –0.47 to 0.23; I²=75%; P<.001), clinical reasoning (SMD=0.29, 95% CI –0.26 to 0.85; I²=88%; P<.001), and communication skills (SMD=–0.02, 95% CI –0.62 to 0.58; I²=86%; P<.001).
  • Trial sequential analysis indicated insufficient sample size for a definitive judgment on clinical reasoning, while procedural skills reached the required information size for a conclusive statistically insignificant result; knowledge and communication skills reached the “inner wedge” area.
  • For procedural skills, subgroup analysis showed virtual simulations were less effective among nursing participants (SMD=–0.55, 95% CI –1.07 to –0.03; I²=69%; P=.04).
  • Univariate meta-regression detected a positive effect of publication year on communication skill scores (β=.09; P=.02).

Methods

  • Following Cochrane methodology and PRISMA guidelines, the authors searched the Cochrane Library, PubMed, Embase, and Educational Resource Information Centre databases for English-language randomized controlled trials up to August 2024; the protocol was registered in PROSPERO (CRD42023466622).
  • Two authors independently selected studies, extracted data, and assessed risk of bias; 27 studies with 1480 participants were included, comparing virtual simulations with mannequins or real persons in pre- and postregistration medical or nursing participants.
  • Pooled estimates used random-effects models with standardized mean differences; heterogeneity was assessed with Cochran Q and I², and explored using leave-one-out, subgroup, and univariate meta-regression analyses; trial sequential analysis was also performed.

Limitations

  • The paper states that researchers in several studies were not blinded and might introduce bias when assessing participants; certain subgroup or meta-regression analyses were not feasible or included only one entry due to the limited number of studies; and heterogeneity could not be fully explained by leave-one-out, subgroup, or meta-regression analyses.
  • Evident from the text: heterogeneity was substantial to considerable for most outcomes (I²=67%-88%), allocation concealment was not explicitly mentioned in most studies, and many risk-of-bias domains were rated unclear, including reporting bias due to absent protocols.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/56195

Cites

71 of 71 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) flow diagram.
Figure 1 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) flow diagram.CC BY · PubMed Central
Figure 2 Risk of bias summary [16,18,30-54]. +: low risk of bias; –: high risk of bias; ?: unclear risk of bias.
Figure 2 Risk of bias summary [16,18,30-54]. +: low risk of bias; –: high risk of bias; ?: unclear risk of bias.CC BY · PubMed Central
Figure 3 Forest plot of virtual simulations versus mannequins or real persons for knowledge [32,35,37,41,44-46,50].
Figure 3 Forest plot of virtual simulations versus mannequins or real persons for knowledge [32,35,37,41,44-46,50].CC BY · PubMed Central
Figure 4 Forest plot of virtual simulations versus mannequins or real persons for procedural skills [16,18,32-34,36,37,42,49,51].
Figure 4 Forest plot of virtual simulations versus mannequins or real persons for procedural skills [16,18,32-34,36,37,42,49,51].CC BY · PubMed Central
Figure 5 Forest plot of virtual simulations versus mannequins or real persons for clinical reasoning [38,40,43,45,47,48,50,52,53].
Figure 5 Forest plot of virtual simulations versus mannequins or real persons for clinical reasoning [38,40,43,45,47,48,50,52,53].CC BY · PubMed Central
Figure 6 Forest plot of virtual simulations versus mannequins or real persons for communication skills [30,31,39,52,54].
Figure 6 Forest plot of virtual simulations versus mannequins or real persons for communication skills [30,31,39,52,54].CC BY · PubMed Central

Verification

Claims the paper makes

19 Valid · 1 Not valid · 11 Not checked · 91 Own results and interpretations

40 more claims · 1 with their cited sources checked

Create a free account to see all 122 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 40
  • Probably not a substantial claim. 50
  • The quote only partly states this claim. 4

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 5

Citations

Cites

Cited by

See all 28

Metadata

Authors
Jiang, N., Zhang, Y., Liang, S., Lyu, X., Chen, S., Huang, X., & Pan, H.
Affiliations
Chinese Academy of Medical Sciences & Peking Union Medical College (CN)
Corresponding author
Chen, Shi
Venue
Journal of Medical Internet Research
Published
2024-12-05
Volume
26
Pages
e56195
ISSN
1438-8871 (ISSN-L)
OpenAlex
W4405067277
Cited by
28
References
65
Citations by year
2026: 17 · 2025: 10 · 2024: 1
FWCI
7.90
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
  • Peking Union Medical College Hospital
  • Peking Union Medical College
Topics
  • Social Sciences › Education › Simulation-Based Education in Healthcare
  • Health Sciences › Medicine › Surgery › Surgical Simulation and Training
  • Social Sciences › Education › Innovations in Medical Education
Keywords
virtual simulation, mannequins, medical education, nursing education, clinical competence, procedural skills, clinical reasoning, communication skills, knowledge acquisition, simulation-based training, pre-registration students, trial sequential analysis, publication bias, cost-effectiveness, health workforce shortage, quality of care
MeSH terms
Clinical Competence, Education, Medical, Education, Nursing, Humans, Manikins, Randomized Controlled Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.