Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials
Open accessPublished version of a preprint
The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index; 21 of the 41 sources its claims cite were checked for support.
Abstract
Background Virtual simulation (VS) is a developing education approach with the recreation of reality using digital technology. The teaching effectiveness of VSs compared to mannequins and real persons (RPs) has never been investigated in medical and nursing education. Objective This study aims to compare VSs and mannequins or RPs in improving the following clinical competencies: knowledge, procedural skills, clinical reasoning, and communication skills. Methods Following Cochrane methodology, a meta-analysis was conducted on the effectiveness of VSs in pre- and postregistration medical or nursing participants. The Cochrane Library, PubMed, Embase, and Educational Resource Information Centre databases were searched to identify English-written randomized controlled trials up to August 2024. Two authors independently selected studies, extracted data, and assessed the risk of bias. All pooled estimates were based on random-effects models and assessed by trial sequential analyses. Leave-one-out, subgroup, and univariate meta-regression analyses were performed to explore sources of heterogeneity. Results A total of 27 studies with 1480 participants were included. Overall, there were no significant differences between VSs and mannequins or RPs in improving knowledge (standard mean difference [SMD]=0.08; 95% CI –0.30 to 0.47; I2=67%; P=.002), procedural skills (SMD=–0.12; 95% CI –0.47 to 0.23; I2=75%; P<.001), clinical reasoning (SMD=0.29; 95% CI –0.26 to 0.85; I2=88%; P<.001), and communication skills (SMD=–0.02; 95% CI: –0.62 to 0.58; I2=86%; P<.001). Trial sequential analysis for clinical reasoning indicated an insufficient sample size for a definitive judgment. For procedural skills, subgroup analyses showed that VSs were less effective among nursing participants (SMD=–0.55; 95% CI –1.07 to –0.03; I2=69%; P=.04). Univariate meta-regression detected a positive effect of publication year (β=.09; P=.02) on communication skill scores. Conclusions Given favorable cost-utility plus high flexibility regarding time and space, VSs are viable alternatives to traditional face-to-face learning modalities. The comparative effectiveness of VSs deserves to be followed up with the emergence of new technology. In addition, further investigation of VSs with different design features will provide novel insights to drive education reform. Trial Registration PROSPERO CRD42023466622; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=466622
Overview
CoChat Summary
TL;DR
This meta-analysis of 27 randomized controlled trials with 1480 participants found no significant differences between virtual simulations and mannequins or real persons for knowledge, procedural skills, clinical reasoning, or communication skills in medical and nursing education. Virtual simulations were less effective for procedural skills among nursing participants, while communication skill effects increased with later publication year.
Key findings
- Across 27 studies and 1480 participants, pooled effects showed no significant differences between virtual simulations and mannequins or real persons for knowledge (SMD=0.08, 95% CI –0.30 to 0.47; I²=67%; P=.002), procedural skills (SMD=–0.12, 95% CI –0.47 to 0.23; I²=75%; P<.001), clinical reasoning (SMD=0.29, 95% CI –0.26 to 0.85; I²=88%; P<.001), and communication skills (SMD=–0.02, 95% CI –0.62 to 0.58; I²=86%; P<.001).
- Trial sequential analysis indicated insufficient sample size for a definitive judgment on clinical reasoning, while procedural skills reached the required information size for a conclusive statistically insignificant result; knowledge and communication skills reached the “inner wedge” area.
- For procedural skills, subgroup analysis showed virtual simulations were less effective among nursing participants (SMD=–0.55, 95% CI –1.07 to –0.03; I²=69%; P=.04).
- Univariate meta-regression detected a positive effect of publication year on communication skill scores (β=.09; P=.02).
Methods
- Following Cochrane methodology and PRISMA guidelines, the authors searched the Cochrane Library, PubMed, Embase, and Educational Resource Information Centre databases for English-language randomized controlled trials up to August 2024; the protocol was registered in PROSPERO (CRD42023466622).
- Two authors independently selected studies, extracted data, and assessed risk of bias; 27 studies with 1480 participants were included, comparing virtual simulations with mannequins or real persons in pre- and postregistration medical or nursing participants.
- Pooled estimates used random-effects models with standardized mean differences; heterogeneity was assessed with Cochran Q and I², and explored using leave-one-out, subgroup, and univariate meta-regression analyses; trial sequential analysis was also performed.
Limitations
- The paper states that researchers in several studies were not blinded and might introduce bias when assessing participants; certain subgroup or meta-regression analyses were not feasible or included only one entry due to the limited number of studies; and heterogeneity could not be fully explained by leave-one-out, subgroup, or meta-regression analyses.
- Evident from the text: heterogeneity was substantial to considerable for most outcomes (I²=67%-88%), allocation concealment was not explicitly mentioned in most studies, and many risk-of-bias domains were rated unclear, including reporting bias due to absent protocols.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 Risk of bias summary [16,18,30-54]. +: low risk of bias; –: high risk of bias; ?: unclear risk of bias.](https://figures.cochat.ai/paper-figures/82664113fdf146e8923cbea021df6e4c/effectiveness-of-virtual-simulations-versus-mannequins-and-real-persons-in-figure-2.jpg)
![Figure 3 Forest plot of virtual simulations versus mannequins or real persons for knowledge [32,35,37,41,44-46,50].](https://figures.cochat.ai/paper-figures/670852939de24ad7ba5e0abfbe994380/effectiveness-of-virtual-simulations-versus-mannequins-and-real-persons-in-figure-3.jpg)
![Figure 4 Forest plot of virtual simulations versus mannequins or real persons for procedural skills [16,18,32-34,36,37,42,49,51].](https://figures.cochat.ai/paper-figures/ad30f44d66fd4ce8b64ed1f94cf18855/effectiveness-of-virtual-simulations-versus-mannequins-and-real-persons-in-figure-4.jpg)
![Figure 5 Forest plot of virtual simulations versus mannequins or real persons for clinical reasoning [38,40,43,45,47,48,50,52,53].](https://figures.cochat.ai/paper-figures/851df484021046c99714a1710142acc8/effectiveness-of-virtual-simulations-versus-mannequins-and-real-persons-in-figure-5.jpg)
![Figure 6 Forest plot of virtual simulations versus mannequins or real persons for communication skills [30,31,39,52,54].](https://figures.cochat.ai/paper-figures/a3143925246f43a5844ca7a1afbe3ad2/effectiveness-of-virtual-simulations-versus-mannequins-and-real-persons-in-figure-6.jpg)
Verification
Claims the paper makes
19 Valid · 1 Not valid · 11 Not checked · 91 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While simulation-based education may bring downstream benefits of reducing medical errors and improving patient care, the current level of evidence is still low due to a lack of high-quality studies with patient-centered outcomes [8-12].”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Introduction
What the sources say
- Not checked — no full text of the sourcecites [8] source
- Not checked — no full text of the sourcecites [9] source
- Partly supportedcites [10] sourceExperimental — accuracy not yet measured
The Study quality and Cadaveric models used for simulation training sections report that most included studies had weak methodology and only one of 51 measured patient outcomes. The Discussion likewise finds little evidence that skills transfer into the operating theatre. These findings concern cadaveric simulation in postgraduate surgical training, not simulation-based education generally; the paper does not establish a reduction in medical errors. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [11] source
- Supportedcites [12] source
The Introduction reports that patient outcomes appear in only 0–5% of medical education studies. The Discussion identifies 50 simulation studies reporting patient outcomes and describes frequent statistical errors and other methodological weaknesses; it also reports downstream benefits while cautioning that these conclusions are tempered by study limitations. The indexed passages do not provide page numbers. (LLM, conf=0.88)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In medical and nursing education, simulation-based teaching has demonstrated superior effectiveness compared to didactic teaching [4,5]”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Introduction
What the sources say
- Partly supportedcites [4] sourceExperimental — accuracy not yet measured
The Conclusions (page not provided) find simulation-based medical education with deliberate practice superior to traditional clinical education for specific clinical skill-acquisition goals. The Results describe studies of medical learners, not nursing education, and the comparator is traditional clinical education rather than didactic teaching. The claim extends the finding beyond the population and comparison studied. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [5] source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“they can provide a more realistic experience with the aid of artificial intelligence (AI) and virtual reality (VR) when certain pathological findings cannot be readily expressed [16-18].”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Introduction
What the sources say
- Partly supportedcites [16] sourceExperimental — accuracy not yet measured
The Background and Methods describe VR presenting abnormal neurological findings that standardized patients cannot express, and the Discussion says this can make cases similar to those of actual patients. The paper does not attribute that capability to AI, and it reports similar—not greater—perceived realness for VRNET and standardized patients. Page numbers are unavailable in our copy. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [17] source
- Not checked — no full text of the sourcecites [18] source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Neither of these groups showed a significant difference between VSs and mannequins or RPs (SMD=–0.12; 95% CI –0.75 to 0.51; I2=88%; P<.001 for undergraduate and SMD=0.93; 95% CI –0.50 to 2.36 for nonundergraduate), with no significant subgroup difference at P=.19.”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Results > Effects of Interventions
What the sources say
- Partly supportedcites [30] sourceExperimental — accuracy not yet measured
The Results (Empirically assessed communication scenario scores; page unavailable) report no statistically significant difference between VR and role-play for the clinician cohort (P = 0.17). The paper does not report a nonundergraduate subgroup analysis, a mannequin comparison, or the attributed SMD and confidence interval. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Neither of these groups showed a significant difference between VSs and mannequins or RPs (SMD=–0.12; 95% CI –0.75 to 0.51; I2=88%; P<.001 for undergraduate and SMD=0.93; 95% CI –0.50 to 2.36 for nonundergraduate), with no significant subgroup difference at P=.19.”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Results > Effects of Interventions
What the sources say
- Not checked — no full text of the sourcecites [31] source
- Not checked — no full text of the sourcecites [39] source
- Not checked — no full text of the sourcecites [52] source
- Partly supportedcites [54] sourceExperimental — accuracy not yet measured
The Team Communication Performance and Teamwork Attitudes sections report no significant difference between virtual and live simulation groups of undergraduate medical and nursing students. The paper is a single randomized trial, however, and does not report the claimed undergraduate subgroup analysis, pooled SMD, confidence interval, or I² statistic. (LLM, conf=0.88)
plainn = 4 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
40 more claims · 1 with their cited sources checked
Create a free account to see all 122 claims and every source verdict.
- It may state two separate claims as one. 40
- Probably not a substantial claim. 50
- The quote only partly states this claim. 4
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Researchers have found traditional high-fidelity simulation with mannequins and emerging novel platforms, such as VR, to be effective in promoting learning outcomes in students ( Jiang et al., 2024 ).”
From Feasibility and acceptability of incorporating artificial intelligence into simulation education (2025) · cites this paper as Jiang et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Discussion > Principal Findings
“Among nursing participants, VSs were more effective in improving procedural skills.”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Discussion > Comparison With Prior Work
“Subgroup analyses showed that VSs were less effective among nursing participants.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“participants in 14 studies were undergraduate students [16,31,35,39,42-46,50-54]; participants in 8 studies were graduate students or clinical staff [18,32-34,36-38,40,41,49]; and the remaining 2 studies enrolled mixed undergraduate and nonundergraduate participants [30,48].”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Results > Included Studies
Reported 27 · recomputed 24
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“13 studies compared VSs and mannequins [33-36,38,40,44-51]; 10 studies compared VSs and RPs [16,18,30,31,37,39,41-43,52-54]; and the remaining study included mannequins and RPs as comparison groups simultaneously [32].”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Results > Included Studies
Reported 27 · recomputed 24
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Results > Effects of Interventions
“the pooled effect size did not reflect a significant difference between VSs and mannequins or RPs on the outcome of procedural skills (SMD=0.29; 95% CI –0.26 to 0.85; I2=88%; P<.01).”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Results > Effects of Interventions
“A total of 9 studies assessed the outcome of clinical reasoning [38,40,43,45,47,48,50,52,53].”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Results > Effects of Interventions
“the pooled effect size did not reflect a significant difference between VSs and mannequins or RPs on the outcome of procedural skills (SMD=–0.02; 95% CI –0.62 to 0.58; I2=86%; P<.01).”
§ Effectiveness of Virtual Simulations Versus Mannequins and Real Persons in Medical and Nursing Education: Meta-Analysis and Trial Sequential Analysis of Randomized Controlled Trials > Results > Effects of Interventions
“A total of 5 studies assessed the outcome of communication skills [30,31,39,52,54].”
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Wan, X., Wang, W., Liu, J., & Tong, T. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-135 · Cited by 11,735 · cited 1×
“For studies that reported outcomes as median values and IQRs or as mean values and CIs, these measures were converted to mean values and SDs using methods described in previous literature [25,26] .”
A revision of bloom's taxonomy: an overview
Krathwohl, D. R. (2002) · Theory Into Practice · doi:10.1207/s15430421tip4104_2 · Cited by 8,132
How to perform a meta-analysis with R: a practical tutorial
Balduzzi, S., Rücker, G., & Schwarzer, G. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300117 · Cited by 5,827 · cited 1×
“All data analyses were performed using package meta of R software (version 4.2.3; R Foundation for Statistical Computing) [29] and TSA software (version 0.9.5.10 beta; Copenhagen Trial Unit).”
InfluentialMcGaghie, W. C., Issenberg, S. B., Cohen, E. R., Barsuk, J. H., & Wayne, D. B. (2011) · Academic Medicine · doi:10.1097/acm.0b013e318217e119 · Cited by 1,866 · cited 1×
“In medical and nursing education, simulation-based teaching has demonstrated superior effectiveness compared to didactic teaching [4,5]; moreover, it reduces anxiety and increases the confidence of students entering practice [6,7].”
How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence
Cummings, J. J., & Bailenson, J. N. (2016) · Media Psychology · doi:10.1080/15213269.2015.1015740 · Cited by 1,749
Trial Sequential Analysis in systematic reviews with meta-analysis
Wetterslev, J., Jakobsen, J. C., & Gluud, C. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0315-7 · Cited by 1,189 · cited 1×
“%) and used these values to calculate trial sequential monitoring boundaries, futility boundaries, and the required information size (RIS) [28].”
Learning science in immersive virtual reality.
Parong, J., & Mayer, R. E. (2018) · Journal of Educational Psychology · doi:10.1037/edu0000241 · Cited by 1,088
Simulation-Based Medical Education: An Ethical Imperative
Ziv, A., Wolpe, P. R., Small, S. D., & Glick, S. (2003) · Academic Medicine · doi:10.1097/00001888-200308000-00006 · Cited by 1,008
Kyaw, B. M., Saxena, N., Posadzki, P., Vseteckova, J., Nikolaou, C. K., George, P. P., Divakar, U., Masiello, I., Kononowicz, A. A., Zary, N., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/12959 · Cited by 859 · cited 3×
“…SPs), digital education is at least as effective in training medical students’ communication skills [15]; virtual patients (VPs) can more effectively improve skills and can be at least as effective in enhancing knowledge [21]; VR can more effectively improve not only skills but also knowledge [22].”
Healthcare Simulation Dictionary
(2020) · Agency for Healthcare Research and Quality (US) eBooks · doi:10.23970/simulationv2 · Cited by 613
Kononowicz, A. A., Woodham, L. A., Edelbring, S., Stathakarou, N., Davies, D., Saxena, N., Tudor Car, L., Carlstedt-Duke, J., Car, J., & Zary, N. (2019) · Journal of Medical Internet Research · doi:10.2196/14676 · Cited by 600 · cited 4×
“…SPs), digital education is at least as effective in training medical students’ communication skills [15]; virtual patients (VPs) can more effectively improve skills and can be at least as effective in enhancing knowledge [21]; VR can more effectively improve not only skills but also knowledge [22].”
InfluentialComputerized Virtual Patients in Health Professions Education: A Systematic Review and Meta-Analysis
Cook, D. A., Erwin, P. J., & Triola, M. M. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181edfe13 · Cited by 570
Clinical virtual simulation in nursing education: randomized controlled trial
Padilha, J. M., Machado, P. P., Ribeiro, A., Ramos, J., & Costa, P. (2019) · Journal of Medical Internet Research · doi:10.2196/11529 · Cited by 550 · cited 3×
“…analysis by discipline (Figure S3a in Multimedia Appendix 4 [32,35,37,41,44-46,50]) , 4 studies [ For subgroup analysis by level (Figure S3b in Multimedia Appendix 4 [32,35,37,41,44-46,50 ]) , 5 studies For subgroup analysis by comparison (Figure S3c in Multimedia Appendix 4 [32,35,37,41,44-46,50”
Effectiveness of patient simulation in nursing education: Meta-analysis
Shin, S., Park, J.-H., & Kim, J.-H. (2015) · Nurse Education Today · doi:10.1016/j.nedt.2014.09.009 · Cited by 543 · cited 1×
“In medical and nursing education, simulation-based teaching has demonstrated superior effectiveness compared to didactic teaching [4,5]; moreover, it reduces anxiety and increases the confidence of students entering practice [6,7].”
Effectiveness of virtual reality in nursing education: meta-analysis
Chen, F.-Q., Leng, Y.-F., Ge, J.-F., Wang, D.-W., Li, C., Chen, B., & Sun, Z.-L. (2020) · Journal of Medical Internet Research · doi:10.2196/18290 · Cited by 535 · cited 1×
“This result aligns with a previous meta-analysis comparing 2D VSs with traditional education methods [21], while other analyses suggested that 3D VSs were more effective [22,55-57].”
Patient Outcomes in Simulation-Based Medical Education: A Systematic Review
Zendejas, B., Brydges, R., Wang, A. T., & Cook, D. A. (2013) · Journal of General Internal Medicine · doi:10.1007/s11606-012-2264-5 · Cited by 384 · cited 1×
“While simulation-based education may bring downstream benefits of reducing medical errors and improving patient care, the current level of evidence is still low due to a lack of high-quality studies with patient-centered outcomes [8-12].”
The use of elearning in medical education: a review of the current situation
Choules, A. P. (2007) · Postgraduate Medical Journal · doi:10.1136/pgmj.2006.054189 · Cited by 306 · cited 2×
“For educators, VSs can potentially save costs due to reduced instructor time, manpower, and space resources [19].”
Effectiveness of virtual reality in nursing education: a systematic review and meta-analysis
Liu, K., Zhang, W., Li, W., Wang, T., & Zheng, Y. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04662-x · Cited by 284 · cited 2×
“These results were inconsistent with previous meta-analyses comparing VSs with traditional education methods [21,22,57].”
Kyaw, B. M., Posadzki, P., Paddock, S., Car, J., Campbell, J., & Tudor Car, L. (2019) · Journal of Medical Internet Research · doi:10.2196/12967 · Cited by 257 · cited 3×
“…in-class group discussions, mannequins, SPs), digital education is at least as effective in training medical students’ communication skills [15]; virtual patients (VPs) can more effectively improve skills and can be at least as effective in enhancing knowledge [21]; VR can more effectively…”
InfluentialRyan, G. V., Callaghan, S., Rafferty, A., Higgins, M. F., Mangina, E., & McAuliffe, F. (2022) · Journal of Medical Internet Research · doi:10.2196/30082 · Cited by 242 · cited 1×
“Second, immersive technology (ie, devices that provide a sense of realism and immersion in the computer-generated world through sensory stimuli) such as VR promotes students’ interest with enhanced satisfaction, self-efficacy, and engagement [59].”
Simulation in medical education
Khan, K., Pattison, T., & Sherwood, M. (2011) · Medical Teacher · doi:10.3109/0142159x.2010.519412 · Cited by 218
The Role for Virtual Patients in the Future of Medical Education
Berman, N. B., Durning, S. J., Fischer, M. R., Huwendiek, S., & Triola, M. M. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001146 · Cited by 211
Virtual Reality Triage Training Provides a Viable Solution for Disaster‐preparedness
Andreatta, P. B., Maslowski, E., Petty, S., Shim, W., Marsh, M., Hall, T., Stern, S., & Frankel, J. (2010) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2010.00728.x · Cited by 209 · cited 5×
“The Egger test showed For subgroup analysis by discipline (Figure S3a in Multimedia Appendix 4 [32,35,37,41,44-46,50]) , 4 studies [ For subgroup analysis by level (Figure S3b in Multimedia Appendix 4 [32,35,37,41,44-46,50 ]) , 5 studies For subgroup analysis by comparison (Figure S3c in Multimedia…”
InfluentialCobbett, S., & Snelgrove-Clarke, E. (2016) · Nurse Education Today · doi:10.1016/j.nedt.2016.08.004 · Cited by 208 · cited 4×
“…analysis by discipline (Figure S3a in Multimedia Appendix 4 [32,35,37,41,44-46,50]) , 4 studies [ For subgroup analysis by level (Figure S3b in Multimedia Appendix 4 [32,35,37,41,44-46,50 ]) , 5 studies For subgroup analysis by comparison (Figure S3c in Multimedia Appendix 4 [32,35,37,41,44-46,50”
InfluentialEfficacy of virtual patients in medical education: A meta-analysis of randomized studies
Consorti, F., Mancuso, R., Nocioni, M., & Piccolo, A. (2012) · Computers & Education · doi:10.1016/j.compedu.2012.04.017 · Cited by 203
Woon, A. P. N., Mok, W. Q., Chieng, Y. J. S., Zhang, H. M., Ramos, P., Mustadi, H. B., & Lau, Y. (2021) · Nurse Education Today · doi:10.1016/j.nedt.2020.104655 · Cited by 193
Liaw, S. Y., Ooi, S. W., Rusli, K. D. B., Lau, T. C., Tam, W. W. S., & Chua, W. L. (2020) · Journal of Medical Internet Research · doi:10.2196/17279 · Cited by 191 · cited 9×
“In Figure 6 [30,31,39,52,54], the pooled effect size did not reflect a significant difference between VSs and mannequins or RPs on the outcome of procedural skills (SMD=–0.02; 95 % CI –0.62 to 0.58; I 2 =86%; P <.01).”
InfluentialCost-Utility Analysis of Virtual and Mannequin-Based Simulation
Haerling, K. A. (2018) · Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0000000000000280 · Cited by 176 · cited 6×
“For subgroup analysis by comparison (Figure S9c in Multimedia Appendix 4 [38,40,43,45,47,48,50,52,53]), 6 studies [38,40,45,47,48,50] and 3 studies [43,52,53] included mannequin and RP arms, respectively.”
InfluentialVirtual Simulation to Enhance Clinical Reasoning in Nursing: A Systematic Review and Meta-analysis
Sim, J. J. M., Rusli, K. D. B., Seah, B., Levett-Jones, T., Lau, Y., & Liaw, S. Y. (2022) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2022.05.006 · Cited by 171
Digital Education in Health Professions: The Need for Overarching Evidence Synthesis
Car, J., Carlstedt-Duke, J., Tudor Car, L., Posadzki, P., Whiting, P., Zary, N., Atun, R., Majeed, A., Campbell, J., & Digital Health Education Collaboration (2019) · Journal of Medical Internet Research · doi:10.2196/12913 · Cited by 166
Do medical students respond empathetically to a virtual patient?
Deladisma, A. M., Cohen, M., Stevens, A., Wagner, P., Lok, B., Bernard, T., Oxendine, C., Schumacher, L., Johnsen, K., Dickerson, R., et al. (2007) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2007.01.021 · Cited by 160 · cited 7×
“In Figure 6 [30,31,39,52,54], the pooled effect size did not reflect a significant difference between VSs and mannequins or RPs on the outcome of procedural skills (SMD=–0.02; 95 % CI –0.62 to 0.58; I 2 =86%; P <.01).”
InfluentialYoungblood, P., Harter, P. M., Srivastava, S., Moffett, S. E., Heinrichs, W. L., & Dev, P. (2008) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0b013e31817bedf7 · Cited by 154 · cited 6×
“For subgroup analysis by comparison (Figure S9c in Multimedia Appendix 4 [38,40,43,45,47,48,50,52,53]), 6 studies [38,40,45,47,48,50] and 3 studies [43,52,53] included mannequin and RP arms, respectively.”
InfluentialLiaw, S. Y., Chan, S. W.-C., Chen, F.-G., Hooi, S. C., & Siau, C. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3322 · Cited by 147 · cited 6×
“For subgroup analysis by comparison (Figure S9c in Multimedia Appendix 4 [38,40,43,45,47,48,50,52,53]), 6 studies [38,40,45,47,48,50] and 3 studies [43,52,53] included mannequin and RP arms, respectively.”
InfluentialTrends and Innovations of Simulation for Twenty First Century Medical Education
Herrera-Aliaga, E., & Estrada, L. D. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.619769 · Cited by 146 · cited 1×
“Clinical simulation, a technique that replicates real experiences with guided experiences, has thereby gained popularity in the last 2 decades and is currently the main tool for training health care professionals [2].”
Virtual reality simulation for nursing education: effectiveness and feasibility
Kiegaldie, D., & Shaw, L. (2023) · BMC Nursing · doi:10.1186/s12912-023-01639-5 · Cited by 145 · cited 1×
“First, digital platforms or software provide repeatable, information-rich feedback for learning [58].”
Plackett, R., Kassianos, A. P., Mylan, S., Kambouri, M., Raine, R., & Sheringham, J. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03410-x · Cited by 143
Lost in space? Cognitive fit and cognitive load in 3D virtual environments
van der Land, S. F., Schouten, A. P., Feldberg, F., van den Hooff, B., & Huysman, M. (2012) · Computers in Human Behavior · doi:10.1016/j.chb.2012.09.006 · Cited by 122
Jung, E.-Y., Park, D. K., Lee, Y. H., Jo, H. S., Lim, Y. S., & Park, R. W. (2011) · Nurse Education Today · doi:10.1016/j.nedt.2011.05.012 · Cited by 119 · cited 1×
“A total of 10 studies assessed the outcome of procedural skills [16,18,32-34, 36 ,37,42,49,51].”
Systematic review of the current status of cadaveric simulation for surgical training
James, H. K., Chapman, A. W., Pattison, G. T. R., Griffin, D. R., & Fisher, J. D. (2019) · British Journal of Surgery · doi:10.1002/bjs.11325 · Cited by 118
Human‐simulation‐based learning to prevent medication error: A systematic review
Sarfati, L., Ranchon, F., Vantard, N., Schwiertz, V., Larbre, V., Parat, S., Faudel, A., & Rioufol, C. (2019) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12883 · Cited by 113
Gu, Y., Zou, Z., & Chen, X. (2017) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2017.01.005 · Cited by 104 · cited 1×
“A gap exists between digital environments and real circumstances, and the abstraction of procedures may not be transferable to reality [60].”
Oliveira Silva, G., Oliveira, F. S. E., Coelho, A. S. G., Cavalcante, A. M. R. Z., Vieira, F. V. M., Fonseca, L. M. M., Campbell, S. H., & Aredes, N. D. A. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2022.104282 · Cited by 91
LeBlanc, J., Hutchison, C., Hu, Y., & Donnon, T. (2013) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.k.01284 · Cited by 66 · cited 3×
“A total of 10 studies assessed the outcome of procedural skills [16,18,32-34, 36 ,37,42,49,51].”
Plotzky, C., Loessl, B., Kuhnert, B., Friedrich, N., Kugler, C., König, P., & Kunze, C. (2023) · BMC Nursing · doi:10.1186/s12912-023-01384-9 · Cited by 66 · cited 1×
“Notwithstanding, VSs are cost-effective alternatives that enable learners to repeat the training freely without worrying about time, space, and patient harm [61].”
Simsek-Cetinkaya, S., & Cakir, S. K. (2023) · Nurse Education Today · doi:10.1016/j.nedt.2023.105857 · Cited by 65 · cited 5×
“Second, too much immersion may not be desirable as it can lead to cognitive overload and hamper procedure learning [42].”
InfluentialChang, K., Chung, J. W. Y., & WONG, T. K. (2002) · Journal of Clinical Nursing · doi:10.1046/j.1365-2702.2002.00561.x · Cited by 63
Sapkaroski, D., Mundy, M., & Dimmock, M. R. (2022) · Journal of Medical Radiation Sciences · doi:10.1002/jmrs.555 · Cited by 62 · cited 8×
“In Figure 6 [30,31,39,52,54], the pooled effect size did not reflect a significant difference between VSs and mannequins or RPs on the outcome of procedural skills (SMD=–0.02; 95 % CI –0.62 to 0.58; I 2 =86%; P <.01).”
InfluentialSpadaro, S., Karbing, D. S., Fogagnolo, A., Ragazzi, R., Mojoli, F., Astolfi, L., Gioia, A., Marangoni, E., Rees, S. E., & Volta, C. A. (2017) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0000000000000249 · Cited by 61
Edeer, A. D., Dicle, A., CAKAN, E., & CAKIR, Ş. (2012) · CIN Computers Informatics Nursing · doi:10.1097/ncn.0b013e3182419134 · Cited by 60 · cited 8×
“The Egger test showed For subgroup analysis by discipline (Figure S3a in Multimedia Appendix 4 [32,35,37,41,44-46,50]) , 4 studies [ For subgroup analysis by level (Figure S3b in Multimedia Appendix 4 [32,35,37,41,44-46,50 ]) , 5 studies For subgroup analysis by comparison (Figure S3c in Multimedia…”
InfluentialHan, S. G., Kim, Y. D., Kong, T. Y., & Cho, J. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02920-4 · Cited by 50 · cited 6×
“…simulations (eg, mannequins and SPs), VSs can be undertaken flexibly with no limit of time and space [15]; they can provide a more realistic experience with the aid of artificial intelligence (AI) and virtual reality (VR) when certain pathological findings cannot be readily expressed [16-18].”
InfluentialEffect of simulation‐based team training in airway management: a systematic review
Nielsen, R. P., Nikolajsen, L., Paltved, C., & Aagaard, R. (2021) · Anaesthesia · doi:10.1111/anae.15375 · Cited by 47 · cited 1×
“While simulation-based education may bring downstream benefits of reducing medical errors and improving patient care, the current level of evidence is still low due to a lack of high-quality studies with patient-centered outcomes [8-12].”
Students’ performance in the different clinical skills assessed in OSCE: what does it reveal?
Sim, J. H., Aziz, Y. F. A., Mansor, A., Vijayananthan, A., Foong, C. C., & Vadivelu, J. (2015) · Medical Education Online · doi:10.3402/meo.v20.26185 · Cited by 41 · cited 3×
“…of interest were defined as follows: (1) knowledge: remembering and understanding basic concepts, measured by question-based theoretical tests [24]; (2) procedural skills: following a series of steps to accomplish certain tasks, demonstrated by simulation exams [24]; (3) clinical reasoning:…”
InfluentialLiaw, S. Y., Sutini, Chua, W. L., Tan, J. Z., Levett-Jones, T., Ashokka, B., Te Pan, T. L., Lau, S. T., & Ignacio, J. (2023) · Journal of General Internal Medicine · doi:10.1007/s11606-022-07557-7 · Cited by 40 · cited 8×
“For subgroup analysis by comparison (Figure S9c in Multimedia Appendix 4 [38,40,43,45,47,48,50,52,53]), 6 studies [38,40,45,47,48,50] and 3 studies [43,52,53] included mannequin and RP arms, respectively.”
InfluentialO’Rourke, S. R., Branford, K. R., Brooks, T. L., Ives, L. T., Nagendran, A., & Compton, S. N. (2020) · Teaching and Learning in Medicine · doi:10.1080/10401334.2019.1652180 · Cited by 37 · cited 7×
“In Figure 6 [30,31,39,52,54], the pooled effect size did not reflect a significant difference between VSs and mannequins or RPs on the outcome of procedural skills (SMD=–0.02; 95 % CI –0.62 to 0.58; I 2 =86%; P <.01).”
InfluentialJiang, B., Ju, H., Zhao, Y., Yao, L., & Feng, Y. (2018) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0000000000000299 · Cited by 35
Resident performance in basic perioperative transesophageal echocardiography
Weber, U., Zapletal, B., Base, E., et al. (2019) · Medicine · doi:10.1097/md.0000000000017072 · Cited by 34 · cited 5×
“The Egger test showed For subgroup analysis by discipline (Figure S3a in Multimedia Appendix 4 [32,35,37,41,44-46,50]) , 4 studies [ For subgroup analysis by level (Figure S3b in Multimedia Appendix 4 [32,35,37,41,44-46,50 ]) , 5 studies For subgroup analysis by comparison (Figure S3c in Multimedia…”
InfluentialThe Efficacy of Arthroscopic Simulation Training on Clinical Ability: A Systematic Review
Luzzi, A., Hellwinkel, J., O’Connor, M., Crutchfield, C., & Lynch, T. S. (2021) · Arthroscopy · doi:10.1016/j.arthro.2020.09.018 · Cited by 32
Teaching Diagnostic Reasoning to Advanced Practice Nurses: Positives and Negatives
LaManna, J. B., Guido-Sanz, F., Anderson, M., Chase, S. K., Weiss, J. A., & Blackwell, C. W. (2019) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2018.10.006 · Cited by 31 · cited 1×
“It has been suggested that VSs, with a large number of customizable clinical scenarios, are highly suitable for improving clinical assessment and decision-making [63].”
Manikin Versus Web-Based Simulation for Advanced Practice Nursing Students
Johnson, M. P., Hickey, K. T., Scopa-Goldman, J., Andrews, T. D., Boerem, P., Covec, M., & Larson, E. (2014) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2014.02.004 · Cited by 25 · cited 6×
“For subgroup analysis by comparison (Figure S9c in Multimedia Appendix 4 [38,40,43,45,47,48,50,52,53]), 6 studies [38,40,45,47,48,50] and 3 studies [43,52,53] included mannequin and RP arms, respectively.”
InfluentialAhlqvist, J. B., Nilsson, T. A., Hedman, L., Desser, T. S., Dev, P., Johansson, M., Youngblood, P., Cheng, R. P., & Gold, G. E. (2013) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0b013e3182a60a48 · Cited by 25 · cited 4×
“…analysis by discipline (Figure S3a in Multimedia Appendix 4 [32,35,37,41,44-46,50]) , 4 studies [ For subgroup analysis by level (Figure S3b in Multimedia Appendix 4 [32,35,37,41,44-46,50 ]) , 5 studies For subgroup analysis by comparison (Figure S3c in Multimedia Appendix 4 [32,35,37,41,44-46,50”
InfluentialWatmough, S., Box, H., Bennett, N., Stewart, A., & Farrell, M. (2016) · BMC Medical Education · doi:10.1186/s12909-016-0629-x · Cited by 25 · cited 1×
“In medical and nursing education, simulation-based teaching has demonstrated superior effectiveness compared to didactic teaching [4,5]; moreover, it reduces anxiety and increases the confidence of students entering practice [6,7].”
Is Virtual Patient Simulation Superior to Human Patient Simulation: A Randomized Controlled Study
Karaduman, G. Ş., & Başak, T. (2022) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000957 · Cited by 25 · cited 6×
“For subgroup analysis by comparison (Figure S9c in Multimedia Appendix 4 [38,40,43,45,47,48,50,52,53]), 6 studies [38,40,45,47,48,50] and 3 studies [43,52,53] included mannequin and RP arms, respectively.”
InfluentialBumbach, M. D., Culross, B. A., & Datta, S. K. (2022) · CIN: Computers, Informatics, Nursing · doi:10.1097/cin.0000000000000916 · Cited by 22 · cited 1×
“One recent analysis found that VR simulations required 22% less time and 40% lower cost than traditional high-fidelity simulations to achieve the same learning outcomes [20].”
Lee, M., Kim, S.-K., Go, Y., Jeong, H., & Lee, Y. (2024) · Applied Nursing Research · doi:10.1016/j.apnr.2024.151800 · Cited by 20 · cited 7×
“For subgroup analysis by comparison (Figure S9c in Multimedia Appendix 4 [38,40,43,45,47,48,50,52,53]), 6 studies [38,40,45,47,48,50] and 3 studies [43,52,53] included mannequin and RP arms, respectively.”
InfluentialMalik, T. G., Mahboob, U., Khan, R. A., & Alam, R. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05241-4 · Cited by 19 · cited 4×
“…analysis by discipline (Figure S3a in Multimedia Appendix 4 [32,35,37,41,44-46,50]) , 4 studies [ For subgroup analysis by level (Figure S3b in Multimedia Appendix 4 [32,35,37,41,44-46,50 ]) , 5 studies For subgroup analysis by comparison (Figure S3c in Multimedia Appendix 4 [32,35,37,41,44-46,50”
InfluentialPietersen, P. I., Jørgensen, R., Graumann, O., Konge, L., Skaarup, S. H., Lawaetz Schultz, H. H., & Laursen, C. B. (2021) · Respiration · doi:10.1159/000509298 · Cited by 18 · cited 5×
“…simulations (eg, mannequins and SPs), VSs can be undertaken flexibly with no limit of time and space [15]; they can provide a more realistic experience with the aid of artificial intelligence (AI) and virtual reality (VR) when certain pathological findings cannot be readily expressed [16-18].”
InfluentialLessons From Implementing Virtual Simulations: A Multi-Program Evaluation
Turner, S., Harder, N., Vigier, D., Cooper, A., Pinel, K., & Mitchell, K. (2022) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2022.09.001 · Cited by 16 · cited 1×
“Virtual simulation (VS) refers to the 2D or 3D recreation of reality by digital technology, where students are able to interact with digital scenes, instruments, and characters [13,14].”
Dante, A., Marcotullio, A., Masotta, V., Caponnetto, V., La Cerra, C., Bertocchi, L., Petrucci, C., & Alfes, C. M. (2021) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-52287-2_11 · Cited by 8
Tran, M., Ahmad, M., Patel, K. D., Argyriou, O., Davies, A. H., & Shalhoub, J. (2024) · Health Science Reports · doi:10.1002/hsr2.2245 · Cited by 7 · cited 6×
“For subgroup analysis by comparison (Figure S9c in Multimedia Appendix 4 [38,40,43,45,47,48,50,52,53]), 6 studies [38,40,45,47,48,50] and 3 studies [43,52,53] included mannequin and RP arms, respectively.”
Influential
Cited by
Toofaninejad, E., Mirzaei, S., Shakib, A. M., Dashtaki, D. G. M., Raoufian, H., Mirmoghtadaie, Z., & Sohrabi, S. (2025) · Advances in Medical Education and Practice · doi:10.2147/amep.s561949 · Cited by 13
Feasibility and acceptability of incorporating artificial intelligence into simulation education
Swan, B. A., Febres-Cordero, S., Steiger, L., Lisenby, A., Getz, T., Hudson, J., Cole, K., Branch, R., McDermott, C., Fugate, K., et al. (2025) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2025.101739 · Cited by 5
Zhu, X., Wang, Y., Chen, X., Zhang, J., Wu, W., Zhao, J., Li, J., Du, Z., & Gao, Y. (2025) · Nurse Education in Practice · doi:10.1016/j.nepr.2025.104416 · Cited by 6
From traditional to immersive: The impact of 3D technologies on medical student learning
Walecki, P. K., Dąbrowa, J., Więcławek, P., Gogól, K., & Proniewska, K. K. (2024) · Bio-Algorithms and Med-Systems · doi:10.5604/01.3001.0054.9632 · Cited by 8
Ronchi, S., Caruso, R., Fabrizi, D., Luciani, M., Alfes, C. M., Colley, N., Magon, A., Conte, G., Valli, A., Terzoni, S., et al. (2025) · MethodsX · doi:10.1016/j.mex.2025.103391 · Cited by 4
Metadata
- Authors
- Jiang, N., Zhang, Y., Liang, S., Lyu, X., Chen, S., Huang, X., & Pan, H.
- Affiliations
- Chinese Academy of Medical Sciences & Peking Union Medical College (CN)
- Corresponding author
- Chen, Shi
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-12-05
- Volume
- 26
- Pages
- e56195
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11659697
- OpenAlex
- W4405067277
- Cited by
- 28
- References
- 65
- Citations by year
- 2026: 17 · 2025: 10 · 2024: 1
- FWCI
- 7.90
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Peking Union Medical College Hospital
- Peking Union Medical College
- Topics
- Social Sciences › Education › Simulation-Based Education in Healthcare
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Social Sciences › Education › Innovations in Medical Education
- Keywords
- virtual simulation, mannequins, medical education, nursing education, clinical competence, procedural skills, clinical reasoning, communication skills, knowledge acquisition, simulation-based training, pre-registration students, trial sequential analysis, publication bias, cost-effectiveness, health workforce shortage, quality of care
- MeSH terms
- Clinical Competence, Education, Medical, Education, Nursing, Humans, Manikins, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.