Evaluating the Instructional Design and Effect on Knowledge, Teamwork, and Skills of Technology-Enhanced Simulation-Based Training in Obstetrics in Uganda: Stepped-Wedge Cluster Randomized Trial
Open accessPublished version of a preprint
The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Simulation-based training is a common strategy for improving the quality of facility-based maternity services and is often evaluated using Kirkpatrick's theoretical model. The results on the Kirkpatrick levels are closely related to the quality of the instructional design of a training program. The instructional design is generally defined as the "set of prescriptions for teaching methods to improve the quality of instruction with a goal of optimizing learning outcomes." OBJECTIVE: The aim of this study is to evaluate the instructional design of a technology-enhanced simulation-based training in obstetrics, the reaction of participants, and the effect on knowledge, teamwork, and skills in a low-income country. METHODS: A stepped-wedge cluster randomized trial was performed in a university hospital in Kampala, Uganda, with an annual delivery volume of over 31,000. In November 2014, a medical simulation center was installed with a full-body birthing simulator (Noelle S550, Gaumard Scientific), an interactive neonate (Simon S102 Newborn CPR Simulator, Gaumard Scientific), and an audio and video recording system. Twelve local obstetricians were trained and certified as medical simulation trainers. From 2014 to 2016, training was provided to 57 residents in groups of 6 to 9 students. Descriptive statistics were calculated for ten instructional design features of the training course measured by the 42-item ID-SIM (Instructional Design of a Simulation Improved by Monitoring). The Wilcoxon signed rank test was conducted to investigate the differences in scores on knowledge, the Clinical Teamwork Scale, and medical technical skills. RESULTS: The mean scores on the ten instructional design features ranged from 54.9 (95% CI 48.5-61.3) to 84.3 (95% CI 80.9-87.6) out of 100. The highest mean score was given on the feature feedback and the lowest scores on repetitive practice and controlled environment. The overall score for the training day was 92.8 out of 100 (95% CI 89.5-96.1). Knowledge improved significantly, with a test score of 63.4% (95% CI 60.7-66.1) before and 78.9% (95% CI 76.8-81.1) after the training (P<.001). The overall score on the 10-point Clinical Teamwork Scale was 6.0 (95% CI 4.4-7.6) before and 5.9 (95% CI 4.5-7.2) after the training (P=.78). Medical technical skills were scored at 55.5% (95% CI 47.2-63.8) before and 65.6% (95% CI 56.5-74.7) after training (P=.08). CONCLUSIONS: Most instructional design features of a technology-enhanced simulation-based training in obstetrics in a low-income country were scored high, although intervals were large. The overall score for the training day was high, and knowledge did improve after the training program, but no changes in teamwork and (most) medical technical skills were found. The lowest-scored instructional design features may be improved to achieve further learning aims. TRIAL REGISTRATION: ISRCTN Registry ISRCTN98617255; http://www.isrctn.com/ISRCTN98617255. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.1186/s12884-020-03050-3.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The everyday use of hospitals and clinics as clusters is rooted in their pivotal role in healthcare delivery, enabling the implementation and evaluation of interventions to enhance patient outcomes (26, 27, 30–34, 36–47, 49, 50, 52–71, 73–75, 77–81, 83–89, 91–96, 98–102, 104, 105, 107, 109).”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [32]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Health facilities, strategically chosen as clusters in numerous studies, underscore the emphasis on healthcare delivery and interventions within controlled environments.”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [32]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Simple randomization, often using computer-generated sequences, was a standard method employed to determine the order in which clusters received the intervention (26, 30, 32, 35, 38, 47, 58, 65).”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [32]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Several studies focused on implementing health technologies and digital solutions to improve healthcare delivery and patient outcomes (26, 27, 32, 54, 62, 79, 85, 90, 91, 94, 98, 101, 104, 105).”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [32]
The cited source says what the claim attributes to it.
In the citing paper
“Many studies involved interventions centered around training healthcare workers or providing health education to patients and communities (29, 32, 33, 48, 49, 62, 70, 78, 81, 86, 88, 89).”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The role of deliberate practice in the acquisition of expert performance
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993) · Psychological Review · doi:10.1037/0033-295x.100.3.363 · Cited by 9,564 · cited 1×
“Five out of these ten features corresponded to the educational theory of deliberate practice by Ericsson et al [54,55].”
Barry Issenberg, S., Mcgaghie, W. C., Petrusa, E. R., Lee Gordon, D., & Scalese, R. J. (2005) · Medical Teacher · doi:10.1080/01421590500046924 · Cited by 3,966 · cited 4×
“These include feedback, repetitive practice, ranging difficulty levels, defined outcomes, individualized learning, curriculum integration, multiple learning strategies, clinical variation, controlled environment, and simulator validity [42,46].”
InfluentialCook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J., & Hamstra, S. J. (2011) · JAMA · doi:10.1001/jama.2011.1234 · Cited by 2,170 · cited 1×
“Cook et al analyzed over 600 studies in a systematic review and meta-analysis and reported that the distribution of learning activities over more than one day was consistently associated with larger effect sizes [59].”
Deliberate practice and acquisition of expert performance: a general overview
Anders Ericsson, K. (2008) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2008.00227.x · Cited by 1,975 · cited 1×
“Five out of these ten features corresponded to the educational theory of deliberate practice by Ericsson et al [54,55].”
A critical review of simulation‐based medical education research: 2003–2009
McGaghie, W. C., Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2010) · Medical Education · doi:10.1111/j.1365-2923.2009.03547.x · Cited by 1,733
Design and analysis of stepped wedge cluster randomized trials
Hussey, M. A., & Hughes, J. P. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.05.007 · Cited by 1,393 · cited 1×
“For a stepped-wedge design, first the sample size calculation for a standard randomized clinical trial is required [49-51].”
Simulation in healthcare education: A best evidence practical guide. AMEE Guide No. 82
Motola, I., Devine, L. A., Chung, H. S., Sullivan, J. E., & Issenberg, S. B. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.818632 · Cited by 1,180 · cited 1×
“The features were incorporated into two guidelines for designing an effective simulation-based training by the Association for Medical Education in Europe [57,58].”
Cook, D. A., Hamstra, S. J., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J., & Hatala, R. (2013) · Medical Teacher · doi:10.3109/0142159x.2012.714886 · Cited by 709 · cited 1×
“Cook et al confirmed the effectiveness of several of Issenberg’s instructional design features [46,56].”
Newborn mortality and fresh stillbirth rates in Tanzania after helping babies breathe training
Msemo, G., Massawe, A., Mmbando, D., Rusibamayila, N., Manji, K., Kidanto, H. L., Mwizamuholya, D., Ringia, P., Ersdal, H. L., & Perlman, J. (2013) · Pediatrics · doi:10.1542/peds.2012-1795 · Cited by 450 · cited 2×
“Since this review, there have been numerous evaluation studies on the effectiveness of simulation training for obstetric emergencies in low-income and middle-income countries [7-40].”
Applying learning theories and instructional design models for effective instruction
Khalil, M. K., & Elkhider, I. A. (2016) · Advances in Physiology Education · doi:10.1152/advan.00138.2015 · Cited by 372 · cited 1×
“Another name for these prescriptions is affordances with the purpose of maximizing the effect, effectiveness, and usefulness of an educational instrument [44].”
Research works printed but not linked: 2
- Bang A Patel A Bellad R Gisore P Goudar SS Esamai F Liechty EA Meleth S Goco N Niermeyer S Keenan W Kamath-Rayne BD Little GA Clarke SB Flanagan VA Bucher S Jain M Mujawar N Jain V Rukunga J Mahantshetti N Dhaded S Bhandankar M McClure EM Carlo WA Wright LL Hibberd PL Helping Babies Breathe (HBB) training: What happens to knowledge and skills over time? BMC Pregnancy Childbirth 2016 11 22 16 1 364 10.1186/s12884-016-1141-3 27875999 10.1186/s12884-016-1141-3 27875999 PMC5120476 · DOI · Paper page
- Arabi Ali M E Ibrahim SA Ahmed SE MacGinnea F Hawkes G Dempsey E Ryan C Anthony Skills retention in Sudanese village midwives 1 year following Helping Babies Breathe training Arch Dis Child 2016 05 101 5 439 42 10.1136/archdischild-2015-309190 26826172 archdischild-2015-309190 26826172 · DOI
Cited by
Abuhassna, H., & Alnawajha, S. (2023) · European Journal of Investigation in Health Psychology and Education · doi:10.3390/ejihpe13040054 · Cited by 143
“A10 [26] Kirkpatrick’s theoretical model Simulation-based training Uganda Learners A technology-enhanced simulation-based obstetrics training in a low-income nation scored well, although intervals were substantial.”
BackgroundMethodInfluentialVuillaume, L. A., Laudren, G., Bosio, A., Thévenot, P., Pelaccia, T., & Chauvin, A. (2021) · JMIR Serious Games · doi:10.2196/27291 · Cited by 40
van Tetering, A. A. C., Ntuyo, P., Martens, R. P. J., Winter, N., Byamugisha, J. K., Oei, S. G., Fransen, A. F., & der Jagt, M. B. V. D. H. (2023) · Annals of Global Health · doi:10.5334/aogh.3891 · Cited by 34
Self-citationMusie, M. R., Tagutanazvo, O. B., Sepeng, N. V., Mulaudzi, F. M., & Hlongwane, T. (2025) · BMC Medical Education · doi:10.1186/s12909-025-06830-7 · Cited by 24
“Studies [35, 39, 43], suggest that the Kirkpatrick theoretical model may be used to assess the impact of a CPD training program which includes four levels reaction, learning, behaviour and results.”
InfluentialFerreira, J. M. G., Generoso, J. R., Marra, A. R., Deliberato, R. O., Dewan, M. L., Loeb, D., Real, F. J., Collins, K. L., Ely, K., Zackoff, M. W., et al. (2025) · Journal of General Internal Medicine · doi:10.1007/s11606-025-09794-y · Cited by 15
Metadata
- Authors
- van Tetering, A. A. C., Segers, M. H. M., Ntuyo, P., Namagambe, I., van der Hout-van der Jagt, M. B., Byamugisha, J. K., & Oei, S. G.
- Affiliations
- Máxima Medisch Centrum (NL)
- Mulago Hospital (UG)
- ORCID
- Venue
- JMIR Medical Education
- Publisher
- JMIR Publications Inc.
- Published
- 2021-02-05
- Volume
- 7
- Issue
- 1
- Pages
- e17277
- ISSN
- 2369-3762 (ISSN-L)
- PMC
- PMC8081249
- OpenAlex
- W3035292487
- Cited by
- 11
- References
- 66
- Citations by year
- 2026: 3 · 2025: 3 · 2023: 3 · 2022: 1 · 2021: 1
- FWCI
- 0.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Education (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TU/e Research Portal (publishedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- TU/e Research Portal (submittedVersion, other-oa)
- Topics
- Social Sciences › Education › Simulation-Based Education in Healthcare
- Health Sciences › Medicine › Surgery › Surgical Simulation and Training
- Social Sciences › Education › Innovations in Medical Education
- Keywords
- simulation-based training, instructional design, obstetrics, medical education, Kirkpatrick model, clinical skills, obstetric residents, feedback, low-income countries, obstetric care, training simulators, Uganda
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating the Instructional Design and Effect on Knowledge, Teamwork, and Skills of Technology-Enhanced Simulation-Based Training in Obstetrics in Uganda: Stepped-Wedge Cluster Randomized Trial. https://app.cochat.ai/papers/10.2196/17277
- The paper
- van Tetering, A. A. C., Segers, M. H. M., Ntuyo, P., Namagambe, I., van der Hout-van der Jagt, M. B., Byamugisha, J. K., & Oei, S. G. (2021). Evaluating the Instructional Design and Effect on Knowledge, Teamwork, and Skills of Technology-Enhanced Simulation-Based Training in Obstetrics in Uganda: Stepped-Wedge Cluster Randomized Trial. JMIR Medical Education. https://doi.org/10.2196/17277
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.