The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis
Open accessAbstract
INTRODUCTION: Mini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences. METHODS: We searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr's adaptation of Kirkpatrick's four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher's exact test for count data. RESULTS: We identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners' reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees' performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, "quality" and "participant responsiveness", to be associated with the educational impact. CONCLUSIONS: Despite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Tools like rubrics, checklists, or rating scales, such as the Mini-Clinical Evaluation Exercise (Mini-CEX), provide structured criteria for performance evaluation [53][54][55].”
From Formative assessment and feedback in medical education: A practical guide: AMEE Guide No. 189 (2026) · cites this paper as [53]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · cited 1×
“Direct observation of trainees’ performance provides a convenient opportunity to give feedback on observed behavior [10, 11].”
Standards for Reporting Qualitative Research: A Synthesis of Recommendations
O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000388 · Cited by 12,211
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
Flagged first, then by citations
Research works printed but not linked: 1
- G Schwarzer, 2007, Meta: An R package for meta-analysis, R news, 7, 40
Cited by
Sorted by citations
Items 101–117 of 117
de Moraes, A. P. P., Guará, J. P., Lucas, M. Z. B. S. S., Lopes, M. K. D., & da Silva, N. B. P. (2024) · Revista Brasileira de Educação Médica · doi:10.1590/1981-5271v48.3-2023-0228
Ebrahimpour-Sadagheyani, H., Saadati, S. M., & Tatari, F. (2025) · Iranian Journal of Nursing and Midwifery Research · doi:10.4103/ijnmr.ijnmr_312_23
Application and Effectiveness Analysis of DOPS in Pediatric Clinical Skills TeachingPreprint
Lu, N., Jiayu, H., Bi, L., Yipin, W., & Yaling¹, H. (2025) · Research Square · doi:10.21203/rs.3.rs-6299626/v1
zou, H., Xie, Q., & Mao, B. (2025) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205251372109
Equivalence in Student Perceptions: A Study of Rural and Remote Medical Placements
Hall, L., Heal, C. F., Woolley, T., Harte, J., & Hollins, A. (2025) · Australian Journal of Rural Health · doi:10.1111/ajr.70122
Ros, R. G., Ruescas-Nicolau, M.-A., Serrano, N. C., Flor-Rufino, C., Martin-Valenzuela, C. S., & Sánchez-Sánchez, M. L. (2024) · Figshare · doi:10.6084/m9.figshare.25414893
Ros, R. G., Ruescas-Nicolau, M.-A., Serrano, N. C., Flor-Rufino, C., Martin-Valenzuela, C. S., & Sánchez-Sánchez, M. L. (2024) · Figshare · doi:10.6084/m9.figshare.26693596
Ros, R. G., Ruescas-Nicolau, M.-A., Serrano, N. C., Flor-Rufino, C., Martin-Valenzuela, C. S., & Sánchez-Sánchez, M. L. (2024) · Figshare · doi:10.6084/m9.figshare.25415024
Ishizuka, K., Nishizaki, Y., Sekine, M., Shimizu, T., Shikino, K., Yamamoto, Y., Fukui, S., Nojima, M., Ohta, M., & Tokuda, Y. (2026) · BMC Medical Education · doi:10.1186/s12909-026-08941-1
Ou, W., Liu, Z., Cao, M., Wu, C., & Zhao, M. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09055-4
Arbea, L., del Barrio, L. G., Carretero, C., & Gea, A. (2026) · Educación Médica · doi:10.1016/j.edumed.2026.101200
Kurmanalina, M. A., Sumanova, A. M., Хайдарова, Н. Б., Detochkina, V. R., & Aitmukhanbetova, A. M. (2026) · European Journal Of Dental Education · doi:10.1111/eje.70220
Liu, Y., Ling, Z., Zh, Z., Peng, X., & Liu, M. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09784-6
Siddiqui, A., Sabhi, Q., Siddiqui, F., Aleem, A., Ahmad, A., & Haider, S. I. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09955-5
Huang, X., Chen, M., Wang, J., Jin, X., Cai, X., Yang, G., Yang, W., & Huang, X. (2026) · Current Problems in Surgery · doi:10.1016/j.cpsurg.2026.102170
Zeng, X., He, Z., 周晓彤, Fan, J., & Shu, H. (2026) · Research Square · doi:10.21203/rs.3.rs-11346633/v1
Metadata
- Authors
- Lörwald, A. C., Lahner, F.-M., Nouns, Z. M., Berendonk, C., Norcini, J., Greif, R., & Huwendiek, S.
- Affiliations
- University of Bern (CH)
- Foundation for Advancement of International Medical Education and Research (US)
- Corresponding author
- Lörwald, Andrea C.
- ORCID
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-06-04
- Volume
- 13
- Issue
- 6
- Pages
- e0198009
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5986126
- OpenAlex
- W2807114631
- Cited by
- 117
- References
- 58
- Citations by year
- 2026: 9 · 2025: 14 · 2024: 19 · 2023: 15 · 2022: 18 · 2021: 17 · 2020: 18 · 2019: 5 · 2018: 2
- FWCI
- 1.90
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- BORIS (University Library Bern) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Open Access CRIS of the University of Bern
- Topics
- Social Sciences › Education › Innovations in Medical Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Medicine › Rehabilitation › Musculoskeletal Disorders and Rehabilitation
- Keywords
- Mini-CEX, Direct Observation of Procedural Skills, formative assessment, clinical skills assessment, Kirkpatrick model, implementation characteristics, trainee performance, undergraduate medical education, graduate education, participant responsiveness, assessment quality
- MeSH terms
- Clinical Competence, Education, Medical, Educational Measurement, Humans, Outcome Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
