The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis
Open accessAbstract
INTRODUCTION: Mini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences. METHODS: We searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr's adaptation of Kirkpatrick's four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher's exact test for count data. RESULTS: We identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners' reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees' performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, "quality" and "participant responsiveness", to be associated with the educational impact. CONCLUSIONS: Despite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Tools like rubrics, checklists, or rating scales, such as the Mini-Clinical Evaluation Exercise (Mini-CEX), provide structured criteria for performance evaluation [53][54][55].”
From Formative assessment and feedback in medical education: A practical guide: AMEE Guide No. 189 (2026) · cites this paper as [53]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · cited 1×
“Direct observation of trainees’ performance provides a convenient opportunity to give feedback on observed behavior [10, 11].”
Standards for Reporting Qualitative Research: A Synthesis of Recommendations
O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000388 · Cited by 12,211
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
Cochrane Handbook for Systematic Reviews of Interventions
Higgins, J. P. T., Green, S., & Ben Van Den, A. (2020) · International Coaching Psychology Review · doi:10.53841/bpsicpr.2020.15.2.123 · Cited by 5,670
Durlak, J. A., & DuPre, E. P. (2008) · American Journal of Community Psychology · doi:10.1007/s10464-008-9165-0 · Cited by 5,304
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
Competency-based medical education: Theory to practice.
Frank, J. R., Snell, L. S., Cate, O. T., Holmboe, E. S., Carraccio, C., Swing, S. R., Harris, P., Glasgow, N. J., Campbell, C., Dath, D., et al. (2010) · Medical Teacher · doi:10.3109/0142159x.2010.501190 · Cited by 2,883
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · Trials · doi:10.1186/1745-6215-11-32 · Cited by 1,871
Workplace-based assessment as an educational tool: AMEE Guide No. 31
Norcini, J., & Burch, V. (2007) · Medical Teacher · doi:10.1080/01421590701775453 · Cited by 984
Self-citationAssociation Between Funding and Quality of Published Medical Education Research
Reed, D. A., Cook, D. A., Beckman, T. J., Levine, R. B., Kern, D. E., & Wright, S. M. (2007) · JAMA · doi:10.1001/jama.298.9.1002 · Cited by 961
Tools for Direct Observation and Assessment of Clinical Skills of Medical Trainees
Kogan, J. R., Holmboe, E. S., & Hauer, K. E. (2009) · JAMA · doi:10.1001/jama.2009.1365 · Cited by 613
The Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation
Norcini, J. J., Blank, L. L., Arnold, G. K., & Kimball, H. R. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-123-10-199511150-00008 · Cited by 526
Self-citationImpact of workplace based assessment on doctors' education and performance: a systematic review
Miller, A. P., & Archer, J. C. (2010) · BMJ · doi:10.1136/bmj.c5064 · Cited by 462
Cavanaugh, J. (2006) · Journal of the American Statistical Association · doi:10.1198/jasa.2006.s94 · Cited by 440
Workplace-based assessment as an educational tool
Norcini, J. J., Burch, V. C., & Gibbs, T. J. (2007) · Medical Entomology and Zoology · Cited by 266
Self-citationCrossley, J., & Jolly, B. C. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04166.x · Cited by 212
Conducting systematic reviews in medical education: a stepwise approach
Cook, D. A., & West, C. P. (2012) · Medical Education · doi:10.1111/j.1365-2923.2012.04328.x · Cited by 186
van de Ridder, J. M. M., McGaghie, W. C., Stokking, K. M., & ten Cate, O. T. J. (2015) · Medical Education · doi:10.1111/medu.12744 · Cited by 154
Trainee doctors’ views on workplace-based assessments: Are they just a tick box exercise?
Bindal, T., Wall, D., & Goodyear, H. M. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.558140 · Cited by 144
Consequences Validity Evidence: Evaluating the Impact of Educational Assessments
Cook, D. A., & Lineberry, M. C. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001114 · Cited by 143
Mini-clinical evaluation exercise in anaesthesia training
Weller, J. M., Jolly, B. C., Misur, M., Merry, A., Jones, A., Crossley, J., Pedersen, K., & Smith, K. (2009) · British Journal of Anaesthesia · doi:10.1093/bja/aep055 · Cited by 124
Saedon, H., Salleh, S., Balakrishnan, A., Imray, C., & Saedon, M. H. (2012) · BMC Medical Education · doi:10.1186/1472-6920-12-25 · Cited by 124
Assessing the performance of specialist registrars
Wragg, A., Wade, W. B., Fuller, G. N., Cowan, G. O., & Mills, P. G. (2003) · Clinical Medicine · doi:10.7861/clinmedicine.3-2-131 · Cited by 123
Hauer, K. E., Holmboe, E. S., & Kogan, J. R. (2011) · Medical Teacher · doi:10.3109/0142159x.2010.507710 · Cited by 118
de Lima, A. A., Barrero, C., Baratta, S., Costa, Y. C., Bortman, G., Carabajales, J., Conde, D. G., Galli, A., Degrange, G., & van der Vleuten, C. P. M. (2007) · Medical Teacher · doi:10.1080/01421590701352261 · Cited by 99
Nair, B., Alexander, H. G., McGrath, B. P., Parvathy, M. S., Kilsby, E. C., Wenzel, J., Frank, I. B., Pachev, G., & Page, G. G. (2008) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2008.tb01951.x · Cited by 90
Internal medicine residents’ perceptions of the Mini-Clinical Evaluation Exercise
Malhotra, S., Hatala, R. A., & Courneya, C. (2008) · Medical Teacher · doi:10.1080/01421590801946962 · Cited by 79
Weller, J. M., Jones, A., Merry, A., Jolly, B. C., & Saunders, D. (2009) · British Journal of Anaesthesia · doi:10.1093/bja/aep211 · Cited by 77
Weston, P. S. J., & Smith, C. A. (2014) · Medical Teacher · doi:10.3109/0142159x.2013.836267 · Cited by 67
Torre, D. M., SIMPSON, D. E., Elnicki, D. M., Sebastian, J. L., & Holmboe, E. S. (2007) · Teaching and Learning in Medicine · doi:10.1080/10401330701366622 · Cited by 63
DOPS assessment: A study to evaluate the experience and opinions of trainees and assessors
Bindal, N., Goodyear, H. M., Bindal, T., & Wall, D. (2013) · Medical Teacher · doi:10.3109/0142159x.2012.746447 · Cited by 58
Morris, A., Hewitt, J., & Roberts, C. M. (2006) · Postgraduate Medical Journal · doi:10.1136/pgmj.2005.040477 · Cited by 52
Castanelli, D. J., Jowsey, T., Chen, Y., & Weller, J. M. (2016) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-016-0740-9 · Cited by 50
Training in hospitals: what do GP specialist trainees think of workplace-based assessments?
Sabey, A., & Harris, M. F. (2011) · Education for Primary Care · doi:10.1080/14739879.2011.11493974 · Cited by 47
Routledge, R. D. (2005) · Wiley eBooks · doi:10.1002/0470011815.b2a10020 · Cited by 44
Habibi, H., Raiesifar, A., Khaghanizade, M., Hosein, M., Marjan, S. M., & Ebadi, A. (2015) · Global Journal of Health Science · doi:10.5539/gjhs.v7n7p17 · Cited by 37
Brazil, V. A., Ratcliffe, L., Zhang, J., & Davin, L. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.719653 · Cited by 37
Introduction of Mini-CEX in undergraduate dental education in India
Behere, R. (2014) · Education for Health · doi:10.4103/1357-6283.152187 · Cited by 36
Suhoyo, Y., Schönrock-Adema, J., Rahayu, G. R., Kuks, J. B. M., & Cohen‐Schotanus, J. (2014) · Medical Teacher · doi:10.3109/0142159x.2014.917160 · Cited by 26
Implementation of a Mini-CEX Requirement Across All Third-Year Clerkships
Kim, S., Willett, L. R., Noveck, H., Patel, M. S., Walker, J. A., & Terregino, C. A. (2016) · Teaching and Learning in Medicine · doi:10.1080/10401334.2016.1165682 · Cited by 26
An evaluation of the use of the mini-CEX in the foundation programme
Jackson, D., & Wall, D. (2010) · British Journal of Hospital Medicine · doi:10.12968/hmed.2010.71.10.78949 · Cited by 24
Yanting, S. L., Sinnathamby, A. S., Wang, D., Heng, M. T. M., Hao, J. L. W., Lee, S. S., Yeo, S. P., & Samarasekera, D. D. (2016) · Tzu Chi Medical Journal · doi:10.1016/j.tcmj.2016.06.001 · Cited by 18
Weijs, C. A., Coe, J. B., & Hecker, K. G. (2015) · Journal of Veterinary Medical Education · doi:10.3138/jvme.1214-123r1 · Cited by 16
Workplace-Based Assessments in Psychiatry: Evaluation of a Whole Assessment System
Brittlebank, A. D., Archer, J. C., Longson, D., Malik, A., & Bhugra, D. (2013) · Academic Psychiatry · doi:10.1176/appi.ap.11110198 · Cited by 16
Nazari, R., Fateme, H., Hamid, S., & Hojjati, H. (2013) · Europe PMC (PubMed Central) · Cited by 14
Vaughan, B., & Moore, K. M. (2015) · International journal of osteopathic medicine · doi:10.1016/j.ijosm.2015.07.002 · Cited by 13
Hoseini, B. L., Mazloum, S. R., Jafarnejad, F., & Foroughipour, M. (2013) · Europe PMC (PubMed Central) · Cited by 13
The midwifery miniCEX – A valuable clinical assessment tool for midwifery education
Sweet, L. P., Glover, P., & McPhee, T. (2012) · Nurse Education in Practice · doi:10.1016/j.nepr.2012.08.015 · Cited by 13
Making the horse drink: use of mini-CEX in an assessment for learning view
Schuwirth, L. (2012) · Advances in Health Sciences Education · doi:10.1007/s10459-012-9423-7 · Cited by 11
Mini-CEX (Clinical Evaluation Exercise) as an assessment tool at Shifa College of Medicine -
Saeed, N., Tariq, N., & Jaffery, T. (2015) · Rawal Medical Journal · Cited by 4
Complementary and alternative medicine in anesthesia: Are we really going there?
Grocott, H. P. (2015) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-015-0526-5 · Cited by 4
Flagged first, then by citations
Research works printed but not linked: 1
- G Schwarzer, 2007, Meta: An R package for meta-analysis, R news, 7, 40
Cited by
Sorted by citations
Anderson, H. L. K., Kurtz, J. B., & West, D. C. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004366 · Cited by 69
Hejri, S. M., Jalili, M., Masoumi, R., Shirazi, M., Nedjat, S., & Norcini, J. J. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1652732 · Cited by 69
Self-citationLörwald, A. C., Lahner, F.-M., Mooser, B., Perrig, M., Widmer, M. K., Greif, R. T., & Huwendiek, S. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1497784 · Cited by 59
Self-citationGuerrero, J. G., Hafiz, A. H., Eltohamy, N. A. E., Gomma, N. H., & Aljarrah, I. (2021) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2021.06.011 · Cited by 50
Examining the educational impact of the mini-CEX: a randomised controlled study
Martinsen, S. S. S., Espeland, T., Berg, E. A. R., Samstad, E., Lillebo, B., & Slørdahl, T. S. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02670-3 · Cited by 45
Metadata
- Authors
- Lörwald, A. C., Lahner, F.-M., Nouns, Z. M., Berendonk, C., Norcini, J., Greif, R., & Huwendiek, S.
- Affiliations
- University of Bern (CH)
- Foundation for Advancement of International Medical Education and Research (US)
- Corresponding author
- Lörwald, Andrea C.
- ORCID
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-06-04
- Volume
- 13
- Issue
- 6
- Pages
- e0198009
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5986126
- OpenAlex
- W2807114631
- Cited by
- 117
- References
- 58
- Citations by year
- 2026: 9 · 2025: 14 · 2024: 19 · 2023: 15 · 2022: 18 · 2021: 17 · 2020: 18 · 2019: 5 · 2018: 2
- FWCI
- 1.90
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- BORIS (University Library Bern) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Open Access CRIS of the University of Bern
- Topics
- Social Sciences › Education › Innovations in Medical Education
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Medicine › Rehabilitation › Musculoskeletal Disorders and Rehabilitation
- Keywords
- Mini-CEX, Direct Observation of Procedural Skills, formative assessment, clinical skills assessment, Kirkpatrick model, implementation characteristics, trainee performance, undergraduate medical education, graduate education, participant responsiveness, assessment quality
- MeSH terms
- Clinical Competence, Education, Medical, Educational Measurement, Humans, Outcome Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
