CoChat
Sign Up
Cited by 83Innovations in Medical EducationHealth and Medical EducationCompetency Development and Evaluation

Evaluating competency-based medical education: a systematized review of current practices

Open access
Alharbi, Nouf Sulaiman·2024·BMC Medical Education
DOI 10.1186/s12909-024-05609-6PMC PMC11149276
Open this paper in CoChat — free account

Abstract

Background: Few published articles provide a comprehensive overview of the available evidence on the topic of evaluating competency-based medical education (CBME) curricula. The purpose of this review is therefore to synthesize the available evidence on the evaluation practices for competency-based curricula employed in schools and programs for undergraduate and postgraduate health professionals. Method: This systematized review was conducted following the systematic reviews approach with minor modifications to synthesize the findings of published studies that examined the evaluation of CBME undergraduate and postgraduate programs for health professionals. Results: Thirty-eight articles met the inclusion criteria and reported evaluation practices in CBME curricula from various countries and regions worldwide, such as Canada, China, Turkey, and West Africa. 57% of the evaluated programs were at the postgraduate level, and 71% were in the field of medicine. The results revealed variation in reporting evaluation practices, with numerous studies failing to clarify evaluations’ objectives, approaches, tools, and standards as well as how evaluations were reported and communicated. It was noted that questionnaires were the primary tool employed for evaluating programs, often combined with interviews or focus groups. Furthermore, the utilized evaluation standards considered the well-known competencies framework, specialized association guidelines, and accreditation criteria. Conclusion: This review calls attention to the importance of ensuring that reports of evaluation experiences include certain essential elements of evaluation to better inform theory and practice.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12909-024-05609-6

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Systematic evidence indicates that competency-based assessment and interdisciplinary training remain insufficient in medical and health management education.

    “Systematic evidence indicates that competency‑based assessment and interdisciplinary training remain insufficient in medical and health management education [8, 9].”

    From Reforming hospital management education with the P-MASE pedagogy: a randomized controlled trial (2026) · cites this paper as [8]

    The Results (Findings of the included studies; page unavailable) report that 63% of included studies did not state the standards used to evaluate competency-based curricula, and the Discussion identifies gaps in reported evaluation practices. This supports a concern about competency-based curriculum evaluation, but the paper does not establish that interdisciplinary training is insufficient or specifically assess health management education. Its finding about evaluation reporting is also narrower than a claim that competency-based assessment itself is insufficient. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 71

  1. [1]Zaini R Bin Abdulrahman K Alkhotani A Al-Hayani AA Al-Alwani A Jastaniah S Saudi meds: a competence specification for Saudi medical graduates Med Teach 2011 33 7 582 4 10.3109/0142159X.2011.578180 21696288 · DOI
  2. [2]Davis MH, Amin Z, Grande JP, O’Neill AE, Pawlina W, Viggiano TR et al. Case studies in outcome-based education. Med Teach. 2009; 29(7):717–22. 10.1080/01421590701691429 18236260 · DOI
  3. [3]Caccia N, Nakajima A, Kent N. Competency-based medical education: The wave of the future. JOGC. 2015; 37(4):349–53. 10.1016/S1701-2163(15)30286-3 26001689 · DOI
  4. [4]Danilovich N Kitto S Price D Campbell C Hodgson A Hendry P Implementing competency-based medical education in family medicine: a narrative review of current trends in assessment Fam Med 2021 53 9 22 10.22454/FamMed.2021.453158 33471918 · DOI
  5. [5]Hawkins RE, Welcher CM, Holmboe ES, Kirk LM, Norcini JJ, Simons KB et al. Implementation of competency-based medical education: Are we addressing the concerns and challenges? Med Educ. 2015 Oct. 22; 49(11):1086–102. 10.1111/medu.12831 26494062 · DOI · Paper page
  6. [6]Simpson JG, Furnace J, Crosby J, Cumming AD, Evans PA, David MF, Ben et al. The Scottish doctor-learning outcomes for the medical undergraduate in Scotland: A foundation for competent and reflective practitioners. Med Teach. 2002; 24(2):136–43. 10.1080/01421590220120713 12098432 · DOI
  7. [7]Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach. 2007; 29(7):648–54. 10.1080/01421590701392903 18236251 · DOI
  8. [8]Frank JR, Danoff D. The CanMEDS initiative: Implementing an outcomes-based framework of physician competencies. Med Teach. 2007; 29(7):642–7. 10.1080/01421590701746983 18236250 · DOI
  9. [9]van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019 Jul 1 [cited 2022 Mar 19]; 94(7):1002–9. https://pubmed.ncbi.nlm.nih.gov/30973365/ 30973365
  10. [10]Hamza DM, Ross S, Oandasan I. Process and outcome evaluation of a CBME intervention guided by program theory. J Eval Clin Pract. 2020; 26(4):1096–104. 10.1111/jep.13344 PMC7496603 31927788 · DOI
  11. [11]Van Melle E Frank J Holmboe E Dagnone D Stockley D Sherbino J A core components framework for evaluating implementation of competency-based medical education programs Acad Med 2019 94 1 30973365
  12. [12]Giancola SP. Program evaluation: Embedding evaluation into program design and development. SAGE Publications; 2020. https://books.google.com.sa/books?id=H-LFDwAAQBAJ
  13. [13]Morrison J, Evaluation. BMJ. 2003; 326(7385):385. http://www.bmj.com/content/326/7385/385.abstract PMC1125244 12586676
  14. [15]Stufflebeam D. Evaluation models. New Dir Eval. 2001; 2001(89):7–98.
  15. [17]Nouraey P Al-Badi A Riasati M Maata RL Educational program and curriculum evaluation models: a mini systematic review of the recent trends UJER 2020 8 4048 55 10.13189/ujer.2020.080930 · DOI
  16. [18]Sarah Schiekirka, Markus A, Feufel C, Herrmann-Lingen T, Raupach. Evaluation in medical education: a topical review of target parameters, data collection tools and confounding factors. Ger Med Sci. 2015; 13 (16). PMC4576315 26421003
  17. [19]Van Melle E Hall A Schumacher D Kinnear B Gruppen L Thoma B Capturing outcomes of competency-based medical education: the call and the challenge Med Teach 2021 43 1 7 34121596
  18. [20]Lomis KD, Mejicano GC, Caverzagie KJ, Monrad SU, Pusic M, Hauer KE. The critical role of infrastructure and organizational culture in implementing competency-based education and individualized pathways in undergraduate medical education. Med Teach. 2021; 43(sup2):S7–16. 10.1080/0142159X.2021.1924364 34291715 · DOI
  19. [21]Taber S, Frank JR, Harris KA, Glasgow NJ, Iobst W, Talbot M. Identifying the policy implications of competency-based education. Med Teach. 2010; 32(8):687–91. 10.3109/0142159X.2010.500706 20662582 · DOI · Paper page
  20. [22]Zaini RG, bin Abdulrahman KA, Al-Khotani AA, Al-Hayani AMA, Al-Alwan IA, Jastaniah SD. Saudi Meds: A competence specification for Saudi medical graduates. Med Teach. 2011 Jul [cited 2022 Mar 19]; 33(7):582–4. https://pubmed.ncbi.nlm.nih.gov/21696288/ 21696288
  21. [23]Leung WC. Competency based medical training: review * Commentary: The baby is thrown out with the bathwater. BMJ. 2002 Sept 28; 325(7366):693–6. 10.1136/BMJ.325.7366.693 PMC126659 12351364 · DOI
  22. [24]Hall AK, Schumacher DJ, Thoma B, Caretta-Weyer H, Kinnear B, Gruppen L et al. Outcomes of competency-based medical education: A taxonomy for shared language. Med Teach. 2021;43(7):788–93. 10.1080/0142159X.2021.1925643 34038673 · DOI
  23. [27]Sharma R, Gordon M, Dharamsi S, Gibbs T. Systematic reviews in medical education: A practical approach: AMEE Guide 94. Med Teach. 2015; 37(2):108–24. 10.3109/0142159X.2014.970996 25314376 · DOI
  24. [29]Acai A, Cupido N, Weavers A, Saperson K, Ladhani M, Cameron S et al. Competence committees: The steep climb from concept to implementation. Med Educ. 2021; 55(9):1067–77. 10.1111/medu.14585 34152027 · DOI
  25. [30]Zhang P Hamza D Ross S Oandasan I Exploring change after implementation of family medicine residency curriculum reform Fam Med 2019 51 331 7 10.22454/FamMed.2019.427722 30973621 · DOI
  26. [31]Thoma B, Hall A, Clark K, Meshkat N, Cheung W, Desaulniers P et al. Evaluation of a national competency-based assessment system in emergency medicine: a CanDREAM Study. J Grad Med Educ. 2020;12(4). PMC7450748 32879682
  27. [32]Schönwetter D Law D Mazurat R Sileikyte R Nazarko O Assessing graduating dental students’ competencies: the impact of classroom, clinic and externships learning experiences Eur J Dent Educ 2011 15 142 52 10.1111/j.1600-0579.2010.00648.x 21762318 · DOI
  28. [33]Nousiainen M Mironova P Hynes M Takahashi S Reznick R Kraemer W Eight-year outcomes of a competency-based residency training program in orthopedic surgery Med Teach 2018 40 1 13 10.1080/0142159X.2017.1421751 29343150 · DOI
  29. [34]Railer J, Stockley D, Flynn L, Hastings Truelove A, Hussain A. Using outcome harvesting: assessing the efficacy of CBME implementation. J Eval Clin Pract. 2020; 26(4). 32083768
  30. [35]Janssen P Keen L Soolsma J Seymour L Harris S Klein M Perinatal nursing education for single-room maternity care: an evaluation of a competency-based model J Clin Nurs 2005 14 95 101 10.1111/j.1365-2702.2004.01014.x 15656853 · DOI
  31. [36]Goudreau J Pepin J Dubois S Boyer L Larue C Legault A A second generation of the competency-based approach to nursing education Int J Nurs Educ Scholarsh 2009 6 Article15 10.2202/1548-923X.1685 19409071 · DOI
  32. [37]Fahim C Bhandari M Yang I Sonnadara R Development and early piloting of a CanMEDS competency-based feedback tool for surgical grand rounds J Surg Educ 2016 73 3 409 15 10.1016/j.jsurg.2015.12.003 26896145 · DOI
  33. [38]Ellaway R Mackay M Lee S Hofmeister M Malin G Archibald D The impact of a national competency-based medical education initiative in family medicine Acad Med 2018 93 1 10.1097/ACM.0000000000002387 30067538 · DOI
  34. [39]D’Souza L, Jaswal J, Chan F, Johnson M, Tay KY, Fung K et al. Evaluating the impact of an integrated multidisciplinary head & neck competency-based anatomy & radiology teaching approach in radiation oncology: a prospective cohort study. BMC Med Educ. 2014; 14(1):124. 10.1186/1472-6920-14-124 PMC4102036 24969509 · DOI
  35. [40]Crawford L Cofie N Mcewen L Dagnone D Taylor S Perceptions and barriers to competency-based education in Canadian postgraduate medical education J Eval Clin Pract 2020 26 4 1124 31 10.1111/jep.13371 32106354 · DOI
  36. [41]Cox K Smith A Lichtveld M A competency-based approach to expanding the cancer care workforce part III—Improving cancer pain and palliative care competency J Cancer Educ 2012 27 507 14 10.1007/s13187-012-0354-z 22528634 PMC3438406 · DOI
  37. [42]Freedman AM, Simmons S, Lloyd LM, Redd TR, Alperin MM, Salek SS et al. Public health training center evaluation: A framework for using logic models to improve practice and educate the public health workforce. Health Promot Pract. 2014; 15(1 Suppl):80S-8S. http://europepmc.org/abstract/MED/24578370 24578370
  38. [43]Kerfoot B Baker H Volkan K Church P Federman D Masser B Development and initial evaluation of a novel urology curriculum for medical students J Urol 2004 172 278 81 10.1097/01.ju.0000132157.84026.ea 15201794 · DOI
  39. [44]Ketteler E Auyang E Beard K McBride E McKee R Russell J Competency champions in the clinical competency committee: a successful strategy to implement milestone evaluations and competency coaching J Surg Educ 2014 71 36 8 10.1016/j.jsurg.2013.09.012 24411421 · DOI
  40. [45]Lipp M An objectified competency-based course in the management of malocclusion and skeletal problems J Dent Educ 2008 72 543 52 10.1002/j.0022-0337.2008.72.5.tb04518.x 18451077 · DOI
  41. [46]Randolph S Rogers B Ostendorf J Evaluation of an occupational health nursing program through competency achievement on-campus and distance education, 2005 and 2008 AAOHN J 2011 59 387 99 21877671
  42. [47]Stefanidis D Acker C Swiderski D Heniford BT Greene F Challenges during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program J Surg Educ 2008 65 4 7 10.1016/j.jsurg.2007.11.009 18308276 · DOI
  43. [48]Stucke R Sorensen M Rosser A Jung S The surgical consult entrustable professional activity (EPA): defining competence as a basis for evaluation Am J Surg 2018 219 2 253 7 10.1016/j.amjsurg.2018.12.056 30616922 · DOI
  44. [49]Swider S Levin P Ailey S Breakwell S Cowell J Mcnaughton D Matching a graduate curriculum in public/community health nursing to practice competencies: the Rush University experience Public Health Nurs 2006 23 190 5 10.1111/j.1525-1446.2006.230211.x 16684194 · DOI
  45. [50]Taleghani M Solomon E Wathen W Non-graded clinical evaluation of dental students in a competency‐based education program J Dent Educ 2004 68 644 55 10.1002/j.0022-0337.2004.68.6.tb03782.x 15217083 · DOI
  46. [51]Zuilen M Mintzer M Milanez M Kaiser R Rodriguez O Paniagua M A competency-based medical student curriculum targeting key geriatric syndromes Gerontol Geriatr Educ 2008 28 29 45 10.1300/J021v28n03_03 18215986 · DOI
  47. [52]Gibson K Boyle P Black D Bennett M Grimm M McNeil H Enhancing evaluation in an undergraduate medical education program Acad Med 2008 83 787 93 10.1097/ACM.0b013e31817eb8ab 18667897 · DOI
  48. [53]Succar T McCluskey P Grigg J Enhancing medical student education by implementing a competency based ophthalmology curriculum Asia-Pac J Ophthalmol 2017 6 42 6 28161917
  49. [54]Gruber P Gomersall C Joynt G Shields F Chu CM Derrick J Teaching acute care: a course for undergraduates Resuscitation 2007 74 142 9 10.1016/j.resuscitation.2006.11.021 17353081 · DOI
  50. [55]Zhang H, Wang B, Zhang L. Reform of the method for evaluating the teaching of medical linguistics to medical students. Chinese Education & Society. 2014; 47(3):60–4. 10.2753/CED1061-1932470305 · DOI
  51. [56]Koeijers J Busari J Duits AJ A case study of the implementation of a competency-based curriculum in a Caribbean teaching hospital West Indian Med J 2012 61 726 32 23620971
  52. [57]Rotthoff T, Ostapczuk MS, de Bruin J, Kröncke KD, Decking U, Schneider M et al. Development and evaluation of a questionnaire to measure the perceived implementation of the mission statement of a competency based curriculum. BMC Med Educ. 2012; 12(1):109. 10.1186/1472-6920-12-109 PMC3515351 23134815 · DOI
  53. [58]Day S Garcia J Antillon F Wilimas J Mckeon L Carty R A sustainable model for pediatric oncology nursing education in low-income countries Pediatr Blood Cancer 2012 58 163 6 10.1002/pbc.24007 22102623 PMC4676073 · DOI
  54. [59]Lee BH, Chae YM, Hokama T, Kim S. Competency-based learning program in system analysis and design for health professionals. APJPH. 2010; 22(3):299–309. http://www.jstor.org/stable/26723774 21212047
  55. [60]Westein M Vries H Floor-Schreudering A Koster A Buurma H Development of a postgraduate workplace-based curriculum for specialization of community pharmacists using CanMEDS competencies, entrustable professional activities and programmatic assessment Am J Pharm Educ 2018 83 ajpe6863 10.5688/ajpe6863 PMC6718509 31507284 · DOI
  56. [61]de Beer W. Is the RANZCP CPD programme a competency-based educational programme? Australas Psychiatry. 2019; 27(4):404–8. 10.1177/1039856219859279 31264890 · DOI
  57. [62]Battat R Jhonson M Wiseblatt L Renard C Habib L Normil M The Haiti Medical Education Project: Development and analysis of a competency based continuing medical education course in Haiti through distance learning BMC Med Educ 2016 16 275 10.1186/s12909-016-0795-x 27760535 PMC5069788 · DOI
  58. [63]İlhan E Evaluation of competency based medical education curriculum IJPE 2021 17 3 153 68 10.29329/ijpe.2021.346.10 · DOI
  59. [64]Ekenze SO, Ameh EA. Evaluation of relevance of the components of Pediatric Surgery residency training in West Africa. J Pediatr Surg. 2010; 45(4):801–5. https://www.sciencedirect.com/science/article/pii/S0022346809008495 20385291
  60. [65]Iobst WF, Sherbino J, Cate O, Ten, Richardson DL, Dath D, Swing SR et al. Competency-based medical education in postgraduate medical education. Med Teach. 2010; 32(8):651–6. 10.3109/0142159X.2010.500709 20662576 · DOI · Paper page
  61. [66]Vakani F, Jafri W, Jafri F, Ahmad A. Towards a competency-based postgraduate medical education. J Coll Physicians Surg Pak; 22(7): 476–7. 22747875
  62. [67]Pascual TNB, Ros S, Engel-Hills P, Chhem RK. Medical competency in postgraduate medical training programs. Radiology Education. 2012; pp. 29–45. 10.1007/978-3-642-27600-2_4 · DOI
  63. [69]Lindeman B, Kern D, Lipsett P. Step 6 evaluation and feedback. In: Thomas P, Kern D, Hughes M, Tackett S, Chen B, editors. Curriculum development for medical education: a six-step approach. 4th ed. Johns Hopkins University; 2022. pp. 142–97.
  64. [70]Morrison J. ABC of learning and teaching in medicine: Evaluation. BMJ. 2003; 326(7385):385–7. 10.1136/BMJ.326.7385.385 PMC1125244 12586676 · DOI
  65. [71]Wang MC, Ellett CD. Program validation. Topics Early Child Spec Educ. 1982; 1(4):35–49. 10.1177/027112148200100408 · DOI
  66. [73]Toosi M, Modarres M, Amini M, Geranmayeh M. Evaluation model in medical education: A systematic review. 2016. PMC8249974 34250133
  67. [74]Balmer DF, Rama JA, Simpson D. Program evaluation models: Evaluating processes and outcomes in graduate medical education. J Grad Med Educ. 2019; 11(1):99–100. 10.4300/JGME-D-18-01084.1 PMC6375316 30805105 · DOI
  68. [75]Vassar M, Wheeler DL, Davison M, Franklin J. Program evaluation in medical education: An overview of the utilization-focused approach. J Educ Eval Health Prof. 2010; 7:1. 10.3352/jeehp.2010.7.1 PMC2886237 20559515 · DOI
  69. [77]Moreau KA, Eady K. Program evaluation use in graduate medical education. J Grad Med Educ. 2023; 15(1):15–8. 10.4300/JGME-D-22-00397.1 PMC9934821 36817537 · DOI
  70. [78]Durning SJ, Hanson J, Gilliland W, McManigle JM, Waechter D, Pangaro LN. Using qualitative data from a program director’s evaluation form as an outcome measurement for medical school. Mil Med. 2010; 175(6):448–52. 10.7205/MILMED-D-09-00044 20572479 · DOI
  71. [80]Wolfhagen HAP, Gijselaers WH, Dolmans D, Essed G, Schmidt HG. Improving clinical education through evaluation. Med Teach. 2009; 19(2):99–103. 10.3109/01421599709019360 · DOI

Cited by

Sorted by citations

Authors
Alharbi, N. S.
Affiliations
King Saud bin Abdulaziz University for Health Sciences (SA)
Corresponding author
Alharbi, Nouf Sulaiman
ORCID
Alharbi, Nouf Sulaiman
Venue
BMC Medical Education
Publisher
Springer Science and Business Media LLC
Published
2024-06-03
Volume
24
Issue
1
Pages
612-612
ISSN
1472-6920 (ISSN-L)
DOI
10.1186/s12909-024-05609-6
PMC
PMC11149276
OpenAlex
W4399288664
Cited by
83
References
2
Citations by year
2026: 53 · 2025: 28 · 2024: 2
FWCI
7.34
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Education (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
  • Health Sciences › Health Professions › General Health Professions › Health and Medical Education
  • Social Sciences › Psychology › Social Psychology › Competency Development and Evaluation
Keywords
competency-based medical education, curriculum evaluation, assessment practices, graduate medical education, undergraduate medical education, evaluation standards, evaluation reporting, competency framework, accreditation standards, health professions education
MeSH terms
Clinical Competence, Competency-Based Education, Curriculum, Education, Medical, Education, Medical, Undergraduate, Humans, Program Evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review