Milestones 2.0: A Step Forward

Abstract

A key component of the Next Accreditation System are the Educational Milestones for all accredited residency and fellowship programs. Use of the milestones in trainee assessment began in 2013, as a key element of the Accreditation Council for Graduate Medical Education's (ACGME's) Next Accreditation System.1 Milestones are used in resident and fellow outcomes-based assessment based on the 6 general competencies: medical knowledge (MK), patient care (PC), interpersonal and communication skills (ICS), practice-based learning and improvement (PBLI), professionalism (PROF), and systems-based practice (SBP). The 6 competencies were introduced by the ACGME and the American Board of Medical Specialties (ABMS) in 1999.While the competencies were crafted with the intent to provide a shared model of professional development and advance assessment of physicians in training, their implementation faced challenges, as programs, faculty, and trainees lacked an understanding of the meaning of the competencies in the context of their specialty. In response, content experts from each specialty created subcompetencies, using narrative milestones to provide more specific and developmental descriptions of the general competencies. This included representatives from the ACGME Residency Review Committees, ABMS certification boards, program director groups, specialty societies, and residents and fellows. These volunteers collectively gave more than 5000 hours of their time to develop the specialty-specific milestones. These milestones were intended “to create a logical trajectory of professional development in essential elements of competency” and provided a “measurable framework of specialty-specific outcomes.”1 When the milestones were first developed, each specialty had the flexibility to identify subcompetencies and write associated developmental milestones. Specialties reviewed program requirements, certification examination outlines and blueprints, curricula, national competency statements, the literature, and results from national consensus-building exercises to guide their process.2The resulting specialty-specific milestones demonstrate substantial variability both with respect to content and how the developmental progression is operationalized across milestone levels.3 In 2013, the ACGME recognized the implementation of the milestones would be an iterative process, and that a periodic review and revision of the milestones would be beneficial, as experience and research evidence accrued. The ACGME made a commitment to begin this task within 3 to 5 years of Phase 1 implementation, as a part of a continuous quality improvement process. Over the course of the first 3 years of implementation, ACGME Milestone staff attended more than 200 program director meetings, focus groups, and visits to accredited programs and sponsoring institutions to gather feedback about the initial experience with milestones. These meetings helped to outline the revision process for Milestones 2.0.Additional concerns were heard through various channels such as focus groups, interviews, and at the 2nd ACGME Milestones Summit in December 2016, where representatives from all ACGME core specialties were present. Stakeholders expressed dissatisfaction with the inconsistencies in the milestones and subcompetencies.4 In fact, these differences were noted to have “complicated efforts to share assessment tools across programs, and provide comprehensive faculty development across specialties.”4 These differences may hamper, rather than encourage, collaboration in assessment and faculty development activities across specialties.In 2016, the ACGME Department of Milestone Development and Evaluation formalized the plan for the review and revision process. This work began with a review of data collected from the field, the data submitted for the twice-yearly milestone reporting, and learning from quantitative and qualitative research. To date, 40 national and multi-institutional studies about the milestones have been completed, and a bibliography of milestone research is available on the ACGME website.5The quantitative research demonstrated validity evidence for key components of the milestones in several specialties.5 Qualitative research has focused on program directors' and trainees' experience with the milestones, Clinical Competency Committee processes, the value of milestone-based feedback, and milestone design. A thematic analysis of the milestones for the non-PC and non-MK domains, completed in preparation for the revision process, demonstrated wide variation among specialties. The analysis showed that across the 26 core specialties and the transitional year there were more than 230 different ways of describing PROF, 171 for PBLI, 176 for ICS, and 122 for SBP.4Lessons from early research on the milestones has informed the milestone revision process. However, it is important to recognize that much has remained the same. Changes can be grouped into 4 categorie