Cited by 333Innovations in Medical EducationStudent Assessment and FeedbackEducational Assessment and Pedagogy

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Twelve Tips for programmatic assessment

Van Der Vleuten, C.P.M., Schuwirth, L.W.T., Driessen, E.W., Govaerts, M.J.B., Heeneman, S.·2015·Medical Teacher

DOI 10.3109/0142159x.2014.973388

Open this paper in CoChat — free account

Abstract

Programmatic assessment is an integral approach to the design of an assessment program with the intent to optimise its learning function, its decision-making function and its curriculum quality-assurance function. Individual methods of assessment, purposefully chosen for their alignment with the curriculum outcomes and their information value for the learner, the teacher and the organisation, are seen as individual data points. The information value of these individual data points is maximised by giving feedback to the learner. There is a decoupling of assessment moment and decision moment. Intermediate and high-stakes decisions are based on multiple data points after a meaningful aggregation of information and supported by rigorous organisational procedures to ensure their dependability. Self-regulation of learning, through analysis of the assessment information and the attainment of the ensuing learning goals, is scaffolded by a mentoring system. Programmatic assessment-for-learning can be applied to any part of the training continuum, provided that the underlying learning conception is constructivist. This paper provides concrete recommendations for implementation of programmatic assessment.

Integrity signals

Registry

Crossref · 10.3109/0142159x.2014.973388

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The Power of Feedback
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905
Focus on Formative Feedback
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631 · cited 1×
“Mostly, conventional feedback from assessments, that is, grades and pass/fail decisions, are poor information carriers (Shute 2008).”
Background
Deliberate Practice and the Acquisition and Maintenance of Expert Performance in Medicine and Related Domains
Ericsson, K. A. (2004) · Academic Medicine · doi:10.1097/00001888-200410001-00022 · Cited by 2,988
Assessing professional competence: from methods to programmes
Van Der Vleuten, C. P. M., & Schuwirth, L. W. T. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02094.x · Cited by 1,268
Self-citation
The assessment of professional competence: Developments, research and practical implications
Van Der Vleuten, C. P. M. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00596229 · Cited by 1,247
Problem‐based learning: future challenges for educational practice and research
Dolmans, D. H. J. M., de Grave, W. S., Wolfhagen, I. H. A. P., & van der Vleuten, C. P. M. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02205.x · Cited by 1,103
Programmatic assessment: From assessment of learning to assessment for learning
Schuwirth, L. W. T., & Van der Vleuten, C. P. M. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.565828 · Cited by 902
Self-citation
A model for programmatic assessment fit for purpose.
van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652239 · Cited by 892
Self-citation
Program evaluation models and related theories: AMEE Guide No. 67
FRYE, A. W., & Hemmer, P. A. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.668637 · Cited by 641
The assessment of professional competence: building blocks for theory development
van der Vleuten, C. P. M., Schuwirth, L., Scheele, F., Driessen, E. W., & Hodges, B. D. (2010) · Best Practice & Research Clinical Obstetrics & Gynaecology · doi:10.1016/j.bpobgyn.2010.04.001 · Cited by 356
Self-citation

Show all 33 references

Sorted by citations

Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review
Chan, K. S., & Zary, N. (2019) · JMIR Medical Education · doi:10.2196/13930 · Cited by 877
“Programmatic assessment involves the use of an AI system to design an assessment program tailored to optimize learning outcomes and ensure curriculum quality at a systemic level [59].”
2018 Consensus framework for good assessment
Norcini, J. J., Anderson, M. B., Bóllela, V. R., Burch, V. C., Costa, M. J., Duvivier, R. J., Hays, R., Mackay, M. F. P., Roberts, T. E., & Swanson, D. B. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1500016 · Cited by 330
Implementing an Entrustable Professional Activities Framework in Undergraduate Medical Education: Early Lessons From the AAMC Core Entrustable Professional Activities for Entering Residency Pilot
Lomis, K. D., Amiel, J. M., Ryan, M. S., Esposito, K. F., Green, M., Stagnaro‐Green, A., Bull, J., & Mejicano, G. C. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001543 · Cited by 242
A scoping review on the notions of Assessment as Learning (AaL), Assessment for Learning (AfL), and Assessment of Learning (AoL)
Schellekens, L. H., Bok, H. G. J., de Jong, L. H., van der Schaaf, M. F., Kremer, W. D. J., & van der Vleuten, C. P. M. (2021) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2021.101094 · Cited by 229
The R2C2 Model in Residency Education: How Does It Foster Coaching and Promote Feedback Use?
Sargeant, J., Lockyer, J. M., Mann, K., Armson, H., Warren, A., Zetkulic, M., Soklaridis, S., Könings, K. D., Ross, K., Silver, I., et al. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002131 · Cited by 148
A history of assessment in medical education
Schuwirth, L., & van der Vleuten, C. P. M. (2020) · Advances in Health Sciences Education · doi:10.1007/s10459-020-10003-0 · Cited by 155
Self-citation
Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles
Heeneman, S., de Jong, L. H., Dawson, L. J., WILKINSON, T. J., Ryan, A., Tait, G. R., Rice, N. E., Torre, D. M., Freeman, A., & van der Vleuten, C. P. M. (2021) · Medical Teacher · doi:10.1080/0142159x.2021.1957088 · Cited by 150
Self-citation
Learning Analytics in Medical Education Assessment: The Past, the Present, and the Future
Chan, T. M., Sebok‐Syer, S. S., Thoma, B., Wise, A. F., Sherbino, J., & Pusic, M. V. (2018) · AEM Education and Training · doi:10.1002/aet2.10087 · Cited by 126
Feedback and coaching
Atkinson, A. R., Watling, C., & Brand, P. L. P. (2021) · European Journal of Pediatrics · doi:10.1007/s00431-021-04118-8 · Cited by 133
Cognitive load theory: Practical implications and an important challenge
Leppink, J. (2017) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2017.05.003 · Cited by 135
Healthcare Simulation Standards of Best PracticeTM Evaluation of Learning and Performance
McMahon, E., Jimenez, F. A., Lawrence, K., & Victor, J. (2021) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2021.08.016 · Cited by 106
Entrusting Observable Practice Activities and Milestones Over the 36 Months of an Internal Medicine Residency
Warm, E. J., Held, J. D., Hellmann, M., Kelleher, M., Kinnear, B., Lee, C., O’Toole, J. K., Mathis, B. R., Mueller, C. V., Sall, D. R., et al. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001292 · Cited by 89
Crossing the Gap: Using Competency-Based Assessment to Determine Whether Learners Are Ready for the Undergraduate-to-Graduate Transition
Murray, K. E., Lane, J. L., Carraccio, C. L., Glasgow, T. S., Long, M., West, D. C., O’Connor, M., Hobday, P. M., Schwartz, A., & Englander, R. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002535 · Cited by 83
Ten key features of the future medical school—not an impossible dream
Harden, R. M. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1498613 · Cited by 78
What we measure … and what we should measure in medical education
Boulet, J. R., & Durning, S. J. (2018) · Medical Education · doi:10.1111/medu.13652 · Cited by 80
Twelve tips to maximize the value of a clinical competency committee in postgraduate medical education
Kinnear, B., Warm, E. J., & Hauer, K. E. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1474191 · Cited by 73
Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review
Anderson, H. L. K., Kurtz, J. B., & West, D. C. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004366 · Cited by 69
Competency‐based education calls for programmatic assessment: But what does this look like in practice?
Rich, J. V., Young, S. F., Donnelly, C. A., Hall, A. K., Dagnone, D., Weersink, K., Caudle, J. M., Melle, E. V., & Klinger, D. A. (2019) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13328 · Cited by 67
Numbers Encapsulate, Words Elaborate: Toward the Best Use of Comments for Assessment and Feedback on Entrustment Ratings
Ginsburg, S., Watling, C., Schumacher, D. J., Gingerich, A., & Hatala, R. A. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004089 · Cited by 64
The ABCs of entrustable professional activities: an overview of ‘entrustable professional activities’ in medical education
El‐Haddad, C., Damodaran, A. K., McNeil, H. P., & Hu, W. C. Y. (2015) · Internal Medicine Journal · doi:10.1111/imj.12914 · Cited by 63
Multiple‐role mentoring: mentors’ conceptualisations, enactments and role conflicts
Meeuwissen, S. N. E., Stalmeijer, R. E., & Govaerts, M. (2019) · Medical Education · doi:10.1111/medu.13811 · Cited by 62
Self-citation
Artificial intelligence in medical education: Are we ready for it?
Imran, N., & Jawaid, M. (2020) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.36.5.3042 · Cited by 56
USMLE Step 2 CK: Best Predictor of Multimodal Performance in an Internal Medicine Residency
Sharma, A., Schauer, D. P., Kelleher, M., Kinnear, B., Sall, D. R., & Warm, E. J. (2019) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-19-00099.1 · Cited by 60
Online Project-Based Learning and Formative Assessment
Chanpet, P., Chomsuwan, K., & Murphy, E. (2018) · Technology Knowledge and Learning · doi:10.1007/s10758-018-9363-2 · Cited by 80
Assessment of communication skills
van der Vleuten, C. P. M., van den Eertwegh, V., & Giroldi, E. (2019) · Patient Education and Counseling · doi:10.1016/j.pec.2019.07.007 · Cited by 67
Programmatic assessment: the process, rationale and evidence for modern evaluation approaches in medical education
van der Vleuten, C. P. M., Lindemann, I., & Schmidt, L. (2018) · The Medical Journal of Australia · doi:10.5694/mja17.00926 · Cited by 56
Ottawa 2020 consensus statements for programmatic assessment – 2. Implementation and practice
Torre, D. M., Rice, N. E., Ryan, A., Bok, H. G. J., Dawson, L. J., Bierer, S. B., WILKINSON, T. J., Tait, G. R., Laughlin, T., Veerapen, K., et al. (2021) · Medical Teacher · doi:10.1080/0142159x.2021.1956681 · Cited by 59
Self-citation
Twelve tips for embedding assessment for and as learning practices in a programmatic assessment system
Sein, A. S., Rashid, H., Meka, J., Amiel, J. M., & Pluta, W. J. (2020) · Medical Teacher · doi:10.1080/0142159x.2020.1789081 · Cited by 57
How ‘Testing’ Has Become ‘Programmatic Assessment for Learning’
Schuwirth, L., & van der Vleuten, C. P. M. (2018) · Health Professions Education · doi:10.1016/j.hpe.2018.06.005 · Cited by 53
Self-citation
Stakeholder perspectives on workplace-based performance assessment: towards a better understanding of assessor behaviour
de Jonge, L. P. J. W. M., Timmerman, A. A., Govaerts, M., Muris, J. W. M., Muijtjens, A. M. M., Kramer, A. W. M., & van der Vleuten, C. P. M. (2017) · Advances in Health Sciences Education · doi:10.1007/s10459-017-9760-7 · Cited by 51
Self-citation
Fairness in human judgement in assessment: a hermeneutic literature review and conceptual framework
Valentine, N. A., Durning, S. J., Shanahan, E. M., & Schuwirth, L. (2020) · Advances in Health Sciences Education · doi:10.1007/s10459-020-10002-1 · Cited by 51
Self-citation
McMaster Modular Assessment Program (McMAP) Through the Years: Residents' Experience With an Evolving Feedback Culture Over a 3‐year Period
Li, S., Sherbino, J., & Chan, T. M. (2016) · AEM Education and Training · doi:10.1002/aet2.10009 · Cited by 42
Assessment Pearls for Competency-Based Medical Education
Humphrey-Murto, S. M., Wood, T. J., Ross, S., Tavares, W., Kvern, B. L., Sidhu, R. S., Sargeant, J. M., & Touchie, C. (2017) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-17-00365.1 · Cited by 47
Very short answer questions: a viable alternative to multiple choice questions
Puthiaparampil, T., & Rahman, M. M. (2020) · BMC Medical Education · doi:10.1186/s12909-020-02057-w · Cited by 48
Key considerations in planning and designing programmatic assessment in competency-based medical education
Ross, S., Hauer, K. E., Wycliffe-Jones, K., Hall, A. K., Molgaard, L. K., Richardson, D. L., Oswald, A., Bhanji, F., & Collaborators, O. B. O. T. I. (2021) · Medical Teacher · doi:10.1080/0142159x.2021.1925099 · Cited by 49
Development of a Postgraduate Community Pharmacist Specialization Program Using CanMEDS Competencies, and Entrustable Professional Activities
Westein, M. P. D., de Vries, H., Floor‐Schreudering, A., Koster, A. S., & Buurma, H. (2018) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe6863 · Cited by 46
Next Steps in the Implementation of Learning Analytics in Medical Education: Consensus From an International Cohort of Medical Educators
Thoma, B., Warm, E. J., Hamstra, S. J., Cavalcanti, R. B., Pusic, M. V., Shaw, T. J., Verma, A., Frank, J. R., & Hauer, K. E. (2020) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-19-00493.1 · Cited by 44
An international study on the implementation of programmatic assessment: Understanding challenges and exploring solutions
Torre, D. M., Schuwirth, L., van der Vleuten, C. P. M., & Heeneman, S. (2022) · Medical Teacher · doi:10.1080/0142159x.2022.2083487 · Cited by 44
Self-citation
Video-based feedback using real consultations for a formative assessment in communication skills
Döhms, M., Collares, C. F., & Tibério, I. D. F. L. C. (2020) · BMC Medical Education · doi:10.1186/s12909-020-1955-6 · Cited by 49
Using Natural Language Processing to Evaluate the Quality of Supervisor Narrative Comments in Competency-Based Medical Education
Spadafore, M., Yılmaz, Y., Rally, V., Chan, T. M., Russell, M., Thoma, B., Singh, S., Monteiro, S., Pardhan, A., Martin, L. J., et al. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005634 · Cited by 41
On the validity of summative entrustment decisions
Touchie, C., Kinnear, B., Schumacher, D. J., Caretta‐Weyer, H. A., Hamstra, S. J., Hart, D. E., Gruppen, L. D., Ross, S., Warm, E. J., ten Cate, O., et al. (2021) · Medical Teacher · doi:10.1080/0142159x.2021.1925642 · Cited by 41
The learning environment in remediation: a review
Cleland, J., Cilliers, F. J., & Schalkwyk, S. C. V. (2017) · The Clinical Teacher · doi:10.1111/tct.12739 · Cited by 40
Generalizability of Competency Assessment Scores Across and Within Clerkships: How Students, Assessors, and Clerkships Matter
Zaidi, N. L. B., Kreiter, C. D., Castaneda, P., Schiller, J. H., Yang, J., Grum, C. M., Hammoud, M. M., Gruppen, L. D., & Santen, S. A. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002262 · Cited by 40
The power of subjectivity in competency-based assessment
Virk, A., Joshi, A., Mahajan, R., & Singh, T. (2020) · Journal of Postgraduate Medicine · doi:10.4103/jpgm.jpgm_591_20 · Cited by 40
How Trainees Come to Trust Supervisors in Workplace-Based Assessment: A Grounded Theory Study
Castanelli, D. J., Weller, J. M., Molloy, E., & Bearman, M. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004501 · Cited by 39
The pursuit of fairness in assessment: Looking beyond the objective
Valentine, N. A., Durning, S. J., Shanahan, E. M., van der Vleuten, C. P. M., & Schuwirth, L. (2022) · Medical Teacher · doi:10.1080/0142159x.2022.2031943 · Cited by 39
Self-citation
Nuance and Noise: Lessons Learned From Longitudinal Aggregated Assessment Data
Chan, T. M., Sherbino, J., & Mercuri, M. (2017) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-17-00086.1 · Cited by 36
Assessment and Feedback in Higher Education: A Review of Literature for the Higher Education Academy
Jackel, B., Pearce, J., Radloff, A., & Edwards, D. (2017) · ACEReSearch Repository (Australian Council for Educational Research) · Cited by 37
Formative assessment and feedback in medical education: A practical guide: AMEE Guide No. 189
Lipnevich, A. A., Mattern, K., & Feddock, C. (2026) · Medical Teacher · doi:10.1080/0142159x.2025.2569623 · Cited by 37
Programmatic Assessment of Competence in Dietetics: A New Frontier
Palermo, C. E., Gibson, S., Dart, J., Whelan, K., & Hay, M. (2016) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2016.03.022 · Cited by 36
Shadow systems in assessment: how supervisors make progress decisions in practice
Castanelli, D. J., Weller, J. M., Molloy, E., & Bearman, M. (2019) · Advances in Health Sciences Education · doi:10.1007/s10459-019-09913-5 · Cited by 36
Implementing Systematic Faculty Development to Support an EPA-Based Program of Assessment: Strategies, Outcomes, and Lessons Learned
Bray, M. J., Bradley, E. B., Martindale, J. R., & Gusic, M. E. (2020) · Teaching and Learning in Medicine · doi:10.1080/10401334.2020.1857256 · Cited by 41
Status of portfolios in undergraduate medical education in the LCME accredited US medical school
Chertoff, J., Wright, A., Novak, M. A., Fantone, J. C., Fleming, A., Ahmed, T., Green, M. M., Kalet, A. L., Linsenmeyer, M., Jacobs, J. L., et al. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1114595 · Cited by 35
A Reliability Analysis of Entrustment-Derived Workplace-Based Assessments
Kelleher, M., Kinnear, B., Sall, D. R., Schumacher, D. J., Schauer, D. P., Warm, E. J., & Kelcey, B. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002997 · Cited by 33
Driving lesson or driving test?: A metaphor to help faculty separate feedback from assessment
Brand, P. L. P., Jaarsma, D. A. D. C., & van der Vleuten, C. P. M. (2020) · Perspectives on Medical Education · doi:10.1007/s40037-020-00617-w · Cited by 36
Student perspectives on programmatic assessment in a large medical programme: A critical realist analysis
Roberts, C., Khanna, P., Bleasel, J. F., Lane, S., Burgess, A. W., Charles, K. A., Howard, R., O'Mara, D. A., Haq, I., & Rutzou, T. (2022) · Medical Education · doi:10.1111/medu.14807 · Cited by 33
Estruturação de Sistemas para Avaliação Programática do Estudante de Medicina
Troncon, L. E. D. A. (2016) · Revista Brasileira de Educação Médica · doi:10.1590/1981-52712015v40n1e01392015 · Cited by 40
The Impact of Assessment for Learning on Learner Performance in Life Science
Oyinloye, O. M., & Imenda, S. N. (2019) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/108689 · Cited by 39
Designing programmes of assessment: A participatory approach
Jamieson, J., Jenkins, G., BEATTY, S. E., & Palermo, C. E. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1355447 · Cited by 32
On Lampposts, Sneetches, and Stars: A Call to Go Beyond Bibliometrics for Determining Academic Value
Chan, T. M., & Kuehl, D. (2019) · Academic Emergency Medicine · doi:10.1111/acem.13707 · Cited by 32
Programmatic Assessment in Emergency Medicine: Implementation of Best Practices
Perry, M. A., Linn, A. M. J., Munzer, B., Hopson, L. R., Amlong, A., Cole, M., & Santen, S. A. (2017) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-17-00094.1 · Cited by 31
Moving Toward Summative Competency Assessment to Individualize the Postclerkship Phase
Keeley, M., Gusic, M. E., Morgan, H. K., Aagaard, E., & Santen, S. A. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002830 · Cited by 30
Mentors’ Beliefs About Their Roles in Health Care Education: A Qualitative Study of Mentors’ Personal Interpretative Framework
Loosveld, L. M., Gerven, P. W. M. V., Vanassche, E., & Driessen, E. W. (2020) · Academic Medicine · doi:10.1097/acm.0000000000003159 · Cited by 30
Self-citation
Nationwide introduction of a new competency framework for undergraduate medical curricula: a collaborative approach
Sohrmann, M., Berendonk, C., NENDAZ, M. R., Bonvin, R., & Swiss Working Group For Profiles Implementation (2020) · Swiss Medical Weekly · doi:10.57187/smw.2020.20201 · Cited by 30
A philosophical history of programmatic assessment: tracing shifting configurations
Pearce, J., & Tavares, W. (2021) · Advances in Health Sciences Education · doi:10.1007/s10459-021-10050-1 · Cited by 32
Entrustment Unpacked: Aligning Purposes, Stakes, and Processes to Enhance Learner Assessment
Kinnear, B., Warm, E. J., Caretta‐Weyer, H. A., Holmboe, E. S., Turner, D. A., van der Vleuten, C. P. M., & Schumacher, D. J. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004108 · Cited by 29
What Behaviors Define a Good Physician? Assessing and Communicating About Noncognitive Skills
Warm, E. J., Kinnear, B., Lance, S. H., Schauer, D. P., & Brenner, J. M. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004215 · Cited by 29
A practical approach to programmatic assessment design
Timmerman, A. A., & Dijkstra, J. (2017) · Advances in Health Sciences Education · doi:10.1007/s10459-017-9756-3 · Cited by 28
Identifying low test-taking effort during low-stakes tests with the new Test-taking Effort Short Scale (TESS) – development and psychometrics
Schüttpelz-Brauns, K., Kadmon, M., Kiessling, C., Karay, Y., Gestmann, M., & Kämmer, J. E. (2018) · BMC Medical Education · doi:10.1186/s12909-018-1196-0 · Cited by 20
The Clinical Learning Environment and Workplace-Based Assessment
Henry, D., & West, D. C. (2019) · Pediatric Clinics of North America · doi:10.1016/j.pcl.2019.03.010 · Cited by 30
Mobile technologies to support workplace-based assessment for entrustment decisions: Guidelines for programs and educators: AMEE Guide No. 154
Marty, A., Linsenmeyer, M., George, B. C., Young, J. Q., Breckwoldt, J., & ten Cate, O. (2023) · Medical Teacher · doi:10.1080/0142159x.2023.2168527 · Cited by 28
Development of and Preliminary Validity Evidence for the EFeCT Feedback Scoring Tool
Ross, S., Hamza, D. M., Zulla, R. T., Stasiuk, S., & Nichols, D. N. (2022) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-21-00602.1 · Cited by 27
Considerations that will determine if competency-based assessment is a sustainable innovation
Dauphinée, W. D., Boulet, J. R., & Norcini, J. J. (2018) · Advances in Health Sciences Education · doi:10.1007/s10459-018-9833-2 · Cited by 27
Entrustment Decision Making in the Core Entrustable Professional Activities: Results of a Multi-Institutional Study
Brown, D. R., Moeller, J. J., Grbic, D., Biskobing, D. M., Crowe, R., Cutrer, W. B., Green, M. L., Obeso, V. T., Wagner, D. P., Warren, J. B., et al. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004242 · Cited by 26
On the issue of costs in programmatic assessment
van der Vleuten, C. P. M., & Heeneman, S. (2016) · Perspectives on Medical Education · doi:10.1007/s40037-016-0295-z · Cited by 25
Self-citation
Why assessment in medical education needs a solid foundation in modern test theory
Schauber, S. K., Hecht, M., & Nouns, Z. M. (2017) · Advances in Health Sciences Education · doi:10.1007/s10459-017-9771-4 · Cited by 25
Identifying Core Components of EPA Implementation: A Path to Knowing if a Complex Intervention Is Being Implemented as Intended
Carraccio, C. L., Martini, A., Melle, E. V., & Schumacher, D. J. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004075 · Cited by 25
Shaping the right conditions in programmatic assessment: how quality of narrative information affects the quality of high-stakes decision-making
de Jong, L. H., Bok, H. G. J., Schellekens, L. H., Kremer, W. D. J., Jonker, F. H., & van der Vleuten, C. P. M. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03257-2 · Cited by 24
Competency Committees in Undergraduate Medical Education: Approaching Tensions Using a Polarity Management Framework
Monrad, S. U., Mangrulkar, R. S., Woolliscroft, J. O., Daniel, M. M., Hartley, S. E., Gay, T. L., Highet, A., Vijayakumar, N., & Santen, S. A. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002816 · Cited by 24
Ten caveats of learning analytics in health professions education: A consumer’s perspective
ten Cate, O., Dahdal, S., Lambert, T., Neubauer, F. B., Pless, A., Pohlmann, P.-F., van Rijen, H. V. M., & Gurtner, C. (2020) · Medical Teacher · doi:10.1080/0142159x.2020.1733505 · Cited by 24
Blended programmatic assessment for competency based curricula
Mahajan, R., Saiyad, S. N., Virk, A., Joshi, A., & Singh, T. (2021) · Journal of Postgraduate Medicine · doi:10.4103/jpgm.jpgm_1061_20 · Cited by 27
Exploring complexities in the reform of assessment practice: a critical realist perspective
Roberts, C., Khanna, P., Lane, S., Reimann, P., & Schuwirth, L. (2021) · Advances in Health Sciences Education · doi:10.1007/s10459-021-10065-8 · Cited by 24
Self-citation
How Trainees Come to Trust Supervisors in Workplace-Based Assessment
Castanelli, D. J., Weller, J. M., Molloy, E., & Bearman, M. (2021) · Own your potential (DEAKIN) · Cited by 24
Monitoring communication skills progress of medical students: Establishing a baseline has value, predicting the future is difficult
Hanley, K., Gillespie, C. C., Zabar, S. R., Adams, J., & Kalet, A. L. (2018) · Patient Education and Counseling · doi:10.1016/j.pec.2018.09.010 · Cited by 24
Using the Five Core Components of Competency-Based Medical Education to Support Implementation of CBVE
Chaney, K. P., & Hodgson, J. (2021) · Frontiers in Veterinary Science · doi:10.3389/fvets.2021.689356 · Cited by 31
An authentic learner-centered planetary health assignment: A five-year evaluation of student choices to address Sustainable Development Goal 13 (Climate Action)
McLean, M., Phelps, C., Smith, J. K., Maheshwari, N., Veer, V., Bushell, D., Matthews, R. S., Craig, B. M., & Moro, C. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1049932 · Cited by 23
Feasibility of peer assessment and clinical audit to self-regulate the quality of physiotherapy services: a mixed methods study
Maas, M. J. M., der Sanden, M. W. G. N., Driehuis, F., Heerkens, Y. F., van der Vleuten, C. P. M., & van der Wees, P. J. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-013726 · Cited by 21
Programmatic assessment: Can we provide evidence for saturation of information?
de Jong, L. H., Bok, H. G. J., Kremer, W. D. J., & van der Vleuten, C. P. M. (2019) · Medical Teacher · doi:10.1080/0142159x.2018.1555369 · Cited by 21
Entrustable Professional Activities: Reflecting on Where We Are to Define a Path for the Next Decade
Schumacher, D. J., & Turner, D. A. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004097 · Cited by 20
A Call for an Integrated Program of Assessment
Fielding, D. W., & Regehr, G. (2017) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe81477 · Cited by 21
The education avenue of the clinical learning environment: A pragmatic approach
Chan, M., Snell, L. S., & Philibert, I. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1566602 · Cited by 20
Juggling roles and generating solutions; practice-based educators’ perceptions of performance-based assessment of physiotherapy students
O’Connor, A., Cantillon, P., Parker, M., & McCurtin, A. (2018) · Physiotherapy · doi:10.1016/j.physio.2018.11.008 · Cited by 19
A Program of Assessment Model for Point-of-Care Ultrasound Training for Pediatric Critical Care Providers: A Comprehensive Approach to Enhance Competency-Based Point-of-Care Ultrasound Training
Maxson, I., Su, E. R., Brown, K. A., Tcharmtchi, M. H., Ginsburg, S., Bhargava, V., Wenger, J. L., Centers, G. I., Alade, K. H., Leung, S. K., et al. (2023) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000003288 · Cited by 18
Implementing Competence Committees on a National Scale: Design and Lessons Learned
Oswald, A., Dubois, D. G., Snell, L. S., Anderson, R., Karpinski, J., Hall, A. K., Frank, J. R., & Cheung, W. J. (2024) · Perspectives on Medical Education · doi:10.5334/pme.961 · Cited by 19
Preschool Communication: Early Identification of Concerns About Preschool Language Development and Social Participation
Doove, B. M., FERON, F. J. M., van Os, J., & Drukker, M. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2020.546536 · Cited by 30
Challenges and opportunities in cultivating medical students’ competencies: Participatory action research from a hierarchical cultural setting
Findyartini, A., Syah, N. A., Susilo, A. P., Nurokhmanti, H., Qomariyah, N. U., Greviana, N., Ainin, D. Q., Sari, S. M., & Claramita, M. (2023) · Medical Education Online · doi:10.1080/10872981.2023.2185122 · Cited by 23
Transforming Resident Assessment: An Analysis Using Deming’s System of Profound Knowledge
Warm, E. J., Kinnear, B., Kelleher, M., Sall, D. R., & Holmboe, E. S. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002499 · Cited by 18
Development and assessment of an online virtual orthodontic curriculum
Liu, L., Yuan, S., Zhang, W., Wang, Z., Zhao, C., Pan, Y., Yan, B., & Wang, L. (2021) · Journal of Dental Education · doi:10.1002/jdd.12825 · Cited by 17
“We just don’t have the resources”: Supervisor perspectives on introducing workplace-based assessments into medical specialist training in South Africa
Ras, T., Jenkins, L. S., Lazarus, C., van Rensburg, J. J., Cooke, R., Senkubuge, F., Dlova, A. N., Singaram, V. S., Daitz, E., Buch, E., et al. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04840-x · Cited by 21
Ambulatory Long Block: A Model of Precision Education and Assessment for Internal Medicine Residents
Warm, E. J., Kinnear, B., Knopp, M. I., Powers‐Fletcher, M. V., Segev, M., Willauer, A. N., & Schauer, D. P. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005610 · Cited by 16

Authors: Van Der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Govaerts, M. J. B., & Heeneman, S.
Affiliations: Flinders University (AU)
Maastricht University (NL)
ORCID: Schuwirth, L.W.T.
Driessen, E.W.
Govaerts, M.J.B.
Heeneman, S.
Venue: Medical Teacher
Publisher: Informa UK Limited
Published: 2014-11-20
Volume: 37
Issue: 7
Pages: 641-646
ISSN: 0142-159X (ISSN-L)
1466-187X
DOI: 10.3109/0142159x.2014.973388
OpenAlex: W1738312775
Cited by: 333
References: 33
Citations by year: 2026: 32 · 2025: 21 · 2024: 29 · 2023: 42 · 2022: 26 · 2021: 47 · 2020: 33 · 2019: 32 · 2018: 30 · 2017: 22 · 2016: 10 · 2015: 5
FWCI: 12.27
Type: journal article
Language: en
Open access: closed
Locations: Medical Teacher (publishedVersion)
PubMed (publishedVersion)
Research Publications (Maastricht University) (publishedVersion)
Research Publications (Maastricht University) (submittedVersion)
Topics: Social Sciences › Education › Innovations in Medical Education
Social Sciences › Education › Student Assessment and Feedback
Social Sciences › Education › Educational Assessment and Pedagogy
Keywords: programmatic assessment, assessment for learning, assessment design, feedback to learners, curriculum quality assurance, high-stakes decision making, information aggregation, student self-regulation, mentoring, constructivist learning, medical education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Twelve Tips for programmatic assessment

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks