A model for programmatic assessment fit for purpose.
Abstract
We propose a model for programmatic assessment in action, which simultaneously optimises assessment for learning and assessment for decision making about learner progress. This model is based on a set of assessment principles that are interpreted from empirical research. It specifies cycles of training, assessment and learner support activities that are complemented by intermediate and final moments of evaluation on aggregated assessment data points. A key principle is that individual data points are maximised for learning and feedback value, whereas high-stake decisions are based on the aggregation of many data points. Expert judgement plays an important role in the programme. Fundamental is the notion of sampling and bias reduction to deal with the inevitable subjectivity of this type of judgement. Bias reduction is further sought in procedural assessment strategies derived from criteria for qualitative research. We discuss a number of challenges and opportunities around the proposed model. One of its prime virtues is that it enables assessment to move, beyond the dominant psychometric discourse with its focus on individual instruments, towards a systems approach to assessment design underpinned by empirically grounded theory.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CBME and programmatic assessment promote the longitudinal compilation of performance data to coach the learner's continuing development and to make evidence-based decisions about the learner's progress.
“Finally, they promote the longitudinal compilation of performance data to coach the learner's continuing development (assessment for learning) and to make evidence-based decisions about the learner's progress (assessment of learning).[11][12][13][14][15]”
From The R2C2 Model in Residency Education: How Does It Foster Coaching and Promote Feedback Use? (2018) · cites this paper as [12]
Programmatic assessment is designed to optimize learning and reduce exam stress.
“Programmatic assessment (a system of frequent formative assessments also used for end-of-year summative judgements) is designed to optimize learning and reduce exam stress [86].”
From Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education. (2015) · cites this paper as [86]
A novel assessment system has been proposed, by which the utility of assessment can complement the former approach by integrating the results of various types of feedback opportunities programmatically.
“Instead, a novel assessment system has been proposed, by which the utility of assessment can complement the former approach by integrating the results of various types of feedback opportunities programmatically [42].”
From Developing Medical Education Curriculum Reform Strategies to Address the Impact of Generative AI: Qualitative Study (2023) · cites this paper as [42]
Cites
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · see citing works
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882 · see citing works
The assessment of clinical skills/competence/performance
Miller, G. E. (1990) · Academic Medicine · doi:10.1097/00001888-199009000-00045 · Cited by 4,883 · see citing works
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631 · see citing works
Klein, G. A. (2008) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288385 · Cited by 2,266 · see citing works
Harvey, L., & Green, D. (1993) · Assessment & Evaluation in Higher Education · doi:10.1080/0260293930180102 · Cited by 1,281 · see citing works
Assessing professional competence: from methods to programmes
Van Der Vleuten, C. P. M., & Schuwirth, L. W. T. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02094.x · Cited by 1,268 · see citing works
Self-citationLinking Theory and Practice: Changing the Pedagogy of Teacher Education
Korthagen, F. A. J., & Kessels, J. P. A. M. (1999) · Educational Researcher · doi:10.3102/0013189x028004004 · Cited by 1,266 · see citing works
The assessment of professional competence: Developments, research and practical implications
Van Der Vleuten, C. P. M. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00596229 · Cited by 1,247 · see citing works
Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks
Falchikov, N., & Goldfinch, J. (2000) · Review of Educational Research · doi:10.3102/00346543070003287 · Cited by 1,170 · see citing works
A Theory of Unconscious Thought
Dijksterhuis, A., & Nordgren, L. F. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00007.x · Cited by 1,041 · see citing works
The CanMEDS initiative: implementing an outcomes-based framework of physician competencies
Frank, J. R., & Danoff, D. S. (2007) · Medical Teacher · doi:10.1080/01421590701746983 · Cited by 1,005 · see citing works
Self-Assessment in the Health Professions: A Reformulation and Research Agenda
Eva, K. W., & Regehr, G. (2005) · Academic Medicine · doi:10.1097/00001888-200510001-00015 · Cited by 992 · see citing works
Workplace-based assessment as an educational tool: AMEE Guide No. 31
Norcini, J., & Burch, V. (2007) · Medical Teacher · doi:10.1080/01421590701775453 · Cited by 984 · see citing works
What every teacher needs to know about clinical reasoning
Eva, K. W. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.01972.x · Cited by 939 · see citing works
A review of rubric use in higher education
Reddy, Y. M., & Andrade, H. (2010) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930902862859 · Cited by 917 · see citing works
Programmatic assessment: From assessment of learning to assessment for learning
Schuwirth, L. W. T., & Van der Vleuten, C. P. M. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.565828 · Cited by 902 · see citing works
Self-citationEducational strategies in curriculum development: the SPICES model
HARDEN, R. M., SOWDEN, S., & DUNN, W. R. (1984) · Medical Education · doi:10.1111/j.1365-2923.1984.tb01024.x · Cited by 772 · see citing works
Mindful Judgment and Decision Making
Weber, E. U., & Johnson, E. J. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.60.110707.163633 · Cited by 746 · see citing works
An admissions OSCE: the multiple mini-interview
Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004) · Medical Education · doi:10.1046/j.1365-2923.2004.01776.x · Cited by 617 · see citing works
Failure to Fail: The Perspectives of Clinical Supervisors
Dudek, N. L., Marks, M. B., & Regehr, G. (2005) · Academic Medicine · doi:10.1097/00001888-200510001-00023 · Cited by 422 · see citing works
SPECIAL ARTICLE: Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings
Williams, R. G., Klamen, D. L., & McGaghie, W. C. (2003) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1504_11 · Cited by 392 · see citing works
Determinants of linear judgment: A meta-analysis of lens model studies.
Karelaia, N., & Hogarth, R. M. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.3.404 · Cited by 358 · see citing works
Assessment of professionalism: Recommendations from the Ottawa 2010 Conference
Hodges, B. D., Ginsburg, S., Cruess, R. L., Cruess, S. R., Delport, R., Hafferty, F. W., Ho, M., Holmboe, E. S., Holtman, M. C., Ohbu, S., et al. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.577300 · Cited by 347 · see citing works
The assessment of professional competence: building blocks for theory development
van der Vleuten, C. P. M., Schuwirth, L., Scheele, F., Driessen, E. W., & Hodges, B. D. (2010) · Best Practice & Research Clinical Obstetrics & Gynaecology · doi:10.1016/j.bpobgyn.2010.04.001 · Cited by 341 · see citing works
Self-citationPortfolios in medical education: why do they meet with mixed success? A systematic review
Driessen, E. W., van Tartwijk, J., van der Vleuten, C. P. M., & Wass, V. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02944.x · Cited by 337 · see citing works
Self-citationFrom principles to practice: An embedded assessment system
Wilson, M., & Sloane, K. (2000) · Applied Measurement in Education · doi:10.1207/s15324818ame1302_4 · Cited by 333 · see citing works
The processes and dimensions of informed self-assessment: A conceptual model.
Sargeant, J., Armson, H., Chesluk, B., Dornan, T., Eva, K., Holmboe, E., Lockyer, J., Loney, E., Mann, K., & van der Vleuten, C. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181d85a4e · Cited by 318 · see citing works
Pitfalls in the pursuit of objectivity: issues of reliability
van der Vleuten, C. P. M., Norman, G. R., & de Graaff, E. (1991) · Medical Education · doi:10.1111/j.1365-2923.1991.tb00036.x · Cited by 285 · see citing works
Govaerts, M., van der Vleuten, C. P. M., Schuwirth, L., & Muijtjens, A. M. M. (2006) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9043-1 · Cited by 284 · see citing works
Self-citationWorkplace-based assessment as an educational tool
Norcini, J. J., Burch, V. C., & Gibbs, T. J. (2007) · Medical Entomology and Zoology · Cited by 266 · see citing works
van Merriënboer, J. J. G. (2007) · Routledge eBooks · doi:10.4324/9781410618054 · Cited by 255 · see citing works
Norcini, J. J. (2003) · BMJ · doi:10.1136/bmj.326.7392.753 · Cited by 253 · see citing works
The portfolio as a tool for stimulating reflection by student teachers
Mansvelder-Longayroux, D. D., Beijaard, D., & Verloop, N. (2006) · Teaching and Teacher Education · doi:10.1016/j.tate.2006.04.033 · Cited by 239 · see citing works
Medical Education: Theory and Practice
Dornan, T., Mann, K. V., Scherpbier, A. J. J. A., & Spencer, J. (2010) · Newcastle University ePrints (Newcastle Univesity) · doi:10.25405/ncl.34176165 · Cited by 227 · see citing works
Portfolios for assessment and learning: AMEE Guide no. 45
van Tartwijk, J., & Driessen, E. W. (2009) · Medical Teacher · doi:10.1080/01421590903139201 · Cited by 217 · see citing works
Self-citationWorkplace-based assessment: effects of rater expertise
Govaerts, M., Schuwirth, L., van der Vleuten, C. P. M., & Muijtjens, A. M. M. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9250-7 · Cited by 198 · see citing works
Self-citationThe New Handbook of Teacher Evaluation: Assessing Elementary and Secondary School Teachers
Heining‐Boynton, A. L., Millman, J., & Darling‐Hammond, L. (1992) · Modern Language Journal · doi:10.2307/329910 · Cited by 195 · see citing works
The wheel of competency assessment: Presenting quality criteria for competency assessment programs
Baartman, L. K. J., Bastiaens, T. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2006) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2006.04.006 · Cited by 192 · see citing works
Self-citationGood judgments do not require complex cognition
Marewski, J. N., Gaissmaier, W., & Gigerenzer, G. (2009) · Cognitive Processing · doi:10.1007/s10339-009-0337-0 · Cited by 190 · see citing works
Engel, B. T. (1967) · Archives of General Psychiatry · doi:10.1001/archpsyc.1967.01730230058008 · Cited by 185 · see citing works
On the generality of specificity
Eva, K. W. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01563.x · Cited by 182 · see citing works
Dannefer, E. F., & Henson, L. C. (2007) · Academic Medicine · doi:10.1097/acm.0b013e31803ead30 · Cited by 181 · see citing works
The Psychology of Experts An Alternative View
Shanteau, J. (2007) · Springer eBooks · doi:10.1007/978-0-585-34290-0_2 · Cited by 173 · see citing works
Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability
Norman, G. R., van der Vleuten, C., & de Graaff, E. (1991) · Medical Education · doi:10.1111/j.1365-2923.1991.tb00037.x · Cited by 170 · see citing works
The Quality of In-house Medical School Examinations
Józefowicz, R. F., Koeppen, B. M., Case, S. M., Galbraith, R. M., Swanson, D. B., & Glew, R. H. (2002) · Academic Medicine · doi:10.1097/00001888-200202000-00016 · Cited by 164 · see citing works
The mechanism of impact of summative assessment on medical students’ learning
Cilliers, F. J., Schuwirth, L., Adendorff, H. J., Herman, N., & van der Vleuten, C. P. M. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9232-9 · Cited by 164 · see citing works
Self-citation“Directed” self-assessment: Practice and feedback within a social context
Sargeant, J., Mann, K., van der Vleuten, C., & Metsemakers, J. (2008) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.155 · Cited by 158 · see citing works
van Merriënboer, J. J. G., & Sluijsmans, D. M. A. (2009) · Educational Psychology Review · doi:10.1007/s10648-008-9092-5 · Cited by 156 · see citing works
A model of the pre-assessment learning effects of summative assessment in medical education
Cilliers, F. J., Schuwirth, L., Herman, N., Adendorff, H. J., & van der Vleuten, C. P. M. (2011) · Advances in Health Sciences Education · doi:10.1007/s10459-011-9292-5 · Cited by 150 · see citing works
Self-citationThe Schoolteacher's Portfolio: An Essay on Possibilities
Bird, T. (1990) · SAGE Publications Ltd eBooks · doi:10.4135/9781412986250.n15 · Cited by 142 · see citing works
A new framework for designing programmes of assessment
Dijkstra, J., van der Vleuten, C. P. M., & Schuwirth, L. (2009) · Advances in Health Sciences Education · doi:10.1007/s10459-009-9205-z · Cited by 133 · see citing works
Self-citationMedical education and the maintenance of incompetence
Hodges, B. D. (2006) · Medical Teacher · doi:10.1080/01421590601102964 · Cited by 129 · see citing works
Expertise and Decision Support
Powell, P. (1993) · Journal of the Operational Research Society · doi:10.1057/jors.1993.123 · Cited by 112 · see citing works
Developing professional judgment
Coles, C. R. (2002) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340220102 · Cited by 109 · see citing works
Sluijsmans, D. M. A., Brand‐Gruwel, S., van Merriënboer, J. J. G., & Bastiaens, T. J. (2002) · Studies In Educational Evaluation · doi:10.1016/s0191-491x(03)90003-4 · Cited by 99 · see citing works
When enough is enough: a conceptual basis for fair and defensible practice performance assessment
Schuwirth, L., Southgate, L. J., Page, G. G., Paget, N. S., Lescop, J., Lew, S. R., Wade, W. B., & Maldonado, M. B. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01313.x · Cited by 95 · see citing works
Self-citationCross institutional collaboration in assessment: a case on progress testing
van der Vleuten, C. P. M., Schuwirth, L., Muijtjens, A. M. M., Thoben, A. J. N. M., Cohen‐Schotanus, J., & van der Vleuten, C. P. M. (2004) · Medical Teacher · doi:10.1080/01421590400016464 · Cited by 79 · see citing works
Self-citationDETERMINING THE QUALITY OF COMPETENCE ASSESSMENT PROGRAMS: A SELF-EVALUATION PROCEDURE
Baartman, L. K. J., Prins, F. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2007) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2007.07.004 · Cited by 69 · see citing works
Self-citationEmbo, M. P. C., Driessen, E. W., Valcke, M., & van der Vleuten, C. P. M. (2010) · Medical Teacher · doi:10.3109/0142159x.2010.490281 · Cited by 63 · see citing works
Self-citationKadri, H. M. A., Al-Moamary, M. S., & van der Vleuten, C. P. M. (2009) · BMC Research Notes · doi:10.1186/1756-0500-2-263 · Cited by 50 · see citing works
Research in assessment: Consensus statement and recommendations from the Ottawa 2010 Conference
Schuwirth, L., Colliver, J. A., Gruppen, L. D., Kreiter, C. D., Mennin, S. P., Onishi, H., Pangaro, L. N., Ringsted, C. V., Swanson, D. B., van der Vleuten, C. P. M., et al. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.551558 · Cited by 47 · see citing works
Self-citationThe Education and Training of Future Physicians
Cavalcanti, R. B., & Detsky, A. S. (2011) · JAMA · doi:10.1001/jama.2011.1232 · Cited by 42 · see citing works
Korthagen, F. A. J., & Wubbels, T. (2001) · Routledge eBooks · doi:10.4324/9781410600523-8 · Cited by 40 · see citing works
Flexible electronic feedback using the virtues of progress testing
Muijtjens, A. M. M., Timmermans, I., Donkers, J. H. H. L. M., Peperkamp, R., Medema, H., Cohen‐Schotanus, J., Thoben, A. J. N. M., Wenink, A. C. G., & van der Vleuten, C. P. M. (2010) · Medical Teacher · doi:10.3109/0142159x.2010.486058 · Cited by 35 · see citing works
van Merriënboer, J. J. G., Kirschner, P. A., & Frèrejean, J. (2024) · Routledge eBooks · doi:10.4324/9781003322481 · Cited by 33 · see citing works
Quality assurance in test construction: The approach of a multidisciplinary central test committee
Verhoeven, B. H., Verwijnen, G. M., Scherpbier, A. J. J. A., Schuwirth, L., & van der Vleuten, C. P. M. (1999) · Cited by 19 · see citing works
Self-citationThe wheel of competency assessment
Baartman, L. K. J., Bastiaens, T. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2005) · Utrecht University Repository (Utrecht University) · Cited by 6 · see citing works
Self-citationClinical and statistical prediction after 50 years: a dedication to Paul Meehl
Morera, O. F., & Dawes, R. M. (2006) · Journal of Behavioral Decision Making · doi:10.1002/bdm.538 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Boscardin, C. K., Gin, B., Golde, P. B., & Hauer, K. E. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005439 · Cited by 533 · see citing works
Core principles of assessment in competency-based medical education.
Lockyer, J., Carraccio, C., Chan, M.-K., Hart, D., Smee, S., Touchie, C., Holmboe, E. S., Frank, J. R., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315082 · Cited by 493 · see citing works
Assessment, feedback and the alchemy of learning
Watling, C., & Ginsburg, S. (2018) · Medical Education · doi:10.1111/medu.13645 · Cited by 434 · see citing works
Implementation of competency‐based medical education: are we addressing the concerns and challenges?
Hawkins, R. E., Welcher, C. M., Holmboe, E. S., Kirk, L. M., Norcini, J. J., Simons, K. B., & Skochelak, S. E. (2015) · Medical Education · doi:10.1111/medu.12831 · Cited by 409 · see citing works
Clinical Reasoning Assessment Methods: A Scoping Review and Practical Guidance
Daniel, M. M., Rencic, J. J., Durning, S. J., Holmboe, E. S., Santen, S. A., Lang, V. J., Ratcliffe, T. A., Gordon, D., Heist, B. S., Lubarsky, S., et al. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002618 · Cited by 358 · see citing works
Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education.
Lefroy, J., Watling, C., Teunissen, P. W., & Brand, P. (2015) · Perspectives on Medical Education · doi:10.1007/s40037-015-0231-7 · Cited by 347 · see citing works
Twelve Tips for programmatic assessment
Van Der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Govaerts, M. J. B., & Heeneman, S. (2015) · Medical Teacher · doi:10.3109/0142159x.2014.973388 · Cited by 333 · see citing works
Self-citationIs the OSCE a feasible tool to assess competencies in undergraduate medical education?
Patrício, M. F., Julião, M., Fareleira, F., & Carneiro, A. V. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.774330 · Cited by 309 · see citing works
Programmatic assessment of competency-based workplace learning: when theory meets practice
Bok, H. G., Teunissen, P. W., Favier, R. P., Rietbroek, N. J., Theyse, L. F., Brommer, H., Haarhuis, J. C., van Beukelen, P., van der Vleuten, C. P., & Jaarsma, D. A. (2013) · BMC Medical Education · doi:10.1186/1472-6920-13-123 · Cited by 303 · see citing works
A call to action: The controversy of and rationale for competency-based medical education
Holmboe, E. S., Sherbino, J., Englander, R., Snell, L. S., Frank, J. R., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315067 · Cited by 276 · see citing works
Capturing the complexity of differentiated instruction
van Geel, M. J. M., Keuning, T., Frèrejean, J., Dolmans, D. H. J. M., van Merriënboer, J. J. G., & Visscher, A. J. (2018) · School Effectiveness and School Improvement · doi:10.1080/09243453.2018.1539013 · Cited by 220 · see citing works
The impact of programmatic assessment on student learning: theory versus practice
Heeneman, S., Pool, A. O., Schuwirth, L., van der Vleuten, C. P. M., & Driessen, E. W. (2015) · Medical Education · doi:10.1111/medu.12645 · Cited by 213 · see citing works
Self-citationCompetency‐based education: programme design and challenges to implementation
Gruppen, L. D., Burkhardt, J., Fitzgerald, J. T., Funnell, M. M., Haftel, H. M., Lypson, M. L., Mullan, P. B., Santen, S. A., Sheets, K. J., Stalburg, C. M., et al. (2016) · Medical Education · doi:10.1111/medu.12977 · Cited by 185 · see citing works
The Power of Subjectivity in the Assessment of Medical Trainees
ten Cate, O., & Regehr, G. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002495 · Cited by 179 · see citing works
Dolmans, D. H. J. M., & Tigelaar, D. E. H. (2012) · Medical Teacher · doi:10.3109/0142159x.2011.595437 · Cited by 172 · see citing works
Self-citationSchellekens, L. H., Bok, H. G. J., de Jong, L. H., van der Schaaf, M. F., Kremer, W. D. J., & van der Vleuten, C. P. M. (2021) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2021.101094 · Cited by 229 · see citing works
Kogan, J. R., Hatala, R. A., Hauer, K. E., & Holmboe, E. S. (2017) · Perspectives on Medical Education · doi:10.1007/s40037-017-0376-7 · Cited by 161 · see citing works
Does authentic assessment assure academic integrity? Evidence from contract cheating data
Ellis, C., van Haeringen, K., Harper, R., Bretag, T., Zucker, I. M., McBride, S., Rozenberg, P., Newton, P. M., & Saddiqui, S. (2019) · Higher Education Research & Development · doi:10.1080/07294360.2019.1680956 · Cited by 162 · see citing works
Assessment of Clinical Skills With Standardized Patients: State of the Art Revisited
Swanson, D. B., & van der Vleuten, C. P. M. (2013) · Teaching and Learning in Medicine · doi:10.1080/10401334.2013.842916 · Cited by 150 · see citing works
Competency-Based, Time-Variable Education in the Health Professions: Crossroads
Lucey, C. R., Thibault, G. E., & ten Cate, O. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002080 · Cited by 149 · see citing works
The R2C2 Model in Residency Education: How Does It Foster Coaching and Promote Feedback Use?
Sargeant, J., Lockyer, J. M., Mann, K., Armson, H., Warren, A., Zetkulic, M., Soklaridis, S., Könings, K. D., Ross, K., Silver, I., et al. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002131 · Cited by 148 · see citing works
Stocker, M., Burmester, M., & Allen, M. L. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-69 · Cited by 146 · see citing works
e-Portfolios enhancing students’ self-directed learning: A systematic review of influencing factors
Beckers, J., Dolmans, D. H. J. M., & van Merriënboer, J. J. G. (2016) · Australasian Journal of Educational Technology · doi:10.14742/ajet.2528 · Cited by 151 · see citing works
A history of assessment in medical education
Schuwirth, L., & van der Vleuten, C. P. M. (2020) · Advances in Health Sciences Education · doi:10.1007/s10459-020-10003-0 · Cited by 149 · see citing works
Self-citationSchut, S., Driessen, E. W., van Tartwijk, J., van der Vleuten, C. P. M., & Heeneman, S. (2018) · Medical Education · doi:10.1111/medu.13532 · Cited by 141 · see citing works
Self-citationGuidelines: The dos, don’ts and don’t knows of remediation in medical education
Chou, C. L., Kalet, A. L., Costa, M. J., Cleland, J., & Winston, K. (2019) · Perspectives on Medical Education · doi:10.1007/s40037-019-00544-5 · Cited by 144 · see citing works
Technology Enhanced Assessment (TEA) in COVID 19 Pandemic
Khan, R. A., & Jawaid, M. (2020) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.36.covid19-s4.2795 · Cited by 141 · see citing works
Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles
Heeneman, S., de Jong, L. H., Dawson, L. J., WILKINSON, T. J., Ryan, A., Tait, G. R., Rice, N. E., Torre, D. M., Freeman, A., & van der Vleuten, C. P. M. (2021) · Medical Teacher · doi:10.1080/0142159x.2021.1957088 · Cited by 150 · see citing works
Barriers to the uptake and use of feedback in the context of summative assessment
Harrison, C., Könings, K. D., Schuwirth, L., Wass, V., & van der Vleuten, C. P. M. (2014) · Advances in Health Sciences Education · doi:10.1007/s10459-014-9524-6 · Cited by 138 · see citing works
Self-citationSchut, S., Maggio, L. A., Heeneman, S., van Tartwijk, J., van der Vleuten, C. P. M., & Driessen, E. W. (2020) · Perspectives on Medical Education · doi:10.1007/s40037-020-00625-w · Cited by 132 · see citing works
Self-citationLearning Analytics in Medical Education Assessment: The Past, the Present, and the Future
Chan, T. M., Sebok‐Syer, S. S., Thoma, B., Wise, A. F., Sherbino, J., & Pusic, M. V. (2018) · AEM Education and Training · doi:10.1002/aet2.10087 · Cited by 126 · see citing works
Validity matters more than cheating
Dawson, P., Bearman, M., Dollinger, M., & Boud, D. J. (2024) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2024.2386662 · Cited by 177 · see citing works
Support for assessment practice: developing the Assessment Design Decisions Framework
Bearman, M., Dawson, P., Boud, D. J., Bennett, S., Hall, M., & Molloy, E. (2016) · Teaching in Higher Education · doi:10.1080/13562517.2016.1160217 · Cited by 125 · see citing works
Harrison, C., Könings, K. D., Dannefer, E. F., Schuwirth, L., Wass, V., & van der Vleuten, C. P. M. (2016) · Perspectives on Medical Education · doi:10.1007/s40037-016-0297-x · Cited by 124 · see citing works
Self-citationBing‐You, R. G., Varaklis, K., Hayes, V., Trowbridge, R. L., Kemp, H., & McKelvy, D. J. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001927 · Cited by 122 · see citing works
Reviewing Residents’ Competence
Hauer, K. E., Chesluk, B. J., Iobst, W. F., Holmboe, E. S., Baron, R. B., Boscardin, C., ten Cate, O., & O’Sullivan, P. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000736 · Cited by 121 · see citing works
The use of programmatic assessment in the clinical workplace: A Maastricht case report
Driessen, E. W., van Tartwijk, J., Govaerts, M., Teunissen, P., & van der Vleuten, C. P. M. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652242 · Cited by 116 · see citing works
Self-citationSchuwirth, L., & Ash, J. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.787140 · Cited by 118 · see citing works
Self-citationImplementation of Competency-Based Pharmacy Education (CBPE)
Koster, A. S., Schalekamp, T., & Meijerman, I. (2017) · Pharmacy · doi:10.3390/pharmacy5010010 · Cited by 111 · see citing works
Driessen, E., & Scheele, F. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.798403 · Cited by 110 · see citing works
Self-citationForsberg, E., Ziegert, K., Hult, H., & Fors, U. (2013) · Nurse Education Today · doi:10.1016/j.nedt.2013.07.010 · Cited by 107 · see citing works
Changing the culture of assessment: the dominance of the summative assessment paradigm
Harrison, C., Könings, K. D., Schuwirth, L., Wass, V., & van der Vleuten, C. P. M. (2017) · BMC Medical Education · doi:10.1186/s12909-017-0912-5 · Cited by 104 · see citing works
Self-citationThe Development of Entrustable Professional Activities for Internal Medicine Residency Training
Caverzagie, K. J., Cooney, T. G., Hemmer, P. A., & Berkowitz, L. R. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000564 · Cited by 100 · see citing works
Believing is seeing: how people's beliefs influence goals, emotions and behaviour
Teunissen, P. W., & Bok, H. G. J. (2013) · Medical Education · doi:10.1111/medu.12228 · Cited by 99 · see citing works
Assessment quality and practices in secondary PE in the Netherlands
Borghouts, L. B., Slingerland, M., & Haerens, L. L. (2016) · Physical Education and Sport Pedagogy · doi:10.1080/17408989.2016.1241226 · Cited by 112 · see citing works
The do’s, don’t and don’t knows of supporting transition to more independent practice
Yardley, S. J., Westerman, M., Bartlett, M., Walton, J. M., Smith, J. M., & Peile, E. (2018) · Perspectives on Medical Education · doi:10.1007/s40037-018-0403-3 · Cited by 100 · see citing works
Assessment in the context of problem-based learning
van der Vleuten, C. P. M., & Schuwirth, L. (2019) · Advances in Health Sciences Education · doi:10.1007/s10459-019-09909-1 · Cited by 109 · see citing works
Self-citationR2C2 in action: Testing an evidence-based model to facilitate feedback and coaching in residency.
Sargeant, J., Mann, K., Manos, S., Epstein, I., Warren, A., Shearer, C., & Boudreau, M. (2017) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-16-00398.1 · Cited by 98 · see citing works
Entrustable professional activities in entry‐level health professional education: A scoping review
Bramley, A. L., & McKenna, L. (2021) · Medical Education · doi:10.1111/medu.14539 · Cited by 96 · see citing works
What programmatic assessment in medical education can learn from healthcare
Schuwirth, L., van der Vleuten, C. P. M., & Durning, S. J. (2017) · Perspectives on Medical Education · doi:10.1007/s40037-017-0345-1 · Cited by 94 · see citing works
Self-citationBullock, J. L., Lai, C. J., Lockspeiser, T. M., O’Sullivan, P., Aronowitz, P. B., Dellmore, D., Fung, C.-C., Knight, C. L., & Hauer, K. E. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002905 · Cited by 94 · see citing works
Khalaf, K., El‐Kishawi, M., Moufti, M. A., & Kawas, S. A. (2020) · Medical Education Online · doi:10.1080/10872981.2020.1826861 · Cited by 99 · see citing works
Warm, E. J., Held, J. D., Hellmann, M., Kelleher, M., Kinnear, B., Lee, C., O’Toole, J. K., Mathis, B. R., Mueller, C. V., Sall, D. R., et al. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001292 · Cited by 89 · see citing works
Enhanced Requirements for Assessment in a Competency-Based, Time-Variable Medical Education System
Gruppen, L. D., ten Cate, O., Lingard, L., Teunissen, P. W., & Kogan, J. R. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002066 · Cited by 88 · see citing works
Exploring the characteristics and context that allow Master Adaptive Learners to thrive
Cutrer, W. B., Atkinson, H. G., Friedman, E. S., Deiorio, N. M., Gruppen, L. D., Dekhtyar, M., & Pusic, M. V. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1484560 · Cited by 92 · see citing works
Driessen, E. W. (2016) · Advances in Health Sciences Education · doi:10.1007/s10459-016-9679-4 · Cited by 86 · see citing works
Self-citationThe impact of entrustment assessments on feedback and learning: Trainee perspectives
Martin, L. J., Sibbald, M., Vegas, D. B., Russell, D., & Govaerts, M. (2019) · Medical Education · doi:10.1111/medu.14047 · Cited by 87 · see citing works
Do coursework summative assessments predict clinical performance? A systematic review
Terry, R., Hing, W. A., Orr, R. M., & Milne, N. (2017) · BMC Medical Education · doi:10.1186/s12909-017-0878-3 · Cited by 85 · see citing works
van den Eertwegh, V., van Dalen, J., van Dulmen, S., van der Vleuten, C. P. M., & Scherpbier, A. J. J. A. (2014) · Patient Education and Counseling · doi:10.1016/j.pec.2014.01.002 · Cited by 85 · see citing works
Twelve tips for teaching with concept maps in medical education
Torre, D. M., Durning, S. J., & Daley, B. J. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.759644 · Cited by 85 · see citing works
Identifying Feedback That Has Impact
Henderson, M., Ajjawi, R., Boud, D. J., & Molloy, E. (2019) · Springer eBooks · doi:10.1007/978-3-030-25112-3_2 · Cited by 92 · see citing works
Theoretical considerations on programmatic assessment
Torre, D. M., Schuwirth, L., & van der Vleuten, C. P. M. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1672863 · Cited by 84 · see citing works
Self-citationChung, M. P., Thang, C. K., Vermillion, M., Fried, J. M., & Uijtdehaage, S. H. J. (2018) · Medical Education Online · doi:10.1080/10872981.2018.1478170 · Cited by 82 · see citing works
Student perspectives on assessment: Experience in a competency-based portfolio system
Altahawi, F. F., Sisk, B. A., Poloskey, S. L., Hicks, C. W., & Dannefer, E. F. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652243 · Cited by 80 · see citing works
What we measure … and what we should measure in medical education
Boulet, J. R., & Durning, S. J. (2018) · Medical Education · doi:10.1111/medu.13652 · Cited by 77 · see citing works
Shimizu, I., Kasai, H., Shikino, K., Araki, N., Takahashi, Z., Onodera, M., Kimura, Y., Tsukamoto, T., Yamauchi, K., Asahina, M., et al. (2023) · JMIR Medical Education · doi:10.2196/53466 · Cited by 81 · see citing works
Kusurkar, R. A., Orsini, C. A., Somra, S., Artino, A. R., Daelmans, H. E. M., Schoonmade, L. J., & van der Vleuten, C. P. M. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005263 · Cited by 76 · see citing works
Video-Based Assessment in Surgical Education: A Scoping Review
McQueen, S. A., McKinnon, V., VanderBeek, L., McCarthy, C., & Sonnadara, R. (2019) · Journal of surgical education · doi:10.1016/j.jsurg.2019.05.013 · Cited by 75 · see citing works
Beyond assessment of learning toward assessment for learning: Educating tomorrow's physicians
Dannefer, E. F. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.787141 · Cited by 72 · see citing works
Kinnear, B., Warm, E. J., & Hauer, K. E. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1474191 · Cited by 71 · see citing works
Anderson, H. L. K., Kurtz, J. B., & West, D. C. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004366 · Cited by 69 · see citing works
Core Clerkship Grading: The Illusion of Objectivity
Hauer, K. E., & Lucey, C. R. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002413 · Cited by 68 · see citing works
Pinilla, S., Lenouvel, E., Cantisani, A., Klöppel, S., Strik, W. K., Huwendiek, S., & Nissen, C. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02608-9 · Cited by 71 · see citing works
Hauer, K. E., Park, Y. S., Bullock, J. L., & Tekian, A. S. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005245 · Cited by 67 · see citing works
Beyond NAVMEC: Competency-Based Veterinary Education and Assessment of the Professional Competencies
Hodgson, J. L., Pelzer, J. M., & Inzana, K. D. (2013) · Journal of Veterinary Medical Education · doi:10.3138/jvme.1012-092r · Cited by 67 · see citing works
Rich, J. V., Young, S. F., Donnelly, C. A., Hall, A. K., Dagnone, D., Weersink, K., Caudle, J. M., Melle, E. V., & Klinger, D. A. (2019) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13328 · Cited by 67 · see citing works
Refocusing portfolio assessment: Curating for feedback and portrayal
Clarke, J., & Boud, D. J. (2016) · Innovations in Education and Teaching International · doi:10.1080/14703297.2016.1250664 · Cited by 66 · see citing works
Duijn, C. C. M. A., Welink, L. S., Bok, H. G. J., & ten Cate, O. (2018) · Perspectives on Medical Education · doi:10.1007/s40037-018-0430-0 · Cited by 65 · see citing works
Assessment quality in tertiary education: An integrative literature review
Leeuwenkamp, K. J. G.-V., Brinke, D. J.-T., & Kester, L. (2017) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2017.08.001 · Cited by 73 · see citing works
Boscardin, C., Fergus, K. B., Hellevig, B., & Hauer, K. E. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1396306 · Cited by 64 · see citing works
Peer assessment in the objective structured clinical examination: A scoping review
Khan, R., Payne, M. W. C., & Chahine, S. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1309375 · Cited by 63 · see citing works
Validity evidence for programmatic assessment in competency-based education
Bok, H. G. J., de Jong, L. H., O’Neill, T., Maxey, C., & Hecker, K. G. (2018) · Perspectives on Medical Education · doi:10.1007/s40037-018-0481-2 · Cited by 63 · see citing works
Assessment of communication skills
van der Vleuten, C. P. M., van den Eertwegh, V., & Giroldi, E. (2019) · Patient Education and Counseling · doi:10.1016/j.pec.2019.07.007 · Cited by 67 · see citing works
Heeneman, S., Schut, S., Donkers, J. H. H. L. M., van der Vleuten, C. P. M., & Muijtjens, A. M. M. (2016) · Medical Teacher · doi:10.1080/0142159x.2016.1230183 · Cited by 58 · see citing works
Systematic review and narrative synthesis of competency-based medical education in anaesthesia
Weller, J. M., Naik, V. N., & Diego, R. J. S. (2020) · British Journal of Anaesthesia · doi:10.1016/j.bja.2019.10.025 · Cited by 61 · see citing works
Frèrejean, J., van Geel, M. J. M., Keuning, T., Dolmans, D. H. J. M., van Merriënboer, J. J. G., & Visscher, A. J. (2021) · Instructional Science · doi:10.1007/s11251-021-09540-x · Cited by 63 · see citing works
Between trust and control: Teachers' assessment conceptualisations within programmatic assessment
Schut, S., Heeneman, S., Bierer, S. B., Driessen, E. W., van Tartwijk, J., & van der Vleuten, C. P. M. (2020) · Medical Education · doi:10.1111/medu.14075 · Cited by 57 · see citing works
Self-citationTrust, power and learning in workplace‐based assessment: The trainee perspective
Castanelli, D. J., Weller, J. M., Molloy, E., & Bearman, M. (2021) · Medical Education · doi:10.1111/medu.14631 · Cited by 56 · see citing works
Knowledge, skills and beetles: respecting the privacy of private experiences in medical education
Veen, M., Skelton, J. R., & de la Croix, A. (2020) · Perspectives on Medical Education · doi:10.1007/s40037-020-00565-5 · Cited by 56 · see citing works
Ottawa 2020 consensus statements for programmatic assessment – 2. Implementation and practice
Torre, D. M., Rice, N. E., Ryan, A., Bok, H. G. J., Dawson, L. J., Bierer, S. B., WILKINSON, T. J., Tait, G. R., Laughlin, T., Veerapen, K., et al. (2021) · Medical Teacher · doi:10.1080/0142159x.2021.1956681 · Cited by 55 · see citing works
Feedback-giving behaviour in performance evaluations during clinical clerkships
Bok, H. G. J., Jaarsma, D. A. D. C., Spruijt, A., van Beukelen, P., van der Vleuten, C. P. M., & Teunissen, P. W. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1017448 · Cited by 54 · see citing works
Meaningful feedback through a sociocultural lens
Ramani, S., Könings, K. D., Ginsburg, S., & van der Vleuten, C. P. M. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1656804 · Cited by 55 · see citing works
Deconstructing programmatic assessment
WILKINSON, T. J., & Tweed, M. (2018) · Advances in Medical Education and Practice · doi:10.2147/amep.s144449 · Cited by 53 · see citing works
Zimmermann, K., Holzinger, I. I. B., Ganassi, L., Esslinger, P., Pilgrim, S., Allen, M. L., Burmester, M., & Stocker, M. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0472-5 · Cited by 52 · see citing works
Competence Assessment as Learner Support in Education
van der Vleuten, C. P. M., Sluijsmans, D. M. A., & Brinke, D. J.-T. (2016) · Technical and vocational education and training · doi:10.1007/978-3-319-41713-4_28 · Cited by 52 · see citing works
Some assembly required: tracing the interpretative work of Clinical Competency Committees
Pack, R., Lingard, L. A., Watling, C., Chahine, S., & Cristancho, S. (2019) · Medical Education · doi:10.1111/medu.13884 · Cited by 52 · see citing works
The Transformational Path Ahead: Competency-Based Medical Education in Family Medicine
Holmboe, E. S. (2021) · Family Medicine · doi:10.22454/fammed.2021.296914 · Cited by 51 · see citing works
How ‘Testing’ Has Become ‘Programmatic Assessment for Learning’
Schuwirth, L., & van der Vleuten, C. P. M. (2018) · Health Professions Education · doi:10.1016/j.hpe.2018.06.005 · Cited by 53 · see citing works
Self-citationDevine, O., Harborne, A. C., & McManus, I. C. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0428-9 · Cited by 50 · see citing works
Embo, M. P. C., Driessen, E. W., Valcke, M., & van der Vleuten, C. P. M. (2014) · Nurse Education Today · doi:10.1016/j.nedt.2014.11.022 · Cited by 50 · see citing works
Self-citation
- Authors
- van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J.
- Affiliations
- Flinders University (AU)
- Maastricht University (NL)
- Leiden University (NL)
- University of Applied Sciences Utrecht (NL)
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2012-02-25
- Volume
- 34
- Issue
- 3
- Pages
- 205-214
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W2163467994
- Cited by
- 892
- References
- 69
- Citations by year
- 2026: 140 · 2025: 62 · 2024: 70 · 2023: 74 · 2022: 72 · 2021: 71 · 2020: 59 · 2019: 66 · 2018: 69 · 2017: 56 · 2016: 50 · 2015: 32 · 2014: 40 · 2013: 21 · 2012: 9
- FWCI
- 31.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Teacher (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Keywords
- programmatic assessment, assessment for learning, student progress, evaluation principles, expert opinion, bias mitigation, sampling, feedback, medical education
- MeSH terms
- Decision Making, Educational Measurement, Humans, Program Evaluation, Models, Educational
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
