Assessing professional competence: from methods to programmes

Van Der Vleuten, Cees P M; Schuwirth, Lambert W T

doi:10.1111/j.1365-2929.2005.02094.x

Cited by 1,268Innovations in Medical EducationStudent Assessment and FeedbackCompetency Development and Evaluation

Assessing professional competence: from methods to programmes

Van Der Vleuten, Cees P M, Schuwirth, Lambert W T·2005·Medical Education

DOI 10.1111/j.1365-2929.2005.02094.x

Open this paper in CoChat — free account

Abstract

Introduction We use a utility model to illustrate that, firstly, selecting an assessment method involves context‐dependent compromises, and secondly, that assessment is not a measurement problem but an instructional design problem, comprising educational, implementation and resource aspects. In the model, assessment characteristics are differently weighted depending on the purpose and context of the assessment. Empirical and theoretical developments Of the characteristics in the model, we focus on reliability, validity and educational impact and argue that they are not inherent qualities of any instrument. Reliability depends not on structuring or standardisation but on sampling. Key issues concerning validity are authenticity and integration of competencies. Assessment in medical education addresses complex competencies and thus requires quantitative and qualitative information from different sources as well as professional judgement. Adequate sampling across judges, instruments and contexts can ensure both validity and reliability. Despite recognition that assessment drives learning, this relationship has been little researched, possibly because of its strong context dependence. Assessment as instructional design When assessment should stimulate learning and requires adequate sampling, in authentic contexts, of the performance of complex competencies that cannot be broken down into simple parts, we need to make a shift from individual methods to an integral programme, intertwined with the education programme. Therefore, we need an instructional design perspective. Implications for development and research Programmatic instructional design hinges on a careful description and motivation of choices, whose effectiveness should be measured against the intended outcomes. We should not evaluate individual methods, but provide evidence of the utility of the assessment programme as a whole.

Integrity signals

Registry

Crossref · 10.1111/j.1365-2929.2005.02094.x

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Programmatic assessment emphasizes progress and learning from feedback and builds it into the system.
“In the new movement towards programmatic assessment, progress and learning from feedback is emphasized and built into the system [74, 75].”
From Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education. (2015) · cites this paper as [74]

The assessment of clinical skills/competence/performance
Miller, G. E. (1990) · Academic Medicine · doi:10.1097/00001888-199009000-00045 · Cited by 4,883 · see citing works
Defining and Assessing Professional Competence
Epstein, R. M. (2002) · JAMA · doi:10.1001/jama.287.2.226 · Cited by 3,114 · see citing works
Standards for educational and psychological testing
(1985) · PsycEXTRA Dataset · doi:10.1037/e500832009-001 · Cited by 1,493 · see citing works
Validity: on the meaningful interpretation of assessment data
Downing, S. M. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01594.x · Cited by 1,434 · see citing works
The interplay of evidence and consequences in the validation of performance assessments
Messick, S. (1994) · Educational Researcher · doi:10.3102/0013189x023002013 · Cited by 1,315 · see citing works
The assessment of professional competence: Developments, research and practical implications
Van Der Vleuten, C. P. M. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00596229 · Cited by 1,247 · see citing works
The use of self-, peer and co-assessment in higher education: A review
Dochy, F., Segers, M., & Sluijsmans, D. (1999) · Studies in Higher Education · doi:10.1080/03075079912331379935 · Cited by 1,206 · see citing works
Complex, Performance-Based Assessment: Expectations and Validation Criteria
Linn, R. L., Baker, E., & Dunbar, S. (1991) · Educational Researcher · doi:10.3102/0013189x020008015 · Cited by 1,043 · see citing works
The Mini-CEX: A Method for Assessing Clinical Skills
Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-6-200303180-00012 · Cited by 730 · see citing works
Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination
Regehr, G., MacRae, H., Reznick, R. K., & Szalay, D. (1998) · Academic Medicine · doi:10.1097/00001888-199809000-00020 · Cited by 712 · see citing works
The Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation
Norcini, J. J., Blank, L. L., Arnold, G. K., & Kimball, H. R. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-123-10-199511150-00008 · Cited by 526 · see citing works
The relationship between competence and performance: implications for assessing practice performance
Rethans, J.-J., Norcini, J. J., Maldonado, M. B., Blackmore, D. E., Jolly, B. C., LaDuca, T., Lew, S. R., Page, G. G., & Southgate, L. J. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01316.x · Cited by 405 · see citing works
SPECIAL ARTICLE: Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings
Williams, R. G., Klamen, D. L., & McGaghie, W. C. (2003) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1504_11 · Cited by 392 · see citing works
Performance-Based Assessment: Lessons From the Health Professions
Swanson, D. B., Norman, G. R., & Linn, R. L. (1995) · Educational Researcher · doi:10.3102/0013189x024005005 · Cited by 287 · see citing works
Pitfalls in the pursuit of objectivity: issues of reliability
van der Vleuten, C. P. M., Norman, G. R., & de Graaff, E. (1991) · Medical Education · doi:10.1111/j.1365-2923.1991.tb00036.x · Cited by 285 · see citing works
Multisource feedback in the assessment of physician competencies
Lockyer, J. (2003) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340230103 · Cited by 284 · see citing works
Developments in outcome-based education
Harden, R. M. (2002) · Medical Teacher · doi:10.1080/01421590220120669 · Cited by 276 · see citing works
The rising cost of NIH-funded biomedical research?
Kennedy, T. J. (1990) · Academic Medicine · doi:10.1097/00001888-199002000-00001 · Cited by 262 · see citing works
Peer assessment of competence
Norcini, J. J. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01536.x · Cited by 244 · see citing works
Assessing health professionals
Crossley, J., Humphris, G. M., & Jolly, B. C. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01294.x · Cited by 209 · see citing works
The use of qualitative research criteria for portfolio assessment as an alternative to reliability evaluation: a case study
Driessen, E. W., van der Vleuten, C. P. M., Schuwirth, L., van Tartwijk, J., & Vermunt, J. D. H. M. (2005) · Medical Education · doi:10.1111/j.1365-2929.2004.02059.x · Cited by 207 · see citing works
Self-citation
The use of clinical simulations in assessment
Schuwirth, L., & van der Vleuten, C. P. M. (2003) · Medical Education · doi:10.1046/j.1365-2923.37.s1.8.x · Cited by 205 · see citing works
Self-citation
Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility
Ram, Grol, Rethans, Schouten, Vleuten, & Kester (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00348.x · Cited by 170 · see citing works
The Quality of In-house Medical School Examinations
Józefowicz, R. F., Koeppen, B. M., Case, S. M., Galbraith, R. M., Swanson, D. B., & Glew, R. H. (2002) · Academic Medicine · doi:10.1097/00001888-200202000-00016 · Cited by 164 · see citing works
Validity and the OSCE
Hodges, B. D. (2003) · Medical Teacher · doi:10.1080/01421590310001002836 · Cited by 155 · see citing works
Changing education, changing assessment, changing research?
Schuwirth, L., & van der Vleuten, C. P. M. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.01851.x · Cited by 146 · see citing works
Self-citation
Evaluating the Consequential Validity of New Modes of Assessment: The Influence of Assessment on Learning, Including Pre-, Post-, and True Assessment Effects
Gielen, S., Dochy, F. J. R. C., & Dierick, S. (2006) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-48125-1_3 · Cited by 120 · see citing works
Standardized or real patients to test clinical competence? The long case revisited
Wass, V., Jones, R. F., & van der Vleuten, C. P. M. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00928.x · Cited by 120 · see citing works
Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence
Norcini, J. J., Swanson, D. B., Grosso, L. J., & WEBSTER, G. D. (1985) · Medical Education · doi:10.1111/j.1365-2923.1985.tb01314.x · Cited by 112 · see citing works
ASSESSMENT OF CLINICAL COMPETENCE: WRITTEN AND COMPUTER‐BASED SIMULATIONS
Swanson, D. B., Norcini, J. J., & Grosso, L. J. (1987) · Assessment & Evaluation in Higher Education · doi:10.1080/0260293870120307 · Cited by 109 · see citing works
Developing professional judgment
Coles, C. R. (2002) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340220102 · Cited by 109 · see citing works
Composite undergraduate clinical examinations: how should the components be combined to maximize reliability?
Wass, V., McGibbon, D., & van der Vleuten, C. P. M. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00929.x · Cited by 92 · see citing works
Assessing Studentsʼ Performances in a Competency-based Curriculum
Smith, S. R., Dollase, R. H., & Boss, J. A. (2003) · Academic Medicine · doi:10.1097/00001888-200301000-00019 · Cited by 88 · see citing works
Clinical Performance Assessments
PETRUSA, E. R. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_26 · Cited by 76 · see citing works
Portfolios – attempting to measure the unmeasurable?
Snadden, D. (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00446.x · Cited by 69 · see citing works
Adapting the Key Features Examination for a clinical clerkship
Hatala, R. A., & Norman, G. R. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01067.x · Cited by 53 · see citing works
Assessment of Clinical Performance: In-Training Evaluation
Turnbull, J., & van Barneveld, C. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_29 · Cited by 42 · see citing works
Reproducibility of clinical performance assessment in practice using incognito standardized patients
GORTER, S. L., Rethans, J., van der Heijde, D. M. F. M., Scherpbier, A. J. J. A., Houben, H., van der Vleuten, C. P. M., & van der Linden, S. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01296.x · Cited by 32 · see citing works
Student Assessment
Mennin, S. P., & Kalishman, S. G. (1998) · Academic Medicine · doi:10.1097/00001888-199809000-00035 · Cited by 32 · see citing works
How to introduce incognito standardized patients into outpatient clinics of specialists in rheumatology
Gorter, J.-J. R. S. L. (2001) · Medical Teacher · doi:10.1080/014215931048 · Cited by 22 · see citing works
Optimising New Modes of Assessment: In search of Qualities and Standards
Segers, M. S. R., Dochy, F. J. R. C., & Cascallar, E. C. (2003) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-48125-1 · Cited by 21 · see citing works
The Influence of Testing Context and Clinical Rotation Order on Students’ OSCE Performance
Blaskiewicz, R., Park, R. S., Chibnall, J. T., & Powell, J. K. (2004) · Academic Medicine · doi:10.1097/00001888-200406000-00020 · Cited by 15 · see citing works
Managing the Curriculum and Managing Change
Davis, W. K., & White, C. B. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_34 · Cited by 12 · see citing works
Reliability of the fellowship examination of the royal Australian college of general practitioners
Hays, R., Fabb, W. E., & van der VIeuten, C. P. M. (1995) · Teaching and Learning in Medicine · doi:10.1080/10401339509539709 · Cited by 11 · see citing works

Flagged first, then by citations

Sorted by citations

Programmatic assessment: From assessment of learning to assessment for learning
Schuwirth, L. W. T., & Van der Vleuten, C. P. M. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.565828 · Cited by 902 · see citing works
Self-citation
A model for programmatic assessment fit for purpose.
van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652239 · Cited by 892 · see citing works
Self-citation
Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99
ten Cate, O., Chen, H. C., Hoff, R. G., Peters, H., Bok, H. G. J., & van der Schaaf, M. F. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1060308 · Cited by 792 · see citing works
Achieving desired results and improved outcomes: Integrating planning and assessment throughout learning activities
Moore, D. E., Green, J. S., & Gallis, H. A. (2009) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.20001 · Cited by 743 · see citing works
Tools for Direct Observation and Assessment of Clinical Skills of Medical Trainees
Kogan, J. R., Holmboe, E. S., & Hauer, K. E. (2009) · JAMA · doi:10.1001/jama.2009.1365 · Cited by 602 · see citing works
Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference
Norcini, J. J., Anderson, M. B., Bóllela, V. R., Burch, V. C., Costa, M. J., Duvivier, R. J., Galbraith, R. M., Hays, R., Kent, A., Perrott, V., et al. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.551559 · Cited by 587 · see citing works
“Continuity” as an Organizing Principle for Clinical Education Reform
Hirsh, D. A., Ogur, B., Thibault, G. E., & Cox, M. (2007) · New England Journal of Medicine · doi:10.1056/nejmsb061660 · Cited by 540 · see citing works
Core principles of assessment in competency-based medical education.
Lockyer, J., Carraccio, C., Chan, M.-K., Hart, D., Smee, S., Touchie, C., Holmboe, E. S., Frank, J. R., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315082 · Cited by 493 · see citing works
Entrustment Decision Making in Clinical Training
ten Cate, O., Hart, D., Ankel, F., Busari, J., Englander, R., Glasgow, N., Holmboe, E., Iobst, W., Lovell, E., Snell, L. S., et al. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001044 · Cited by 478 · see citing works
Implementation of competency‐based medical education: are we addressing the concerns and challenges?
Hawkins, R. E., Welcher, C. M., Holmboe, E. S., Kirk, L. M., Norcini, J. J., Simons, K. B., & Skochelak, S. E. (2015) · Medical Education · doi:10.1111/medu.12831 · Cited by 409 · see citing works
Advancing competency-based medical education: A charter for clinician–educators.
Carraccio, C., Englander, R., Van Melle, E., ten Cate, O., Lockyer, J., Chan, M.-K., Frank, J. R., & Snell, L. S. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001048 · Cited by 372 · see citing works
Assessment methods in medical education
Norcini, J. J., & McKinley, D. W. (2007) · Teaching and Teacher Education · doi:10.1016/j.tate.2006.12.021 · Cited by 366 · see citing works
Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education.
Lefroy, J., Watling, C., Teunissen, P. W., & Brand, P. (2015) · Perspectives on Medical Education · doi:10.1007/s40037-015-0231-7 · Cited by 347 · see citing works
Twelve Tips for programmatic assessment
Van Der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Govaerts, M. J. B., & Heeneman, S. (2015) · Medical Teacher · doi:10.3109/0142159x.2014.973388 · Cited by 333 · see citing works
Self-citation
The assessment of professional competence: building blocks for theory development
van der Vleuten, C. P. M., Schuwirth, L., Scheele, F., Driessen, E. W., & Hodges, B. D. (2010) · Best Practice & Research Clinical Obstetrics & Gynaecology · doi:10.1016/j.bpobgyn.2010.04.001 · Cited by 341 · see citing works
Self-citation
Self-monitoring in clinical practice: A challenge for medical educators
Epstein, R. M., Siegel, D. J., & Silberman, J. (2008) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.149 · Cited by 333 · see citing works
Is the OSCE a feasible tool to assess competencies in undergraduate medical education?
Patrício, M. F., Julião, M., Fareleira, F., & Carneiro, A. V. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.774330 · Cited by 309 · see citing works
Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks
Baartman, L. K. J., Bastiaens, T. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2007) · Educational Research Review · doi:10.1016/j.edurev.2007.06.001 · Cited by 328 · see citing works
Programmatic assessment of competency-based workplace learning: when theory meets practice
Bok, H. G., Teunissen, P. W., Favier, R. P., Rietbroek, N. J., Theyse, L. F., Brommer, H., Haarhuis, J. C., van Beukelen, P., van der Vleuten, C. P., & Jaarsma, D. A. (2013) · BMC Medical Education · doi:10.1186/1472-6920-13-123 · Cited by 303 · see citing works
Reflection: a link between receiving and using assessment feedback
Sargeant, J. M., Mann, K. V., van der Vleuten, C. P., & Metsemakers, J. F. (2009) · Advances in Health Sciences Education · doi:10.1007/s10459-008-9124-4 · Cited by 296 · see citing works
Assessment in the post-psychometric era: Learning to love the subjective and collective
Hodges, B. D. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.789134 · Cited by 283 · see citing works
Broadening Perspectives on Clinical Performance Assessment: Rethinking the Nature of In-training Assessment
Govaerts, M., van der Vleuten, C. P. M., Schuwirth, L., & Muijtjens, A. M. M. (2006) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9043-1 · Cited by 284 · see citing works
Self-citation
A call to action: The controversy of and rationale for competency-based medical education
Holmboe, E. S., Sherbino, J., Englander, R., Snell, L. S., Frank, J. R., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315067 · Cited by 276 · see citing works
Mastery Learning
McGaghie, W. C. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000911 · Cited by 273 · see citing works
Using Virtual Reality Simulation Environments to Assess Competence for Emergency Medicine Learners
McGrath, J. L., Taekman, J. M., Dev, P., Danforth, D. R., Mohan, D., Kman, N. E., Crichlow, A., & Bond, W. F. (2017) · Academic Emergency Medicine · doi:10.1111/acem.13308 · Cited by 249 · see citing works
Seeing the ‘black box’ differently: assessor cognition from three research perspectives
Gingerich, A., Kogan, J. R., Yeates, P., Govaerts, M., & Holmboe, E. S. (2014) · Medical Education · doi:10.1111/medu.12546 · Cited by 245 · see citing works
Opening the black box of clinical skills assessment via observation: a conceptual model
Kogan, J. R., Conforti, L. N., Bernabeo, E. C., Iobst, W. F., & Holmboe, E. S. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04025.x · Cited by 235 · see citing works
From Flexner to Competencies
Carraccio, C. L., & Englander, R. (2013) · Academic Medicine · doi:10.1097/acm.0b013e318299396f · Cited by 231 · see citing works
Assessment for selection for the health care professions and specialty training: Consensus statement and recommendations from the Ottawa 2010 Conference
Prideaux, D. J., Roberts, C., Eva, K. W., Centeno, A., McCrorie, P., McManus, I. C., Patterson, F., POWIS, D. A., Tekian, A. S., & Wilkinson, D. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.551560 · Cited by 221 · see citing works
Good questions, good answers: construct alignment improves the performance of workplace‐based assessment scales
Crossley, J., Johnson, G., Booth, J., & Wade, W. B. (2011) · Medical Education · doi:10.1111/j.1365-2923.2010.03913.x · Cited by 217 · see citing works
Assessing Dental Students’ Competence: Best Practice Recommendations in the Performance Assessment Literature and Investigation of Current Practices in Predoctoral Dental Education
Albino, J. E. N., Young, S. K., Neumann, L. M., Kramer, G. A., Andrieu, S. C., Henson, L. C., Horn, B. D., & Hendricson, W. D. (2008) · Journal of Dental Education · doi:10.1002/j.0022-0337.2008.72.12.tb04620.x · Cited by 215 · see citing works
Workplace-based assessment: effects of rater expertise
Govaerts, M., Schuwirth, L., van der Vleuten, C. P. M., & Muijtjens, A. M. M. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9250-7 · Cited by 198 · see citing works
Self-citation
Rater-Based Assessments as Social Judgments: Rethinking the Etiology of Rater Errors
Gingerich, A., Regehr, G., & Eva, K. W. (2011) · Academic Medicine · doi:10.1097/acm.0b013e31822a6cf8 · Cited by 196 · see citing works
A Tea-Steeping or i-Doc Model for Medical Education?
Hodges, B. D. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181f12f32 · Cited by 193 · see citing works
The wheel of competency assessment: Presenting quality criteria for competency assessment programs
Baartman, L. K. J., Bastiaens, T. J., Kirschner, P. A., & van der Vleuten, C. P. M. (2006) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2006.04.006 · Cited by 192 · see citing works
Seeing the same thing differently
Yeates, P., O'Neill, P. A., Mann, K. V., & Eva, K. W. (2012) · Advances in Health Sciences Education · doi:10.1007/s10459-012-9372-1 · Cited by 185 · see citing works
The Portfolio Approach to Competency-Based Assessment at the Cleveland Clinic Lerner College of Medicine
Dannefer, E. F., & Henson, L. C. (2007) · Academic Medicine · doi:10.1097/acm.0b013e31803ead30 · Cited by 181 · see citing works
Faculty staff perceptions of feedback to residents after direct observation of clinical skills
Kogan, J. R., Conforti, L. N., Bernabeo, E. C., Durning, S. J., Hauer, K. E., & Holmboe, E. S. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04137.x · Cited by 180 · see citing works
Towards a program of assessment for health professionals: from training into practice
Eva, K. W., Bordage, G., Campbell, C. B., Galbraith, R. M., Ginsburg, S., Holmboe, E. S., & Regehr, G. (2015) · Advances in Health Sciences Education · doi:10.1007/s10459-015-9653-6 · Cited by 181 · see citing works
Integrating professionalism into the curriculum: AMEE Guide No. 61
O’Sullivan, H., van Mook, W., Fewtrell, R., & Wass, V. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.655610 · Cited by 179 · see citing works
Script concordance testing: a review of published validity evidence
Lubarsky, S., Charlin, B., Cook, D. A., Chalk, C., & van der Vleuten, C. P. M. (2011) · Medical Education · doi:10.1111/j.1365-2923.2010.03863.x · Cited by 171 · see citing works
Advancing Resident Assessment in Graduate Medical Education
Swing, S. R., Clyman, S. G., Holmboe, E. S., & Williams, R. G. (2009) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-09-00010.1 · Cited by 167 · see citing works
Programmatic assessment and Kane’s validity perspective
Schuwirth, L., & van der Vleuten, C. P. M. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04098.x · Cited by 165 · see citing works
Self-citation
Conceptual framework for performance assessment: Competency, competence and performance in the context of assessments in healthcare – Deciphering the terminology
Khan, K., & Ramachandran, S. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.722707 · Cited by 162 · see citing works
Fidelity is not easy! Challenges and guidelines for assessing fidelity in complex interventions
Ginsburg, L. R., Hoben, M., Easterbrook, A., Anderson, R. A., Estabrooks, C. A., & Norton, P. (2021) · Trials · doi:10.1186/s13063-021-05322-5 · Cited by 157 · see citing works
International consensus statement on the assessment of interprofessional learning outcomes
Rogers, G. D., Thistlethwaite, J. E., Anderson, L., Dahlgren, M. A., Grymonpre, R., Moran, M., & Samarasekera, D. D. (2016) · Medical Teacher · doi:10.1080/0142159x.2017.1270441 · Cited by 156 · see citing works
Guidelines: The do’s, don’ts and don’t knows of direct observation of clinical skills in medical education
Kogan, J. R., Hatala, R. A., Hauer, K. E., & Holmboe, E. S. (2017) · Perspectives on Medical Education · doi:10.1007/s40037-017-0376-7 · Cited by 161 · see citing works
Challenges in multisource feedback: intended and unintended outcomes
Sargeant, J., Mann, K., Sinclair, D., Van der Vleuten, C., & Metsemakers, J. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02769.x · Cited by 151 · see citing works
Evolving concepts of assessment in a competency-based world
Harris, P., Bhanji, F., Topps, M. H., Ross, S., Lieberman, S. A., Frank, J. R., Snell, L. S., Sherbino, J., & Collaborators, O. B. O. T. I. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1315071 · Cited by 155 · see citing works
Assessment of Clinical Skills With Standardized Patients: State of the Art Revisited
Swanson, D. B., & van der Vleuten, C. P. M. (2013) · Teaching and Learning in Medicine · doi:10.1080/10401334.2013.842916 · Cited by 150 · see citing works
Reconceptualizing Variable Rater Assessments as Both an Educational and Clinical Care Problem
Kogan, J. R., Conforti, L. N., Iobst, W. F., & Holmboe, E. S. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000221 · Cited by 148 · see citing works
A history of assessment in medical education
Schuwirth, L., & van der Vleuten, C. P. M. (2020) · Advances in Health Sciences Education · doi:10.1007/s10459-020-10003-0 · Cited by 149 · see citing works
Self-citation
Mapping the dark matter of context: A conceptual scoping review.
Bates, J., & Ellaway, R. H. (2016) · Medical Education · doi:10.1111/medu.13034 · Cited by 144 · see citing works
LAP-VEGaS Practice Guidelines for Reporting of Educational Videos in Laparoscopic Surgery
Celentano, V., Smart, N. J., McGrath, J. S., Cahill, R. A., Spinelli, A., Obermair, A., Hasegawa, H., Lal, P., Almoudaris, A. M., Hitchins, C. R., et al. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002725 · Cited by 141 · see citing works
Curriculum structure, content, learning and assessment in European undergraduate dental education - update 2010
Manogue, M. R., McLoughlin, J., CHRISTERSSON, C. E., Delap, E., Lindh, C., Schoonheim‐Klein, M., & Plasschaert, A. (2011) · European Journal Of Dental Education · doi:10.1111/j.1600-0579.2011.00699.x · Cited by 142 · see citing works
The use of progress testing
Schuwirth, L., & van der Vleuten, C. P. M. (2012) · Perspectives on Medical Education · doi:10.1007/s40037-012-0007-2 · Cited by 142 · see citing works
Self-citation
Stakes in the eye of the beholder: an international study of learners’ perceptions within programmatic assessment
Schut, S., Driessen, E. W., van Tartwijk, J., van der Vleuten, C. P. M., & Heeneman, S. (2018) · Medical Education · doi:10.1111/medu.13532 · Cited by 141 · see citing works
A Handbook for Teaching and Learning in Higher Education
(2003) · Routledge eBooks · doi:10.4324/9780203416877 · Cited by 142 · see citing works
Assessment of competence and progressive independence in postgraduate clinical training
Dijksterhuis, M. G. K., Voorhuis, M., Teunissen, P. W., Schuwirth, L., ten Cate, O., Braat, D. D. M., & Scheele, F. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03509.x · Cited by 140 · see citing works
Self-citation
How to construct and implement script concordance tests: insights from a systematic review
Dory, V., Gagnon, R. J., Vanpee, D., & Charlin, B. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04211.x · Cited by 139 · see citing works
In-training assessment using direct observation of single-patient encounters: a literature review
Pelgrim, E. A. M., Kramer, A. W. M., Mokkink, H. G. A., van den Elsen, L., Grol, R. P. T. M., & van der Vleuten, C. P. M. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9235-6 · Cited by 141 · see citing works
A study on stress, stressors and coping strategies among Malaysian medical students.
Yusoff, M. S. B., Yee, L. Y., Wei, L. H., Meng, L. H., Bin, L. X., Siong, T. C., & Rahim, A. F. A. (2011) · International Journal of Students Research · doi:10.5549/ijsr.1.2.45-50 · Cited by 139 · see citing works
Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles
Heeneman, S., de Jong, L. H., Dawson, L. J., WILKINSON, T. J., Ryan, A., Tait, G. R., Rice, N. E., Torre, D. M., Freeman, A., & van der Vleuten, C. P. M. (2021) · Medical Teacher · doi:10.1080/0142159x.2021.1957088 · Cited by 150 · see citing works
Improving patient safety through training in non-technical skills
Flin, R. H., & Patey, R. E. (2009) · BMJ · doi:10.1136/bmj.b3595 · Cited by 137 · see citing works
Barriers to the uptake and use of feedback in the context of summative assessment
Harrison, C., Könings, K. D., Schuwirth, L., Wass, V., & van der Vleuten, C. P. M. (2014) · Advances in Health Sciences Education · doi:10.1007/s10459-014-9524-6 · Cited by 138 · see citing works
Self-citation
A new framework for designing programmes of assessment
Dijkstra, J., van der Vleuten, C. P. M., & Schuwirth, L. (2009) · Advances in Health Sciences Education · doi:10.1007/s10459-009-9205-z · Cited by 133 · see citing works
Self-citation
Should essays and other “open-ended”-type questions retain a place in written summative assessment in clinical medicine?
Hift, R. (2014) · BMC Medical Education · doi:10.1186/s12909-014-0249-2 · Cited by 136 · see citing works
Where the rubber meets the road — An integrative review of programmatic assessment in health care professions education
Schut, S., Maggio, L. A., Heeneman, S., van Tartwijk, J., van der Vleuten, C. P. M., & Driessen, E. W. (2020) · Perspectives on Medical Education · doi:10.1007/s40037-020-00625-w · Cited by 132 · see citing works
Key tips for teaching in the clinical setting
Burgess, A. W., van Diggele, C., Roberts, C., & Mellis, C. (2020) · BMC Medical Education · doi:10.1186/s12909-020-02283-2 · Cited by 138 · see citing works
Summative assessments are more powerful drivers of student learning than resource intensive teaching formats
Raupach, T., Brown, J., Anders, S., Hasenfuss, G. P., & Harendza, S. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-61 · Cited by 133 · see citing works
The Hidden Value of Narrative Comments for Assessment: A Quantitative Reliability Analysis of Qualitative Data
Ginsburg, S., van der Vleuten, C. P. M., & Eva, K. W. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001669 · Cited by 125 · see citing works
Towards competence‐based VET: dealing with the pitfalls
Biemans, H. J. A., Wesselink, R., Gulikers, J. T. M., Schaafsma, S. C. T., Verstegen, J. A. A. M., & Mulder, M. (2009) · Journal of Vocational Education and Training · doi:10.1080/13636820903194682 · Cited by 125 · see citing works
Beyond “formative”: assessments to enrich student learning
Kulasegaram, K. M., & Rangachari, P. K. (2018) · AJP Advances in Physiology Education · doi:10.1152/advan.00122.2017 · Cited by 143 · see citing works
The Top-Cited Articles in Medical Education
Azer, S. A. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000780 · Cited by 126 · see citing works
Twelve tips for implementing tools for direct observation of medical trainees’ clinical skills during patient encounters
Hauer, K. E., Holmboe, E. S., & Kogan, J. R. (2011) · Medical Teacher · doi:10.3109/0142159x.2010.507710 · Cited by 118 · see citing works
Perspective: Viewing “Strugglers” Through a Different Lens: How a Self-Regulated Learning Perspective Can Help Medical Educators With Assessment and Remediation
Durning, S. J., Cleary, T. J., Sandars, J. E., Hemmer, P. A., Kokotailo, P. K., & Artino, A. R. (2011) · Academic Medicine · doi:10.1097/acm.0b013e31820dc384 · Cited by 119 · see citing works
Assessing tomorrow's learners: In competency-based education only a radically different holistic method of assessment will work. Six things we could forget
Schuwirth, L., & Ash, J. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.787140 · Cited by 118 · see citing works
Self-citation
Driving Care Quality
Carraccio, C. L., Englander, R., Holmboe, E. S., & Kogan, J. R. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000985 · Cited by 116 · see citing works
Viewpoint: Competency in Surgical Residency Training: Defining and Raising the Bar
Bhatti, N. I., & Cummings, C. W. (2007) · Academic Medicine · doi:10.1097/acm.0b013e3180555bfb · Cited by 113 · see citing works
A hypothesis‐driven physical examination learning and assessment procedure for medical students: initial validity evidence
Yudkowsky, R., Otaki, J., Lowenstein, T., Riddle, J. M., Nishigori, H., & Bordage, G. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03379.x · Cited by 112 · see citing works
Critical Thinking in Health Professions Education: Summary and Consensus Statements of the Millennium Conference 2011
Huang, G. C., Newman, L. R., & Schwartzstein, R. M. (2014) · Teaching and Learning in Medicine · doi:10.1080/10401334.2013.857335 · Cited by 111 · see citing works
Learning Gain in Higher Education
McGrath, C. H., Guerin, B., Harte, E., Frearson, M., & Manville, C. (2021) · International Perspectives on Higher Education Research · doi:10.7249/rr996 · Cited by 87 · see citing works
Nutrition Competencies in Health Professionals' Education and Training: A New Paradigm
Kris‐Etherton, P. M., Akabas, S., Douglas, P., Kohlmeier, M., Laur, C. V., Lenders, C. M., Levy, M. D., Nowson, C. A., Ray, S., PRATT, C. A., et al. (2015) · Advances in Nutrition · doi:10.3945/an.114.006734 · Cited by 109 · see citing works
Perspective: Competencies, Outcomes, and Controversy—Linking Professional Activities to Competencies to Improve Resident Education and Practice
Jones, M. D., Rosenberg, A. A., Gilhooly, J. T., & Carraccio, C. L. (2010) · Academic Medicine · doi:10.1097/acm.0b013e31820442e9 · Cited by 109 · see citing works
Assessment of professionalism: A consolidation of current thinking
Goldie, J. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.714888 · Cited by 107 · see citing works
OECD Reviews of Evaluation and Assessment in Education: Australia 2011
Shewbridge, C., Herman, J. L., Donaldson, G., & Santiago, P. (2011) · DIAL (Catholic University of Leuven) · doi:10.1787/9789264116672-en · Cited by 107 · see citing works
Objective structured clinical examination (OSCE) revisited
Gupta, P., Dewan, P., & Singh, T. (2010) · Indian Pediatrics · doi:10.1007/s13312-010-0155-6 · Cited by 101 · see citing works
Challenges for educationalists
Schuwirth, L., & van der Vleuten, C. P. M. (2006) · BMJ · doi:10.1136/bmj.38952.701875.94 · Cited by 101 · see citing works
Self-citation
The Development of Entrustable Professional Activities for Internal Medicine Residency Training
Caverzagie, K. J., Cooney, T. G., Hemmer, P. A., & Berkowitz, L. R. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000564 · Cited by 100 · see citing works
How much do medical students know about infection control?
Mann, C. M., & Wood, A. (2006) · Journal of Hospital Infection · doi:10.1016/j.jhin.2006.06.030 · Cited by 98 · see citing works
Assessment in the context of problem-based learning
van der Vleuten, C. P. M., & Schuwirth, L. (2019) · Advances in Health Sciences Education · doi:10.1007/s10459-019-09909-1 · Cited by 109 · see citing works
Self-citation
Comparing students’ perceived and actual competence in higher vocational education
Baartman, L. K. J., & Ruijs, L. (2011) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2011.553274 · Cited by 99 · see citing works
Validity, reliability, feasibility and satisfaction of the Mini-Clinical Evaluation Exercise (Mini-CEX) for cardiology residency training
de Lima, A. A., Barrero, C., Baratta, S., Costa, Y. C., Bortman, G., Carabajales, J., Conde, D. G., Galli, A., Degrange, G., & van der Vleuten, C. P. M. (2007) · Medical Teacher · doi:10.1080/01421590701352261 · Cited by 98 · see citing works
Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program
Mišković, D., Wyles, S. M., Carter, F., Coleman, M., & Hanna, G. B. (2010) · Surgical Endoscopy · doi:10.1007/s00464-010-1329-y · Cited by 94 · see citing works
What programmatic assessment in medical education can learn from healthcare
Schuwirth, L., van der Vleuten, C. P. M., & Durning, S. J. (2017) · Perspectives on Medical Education · doi:10.1007/s40037-017-0345-1 · Cited by 94 · see citing works
Self-citation
The Illusion of Competency Versus the Desirability of Expertise: Seeking a Common Standard for Support Professions in Sport
Collins, D. J., Burke, V., Martindale, A., & Cruickshank, A. (2014) · Sports Medicine · doi:10.1007/s40279-014-0251-1 · Cited by 94 · see citing works
Doctor performance assessment in daily practise: does it help doctors or not? A systematic review
Overeem, K., Faber, M. J., Arah, O. A., Elwyn, G. J., Lombarts, K. M. J. M. H., Wollersheim, H. C. H., & Grol, R. P. T. M. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02897.x · Cited by 94 · see citing works
Very‐short‐answer questions: reliability, discrimination and acceptability
Sam, A. H., Field, S. M., Collares, C. F., van der Vleuten, C. P. M., Wass, V., Melville, C. R., Harris, J., & Meeran, K. M. (2018) · Medical Education · doi:10.1111/medu.13504 · Cited by 97 · see citing works
Measuring professional identity formation early in medical school
Kalet, A. L., Buckvar-Keltz, L., Harnik, V., Monson, V. E., Hubbard, S. M., Crowe, R., Song, H. S., & Yingling, S. L. (2016) · Medical Teacher · doi:10.1080/0142159x.2017.1270437 · Cited by 90 · see citing works
Tools for Measuring Clinical Ultrasound Competency: Recommendations From the Ultrasound Competency Work Group
Damewood, S. C., Leo, M. M., Bailitz, J. M., Gottlieb, M. A., Liu, R., Hoffmann, B., & Gaspari, R. (2019) · AEM Education and Training · doi:10.1002/aet2.10368 · Cited by 87 · see citing works

AN OVERVIEW OF ASSESSMENT IN MEDICAL EDUCATION: RELIABILITY, VALIDITY, EDUCATIONAL CONSEQUENCES
van der Vleuten, P. C. (2002) · Tıp Eğitimi Dünyası
How to Design a Useful Test
Schuwirth, L., & van der Vleuten, C. P. M. (2018) · Wiley eBooks · doi:10.1002/9781119373780.ch20 · Cited by 24 · see citing works
Self-citation
A new framework for designing programmes of assessment
Dijkstra, J., van der Vleuten, C. P. M., & Schuwirth, L. (2009) · Advances in Health Sciences Education · doi:10.1007/s10459-009-9205-z · Cited by 133 · see citing works
Self-citation
Medical Education, Assessment in
Govaerts, M. (2014) · Wiley eBooks · doi:10.1002/9781118410868.wbehibs426 · Cited by 2 · see citing works
Assessment of Professional Competence
Tigelaar, D. E. H., & van der Vleuten, C. P. M. (2014) · Springer international handbooks of education · doi:10.1007/978-94-017-8902-8_45 · Cited by 23 · see citing works

Authors: Van Der Vleuten, C. P. M., & Schuwirth, L. W. T.
Affiliations: Maastricht University (NL)
ORCID: Schuwirth, Lambert W T
Venue: Medical Education
Publisher: Wiley
Published: 2005-02-23
Volume: 39
Issue: 3
Pages: 309-317
ISSN: 0308-0110 (ISSN-L)
1365-2923
DOI: 10.1111/j.1365-2929.2005.02094.x
OpenAlex: W2163175447
Cited by: 1,268
References: 44
Citations by year: 2026: 87 · 2025: 46 · 2024: 51 · 2023: 66 · 2022: 66 · 2021: 92 · 2020: 63 · 2019: 76 · 2018: 59 · 2017: 66 · 2016: 65 · 2015: 75 · 2014: 72 · 2013: 64 · 2012: 47
FWCI: 33.42
Type: journal article
Language: en
Open access: closed
Licence: onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations: Medical Education (publishedVersion)
PubMed (publishedVersion)
Northumbria Research Link (Northumbria University) (acceptedVersion)
Topics: Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
Social Sciences › Education › Student Assessment and Feedback
Social Sciences › Psychology › Social Psychology › Competency Development and Evaluation
Keywords: professional competence assessment, assessment methods, assessment reliability, assessment validity, sampling, authenticity, competency-based education
MeSH terms: Clinical Competence, Education, Medical, Undergraduate, Learning, Reproducibility of Results, Program Evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Assessing professional competence: from methods to programmes

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks