The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Assessing professional competence: from methods to programmes
Abstract
Introduction We use a utility model to illustrate that, firstly, selecting an assessment method involves context‐dependent compromises, and secondly, that assessment is not a measurement problem but an instructional design problem, comprising educational, implementation and resource aspects. In the model, assessment characteristics are differently weighted depending on the purpose and context of the assessment. Empirical and theoretical developments Of the characteristics in the model, we focus on reliability, validity and educational impact and argue that they are not inherent qualities of any instrument. Reliability depends not on structuring or standardisation but on sampling. Key issues concerning validity are authenticity and integration of competencies. Assessment in medical education addresses complex competencies and thus requires quantitative and qualitative information from different sources as well as professional judgement. Adequate sampling across judges, instruments and contexts can ensure both validity and reliability. Despite recognition that assessment drives learning, this relationship has been little researched, possibly because of its strong context dependence. Assessment as instructional design When assessment should stimulate learning and requires adequate sampling, in authentic contexts, of the performance of complex competencies that cannot be broken down into simple parts, we need to make a shift from individual methods to an integral programme, intertwined with the education programme. Therefore, we need an instructional design perspective. Implications for development and research Programmatic instructional design hinges on a careful description and motivation of choices, whose effectiveness should be measured against the intended outcomes. We should not evaluate individual methods, but provide evidence of the utility of the assessment programme as a whole.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In the new movement towards programmatic assessment, progress and learning from feedback is emphasized and built into the system [74, 75].”
From Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education. (2015) · cites this paper as [74]
Citations
Cites
The assessment of clinical skills/competence/performance
Miller, G. E. (1990) · Academic Medicine · doi:10.1097/00001888-199009000-00045 · Cited by 4,883
Defining and Assessing Professional Competence
Epstein, R. M. (2002) · JAMA · doi:10.1001/jama.287.2.226 · Cited by 3,114
Standards for educational and psychological testing
(1985) · PsycEXTRA Dataset · doi:10.1037/e500832009-001 · Cited by 1,493
Validity: on the meaningful interpretation of assessment data
Downing, S. M. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01594.x · Cited by 1,434
The interplay of evidence and consequences in the validation of performance assessments
Messick, S. (1994) · Educational Researcher · doi:10.3102/0013189x023002013 · Cited by 1,315
The assessment of professional competence: Developments, research and practical implications
Van Der Vleuten, C. P. M. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00596229 · Cited by 1,247
The use of self-, peer and co-assessment in higher education: A review
Dochy, F., Segers, M., & Sluijsmans, D. (1999) · Studies in Higher Education · doi:10.1080/03075079912331379935 · Cited by 1,206
Complex, Performance-Based Assessment: Expectations and Validation Criteria
Linn, R. L., Baker, E., & Dunbar, S. (1991) · Educational Researcher · doi:10.3102/0013189x020008015 · Cited by 1,043
The Mini-CEX: A Method for Assessing Clinical Skills
Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-6-200303180-00012 · Cited by 730
Regehr, G., MacRae, H., Reznick, R. K., & Szalay, D. (1998) · Academic Medicine · doi:10.1097/00001888-199809000-00020 · Cited by 712
The Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation
Norcini, J. J., Blank, L. L., Arnold, G. K., & Kimball, H. R. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-123-10-199511150-00008 · Cited by 526
The relationship between competence and performance: implications for assessing practice performance
Rethans, J.-J., Norcini, J. J., Maldonado, M. B., Blackmore, D. E., Jolly, B. C., LaDuca, T., Lew, S. R., Page, G. G., & Southgate, L. J. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01316.x · Cited by 405
SPECIAL ARTICLE: Cognitive, Social and Environmental Sources of Bias in Clinical Performance Ratings
Williams, R. G., Klamen, D. L., & McGaghie, W. C. (2003) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1504_11 · Cited by 392
Performance-Based Assessment: Lessons From the Health Professions
Swanson, D. B., Norman, G. R., & Linn, R. L. (1995) · Educational Researcher · doi:10.3102/0013189x024005005 · Cited by 287
Pitfalls in the pursuit of objectivity: issues of reliability
van der Vleuten, C. P. M., Norman, G. R., & de Graaff, E. (1991) · Medical Education · doi:10.1111/j.1365-2923.1991.tb00036.x · Cited by 285
Multisource feedback in the assessment of physician competencies
Lockyer, J. (2003) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340230103 · Cited by 284
Developments in outcome-based education
Harden, R. M. (2002) · Medical Teacher · doi:10.1080/01421590220120669 · Cited by 276
The rising cost of NIH-funded biomedical research?
Kennedy, T. J. (1990) · Academic Medicine · doi:10.1097/00001888-199002000-00001 · Cited by 262
Norcini, J. J. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01536.x · Cited by 244
Assessing health professionals
Crossley, J., Humphris, G. M., & Jolly, B. C. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01294.x · Cited by 209
Driessen, E. W., van der Vleuten, C. P. M., Schuwirth, L., van Tartwijk, J., & Vermunt, J. D. H. M. (2005) · Medical Education · doi:10.1111/j.1365-2929.2004.02059.x · Cited by 207
Self-citationThe use of clinical simulations in assessment
Schuwirth, L., & van der Vleuten, C. P. M. (2003) · Medical Education · doi:10.1046/j.1365-2923.37.s1.8.x · Cited by 205
Self-citationRam, Grol, Rethans, Schouten, Vleuten, & Kester (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00348.x · Cited by 170
The Quality of In-house Medical School Examinations
Józefowicz, R. F., Koeppen, B. M., Case, S. M., Galbraith, R. M., Swanson, D. B., & Glew, R. H. (2002) · Academic Medicine · doi:10.1097/00001888-200202000-00016 · Cited by 164
Hodges, B. D. (2003) · Medical Teacher · doi:10.1080/01421590310001002836 · Cited by 155
Changing education, changing assessment, changing research?
Schuwirth, L., & van der Vleuten, C. P. M. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.01851.x · Cited by 146
Self-citationGielen, S., Dochy, F. J. R. C., & Dierick, S. (2006) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-48125-1_3 · Cited by 120
Standardized or real patients to test clinical competence? The long case revisited
Wass, V., Jones, R. F., & van der Vleuten, C. P. M. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00928.x · Cited by 120
Norcini, J. J., Swanson, D. B., Grosso, L. J., & WEBSTER, G. D. (1985) · Medical Education · doi:10.1111/j.1365-2923.1985.tb01314.x · Cited by 112
ASSESSMENT OF CLINICAL COMPETENCE: WRITTEN AND COMPUTER‐BASED SIMULATIONS
Swanson, D. B., Norcini, J. J., & Grosso, L. J. (1987) · Assessment & Evaluation in Higher Education · doi:10.1080/0260293870120307 · Cited by 109
Developing professional judgment
Coles, C. R. (2002) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340220102 · Cited by 108
Wass, V., McGibbon, D., & van der Vleuten, C. P. M. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00929.x · Cited by 92
Assessing Studentsʼ Performances in a Competency-based Curriculum
Smith, S. R., Dollase, R. H., & Boss, J. A. (2003) · Academic Medicine · doi:10.1097/00001888-200301000-00019 · Cited by 88
Clinical Performance Assessments
PETRUSA, E. R. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_26 · Cited by 76
Portfolios – attempting to measure the unmeasurable?
Snadden, D. (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00446.x · Cited by 69
Adapting the Key Features Examination for a clinical clerkship
Hatala, R. A., & Norman, G. R. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01067.x · Cited by 53
Assessment of Clinical Performance: In-Training Evaluation
Turnbull, J., & van Barneveld, C. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_29 · Cited by 42
Reproducibility of clinical performance assessment in practice using incognito standardized patients
GORTER, S. L., Rethans, J., van der Heijde, D. M. F. M., Scherpbier, A. J. J. A., Houben, H., van der Vleuten, C. P. M., & van der Linden, S. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01296.x · Cited by 32
Mennin, S. P., & Kalishman, S. G. (1998) · Academic Medicine · doi:10.1097/00001888-199809000-00035 · Cited by 32
Gorter, J.-J. R. S. L. (2001) · Medical Teacher · doi:10.1080/014215931048 · Cited by 22
Optimising New Modes of Assessment: In search of Qualities and Standards
Segers, M. S. R., Dochy, F. J. R. C., & Cascallar, E. C. (2003) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-48125-1 · Cited by 21
The Influence of Testing Context and Clinical Rotation Order on Students’ OSCE Performance
Blaskiewicz, R., Park, R. S., Chibnall, J. T., & Powell, J. K. (2004) · Academic Medicine · doi:10.1097/00001888-200406000-00020 · Cited by 15
Managing the Curriculum and Managing Change
Davis, W. K., & White, C. B. (2002) · Springer international handbooks of education · doi:10.1007/978-94-010-0462-6_34 · Cited by 12
Reliability of the fellowship examination of the royal Australian college of general practitioners
Hays, R., Fabb, W. E., & van der VIeuten, C. P. M. (1995) · Teaching and Learning in Medicine · doi:10.1080/10401339509539709 · Cited by 11
Research works printed but not linked: 5
- Van Merriënboer JJG, 1997, Training Complex Cognitive Skills.
- Gibbs G, 2002, How Assessment Influences Student Learning – a Conceptual Overview:
- Epstein RM, 2001, Comprehensive Assessment Manual
- Swanson DB, 1987, Further Developments in Assessing Clinical Competence, 13
- Van Der Vleuten CPM, 1988, Reliability (Generalisability) of the Maastricht Skills Test.
Cited by
Sorted by citations
Programmatic assessment: From assessment of learning to assessment for learning
Schuwirth, L. W. T., & Van der Vleuten, C. P. M. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.565828 · Cited by 902
Self-citationA model for programmatic assessment fit for purpose.
van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652239 · Cited by 892
Self-citationten Cate, O., Chen, H. C., Hoff, R. G., Peters, H., Bok, H. G. J., & van der Schaaf, M. F. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1060308 · Cited by 851
Moore, D. E., Green, J. S., & Gallis, H. A. (2009) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.20001 · Cited by 743
Tools for Direct Observation and Assessment of Clinical Skills of Medical Trainees
Kogan, J. R., Holmboe, E. S., & Hauer, K. E. (2009) · JAMA · doi:10.1001/jama.2009.1365 · Cited by 613
Metadata
- Authors
- Van Der Vleuten, C. P. M., & Schuwirth, L. W. T.
- Affiliations
- Maastricht University (NL)
- Venue
- Medical Education
- Publisher
- Wiley
- Published
- 2005-02-23
- Volume
- 39
- Issue
- 3
- Pages
- 309-317
- ISSN
- 0308-0110 (ISSN-L)
- 1365-2923
- OpenAlex
- W2163175447
- Cited by
- 1,268
- References
- 44
- Citations by year
- 2026: 87 · 2025: 46 · 2024: 51 · 2023: 66 · 2022: 66 · 2021: 92 · 2020: 63 · 2019: 76 · 2018: 59 · 2017: 66 · 2016: 65 · 2015: 75 · 2014: 72 · 2013: 64 · 2012: 47
- FWCI
- 33.42
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Education (publishedVersion)
- PubMed (publishedVersion)
- Northumbria Research Link (Northumbria University) (acceptedVersion)
- Topics
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Competency Development and Evaluation
- Keywords
- professional competence assessment, assessment methods, assessment reliability, assessment validity, sampling, authenticity, competency-based education
- MeSH terms
- Clinical Competence, Education, Medical, Undergraduate, Learning, Reproducibility of Results, Program Evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review