What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research

Driessen, Erik; Scheele, Fedde

doi:10.3109/0142159x.2013.798403

Cited by 110Innovations in Medical EducationClinical Reasoning and Diagnostic SkillsStudent Assessment and Feedback

What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research

Driessen, Erik, Scheele, Fedde·2013·Medical Teacher

DOI 10.3109/0142159x.2013.798403

Open this paper in CoChat — free account

Abstract

Workplace-based assessment is more commonly given a lukewarm than a warm welcome by its prospective users. In this article, we summarise the workplace-based assessment literature as well as our own experiences with workplace-based assessment to derive lessons that can facilitate acceptance of workplace-based assessment in postgraduate specialty training. We propose to shift the emphasis in workplace-based assessment from assessment of trainee performance to the learning of trainees. Workplace-based assessment should focus on supporting supervisors in taking entrustment decisions by complementing their "gut feeling" with information from assessments and focus less on assessment and testability. One of the most stubborn problems with workplace-based assessment is the absence of observation of trainees and the lack of feedback based on observations. Non-standardised observations are used to organise feedback. To make these assessments meaningful for learning, it is essential that they are not perceived as summative by their users, that they provide narrative feedback for the learner and that there is a form of facilitation that helps to integrate the feedback in trainees' self-assessments.

Integrity signals

Registry

Crossref · 10.3109/0142159x.2013.798403

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Grades have the potential to demotivate or to reduce effort.
“Because of the potential for grades to demotivate or to reduce effort, it has been suggested that it might be wisest to avoid giving grades except when formally assessing the learner (in infrequent ‘high stakes’ assessments) [58].”
From Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education. (2015) · cites this paper as [58]

The Power of Feedback
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · see citing works
Focus on Formative Feedback
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631 · see citing works
Strangers to ourselves: discovering the adaptive unconscious
(2003) · Choice Reviews Online · doi:10.5860/choice.40-3712 · Cited by 1,641 · see citing works
Immune neglect: A source of durability bias in affective forecasting.
Gilbert, D. T., Pinel, E. C., Wilson, T. D., Blumberg, S. J., & Wheatley, T. P. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.75.3.617 · Cited by 1,547 · see citing works
Viewpoint: Competency-Based Postgraduate Training: Can We Bridge the Gap between Theory and Clinical Practice?
ten Cate, O., & Scheele, F. (2007) · Academic Medicine · doi:10.1097/acm.0b013e31805559c7 · Cited by 1,162 · see citing works
Self-citation
Workplace-based assessment as an educational tool: AMEE Guide No. 31
Norcini, J., & Burch, V. (2007) · Medical Teacher · doi:10.1080/01421590701775453 · Cited by 984 · see citing works
A model for programmatic assessment fit for purpose.
van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652239 · Cited by 892 · see citing works
Self-citation
State of the science in health professional education: effective feedback
Archer, J. C. (2010) · Medical Education · doi:10.1111/j.1365-2923.2009.03546.x · Cited by 783 · see citing works
Impact of workplace based assessment on doctors' education and performance: a systematic review
Miller, A. P., & Archer, J. C. (2010) · BMJ · doi:10.1136/bmj.c5064 · Cited by 462 · see citing works
Failure to Fail: The Perspectives of Clinical Supervisors
Dudek, N. L., Marks, M. B., & Regehr, G. (2005) · Academic Medicine · doi:10.1097/00001888-200510001-00023 · Cited by 422 · see citing works
The assessment of professional competence: building blocks for theory development
van der Vleuten, C. P. M., Schuwirth, L., Scheele, F., Driessen, E. W., & Hodges, B. D. (2010) · Best Practice & Research Clinical Obstetrics & Gynaecology · doi:10.1016/j.bpobgyn.2010.04.001 · Cited by 341 · see citing works
Self-citation
Portfolios in medical education: why do they meet with mixed success? A systematic review
Driessen, E. W., van Tartwijk, J., van der Vleuten, C. P. M., & Wass, V. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02944.x · Cited by 337 · see citing works
Self-citation
Implementing workplace‐based assessment across the medical specialties in the United Kingdom
Wilkinson, J. R., Crossley, J. G. M., Wragg, A., Mills, P., Cowan, G., & Wade, W. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03010.x · Cited by 272 · see citing works
Workplace-based assessment as an educational tool
Norcini, J. J., Burch, V. C., & Gibbs, T. J. (2007) · Medical Entomology and Zoology · Cited by 266 · see citing works
Learning from clinical work: the roles of learning cues and credibility judgements
Watling, C., Driessen, E., van der Vleuten, C. P. M., & Lingard, L. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04126.x · Cited by 261 · see citing works
Self-citation
The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12
Tochel, C., Haig, A., Hesketh, A., Cadzow, A., Beggs, K., Colthart, I., & Peacock, H. (2009) · Medical Teacher · doi:10.1080/01421590902883056 · Cited by 251 · see citing works
Medical Education: Theory and Practice
Dornan, T., Mann, K. V., Scherpbier, A. J. J. A., & Spencer, J. (2010) · Newcastle University ePrints (Newcastle Univesity) · doi:10.25405/ncl.34176165 · Cited by 227 · see citing works
Introducing competency-based postgraduate medical education in the Netherlands
Scheele, F., Teunissen, P. W., van Luijk, S. J., Heineman, E., Fluit, C. R. M. G., Mulder, H., Meininger, A., Wijnen‐Meijer, M., Glas, G. G., Sluiter, H. E., et al. (2008) · Medical Teacher · doi:10.1080/01421590801993022 · Cited by 220 · see citing works
Self-citation
Trust, competence, and the supervisor's role in postgraduate training
ten Cate, O. (2006) · BMJ · doi:10.1136/bmj.38938.407569.94 · Cited by 218 · see citing works
Good questions, good answers: construct alignment improves the performance of workplace‐based assessment scales
Crossley, J., Johnson, G., Booth, J., & Wade, W. B. (2011) · Medical Education · doi:10.1111/j.1365-2923.2010.03913.x · Cited by 217 · see citing works
Making sense of work‐based assessment: ask the right questions, in the right way, about the right things, of the right people
Crossley, J., & Jolly, B. C. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04166.x · Cited by 212 · see citing works
Strangers to Ourselves: Discovering the Adaptive Unconscious
Wilson, T. D. (2004) · doi:10.2307/j.ctvjghvsk · Cited by 201 · see citing works
The impact of mentoring during postgraduate training on doctors’ career success
Stamm, M., & Buddeberg-Fischer, B. (2011) · Medical Education · doi:10.1111/j.1365-2923.2010.03857.x · Cited by 160 · see citing works
The self critical doctor: helping students become more reflective
Driessen, E. W., van Tartwijk, J., & Dornan, T. (2008) · BMJ · doi:10.1136/bmj.39503.608032.ad · Cited by 153 · see citing works
Self-citation
Assessment of competence and progressive independence in postgraduate clinical training
Dijksterhuis, M. G. K., Voorhuis, M., Teunissen, P. W., Schuwirth, L., ten Cate, O., Braat, D. D. M., & Scheele, F. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03509.x · Cited by 140 · see citing works
Self-citation
The use of programmatic assessment in the clinical workplace: A Maastricht case report
Driessen, E. W., van Tartwijk, J., Govaerts, M., Teunissen, P., & van der Vleuten, C. P. M. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652242 · Cited by 116 · see citing works
Self-citation
Perspective: Competencies, Outcomes, and Controversy—Linking Professional Activities to Competencies to Improve Resident Education and Practice
Jones, M. D., Rosenberg, A. A., Gilhooly, J. T., & Carraccio, C. L. (2010) · Academic Medicine · doi:10.1097/acm.0b013e31820442e9 · Cited by 109 · see citing works
Composite reliability of a workplace-based assessment toolbox for postgraduate medical education
van Loon, J., Overeem, K., Donkers, J. H. H. L. M., van der Vleuten, C. P. M., & Driessen, E. W. (2013) · Advances in Health Sciences Education · doi:10.1007/s10459-013-9450-z · Cited by 104 · see citing works
Self-citation
A critical time for medical education: the perils of competence-based reform of the curriculum
Malone, K., & Supri, S. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9247-2 · Cited by 101 · see citing works
British surgeons' experiences of mandatory online workplace-based assessment
Pereira, E. A. C., & Dean, B. J. F. (2009) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2009.080398 · Cited by 81 · see citing works
Student perspectives on assessment: Experience in a competency-based portfolio system
Altahawi, F. F., Sisk, B. A., Poloskey, S. L., Hicks, C. W., & Dannefer, E. F. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652243 · Cited by 80 · see citing works
Workplace‐based assessments of junior doctors: do scores predict training difficulties?
Mitchell, C., Bhat, S., Herbert, A., & Baker, P. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04056.x · Cited by 53 · see citing works
How ratings vary by staff group in multi‐source feedback assessment of junior doctors
Bullock, A. D., HASSELL, A. B., Markham, W. A., Wall, D. W., & Whitehouse, A. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03333.x · Cited by 53 · see citing works
Training in hospitals: what do GP specialist trainees think of workplace-based assessments?
Sabey, A., & Harris, M. F. (2011) · Education for Primary Care · doi:10.1080/14739879.2011.11493974 · Cited by 47 · see citing works
Editorial – Checklists vs. Ratings, the Illusion of Objectivity, the Demise of Skills and the Debasement of Evidence
Norman, G. R. (2005) · Advances in Health Sciences Education · doi:10.1007/s10459-005-4723-9 · Cited by 43 · see citing works
Get yourself a mentor
Driessen, E. W., Overeem, K., & van der Vleuten, C. P. M. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.03948.x · Cited by 30 · see citing works
Self-citation
From theory to actual practice: Creation and application of milestones in an internal medicine residency program, 2004–2010
Meade, L. B., Borden, S., McArdle, P. J., Rosenblum, M. J., Picchioni, M., & Hinchey, K. T. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.689441 · Cited by 24 · see citing works
Invited commentary: The ACGME “Outcomes Project”
Pellegrini, C. A. (2002) · Surgery · doi:10.1067/msy.2002.120666 · Cited by 21 · see citing works
Quality assurance of medical education in the Netherlands: programme or systems accreditation?
Hillen, H. F. P. (2010) · Tijdschrift voor Medisch Onderwijs · doi:10.1007/s12507-010-0013-5 · Cited by 12 · see citing works
Relationship between increased serum activity of alanine aminotransferase and metabolic parameters in non-alcoholic population
Wang, N.-S. (2008) · doi:10.3760/cma.j.issn.1671-7368.2008.11.007 · Cited by 1 · see citing works

Flagged first, then by citations

1 of the 41 references couldn’t be matched to a known paper.

Sorted by citations

Assessment, feedback and the alchemy of learning
Watling, C., & Ginsburg, S. (2018) · Medical Education · doi:10.1111/medu.13645 · Cited by 434 · see citing works
Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education.
Lefroy, J., Watling, C., Teunissen, P. W., & Brand, P. (2015) · Perspectives on Medical Education · doi:10.1007/s40037-015-0231-7 · Cited by 347 · see citing works
Entrustability Scales
Rekman, J. F., Gofton, W. T., Dudek, N. L., Gofton, T., & Hamstra, S. J. (2015) · Academic Medicine · doi:10.1097/acm.0000000000001045 · Cited by 238 · see citing works
The impact of programmatic assessment on student learning: theory versus practice
Heeneman, S., Pool, A. O., Schuwirth, L., van der Vleuten, C. P. M., & Driessen, E. W. (2015) · Medical Education · doi:10.1111/medu.12645 · Cited by 213 · see citing works
Self-citation
Identifying coaching skills to improve feedback use in postgraduate medical education
Armson, H., Lockyer, J. M., Zetkulic, M., Könings, K. D., & Sargeant, J. (2019) · Medical Education · doi:10.1111/medu.13818 · Cited by 115 · see citing works
Experiences with EPAs, potential benefits and pitfalls
van Loon, K. A., Driessen, E. W., Teunissen, P. W., & Scheele, F. (2014) · Medical Teacher · doi:10.3109/0142159x.2014.909588 · Cited by 112 · see citing works
Self-citation
R2C2 in action: Testing an evidence-based model to facilitate feedback and coaching in residency.
Sargeant, J., Mann, K., Manos, S., Epstein, I., Warren, A., Shearer, C., & Boudreau, M. (2017) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-16-00398.1 · Cited by 98 · see citing works
Feedback in Medical Education: An Evidence-based Guide to Best Practices from the Council of Residency Directors in Emergency Medicine
Natesan, S. M., Jordan, J., Sheng, A. Y., Carmelli, G., Barbas, B., King, A., Gore, K., Estes, M., & Gottlieb, M. A. (2023) · Western Journal of Emergency Medicine · doi:10.5811/westjem.56544 · Cited by 99 · see citing works
Entrusting Observable Practice Activities and Milestones Over the 36 Months of an Internal Medicine Residency
Warm, E. J., Held, J. D., Hellmann, M., Kelleher, M., Kinnear, B., Lee, C., O’Toole, J. K., Mathis, B. R., Mueller, C. V., Sall, D. R., et al. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001292 · Cited by 89 · see citing works
Managing tensions in assessment: moving beyond either–or thinking
Govaerts, M., van der Vleuten, C. P. M., & Holmboe, E. S. (2018) · Medical Education · doi:10.1111/medu.13656 · Cited by 88 · see citing works
Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review
Anderson, H. L. K., Kurtz, J. B., & West, D. C. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004366 · Cited by 69 · see citing works
Constructing a Shared Mental Model for Faculty Development for the Core Entrustable Professional Activities for Entering Residency
Favreau, M. A., Tewksbury, L. R., Lupi, C. S., Cutrer, W. B., Jokela, J. A., & Yarris, L. M. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001511 · Cited by 66 · see citing works
Multiple‐role mentoring: mentors’ conceptualisations, enactments and role conflicts
Meeuwissen, S. N. E., Stalmeijer, R. E., & Govaerts, M. (2019) · Medical Education · doi:10.1111/medu.13811 · Cited by 62 · see citing works
Feedback-giving behaviour in performance evaluations during clinical clerkships
Bok, H. G. J., Jaarsma, D. A. D. C., Spruijt, A., van Beukelen, P., van der Vleuten, C. P. M., & Teunissen, P. W. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1017448 · Cited by 54 · see citing works
ADEA‐ADEE Shaping the Future of Dental Education III
Tonni, I., Gadbury‐Amyot, C. C., Govaerts, M., ten Cate, O., Davis, J., García, L. T., & Valachovic, R. W. (2020) · Journal of Dental Education · doi:10.1002/jdd.12024 · Cited by 53 · see citing works
The Reliability of Multisource Feedback in Competency-Based Assessment Programs
van Loon, J., Overeem, K., Govaerts, M., Verhoeven, B. H., van der Vleuten, C. P. M., & Driessen, E. W. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000763 · Cited by 52 · see citing works
Self-citation
Perceptions of purpose, value, and process of the mini-Clinical Evaluation Exercise in anesthesia training
Castanelli, D. J., Jowsey, T., Chen, Y., & Weller, J. M. (2016) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-016-0740-9 · Cited by 50 · see citing works
Thresholds of Principle and Preference
Apramian, T., Cristancho, S., Watling, C., Ott, M. C., & Lingard, L. A. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000909 · Cited by 50 · see citing works
Patterns of direct observation and their impact during residency: general practice supervisors’ views
Rietmeijer, C. B. T., Huisman, D., Blankenstein, A. H., de Vries, H., Scheele, F., Kramer, A. W. M., & Teunissen, P. W. (2018) · Medical Education · doi:10.1111/medu.13631 · Cited by 41 · see citing works
Self-citation
Supervised learning events in the Foundation Programme: a UK-wide narrative interview study
Rees, C. E., Cleland, J., Dennis, A. A., Kelly, N. A. A., Mattick, K. L., & Monrouxe, L. V. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005980 · Cited by 41 · see citing works
Evidence-Informed Facilitated Feedback: The R2C2 Feedback Model
Sargeant, J. M., Armson, H., Driessen, E. W., Holmboe, E. S., Könings, K. D., Lockyer, J., Lynn, L. A., Mann, K. V., Ross, K., Silver, I., et al. (2016) · MedEdPORTAL · doi:10.15766/mep_2374-8265.10387 · Cited by 39 · see citing works
Self-citation
A novel workplace-based assessment for competency-based decisions and learner feedback
Hicks, P. J., Margolis, M. J., Carraccio, C. L., Clauser, B. E., Donnelly, K. M., Fromme, H. B., Gifford, K. A., Poynter, S. E., Schumacher, D. J., Schwartz, A., et al. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1461204 · Cited by 40 · see citing works
Factors Influencing Mini-CEX Rater Judgments and Their Practical Implications: A Systematic Literature Review
Lee, V., Brain, K., & Martin, J. A. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001537 · Cited by 39 · see citing works
The Development of Professional Identity
Cruess, S. R., & Cruess, R. L. (2018) · Wiley eBooks · doi:10.1002/9781119373780.ch17 · Cited by 37 · see citing works
Validity Evidence for Assessing Entrustable Professional Activities During Undergraduate Medical Education
Violato, C., Cullen, M. J., Englander, R., Murray, K. E., Hobday, P. M., Borman-Shoap, E. C., & Ersan, O. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004090 · Cited by 35 · see citing works
Relatively speaking: contrast effects influence assessors’ scores and narrative feedback
Yeates, P., Cardell, J., Byrne, G., & Eva, K. W. (2015) · Medical Education · doi:10.1111/medu.12777 · Cited by 34 · see citing works
From opening the ‘black box’ to looking behind the curtain: cognition and context in assessor-based judgements
Lee, V., Brain, K., & Martin, J. A. (2018) · Advances in Health Sciences Education · doi:10.1007/s10459-018-9851-0 · Cited by 30 · see citing works
Fostering Dental Students’ Academic Achievements and Reflection Skills Through Clinical Peer Assessment and Feedback
Tricio, J., Woolford, M. J., & Escudier, M. P. (2016) · Journal of Dental Education · doi:10.1002/j.0022-0337.2016.80.8.tb06171.x · Cited by 28 · see citing works
Assessing family medicine trainees – what can we learn from the European neighbours?
Flum, E., Maagaard, R., Godycki-Ćwirko, M., Scarborough, N., Scherpbier, N. D., Ledig, T., Roos, M., & Steinhäuser, J. (2015) · Europe PMC (PubMed Central) · doi:10.3205/zma000963 · Cited by 26 · see citing works
A model of engagement in reflective writing-based portfolios: Interactions between points of vulnerability and acts of adaptability
Arntfield, S., Parlett, B., Meston, C. N., Apramian, T., & Lingard, L. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1009426 · Cited by 23 · see citing works
The influence of students’ prior clinical skills and context characteristics on mini-CEX scores in clerkships – a multilevel analysis
Rogausch, A., Beyeler, C., Montagne, S., Jucker-Kupper, P., Berendonk, C., Huwendiek, S., Gemperli, A., & Himmel, W. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0490-3 · Cited by 23 · see citing works
What difficulties do faculty members face when conducting workplace-based assessments in undergraduate clerkships?
Daelmans, H. E. M., der Vossen, M. M., Croiset, G., & Kusurkar, R. A. (2016) · International Journal of Medical Education · doi:10.5116/ijme.5689.3c7f · Cited by 23 · see citing works
Obstetrics and gynaecology training in Europe needs a next step
Scheele, F., Novak, Ž., Vetter, K. J., Caccia, N. C., & Goverde, A. J. (2014) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2014.04.014 · Cited by 22 · see citing works
Self-citation
Entrustables and Entrustment: Through the Looking Glass at the Clinical Making of a Nurse Practitioner
Bargagliotti, L. A., & Davenport, D. (2017) · The Journal for Nurse Practitioners · doi:10.1016/j.nurpra.2017.05.018 · Cited by 22 · see citing works
Examining the Effects of Narrative Commentary on Evaluators’ Summative Assessments of Resident Performance
Lefebvre, C., Hiestand, B. C., Glass, C., Masneri, D. A., Hosmer, K., Hunt, M. R., & Hartman, N. D. (2018) · Evaluation & the Health Professions · doi:10.1177/0163278718820415 · Cited by 22 · see citing works
Implementation of workplace-based assessment in general practice
Prins, S. H., Brøndt, S. G., & Malling, B. V. (2019) · Education for Primary Care · doi:10.1080/14739879.2019.1588788 · Cited by 22 · see citing works
Resident-Sensitive Quality Measures in the Pediatric Emergency Department: Exploring Relationships With Supervisor Entrustment and Patient Acuity and Complexity
Schumacher, D. J., Holmboe, E. S., Carraccio, C. L., Martini, A., van der Vleuten, C. P. M., Busari, J. O., Sobolewski, B., & Byczkowski, T. L. (2020) · Academic Medicine · doi:10.1097/acm.0000000000003242 · Cited by 21 · see citing works
Key Factors in Clinical Competency Committee Members’ Decisions Regarding Residents’ Readiness to Serve as Supervisors: A National Study
Schumacher, D. J., Martini, A., Bartlett, K. W., King, B., Calaman, S., Garfunkel, L. C., Elliott, S. P., Frohna, J. G., Schwartz, A., & Michelson, C. D. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002469 · Cited by 20 · see citing works
The mini‐clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?
Montagne, S., Rogausch, A., Gemperli, A., Berendonk, C., Jucker-Kupper, P., & Beyeler, C. (2014) · Medical Education · doi:10.1111/medu.12513 · Cited by 20 · see citing works
Encouraging residents’ professional development and career planning: the role of a development-oriented performance assessment
Dijkhuizen, K., Bustraan, J., de Beaufort, A. J., Velthuis, S. I., Driessen, E. W., & Lith, J. M. M. V. (2018) · BMC Medical Education · doi:10.1186/s12909-018-1317-9 · Cited by 19 · see citing works
Self-citation
“Staying in the Game”: How Procedural Variation Shapes Competence Judgments in Surgical Education
Apramian, T., Cristancho, S., Watling, C., Ott, M. C., & Lingard, L. A. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001364 · Cited by 19 · see citing works
Lessons learnt from a pilot of assessment for learning
Cho, S. P., Parry, D., & Wade, W. B. (2014) · Clinical Medicine · doi:10.7861/clinmedicine.14-6-577 · Cited by 18 · see citing works
Striving to thrive or striving to survive: Professional identity constructions of medical trainees in clinical assessment activities
Liao, K., Ajjawi, R., Peng, C., Jenq, C., & Monrouxe, L. V. (2023) · Medical Education · doi:10.1111/medu.15152 · Cited by 17 · see citing works
Entrustable professional activities (EPAs) for postgraduate competency based intensive care medicine training in the Netherlands: The next step towards excellence in intensive care medicine training
van Bockel, E. A. P., Walstock, P. A., van Mook, W., Arbous, M. S., Tepaske, R., van Hemel, T., Müller, M. C. A., Delwig, H., & Tulleken, J. E. (2019) · Journal of Critical Care · doi:10.1016/j.jcrc.2019.09.012 · Cited by 17 · see citing works
Validation of a novel intraoperative assessment tool: The Surgical Procedure Feedback Rubric
Toprak, A., Luhanga, U., Jones, S., WINTHROP, A. L., & McEwen, L. A. (2015) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2015.08.032 · Cited by 16 · see citing works
Entrustable Professional Activities and Learning: The Postgraduate Trainee Perspective
Stephan, A., Cheung, G., & van der Vleuten, C. P. M. (2022) · Academic Psychiatry · doi:10.1007/s40596-022-01712-2 · Cited by 15 · see citing works
Structure, governance and delivery of specialist training programs in periodontology and implant dentistry
Goldstein, M., Donos, N., Teughels, W., Gkranias, N., Temmerman, A., Derks, J., Kuru, B. E., Carra, M. C., Castro, A. B. S., Dereka, X. E., et al. (2024) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.14033 · Cited by 15 · see citing works
Assessment methods in medical specialist assessments in the DACH region – overview, critical examination and recommendations for further development
Thiessen, N., Fischer, M. R., & Huwendiek, S. (2019) · Europe PMC (PubMed Central) · doi:10.3205/zma001286 · Cited by 16 · see citing works
Assessing professional competence: a critical review of the Annual Review of Competence Progression
Woolf, K., Page, M., & Viney, R. (2019) · Journal of the Royal Society of Medicine · doi:10.1177/0141076819848113 · Cited by 14 · see citing works
Overshadowed by Assessment: Understanding Trainee and Supervisor Perspectives on the Oral Case Presentation in Internal Medicine Workplace-Based Assessment
Melvin, L., Rassos, J., Panisko, D. M., Driessen, E. W., Kulasegaram, K. M., & Kuper, A. (2018) · Academic Medicine · doi:10.1097/acm.0000000000002451 · Cited by 13 · see citing works
Self-citation
Development and Evaluation of a New Formative Assessment of Surgical Performance
Davies, R. M., Hadfield-Law, L., & Turner, P. G. (2018) · Journal of surgical education · doi:10.1016/j.jsurg.2018.02.004 · Cited by 12 · see citing works
Role for Assessment in Maintenance of Certification: Physician Perceptions of Assessment
Lockyer, J., Horsley, T., Zeiter, J., & Campbell, C. B. (2015) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.21265 · Cited by 12 · see citing works
Using patients’ charts to assess medical trainees in the workplace: A systematic review
Al-Wassia, H. K., Al‐Wassia, R., Shihata, S., Park, Y. S., & Tekian, A. S. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1006599 · Cited by 12 · see citing works
Foreword: The Next Era of Assessment and Precision Education
Schumacher, D. J., Santen, S. A., Pugh, C. M., & Burk‐Rafel, J. (2023) · Academic Medicine · doi:10.1097/acm.0000000000005609 · Cited by 11 · see citing works
‘Next steps are…’: An exploration of coaching and feedback language in EPA assessment comments
Day, L. B., Rassos, J., Billick, M. J., & Ginsburg, S. (2022) · Medical Teacher · doi:10.1080/0142159x.2022.2098098 · Cited by 11 · see citing works
Advancing Competency-Based Medical Education Through Assessment and Feedback in Breast Imaging
Holbrook, A. I., & Kasales, C. J. (2019) · Academic Radiology · doi:10.1016/j.acra.2019.04.017 · Cited by 11 · see citing works
Validating written feedback in clinical formative assessment
Page, M., Gardner, J., & Booth, J. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2019.1691974 · Cited by 10 · see citing works
What happens under the flag of direct observation, and how that matters: A qualitative study in general practice residency
Rietmeijer, C. B. T., Blankenstein, A. H., Huisman, D., van der Horst, H. E., Kramer, A. W. M., de Vries, H., Scheele, F., & Teunissen, P. W. (2021) · Medical Teacher · doi:10.1080/0142159x.2021.1898572 · Cited by 11 · see citing works
Self-citation
Implementing Core Entrustable Professional Activities in Undergraduate Medical Education: A Psychometric Study
Violato, C., Englander, R., Dale, E. D., & Gauer, J. L. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005907 · Cited by 10 · see citing works
The Future of High-Quality Care Depends on Better Assessment of Physician Performance
Schumacher, D. J., van der Vleuten, C. P. M., & Carraccio, C. L. (2016) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2016.2715 · Cited by 9 · see citing works
Profiling postgraduate workplace-based assessment implementation in Ireland: a retrospective cohort study
Barrett, A., Galvin, R., Steinert, Y., Scherpbier, A. J. J. A., O’Shaughnessy, A., Walsh, G., & Horgan, M. M. (2016) · SpringerPlus · doi:10.1186/s40064-016-1748-x · Cited by 9 · see citing works
Integrating workplace learning, assessment and supervision in health care education
Embo, M. (2015) · doi:10.26481/dis.20150313me · Cited by 9 · see citing works
Supervision for entrustable professional activities
Patel, M., & Baker, P. (2018) · Medical Education · doi:10.1111/medu.13685 · Cited by 8 · see citing works
Reliability testing of a portfolio assessment tool for postgraduate family medicine training in South Africa
Jenkins, L. S., Mash, R. J., & Derese, A. (2013) · African Journal of Primary Health Care & Family Medicine · doi:10.4102/phcfm.v5i1.577 · Cited by 8 · see citing works
Exploring the Reality of Using Patient Experience Data to Provide Resident Feedback: A Qualitative Study of Attending Physician Perspectives
Campbell, S., Goltz, H. H., Njue-Marendes, S., & Dang, B. N. (2016) · The Permanente Journal · doi:10.7812/tpp/15-154 · Cited by 8 · see citing works
Preceptors' considerations when entrusting professional activities to postgraduate nursing students: A qualitative study
Kleijer, E. F. W., Schuurmans, M. J., ten Cate, O., & Pool, I. A. (2023) · Nurse Education Today · doi:10.1016/j.nedt.2023.105799 · Cited by 7 · see citing works
Computer-assisted simulated workplace-based assessment in surgery: application of the universal framework of intraoperative performance within a mixed-reality simulation
Stefan, P., Pfandler, M., Kullmann, A., Eck, U., Koch, A., Mehren, C., von der Heide, A., Weidert, S., Fürmetz, J., Euler, E., et al. (2023) · BMJ Surgery Interventions & Health Technologies · doi:10.1136/bmjsit-2022-000135 · Cited by 7 · see citing works
How to enhance and assess reflection in specialist training: a mixed method validation study of a new tool for global assessment of reflection ability
Lillevang, G., Ibsen, H., Prins, S. H., & Kjær, N. K. (2020) · BMC Medical Education · doi:10.1186/s12909-020-02256-5 · Cited by 6 · see citing works
‘Ticked off’? Can a new outcomes-based postgraduate curriculum utilising programmatic assessment reduce assessment burden in Intensive Care Medicine?
Smith, C. D., & Patel, M. (2022) · Journal of the Intensive Care Society · doi:10.1177/17511437211061642 · Cited by 5 · see citing works
Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study
Holm, E., Al-Bayati, S. J. L., Barfod, T. S., Lembeck, M. A., Pedersen, H., Ramberg, E., Klemmensen, A., & Sørensen‎, J. L. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047019 · Cited by 5 · see citing works
The Discourse and Attempt of Student-Centered Assessment in the Context of Cultural Diversity
Claramita, M., Hidayah, R. N., Suhoyo, Y., Sari, S. M., Ainin, D. Q., Syah, N. A., Soemantri, D., Kurniasih, I., Kristina, T. N., Tekian, A. S., et al. (2022) · Springer eBooks · doi:10.1007/978-981-16-7232-3_6 · Cited by 4 · see citing works
Are we generating more assessments without added value? Surgical trainees’ perceptions of and receptiveness to cross-specialty assessment
Burm, S., Sebok‐Syer, S. S., Koughnett, J. A. V., & Watling, C. (2020) · Perspectives on Medical Education · doi:10.1007/s40037-020-00594-0 · Cited by 4 · see citing works
The Alphacrucis College clinical teaching model: An evaluation
Twelves, J. (2019) · International Journal of Christianity & Education · doi:10.1177/2056997119865565 · Cited by 4 · see citing works
Assessing trainee performance: ensuring learner control, supporting development, and maximizing assessment moments
Schumacher, D. J., Teunissen, P. W., Kinnear, B., & Driessen, E. W. (2021) · European Journal of Pediatrics · doi:10.1007/s00431-021-04182-0 · Cited by 3 · see citing works
Self-citation
A workplace-based learning program on clerkship of internal medicine improves medical student’s clinical competence: a retrospective comparative study
Wang, J.-N., Lv, H., Sun, Y., Cao, W., & Jiang, L. (2026) · BMC Medical Education · doi:10.1186/s12909-026-08719-5 · Cited by 2 · see citing works
Assessment and Evaluation of Cardiology Residency Training in Türkiye: A National Survey
Yı́ldı́rı́r, A. (2023) · The Anatolian Journal of Cardiology · doi:10.14744/anatoljcardiol.2023.3282 · Cited by 2 · see citing works
A study of mini-CEX (mini-clinical evaluation exercise) as an assessment tool of postgraduate students in ophthalmology
Shinde, C. A., Shirwadkar, S., Hegde, R., D’Cunha, L., Rao, P. R., Kokate, A., & Raghuwanshi, A. (2023) · D Y Patil Journal of Health Sciences · doi:10.4103/dypj.dypj_5_22 · Cited by 2 · see citing works
“Conversational Advice”: A mixed-methods analysis of medical residents’ experiences co-managing primary care patients with behavioral health providers
Hemming, P., Levine, R. B., & Gallo, J. J. (2017) · Patient Education and Counseling · doi:10.1016/j.pec.2017.07.014 · Cited by 2 · see citing works
Defining competency for Royal Australian and New Zealand College of Obstetricians and Gynaecologists training: An exploratory study of Victorian Integrated Training Program coordinators’ understanding of competency
Pagram, H., Bilszta, J. L. C., & Szabo, R. A. (2021) · Australian and New Zealand Journal of Obstetrics and Gynaecology · doi:10.1111/ajo.13324 · Cited by 2 · see citing works
Application of Classical Test Theory and Item Response Theory to Analyze Multiple Choice Questions
Nasir, M. (2014) · Libraries and Cultural Resources (University of Calgary) · doi:10.11575/prism/24958 · Cited by 2 · see citing works
Can case-based discussions in a group setting be used to assess residents' clinical skills?
Johansen, R. F., Nielsen, R. B., Malling, B. V., & Storm, H. (2021) · International Journal of Medical Education · doi:10.5116/ijme.606a.eb39 · Cited by 1 · see citing works
Implementation of structured feedback in a psychiatry residency program in Canada: a qualitative analysis study
Thakur, A., Darani, S. A., Kalocsai, C., Silver, I., Sockalingam, S., & Soklaridis, S. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1276985 · Cited by 1 · see citing works
Development of a Tool to Assess Surgical Resident Competence On-Call: The Western University Call Assessment Tool (WUCAT)
Mitchell, E. C., Ott, M. C., Ross, D., & Grant, A. L. (2023) · Journal of surgical education · doi:10.1016/j.jsurg.2023.10.001 · Cited by 1 · see citing works
A mixed-methods study of the experience of GP educational supervisors
Elliott, D., Shepherd, A., Hanley, K., & Gambhir, N. (2025) · BJGP Open · doi:10.3399/bjgpo.2024.0106 · Cited by 1 · see citing works
How We Built Workplace Based Assessment-for-Learning in Irish GP Training
Hanley, K., McSwiney, E., Malley, B. O., Barrett, A., & McEllistrem, B. (2025) · Perspectives on Medical Education · doi:10.5334/pme.1428 · Cited by 1 · see citing works
Entrustable Professional Activities: A Primer for Canadian Pharmacy Preceptors
Wilbur, K. (2017) · The Canadian Journal of Hospital Pharmacy · doi:10.4212/cjhp.v70i6.1712 · Cited by 1 · see citing works
The Development and Implementation of New Assessment Tools for the Surgical Clerkship Rotation
Kolar, M., Katsoulas, E., Toprak, A., Suart, T. N., Davidson, L., WINTHROP, A. L., Hastings‐Truelove, A., & Stockley, D. B. (2018) · MedEdPublish · doi:10.15694/mep.2018.000004.1 · Cited by 1 · see citing works
Combining Two Performance-based Assessment Methods of Logbook and DOPS in Field Internship of BSc Anesthesiology Students
Mirhosseini, F., Manoochehri, H., musavi, S., Hasanshiri, F., Bigdeli, S., Moghadam, Z. R., & Hosseini, M. (2017) · Journal of Medical Education Development · doi:10.29252/edcj.10.27.46 · Cited by 1 · see citing works
Relationship of Preceptor Ad-hoc Entrustment Decisions to Students’ Clinical Skills Performance
Basehore, P. M., Wang, S., & Khan, M. (2020) · MedEdPublish · doi:10.15694/mep.2020.000061.1 · Cited by 1 · see citing works
Admission and poor performance of trainees in the postgraduate GP training in the Netherlands
Vermeulen, M. I. (2014) · Utrecht University Repository (Utrecht University) · Cited by 1 · see citing works
THE MINI MILESTONES ASSESSMENT (MINI-MAS), A DIRECT OBSERVATIONAL TOOL TO ASSESS CLINICAL MILESTONES IN THE ERA OF COMPETENCY-BASED EDUCATION
Ladhani, M. (2014) · MacSphere (McMaster University) · doi:10.71548/15186 · Cited by 1 · see citing works
Evaluating Clinical Trainees in the Workplace. On Supervision, Trust and the Role of Competency Committees
Hauer, K. E. (2015) · Utrecht University Repository (Utrecht University)
For peer review only Supervised Learning Events in the Foundation Programme: A UK-wide narrative interview study
Rees, C. E., & Dennis, A. A. (2014)
THE SURGICAL PROCEDURE FEEDBACK RUBRIC FOR ASSESSING RESIDENT PERFORMANCE IN THE OPERATING ROOM: DOES IT WORK?
Toprak, A. (2015) · QSpace (Queen's University Library)
Feedforward interview technique in obstetrics and gynaecology residents: a fact or fallacy.
Sami, S., & Ahmad, A. (2015) · PubMed
A QUALITATIVE STUDY OF ATTENDINGS’ AND RESIDENTS’ PERSPECTIVES ON FEEDBACK IN PEDIATRICS CLINICAL SETTINGS
Luhanga, U. (2015) · QSpace (Queen's University Library)
Can we redesign the MRCGP assessment to support lifelong learning?
Harrison, C. (2019) · Education for Primary Care · doi:10.1080/14739879.2018.1563507
Do Supervised Learning Events reflect clinical competency and support ‘Trainee in Difficulty' identification? Physician Trainees' perceptions.
Tomkins, A., Sherratt, C., & Patel, M. (2019) · Journal of Contemporary Medical Education · doi:10.5455/jcme.20190705075510
Resident-sensitive quality measures
Schumacher (2020) · doi:10.26481/dis.20200319ds
Modern opleiden: oude wijn in nieuwe zakken?
Smeenk, F. W. J. M. (2014) · doi:10.26481/spe.20140424fs

Authors: Driessen, E., & Scheele, F.
Affiliations: Maastricht University (NL)
Sint Lucas Andreas Hospital (NL)
Corresponding author: Driessen, Erik
ORCID: Driessen, Erik
Scheele, Fedde
Venue: Medical Teacher
Publisher: Informa UK Limited
Published: 2013-05-23
Volume: 35
Issue: 7
Pages: 569-574
ISSN: 0142-159X (ISSN-L)
1466-187X
DOI: 10.3109/0142159x.2013.798403
OpenAlex: W2170153256
Cited by: 110
References: 41
Citations by year: 2026: 6 · 2025: 3 · 2024: 2 · 2023: 9 · 2022: 6 · 2021: 8 · 2020: 6 · 2019: 11 · 2018: 13 · 2017: 6 · 2016: 10 · 2015: 18 · 2014: 10 · 2013: 1
FWCI: 13.26
Type: journal article
Language: en
Open access: closed
Locations: Medical Teacher (publishedVersion)
PubMed (publishedVersion)
Research Publications (Maastricht University) (publishedVersion)
Pure Amsterdam UMC (publishedVersion)
Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
Research Publications (Maastricht University) (submittedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion)
Topics: Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Social Sciences › Education › Student Assessment and Feedback
Keywords: workplace-based assessment, graduate medical education, specialist training, entrustment decisions, trainee learning, clinical supervision, narrative feedback, summative assessment, self-assessment
MeSH terms: Attitude of Health Personnel, Clinical Competence, Education, Medical, Graduate, Educational Measurement, Feedback, Humans, Internship and Residency, Workplace

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks