Toward meaningful evaluation of medical trainees: The influence of participants’ perceptions of the process.
Abstract
An essential goal of evaluation is to foster learning. Across the medical education spectrum, evaluation of clinical performance is dominated by subjective feedback to learners based on observation by expert supervisors. Research in non-medical settings has suggested that participants' perceptions of evaluation processes exert considerable influence over whether the feedback they receive actually facilitates learning, but similar research on perceptions of feedback in the medical setting has been limited. In this review, we examine the literature on recipient perceptions of feedback and how those perceptions influence the contribution that feedback makes to their learning. A focused exploration of relevant work on this subject in higher education and industrial psychology settings is followed by a detailed examination of available research on perceptions of evaluation processes in medical settings, encompassing both trainee and evaluator perspectives. We conclude that recipients' and evaluators' perceptions of an evaluation process profoundly affect the usefulness of the evaluation and the extent to which it achieves its goals. Attempts to improve evaluation processes cannot, therefore, be limited to assessment tool modification driven by reliability and validity concerns, but must also take account of the critical issue of feedback reception and the factors that influence it. Given the unique context of clinical performance evaluation in medicine, a research agenda is required that seeks to more fully understand the complexity of the processes of giving, receiving, interpreting, and using feedback as a basis for real progress toward meaningful evaluation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The mechanisms by which learners develop perceptions of feedback are complex and multifaceted.
“The mechanisms by which learners develop perceptions of feedback are complex and multifaceted. 13-15”
From Medical student perceptions of feedback and feedback behaviors within the context of the “educational alliance.” (2017) · cites this paper as [15]
Cites
(1993) · Journal of the American Dietetic Association · doi:10.1016/0002-8223(93)92196-5 · Cited by 12,205 · see citing works
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882 · see citing works
The assessment of clinical skills/competence/performance
Miller, G. E. (1990) · Academic Medicine · doi:10.1097/00001888-199009000-00045 · Cited by 4,883 · see citing works
Davis, D. A., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., & Perrier, L. (2006) · JAMA · doi:10.1001/jama.296.9.1094 · Cited by 2,375 · see citing works
Assessment in medical education
Epstein, R. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmra054784 · Cited by 1,872 · see citing works
Consequences of individual feedback on behavior in organizations.
Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979) · Journal of Applied Psychology · doi:10.1037/0021-9010.64.4.349 · Cited by 1,822 · see citing works
(1992) · Long Range Planning · doi:10.1016/0024-6301(92)90232-q · Cited by 1,296 · see citing works
Journal of General Internal Medicine
(2013) · Journal of General Internal Medicine · doi:10.1007/11606.1525-1497 · Cited by 1,284 · see citing works
Feedback in Clinical Medical Education
Ende, J. (1983) · JAMA · doi:10.1001/jama.1983.03340060055026 · Cited by 1,259 · see citing works
Self-Assessment in the Health Professions: A Reformulation and Research Agenda
Eva, K. W., & Regehr, G. (2005) · Academic Medicine · doi:10.1097/00001888-200510001-00015 · Cited by 992 · see citing works
Yorke, M. (2003) · Higher Education · doi:10.1023/a:1023967026413 · Cited by 973 · see citing works
Exploring the Role of Emotions in Injustice Perceptions and Retaliation.
Barclay, L. J., Skarlicki, D. P., & Pugh, S. D. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.4.629 · Cited by 585 · see citing works
Getting the Message across: The Problem of Communicating Assessment Feedback
Higgins, R., Hartley, P., & Skelton, A. (2001) · Teaching in Higher Education · doi:10.1080/13562510120045230 · Cited by 549 · see citing works
360° feedback: Accuracy, reactions, and perceptions of usefulness.
Brett, J. F., & ATWATER, L. E. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.5.930 · Cited by 474 · see citing works
Failure to Fail: The Perspectives of Clinical Supervisors
Dudek, N. L., Marks, M. B., & Regehr, G. (2005) · Academic Medicine · doi:10.1097/00001888-200510001-00023 · Cited by 422 · see citing works
The Royal College of Physicians and Surgeons of Canada
Lewis, D. S. (1962) · The Canadian Journal of Psychiatry · doi:10.1177/070674377902400512 · Cited by 357 · see citing works
Split Roles in Performance Appraisal
Meyer, H. H., Kay, E., & French, J. R. P. (1981) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-04809-0_6 · Cited by 342 · see citing works
The multitrait-multirater approach to measuring managerial job performance.
Lawler, E. E. (1967) · Journal of Applied Psychology · doi:10.1037/h0025095 · Cited by 304 · see citing works
Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use
Sargeant, J., Mann, K., Sinclair, D., Van der Vleuten, C., & Metsemakers, J. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9039-x · Cited by 274 · see citing works
Employee responses to formal performance appraisal feedback.
Pearce, J. L., & Porter, L. W. (1986) · Journal of Applied Psychology · doi:10.1037/0021-9010.71.2.211 · Cited by 268 · see citing works
Correlates of employee reactions to performance appraisals and appraisal systems.
Dipboye, R. L., & de Pontbriand, R. (1981) · Journal of Applied Psychology · doi:10.1037/0021-9010.66.2.248 · Cited by 250 · see citing works
Why medical educators may be failing at feedback.
Bing-You, R. G. (2009) · JAMA · doi:10.1001/jama.2009.1393 · Cited by 224 · see citing works
An investigation of medical student reactions to feedback: a randomised controlled trial
Boehler, M. L., Rogers, D. A., Schwind, C. J., Mayforth, R. D., Quin, J. A., Williams, R. G., & Dunnington, G. L. (2006) · Medical Education · doi:10.1111/j.1365-2929.2006.02503.x · Cited by 206 · see citing works
Surgery residents and attending surgeons have different perceptions of feedback.
Sender Liberman, A., Liberman, M., Steinert, Y., McLeod, P., & Meterissian, S. (2005) · Medical Teacher · doi:10.1080/0142590500129183 · Cited by 184 · see citing works
The pay-for-performance dilemma
Meyer, H. H. (1975) · Organizational Dynamics · doi:10.1016/0090-2616(75)90029-7 · Cited by 181 · see citing works
Sargeant, J., Mann, K., & Ferrier, S. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02124.x · Cited by 174 · cited 2× · see citing works
“In the continuing medical education arena, Sargeant et al. (2005) studied family physicians’ reactions to multisource feedback and found that recipients perceived negative feedback as inaccurate or not credible, negating any useful effect of such feedback.”
BackgroundFeedback falling on deaf ears: residents' receptivity to feedback tempered by sender credibility
Bing-You, R. G., Paterson, J., & Levine, M. A. (1997) · Medical Teacher · doi:10.3109/01421599709019346 · Cited by 161 · see citing works
Direct Observation in Medical Education: A Review of the Literature and Evidence for Validity
Fromme, H. B., Karani, R., & Downing, S. M. (2009) · Mount Sinai Journal of Medicine A Journal of Translational and Personalized Medicine · doi:10.1002/msj.20123 · Cited by 159 · see citing works
De, S. K., Henke, P. K., Ailawadi, G., Dimick, J. B., & Colletti, L. M. (2004) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2004.08.025 · Cited by 157 · see citing works
Global rating scales in residency education
Gray, J. D. (1996) · Academic Medicine · doi:10.1097/00001888-199601000-00043 · Cited by 151 · see citing works
Rewarding Leadership and Fair Procedures as Determinants of Self-Esteem.
David De Cremer, A., van Knippenberg, B., van Knippenberg, D., Mullenders, D., & Stinglhamber, F. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.1.3 · Cited by 136 · see citing works
Review of instruments for peer assessment of physicians
Evans, R., Elwyn, G. J., & Edwards, A. (2004) · BMJ · doi:10.1136/bmj.328.7450.1240 · Cited by 125 · see citing works
A survey of medical students’ views about the purposes and fairness of assessment
Duffield, K. E., & Spencer, J. A. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01291.x · Cited by 119 · see citing works
Chou, S. H., Cole, G., McLaughlin, K., & Lockyer, J. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03111.x · Cited by 111 · see citing works
Improving in-training evaluation programs
Turnbull, J. M., Gray, J., & MacFadyen, J. (1998) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.1998.00097.x · Cited by 99 · see citing works
Rules of Engagement: Residents’ Perceptions of the In-Training Evaluation Process
Watling, C. J., Kenyon, C. F., Zibrowski, E. M., Schulz, V., Goldszmidt, M. A., Singh, I., Maddocks, H. L., & Lingard, L. (2008) · Academic Medicine · doi:10.1097/acm.0b013e318183e78c · Cited by 86 · see citing works
Self-citationHaber, R. J., & Avins, A. L. (1994) · Journal of General Internal Medicine · doi:10.1007/bf02600028 · Cited by 76 · see citing works
Nutrition assessment and counseling practices
Glanz, K., Tziraki, C., Albright, C. L., & Fernandes, J. (1995) · Journal of General Internal Medicine · doi:10.1007/bf02600234 · Cited by 74 · see citing works
Higgins, R. S. D., Bridges, J., Burke, J. M., O'Donnell, M. A., Cohen, N. M., & Wilkes, S. B. (2004) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2003.09.075 · Cited by 65 · see citing works
Comparing resident measurements to attending surgeon self-perceptions of surgical educators
Claridge, J. A., Calland, J. F., Chandrasekhara, V., Young, J. S., Sanfey, H., & Schirmer, B. D. (2003) · The American Journal of Surgery · doi:10.1016/s0002-9610(02)01421-6 · Cited by 63 · see citing works
An Exploration of Faculty Perspectives on the In-Training Evaluation of Residents
Watling, C., Kenyon, C. F., Schulz, V. N., Goldszmidt, M. A., Zibrowski, E. M., & Lingard, L. A. (2010) · Academic Medicine · doi:10.1097/acm.0b013e3181e19722 · Cited by 61 · see citing works
Self-citationColletti, L. M. (2000) · Journal of Surgical Research · doi:10.1006/jsre.2000.5848 · Cited by 47 · see citing works
EditorialCompetency-based assessment: making it a reality
Davis, M. H., & Harden, R. M. (2003) · Medical Teacher · doi:10.1080/0142159032000153842 · Cited by 45 · see citing works
Daelmans, H. E. M., Overmeer, R., van der Hem-Stokroos, H. H., Scherpbier, A. J. J. A., Stehouwer, C. D. A., & van der Vleuten, C. P. M. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02358.x · Cited by 41 · see citing works
Awad, S. S., Liscum, K. R., Aoki, N., Awad, S., & Berger, D. H. (2002) · Journal of Surgical Research · doi:10.1006/jsre.2002.6401 · Cited by 40 · see citing works
Coaching Medical Students in Receiving Effective Feedback
Bing‐You, R. G., Bertsch, T. F., & Thompson, J. A. (1998) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1004_6 · Cited by 36 · see citing works
Ward Evaluations: Should They Be Abandoned?
Kwolek, C. J., Donnelly, M. B., SLOAN, D. A., et al. (1997) · Journal of Surgical Research · doi:10.1006/jsre.1997.5001 · Cited by 36 · see citing works
The scope of clinical education
Barrows, H. S. (1986) · Academic Medicine · doi:10.1097/00001888-198609000-00004 · Cited by 31 · see citing works
Commentary: Assessment Is an Educational Tool
Krupat, E., & Dienstag, J. L. (2009) · Academic Medicine · doi:10.1097/acm.0b013e31819f7fb9 · Cited by 27 · see citing works
The importance of strong evaluation standards and procedures in training residents
Short, J. (1993) · Academic Medicine · doi:10.1097/00001888-199307000-00003 · Cited by 24 · see citing works
Kolars, J. C., McDonald, F. S., Subhiyah, R. G., et al. (2003) · Clinical Gastroenterology and Hepatology · doi:10.1053/jcgh.2003.50010 · Cited by 18 · see citing works
Assessing Clinical Competence in Physical Medicine & Rehabilitation Residency Programs
Pasquina, P. F., Kelly, S. P., & Hawkins, R. E. (2003) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/01.phm.0000069213.90013.01 · Cited by 13 · see citing works
Summative evaluation on the hospital wards. What do faculty say to learners?
Hasley, P. B., & Arnold, R. M. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-008-9127-1 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The “educational alliance” as a framework for reconceptualizing feedback in medical education.
Telio, S., Ajjawi, R., & Regehr, G. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000560 · Cited by 572 · see citing works
Learning from clinical work: the roles of learning cues and credibility judgements
Watling, C., Driessen, E., van der Vleuten, C. P. M., & Lingard, L. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04126.x · Cited by 261 · see citing works
Self-citationFeedback and the educational alliance: Examining credibility judgements and their consequences.
Telio, S., Regehr, G., & Ajjawi, R. (2016) · Medical Education · doi:10.1111/medu.13063 · Cited by 231 · see citing works
Ramani, S., Post, S. E., Könings, K., Mann, K., Katz, J. T., & van der Vleuten, C. (2017) · Teaching and Learning in Medicine · doi:10.1080/10401334.2016.1244014 · Cited by 138 · see citing works
Encouraging residents to seek feedback.
Delva, D., Sargeant, J., Miller, S., Holland, J., Alexiadis Brown, P., Leblanc, C., Lightfoot, K., & Mann, K. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.806791 · Cited by 118 · see citing works
Bowen, L., Marshall, M., & Murdoch-Eaton, D. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001632 · Cited by 99 · see citing works
Understanding responses to feedback: the potential and limitations of regulatory focus theory
Watling, C., Driessen, E., van der Vleuten, C. P. M., Vanstone, M., & Lingard, L. (2012) · Medical Education · doi:10.1111/j.1365-2923.2012.04209.x · Cited by 99 · see citing works
Self-citationMusic lessons: revealing medicine's learning culture through a comparison with that of music
Watling, C., Driessen, E., van der Vleuten, C. P. M., Vanstone, M., & Lingard, L. (2013) · Medical Education · doi:10.1111/medu.12235 · Cited by 72 · see citing works
Self-citationGarcia, I., James, R. W., Bischof, P., & Baroffio, A. (2017) · Teaching and Learning in Medicine · doi:10.1080/10401334.2017.1279056 · Cited by 60 · see citing works
Feedback mapping—The curricular cornerstone of an “educational alliance.”
Murdoch-Eaton, D., & Bowen, L. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1297892 · Cited by 20 · see citing works
- Authors
- Watling, C. J., & Lingard, L.
- Affiliations
- Western University (CA)
- Corresponding author
- Watling, Christopher J.
- Venue
- Advances in Health Sciences Education
- Published
- 2010-02-09
- Volume
- 17
- Issue
- 2
- Pages
- 183-194
- ISSN
- 1382-4996 (ISSN-L)
- 1573-1677
- OpenAlex
- W1976235441
- Cited by
- 139
- References
- 53
- Citations by year
- 2026: 7 · 2025: 6 · 2024: 7 · 2023: 11 · 2022: 11 · 2021: 15 · 2020: 8 · 2019: 3 · 2018: 11 · 2017: 12 · 2016: 11 · 2015: 15 · 2014: 8 · 2013: 8 · 2012: 6
- FWCI
- 4.44
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Advances in Health Sciences Education (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Education › Student Assessment and Feedback
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- clinical performance assessment, medical trainees, feedback reception, feedback effectiveness, clinical supervision, learner perceptions, evaluator perceptions, assessment reliability, clinical competence assessment
- MeSH terms
- Education, Medical, Graduate, Educational Measurement, Humans, Internship and Residency, Perception, Students, Medical, Teaching, Feedback, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
