Exploring family physicians' reactions to multisource feedback: perceptions of credibility and usefulness
Abstract
Purpose Physician performance is comprised of several domains of professional competence. Multisource feedback (MSF) or 360‐degree feedback is an approach used to assess these, particularly the humanistic and relational competencies. Research studying responses to performance assessment shows that reactions vary and can influence how performance feedback is used. Improvement does not always result, especially when feedback is perceived as negative. This small qualitative study undertook preliminary exploration of physicians' reactions to MSF, and perceptions influencing these and the acceptance and use of their feedback. Methods We held focus groups with 15 family physicians participating in an MSF pilot study. Qualitative analyses included content and constant comparative analyses. Results Participants agreed that the purpose of MSF assessment should be to enhance practice and generally agreed with their patients' feedback. However, responses to medical colleague and co‐worker feedback ranged from positive to negative. Several participants who responded negatively did not agree with their feedback nor were inclined to use it for practice improvement. Reactions were influenced by perceptions of accuracy, credibility and usefulness of feedback. Factors shaping these perceptions included: recruiting credible reviewers, ability of reviewers to make objective assessments, use of the assessment tool and specificity of the feedback. Conclusion Physicians' perceptions of the MSF process and feedback can influence how and if they use the feedback for practice improvement. These findings are important, raising the concern that feedback perceived as negative and not useful will have no or negative results, and highlight questions for further study.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Trainees must perceive feedback as credible in order for it to be influential.
“Do recognize that trainees must perceive feedback as credible in order for it to be influential.”
From Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education. (2015) · cites this paper as [8]
Factors such as the emotional reaction of the learner, the credibility of the educator, or the time interval within which feedback is given may all affect how learners perceive and interpret the feedback message.
“Other factors -such as the emotional reaction of the learner, the credibility of the educator, or the time interval within which feedback is given -may all affect how learners perceive and interpret the feedback message. 14,18-21”
From Medical student perceptions of feedback and feedback behaviors within the context of the “educational alliance.” (2017) · cites this paper as [19]
Cites
(1998) · Sage eBooks · Cited by 38,631 · see citing works
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882 · see citing works
(2021) · Edward Elgar Publishing eBooks · doi:10.4337/9781788976954.00014 · Cited by 4,030 · see citing works
Defining and Assessing Professional Competence
Epstein, R. M. (2002) · JAMA · doi:10.1001/jama.287.2.226 · Cited by 3,114 · see citing works
360° feedback: Accuracy, reactions, and perceptions of usefulness.
Brett, J. F., & ATWATER, L. E. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.5.930 · Cited by 474 · see citing works
SELF‐OTHER AGREEMENT: DOES IT REALLY MATTER?
ATWATER, L. E., Ostroff, C. L., Yammarino, F. J., & Fleenor, J. W. (1998) · Personnel Psychology · doi:10.1111/j.1744-6570.1998.tb00252.x · Cited by 455 · see citing works
Use of Peer Ratings to Evaluate Physician Performance
Ramsey, P. G. (1993) · JAMA · doi:10.1001/jama.1993.03500130069034 · Cited by 352 · see citing works
Off the Track: Why and How Successful Executives Get Derailed
McCall, M. W., & Lombardo, M. M. (1983) · doi:10.35613/ccl.1983.1083 · Cited by 340 · see citing works
Multisource feedback in the assessment of physician competencies
Lockyer, J. (2003) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340230103 · Cited by 284 · see citing works
Job Feedback: Giving, Seeking, and Using Feedback for Performance Improvement
Breaugh, J. A., & London, M. (2004) · Academy of Management Review · doi:10.2307/20159059 · Cited by 272 · see citing works
Employee responses to formal performance appraisal feedback.
Pearce, J. L., & Porter, L. W. (1986) · Journal of Applied Psychology · doi:10.1037/0021-9010.71.2.211 · Cited by 268 · see citing works
A FIVE‐YEAR STUDY OF UPWARD FEEDBACK: WHAT MANAGERS DO WITH THEIR RESULTS MATTERS
Walker, A. G., & Smither, J. W. (1999) · Personnel Psychology · doi:10.1111/j.1744-6570.1999.tb00166.x · Cited by 257 · see citing works
Norcini, J. J. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01536.x · Cited by 244 · see citing works
Measuring the ability to meet family needs in an intensive care unit
Johnson, D. H., Wilson, M., Cavanaugh, B., Bryden, C., Gudmundson, D., & Moodley, O. (1998) · Critical Care Medicine · doi:10.1097/00003246-199802000-00023 · Cited by 232 · see citing works
Ashworth, P. D. (2000) · Estudios pedagógicos · doi:10.4067/s0718-07052000000100007 · Cited by 204 · see citing works
Assessment of physician performance in Alberta: the physician achievement review.
Hall, W. C., Violato, C., Lewkonia, R. M., Lockyer, J., Fidler, H., Toews, J., Jennett, P., Donoff, M. G., & Moores, D. (1999) · Europe PMC (PubMed Central) · Cited by 191 · see citing works
Medical studentsʼ clinical self-assessments
Woolliscroft, J. O., Tenhaken, J. D., Smith, J. M., & Calhoun, J. G. (1993) · Academic Medicine · doi:10.1097/00001888-199304000-00016 · Cited by 170 · see citing works
Feedback falling on deaf ears: residents' receptivity to feedback tempered by sender credibility
Bing-You, R. G., Paterson, J., & Levine, M. A. (1997) · Medical Teacher · doi:10.3109/01421599709019346 · Cited by 161 · see citing works
The Value of Patient and Peer Ratings in Recertification
Lipner, R. S., Blank, L. L., Leas, B. F., & Fortna, G. S. (2002) · Academic Medicine · doi:10.1097/00001888-200210001-00021 · Cited by 132 · see citing works
Changing physiciansʼ practices
Fidler, H., Lockyer, J., Toews, J., & Violato, C. (1999) · Academic Medicine · doi:10.1097/00001888-199906000-00019 · Cited by 125 · see citing works
Woolliscroft, J. O., Howell, J. D., Patel, B. P., & Swanson, D. B. (1994) · Academic Medicine · doi:10.1097/00001888-199403000-00017 · Cited by 99 · see citing works
Violato, C., Marini, A., Toews, J. A., Lockyer, J., & Fidler, H. (1997) · Academic Medicine · doi:10.1097/00001888-199710001-00028 · Cited by 91 · see citing works
Measuring “goodness” in individuals and healthcare systems
Pringle, M., Wilson, T., & Grol, R. (2002) · BMJ · doi:10.1136/bmj.325.7366.704 · Cited by 91 · see citing works
Ratings of the performances of practicing internists by hospital-based registered nurses
Wenrich, M. D., Carline, J. D., Giles, L. M., & Ramsey, P. G. (1993) · Academic Medicine · doi:10.1097/00001888-199309000-00014 · Cited by 85 · see citing works
Use of peer ratings to evaluate physician performance
Ramsey, P. G. (1993) · JAMA · doi:10.1001/jama.269.13.1655 · Cited by 85 · see citing works
Sargeant, J. M., Mann, K. V., Ferrier, S. N., Langille, D. B., Muirhead, P. D., Hayes, V. M., & Sinclair, D. E. (2003) · Academic Medicine · doi:10.1097/00001888-200310001-00014 · Cited by 68 · see citing works
Self-citationJohnson, D. H., & C̆ujec, B. (1998) · Critical Care Medicine · doi:10.1097/00003246-199811000-00020 · Cited by 68 · see citing works
Karl, K. A. (1998) · Academy of Management Perspectives · doi:10.5465/ame.1998.1334026 · Cited by 47 · see citing works
Maximizing the Value of 360‐degree Feedback
(2001) · International Journal of Manpower · doi:10.1108/ijm.2001.22.7.664.4 · Cited by 46 · see citing works
Employee responses to formal performance appraisal feedback.
Pearce, J. L., & Porter, L. W. (1986) · Journal of Applied Psychology · doi:10.1037//0021-9010.71.2.211 · Cited by 32 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Impact of workplace based assessment on doctors' education and performance: a systematic review
Miller, A. P., & Archer, J. C. (2010) · BMJ · doi:10.1136/bmj.c5064 · Cited by 462 · see citing works
Assessment, feedback and the alchemy of learning
Watling, C., & Ginsburg, S. (2018) · Medical Education · doi:10.1111/medu.13645 · Cited by 434 · see citing works
Cheng, A., Nadkarni, V., Mancini, M. B., Hunt, E. A., Sinz, E. H., Merchant, R. M., Donoghue, A. J., Duff, J. P., Eppich, W., Auerbach, M. A., et al. (2018) · Circulation · doi:10.1161/cir.0000000000000583 · Cited by 430 · see citing works
Guidelines: The do’s, don’ts and don’t knows of feedback for clinical education.
Lefroy, J., Watling, C., Teunissen, P. W., & Brand, P. (2015) · Perspectives on Medical Education · doi:10.1007/s40037-015-0231-7 · Cited by 347 · see citing works
Self-monitoring in clinical practice: A challenge for medical educators
Epstein, R. M., Siegel, D. J., & Silberman, J. (2008) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.149 · Cited by 333 · see citing works
Reflection: a link between receiving and using assessment feedback
Sargeant, J. M., Mann, K. V., van der Vleuten, C. P., & Metsemakers, J. F. (2009) · Advances in Health Sciences Education · doi:10.1007/s10459-008-9124-4 · Cited by 296 · see citing works
“Collectively this group appeared to demonstrate saturation of themes regarding reflective responses to MSF and the linkages between reflective activities and subsequent decision-making processes; i.e., new themes and concepts were not emerging.”
MethodInfluentialSelf-citationUnderstanding the influence of emotions and reflection upon multi-source feedback acceptance and use
Sargeant, J., Mann, K., Sinclair, D., Van der Vleuten, C., & Metsemakers, J. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9039-x · Cited by 274 · see citing works
“These were often strong, pervasive and long-lasting, consistent with earlier findings (Boud, 2002; Ashford et al., 2003; Sargeant et al., 2005 ).”
BackgroundResultInfluentialSelf-citationLearning from clinical work: the roles of learning cues and credibility judgements
Watling, C., Driessen, E., van der Vleuten, C. P. M., & Lingard, L. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04126.x · Cited by 261 · see citing works
Group processes in medical education: learning from social identity theory
Burford, B. C. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04099.x · Cited by 256 · see citing works
Brinkman, W. B., Geraghty, S. R., Lanphear, B. P., Khoury, J. C., Gonzalez del Rey, J. A., DeWitt, T. G., & Britto, M. T. (2007) · Archives of Pediatrics & Adolescent Medicine · doi:10.1001/archpedi.161.1.44 · Cited by 204 · see citing works
Beyond individualism: professional culture and its influence on feedback
Watling, C., Driessen, E., van der Vleuten, C. P. M., Vanstone, M., & Lingard, L. (2013) · Medical Education · doi:10.1111/medu.12150 · Cited by 174 · see citing works
Learning culture and feedback: an international study of medical athletes and musicians
Watling, C., Driessen, E., van der Vleuten, C. P. M., & Lingard, L. (2014) · Medical Education · doi:10.1111/medu.12407 · Cited by 162 · see citing works
“Directed” self-assessment: Practice and feedback within a social context
Sargeant, J., Mann, K., van der Vleuten, C., & Metsemakers, J. (2008) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.155 · Cited by 158 · see citing works
Self-citationChallenges in multisource feedback: intended and unintended outcomes
Sargeant, J., Mann, K., Sinclair, D., Van der Vleuten, C., & Metsemakers, J. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02769.x · Cited by 151 · see citing works
Self-citationWatling, C. J., & Lingard, L. (2012) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9223-x · Cited by 139 · see citing works
“In the continuing medical education arena, Sargeant et al. (2005) studied family physicians’ reactions to multisource feedback and found that recipients perceived negative feedback as inaccurate or not credible, negating any useful effect of such feedback.”
BackgroundSaedon, H., Salleh, S., Balakrishnan, A., Imray, C., & Saedon, M. H. (2012) · BMC Medical Education · doi:10.1186/1472-6920-12-25 · Cited by 124 · see citing works
“Saedon et al.BM C M edicalEducation 2012,12:25 Page 5 of 8 http://w w w .biom edcentral.com /1472-6920/12/25 T W H Ja M M C Jo n t ferra ican is, aly ons A qualitative study by Sargeant et al found that doctors did not make changes if feedback from MSF was positive, and only 7 out of thirteen doctors who received nega…”
BackgroundInfluentialTeunissen, P. W. (2015) · Advances in Health Sciences Education · doi:10.1007/s10459-014-9556-y · Cited by 113 · see citing works
Feedback Redefined: Principles and Practice
Ramani, S., Könings, K. D., Ginsburg, S., & van der Vleuten, C. P. M. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-04874-2 · Cited by 125 · see citing works
Johnson, C., Keating, J. L., Boud, D. J., Dalton, M., Kiegaldie, D., Hay, M., McGrath, B. P., McKenzie, W. A., Nair, K., Nestel, D., et al. (2016) · BMC Medical Education · doi:10.1186/s12909-016-0613-5 · Cited by 108 · see citing works
“Their reasons included they did not consider there was a problem, did not believe the educator’s comments were credible or relevant [34, 35], or did not understand what needed improving or how to do it [34, 36].”
BackgroundViolato, C., & Lockyer, J. (2006) · Advances in Health Sciences Education · doi:10.1007/s10459-005-5639-0 · Cited by 104 · see citing works
Ferguson, J., Wakeling, J., & Bowie, P. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-76 · Cited by 102 · see citing works
“The articles by Fidler et al. (1999) [21] and Hall et al. (1999) [22] report on different aspects of the same study and the five articles reported by Sargeant et al. (2003, 2005, 2007, 2008, 2009) [29-33] all relate to the same sample population, from the pilot study reported by Sargeant et al.…”
BackgroundResultInfluentialThe Foundation Programme assessment tools: An opportunity to enhance feedback to trainees?
Carr, S. (2006) · Postgraduate Medical Journal · doi:10.1136/pgmj.2005.042366 · Cited by 103 · see citing works
Bowen, L., Marshall, M., & Murdoch-Eaton, D. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001632 · Cited by 99 · see citing works
The Pharmaceutical Price Regulation Scheme
Collier, J. (2007) · BMJ · doi:10.1136/bmj.39136.464421.be · Cited by 99 · see citing works
mini-PAT (Peer Assessment Tool): A Valid Component of a National Assessment Programme in the UK?
Archer, J. C., Norcini, J. J., Southgate, L. J., Heard, S., & Davies, H. A. (2006) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9033-3 · Cited by 96 · see citing works
Doctors’ perceptions of why 360‐degree feedback does (not) work: a qualitative study
Overeem, K., Wollersheim, H. C. H., Driessen, E. W., Lombarts, K. M. J. M. H., van de Ven, G., Grol, R. P. T. M., & Arah, O. A. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03439.x · Cited by 93 · see citing works
Doctor performance assessment in daily practise: does it help doctors or not? A systematic review
Overeem, K., Faber, M. J., Arah, O. A., Elwyn, G. J., Lombarts, K. M. J. M. H., Wollersheim, H. C. H., & Grol, R. P. T. M. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02897.x · Cited by 94 · see citing works
Unfulfilled promise, untapped potential: Feedback at the crossroads
Watling, C. (2014) · Medical Teacher · doi:10.3109/0142159x.2014.889812 · Cited by 90 · see citing works
“Less attention has been devoted to learners’ perceptions of and responses to feedback or the formal and informal processes that support it, although a growing body of literature has been fruitfully exploring this area (Bing-You & Patterson 1997; Sargeant et al. 2005; Watling et al. 2008).”
BackgroundRules of Engagement: Residents’ Perceptions of the In-Training Evaluation Process
Watling, C. J., Kenyon, C. F., Zibrowski, E. M., Schulz, V., Goldszmidt, M. A., Singh, I., Maddocks, H. L., & Lingard, L. (2008) · Academic Medicine · doi:10.1097/acm.0b013e318183e78c · Cited by 86 · see citing works
Resident Self-Other Assessor Agreement
Lipsett, P. A. (2011) · Archives of Surgery · doi:10.1001/archsurg.2011.172 · Cited by 85 · see citing works
“Physicians also more frequently disagreed with feedback from medical colleagues than that from patients or coworkers in the same office.(21) Further, they found that physicians responded with negative emotions to feedback that was inconsistent with selfperceptions of performance, questioned its credibility, and were n…”
BackgroundMaking sense of feedback experiences: a multi‐school study of medical students' narratives
Urquhart, L. M., Rees, C. E., & Ker, J. S. (2014) · Medical Education · doi:10.1111/medu.12304 · Cited by 83 · see citing works
Hawkins, R. E., Lipner, R. S., Ham, H. P., Wagner, R., & Holmboe, E. S. (2013) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.21201 · Cited by 84 · see citing works
Violato, C., Lockyer, J., & Fidler, H. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03127.x · Cited by 79 · see citing works
Using the 360° multisource feedback model to evaluate teaching and professionalism
Berk, R. A. (2009) · Medical Teacher · doi:10.3109/01421590802572775 · Cited by 78 · see citing works
Using audit and feedback to health professionals to improve the quality and safety of health care
Flottorp, S. A., Jamtvedt, G., Gibis, B., & McKee, M. (2010) · Cited by 77 · see citing works
Murphy, D., Bruce, D., Mercer, S. W., & Eva, K. W. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-008-9104-8 · Cited by 78 · see citing works
Overeem, K., Lombarts, K. M. J. M. H., Arah, O. A., Klazinga, N., Grol, R. P. T. M., & Wollersheim, H. C. H. (2010) · Medical Teacher · doi:10.3109/01421590903144128 · Cited by 76 · see citing works
“For both PAR and ABIM, we gave respondents the opportunity to provide narrative comments to increase the specificity of MSF as recommended in earlier literature (Sargeant et al. 2005).”
MethodAsprey, A., Campbell, J. L., Newbould, J., Cohn, S. R., Carter, M., Davey, A. F., & Roland, M. O. (2013) · British Journal of General Practice · doi:10.3399/bjgp13x664252 · Cited by 73 · see citing works
Feedback providers’ credibility impacts students’ satisfaction with feedback and delayed performance
van de Ridder, J. M. M., Berk, F. C. J., Stokking, K. M., & ten Cate, O. (2014) · Medical Teacher · doi:10.3109/0142159x.2014.970617 · Cited by 69 · see citing works
“Feedback providers influence the degree to which the students accept feedback (Halperin et al. 1976; Ilgen et al. 1979; Sargeant et al. 2005, 2011; Veloski et al. 2006; Watling et al. 2012a, b), their decision to ask for feedback (Vancouver & Morrison 1995), and their strategy to seek feedback (Lee…”
BackgroundBoiko, O. V., Campbell, J. L., Elmore, N., Davey, A. F., Roland, M. O., & Burt, J. (2014) · Health Expectations · doi:10.1111/hex.12298 · Cited by 66 · see citing works
Validity and reliability of the CARE Measure in secondary care
Mercer, S. W., & Murphy, D. (2008) · Clinical Governance An International Journal · doi:10.1108/14777270810912969 · Cited by 64 · see citing works
Burt, J., Campbell, J., Abel, G., Aboulghate, A., Ahmed, F., Asprey, A., Barry, H. E., Beckwith, J., BENSON, J. A., Boiko, O. V., et al. (2017) · Programme Grants for Applied Research · doi:10.3310/pgfar05090 · Cited by 65 · see citing works
“Our identification of issues over the credibility of surveys, and difficulties in the interpretation of feedback, clearly echoes previous work in both primary and secondary care, which suggests widespread scepticism about the robustness of patient surveys.(119,200,205,206) Practice staff were more likely to view resul…”
ResultUser perceptions of multi‐source feedback tools for junior doctors
Burford, B. C., Illing, J. C., Kergon, C., Morrow, G., & Livingston, M. (2010) · Medical Education · doi:10.1111/j.1365-2923.2009.03565.x · Cited by 61 · see citing works
Ramani, S., Könings, K. D., Ginsburg, S., & van der Vleuten, C. P. M. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002971 · Cited by 64 · see citing works
The role of educational theory in continuing medical education: Has it helped us?
Mann, K. V. (2004) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.1340240505 · Cited by 58 · see citing works
Self-citationRamani, S., Könings, K. D., Mann, K. V., & van der Vleuten, C. P. M. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1353071 · Cited by 56 · see citing works
“Supervisors, peers and nonphysician team members, such as nurses or case managers and patients, are all likely to perceive and provide insights on different dimensions of performance, thus enhancing professional development of residents (Sargeant et al. 2005; Ten Cate and Sargeant 2011; Holmboe and Ross 2012; Sargeant…”
BackgroundSelf-citationMeaningful feedback through a sociocultural lens
Ramani, S., Könings, K. D., Ginsburg, S., & van der Vleuten, C. P. M. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1656804 · Cited by 55 · see citing works
Bodle, J. F., Kaufmann, S., Bisson, D., Nathanson, B., & Binney, D. M. (2008) · Medical Teacher · doi:10.1080/01421590701881624 · Cited by 54 · see citing works
“Although assessment by juniors raises issues (Lockyer et al. 2003; Sargeant et al. 2005), as the outcome of the assessment was not under scrutiny, merely the process, their inclusion provided important information.”
BackgroundWorkplace‐based assessments of junior doctors: do scores predict training difficulties?
Mitchell, C., Bhat, S., Herbert, A., & Baker, P. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04056.x · Cited by 53 · see citing works
Hayward, M. F., Curran, V. R., Curtis, B., Schulz, H., & Murphy, S. (2014) · BMC Medical Education · doi:10.1186/s12909-014-0279-9 · Cited by 52 · see citing works
“This method can be used for both formative and summative assessment purposes and has been widely used in postgraduate medical education [9,10], and CME [11].”
MethodThe Reliability of Multisource Feedback in Competency-Based Assessment Programs
van Loon, J., Overeem, K., Govaerts, M., Verhoeven, B. H., van der Vleuten, C. P. M., & Driessen, E. W. (2015) · Academic Medicine · doi:10.1097/acm.0000000000000763 · Cited by 52 · see citing works
“How am I doing?” Teaching medical students to elicit feedback during their clerkships
Milan, F. B., Dyche, L., & Fletcher, J. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.588732 · Cited by 47 · see citing works
“2000), that it is poorly delivered or not perceived as useful (Hewson & Little 1998; Sargeant et al. 2005; Torre et al. 2005; Veloski et al. 2006).”
BackgroundExamining the educational impact of the mini-CEX: a randomised controlled study
Martinsen, S. S. S., Espeland, T., Berg, E. A. R., Samstad, E., Lillebo, B., & Slørdahl, T. S. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02670-3 · Cited by 45 · see citing works
“We know from previous research that credibility is an important determinant of how impactful feedback is [33, 34].”
BackgroundThe Assessment of Emergency Physicians by a Regulatory Authority
Lockyer, J., Violato, C., & Fidler, H. (2006) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2006.tb00293.x · Cited by 42 · see citing works
Paige, J. T., Aaron, D. L., Yang, T., Howell, D. S., HILTON, C. W., Cohn, I., & Chauvin, S. W. (2008) · The American Surgeon · doi:10.1177/000313480807400909 · Cited by 42 · see citing works
Factors predicting doctors’ reporting of performance change in response to multisource feedback
Overeem, K., Wollersheimh, H. C., Arah, O. A., Cruijsberg, J. K., Grol, R. P. T. M., & Lombarts, K. M. J. M. H. (2012) · BMC Medical Education · doi:10.1186/1472-6920-12-52 · Cited by 41 · see citing works
“Doctors also complete a self-rating using questions identical to those on the colleague survey so that scores can be compared [2].”
BackgroundMethodThe quality of feedback during formative OSCEs depends on the tutors’ profile
Perron, N. J., Louis‐Simonet, M., Cerutti, B., Pfarrwaller, E., Sommer, J., & NENDAZ, M. R. (2016) · BMC Medical Education · doi:10.1186/s12909-016-0815-x · Cited by 40 · see citing works
“Feedback seems to be more effective when learners are more oriented towards learning than performance demonstration [4, 6, 9, 10], and when the feedback content matches some of the learners’ self-perceptions [11].”
BackgroundShalhoub, J., Marshall, D. C., & Ippolito, K. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-013417 · Cited by 40 · see citing works
Areemit, R., Cooper, C. M., Wirasorn, K., Paopongsawan, P., Panthongviriyakul, C., & Ramani, S. (2020) · Teaching and Learning in Medicine · doi:10.1080/10401334.2020.1813584 · Cited by 41 · see citing works
A novel workplace-based assessment for competency-based decisions and learner feedback
Hicks, P. J., Margolis, M. J., Carraccio, C. L., Clauser, B. E., Donnelly, K. M., Fromme, H. B., Gifford, K. A., Poynter, S. E., Schumacher, D. J., Schwartz, A., et al. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1461204 · Cited by 40 · see citing works
“The importance of intentional, carefully constructed feedback in medical education has been well documented (Lockyer 2003; Sargeant et al. 2005, 2006; Archer 2010).”
BackgroundA qualitative exploration: questioning multisource feedback in residency education
Yama, B. A., Hodgins, M., Boydell, K., & Schwartz, S. B. (2018) · BMC Medical Education · doi:10.1186/s12909-018-1270-7 · Cited by 37 · see citing works
Specialty‐specific multi‐source feedback: assuring validity, informing training
Davies, H. A., Archer, J. C., Bateman, A. C., Dewar, S., Crossley, J., Grant, J., & Southgate, L. J. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03162.x · Cited by 37 · see citing works
Do trainees value feedback in case-based discussion assessments?
Mehta, F., Brown, J., & Shaw, N. J. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.731100 · Cited by 36 · see citing works
“…2008) Studies describing feedback in the Mini-Clinical Evaluation Exercise and Multi-source feedback assessments have been published, however, there has been little work investigating feedback in CBDs (Hauer 2000; Kogan et al. 2002; Violato et al. 2003; Holemboe et al. 2004; Sargent et al. 2005).”
MethodCapturing patients' views on communication with anaesthetists: the CARE Measure
Mercer, S. W., HATCH, D. J., Murray, A., Murphy, D., & Eva, K. W. (2008) · Clinical Governance An International Journal · doi:10.1108/14777270810867320 · Cited by 35 · see citing works
“This feedback (scores and free text) is a basis on which a peer based discussion may provide individuals with insight and promote change where applicable (Sargeant et al., 2005).”
BackgroundMiles, A., Ginsburg, S., Sibbald, M., Tavares, W., Watling, C., & Stroud, L. F. (2020) · Medical Education · doi:10.1111/medu.14426 · Cited by 34 · see citing works
Experiencing patient-experience surveys: a qualitative study of the accounts of GPs
Edwards, A., Evans, R., White, P., & Elwyn, G. J. (2011) · British Journal of General Practice · doi:10.3399/bjgp11x567072 · Cited by 34 · see citing works
Jang, E. E., Lajoie, S. P., Wagner, M., Xu, Z., Poitras, E., & Naismith, L. M. (2016) · Journal of Educational Computing Research · doi:10.1177/0735633116678995 · Cited by 34 · see citing works
“Learners are more likely to ignore constructive feed-back when it is not aligned with their self-assessments (Eva, 2009; Eva et al., 2012; Sargeant et al., 2005; Sargeant, Mann, Sinclair, Van der Vleuten, & Metsemakers, 2008) highlighting the importance of gathering evidence of students’ confidence…”
BackgroundMultisource Feedback: Can It Meet Criteria for Good Assessment?
Lockyer, J. (2013) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.21171 · Cited by 33 · see citing works
Johnson, C., Keating, J. L., Farlie, M. K., Kent, F., Leech, M. T., & Molloy, E. (2019) · BMC Medical Education · doi:10.1186/s12909-019-1524-z · Cited by 32 · see citing works
“Self-assessments can alert the educator to the potential for a negative emotional reaction and rejection of the educator’s opinion if the learner rates their performance much higher than the educator [43].”
BackgroundWhat Evidence is There for the Use of Workplace-Based Assessment in Surgical Training?
Shalhoub, J., Vesey, A. T., & Fitzgerald, J. E. F. (2014) · Journal of surgical education · doi:10.1016/j.jsurg.2014.03.013 · Cited by 32 · see citing works
“Some of the strongest evidence in this systematic review was derived from detailed focus group data.(22) A total of 4 studies examined mini-CEX, 1 looked at DOPS, and 3 were concerned with multiple WBAs.”
BackgroundDoes source matter? Nurses' and Physicians' perceptions of interprofessional feedback
van Schaik, S. M., O’Sullivan, P., Eva, K. W., Irby, D. M., & Regehr, G. (2016) · Medical Education · doi:10.1111/medu.12850 · Cited by 31 · see citing works
Mazor, K. M., Canavan, C., Farrell, M., Margolis, M. J., & Clauser, B. E. (2008) · Academic Medicine · doi:10.1097/acm.0b013e318183e329 · Cited by 30 · see citing works
Hill, J. J., Asprey, A., Richards, S. H., & Campbell, J. L. (2012) · British Journal of General Practice · doi:10.3399/bjgp12x641429 · Cited by 30 · see citing works
An instrument for evaluating clinical teaching in Japan: content validity and cultural sensitivity
Kikukawa, M., Stalmeijer, R., Emura, S., Roff, S., & Scherpbier, A. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-179 · Cited by 35 · see citing works
“We therefore included three groups of stakeholders: residents, clinical teachers and educational experts, and although we also considered the inclusion of nurses and clerks, we decided against it, because both in Japan and in other parts of the world, it is not always the case that these groups observe residents and c…”
BackgroundKuklick, L., & Lindner, M. A. (2022) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2022.102146 · Cited by 32 · see citing works
Useful but Different: Resident Physician Perceptions of Interprofessional Feedback
Vesel, T. P., OʼBrien, B. C., Henry, D., & van Schaik, S. M. (2016) · Teaching and Learning in Medicine · doi:10.1080/10401334.2016.1146609 · Cited by 29 · see citing works
Hicks, P. J., Margolis, M. J., Poynter, S. E., Chaffinch, C. N., Tenney‐Soeiro, R., Turner, T. L., Waggoner-Fountain, L. A., Lockridge, R., Clyman, S. G., & Schwartz, A. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001057 · Cited by 28 · see citing works
Bogetz, A. L., Rassbach, C. E., Chan, T., & Blankenburg, R. L. (2016) · Academic Pediatrics · doi:10.1016/j.acap.2016.10.020 · Cited by 28 · see citing works
“Several studies on feedback from faculty have reported similar results, confirming the diverse influences on feedback acceptance.(4,5,11,21,22) This study demonstrates that feedback from patients and families may introduce additional and unique challenges that further affect feedback acceptance and use.”
Resultvan Leijen‐Zeelenberg, J. E., van Raak, A., Ruwaard, D., & Vrijhoef, H. J. (2013) · International Journal of Integrated Care · doi:10.5334/ijic.1467 · Cited by 27 · see citing works
Dannefer, E. F., Bierer, S. B., & Gladding, S. P. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652241 · Cited by 27 · see citing works
Self-evaluation: how well do surgery residents judge performance on a rotation?
Gow, K. W. (2013) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2013.01.010 · Cited by 27 · see citing works
Brænd, A. M., Gran, S. F., Frich, J. C., & Lindbæk, M. (2010) · Medical Teacher · doi:10.3109/01421590903516866 · Cited by 27 · see citing works
“nation with self-assessment, are used to promote quality improvement (Lockyer 2003; Sargeant et al. 2005; Wood et al. 2006).”
MethodA critical analysis of mini peer assessment tool (mini-PAT)
Abdulla, A. (2008) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2007.070077 · Cited by 24 · see citing works
Tailor, A., Dubrey, S. W., & Das, S. K. (2014) · Clinical Medicine · doi:10.7861/clinmedicine.14-5-510 · Cited by 24 · see citing works
Good to Great: Using 360-Degree Feedback to Improve Physician Emotional Intelligence
Hammerly, M. E., Harmon, L., & Schwaitzberg, S. D. (2014) · Journal of Healthcare Management · doi:10.1097/00115514-201409000-00009 · Cited by 42 · see citing works
“Ensure that the reviewer selection process is credible and transparent to the physicians (Sargeant et al., 2005).”
BackgroundArmson, H., Perrier, L., Roder, S., Shommu, N. S., Wakefield, J. G., Shaw, E. A., Zahorka, S., Elmslie, T., & Lofft, M. (2020) · Journal of Continuing Education in the Health Professions · doi:10.1097/ceh.0000000000000300 · Cited by 22 · see citing works
“Furthermore, the validity of objective data has been challenged by physicians because of the perception that it is divorced from the realities of clinical practice.(7,9,19) If identified practice gaps are discordant with physicians’ self-perceptions, doubts regarding the credibility, accuracy, or specificity of the as…”
BackgroundResultvan der Meulen, M. W., Smirnova, A., Heeneman, S., Egbrink, M. G. A. O., van der Vleuten, C. P. M., & Lombarts, K. M. J. M. H. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002767 · Cited by 21 · see citing works
McKay, J., Murphy, D. J., Bowie, P., Schmuck, M. L., Lough, M., & Eva, K. W. (2007) · BMJ Quality & Safety · doi:10.1136/qshc.2006.020750 · Cited by 20 · see citing works
Do assessor comments on a multi‐source feedback instrument provide learner‐centred feedback?
Vivekananda‐Schmidt, P., Mackillop, L. H., Crossley, J., & Wade, W. B. (2013) · Medical Education · doi:10.1111/medu.12249 · Cited by 19 · see citing works
Multisource Feedback and Narrative Comments: Polarity, Specificity, Actionability, and CanMEDS Roles
Lockyer, J., Sargeant, J. M., Richards, S. H., Campbell, J. L., & Rivera, L. (2018) · Journal of Continuing Education in the Health Professions · doi:10.1097/ceh.0000000000000183 · Cited by 19 · see citing works
Self-citationA Training Intervention through a 360° Multisource Feedback Model
Sureda, E., Moscoso, S. C., Sanduvete‐Chaves, S., & Sesé, A. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18179137 · Cited by 17 · see citing works
Using Peer Feedback to Promote Clinical Excellence in Hospital Medicine
Rosenthal, M. A., Sharpe, B. A., & Haber, L. A. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-06235-w · Cited by 17 · see citing works
Fornells-Vallés, J. M., Julià, X., Arnau, J., & Martínez-Carretero, J. M. (2008) · Educación Médica · doi:10.4321/s1575-18132008000100003 · Cited by 16 · see citing works
Using Videorecording to Enhance the Development of Novice Researchers' Interviewing Skills
Uhrenfeldt, L., Paterson, B. L., & Hall, E. O. C. (2007) · International Journal of Qualitative Methods · doi:10.1177/160940690700600104 · Cited by 16 · see citing works
“Researchers (Sargeant et al., 2005) in medicine have demonstrated the challenges inherent in an expert faculty member’s providing feedback about a videorecorded interview.”
Defining competency-based evaluation objectives in family medicine: communication skills.
Laughlin, T., Wetmore, S., Allen, T., Brailovsky, C., Crichton, T., Bethune, C., Donoff, M. G., & Lawrence, K. (2012) · Europe PMC (PubMed Central) · Cited by 15 · see citing works
Interventions in Reducing Caesarean Section in the World: A Systematic Review
Moradi, F., Aryankhesal, A., Heidari, M., Soroush, A., & Sadr, S. R. (2019) · Malaysian Journal of Medical Sciences · doi:10.21315/mjms2019.26.5.3 · Cited by 15 · see citing works
“indicated that the reaction and perception of physicians to multi-stage feedbacks can affect their performance (34).”
BackgroundHow does multisource feedback influence residency training? A qualitative case study
Hennel, E. K., Trachsel, A., Subotic, U., Lörwald, A. C., Harendza, S., & Huwendiek, S. (2022) · Medical Education · doi:10.1111/medu.14798 · Cited by 14 · see citing works
“By performing a reflexive thematic analysis, we extracted the influencing factors and the impact of MSF. Results: We found seven influencing factors: MSF is facilitated by the announcement of a clear goal of MSF, the training of raters on the MSF instrument, a longitudinal approach of observation, timing not too early…”
ResultInfluentialLa evaluación de competencias en Medicina
Sureda, E., Palmer, C. R., & Sesé, A. (2017) · Revista Clínica Española · doi:10.1016/j.rce.2017.05.004 · Cited by 14 · see citing works
Workplace-Based Assessment Instruments in the Health Sciences
Magnier, K. M., Dale, V. H. M., & Pead, M. J. (2012) · Journal of Veterinary Medical Education · doi:10.3138/jvme.1211-118r · Cited by 14 · see citing works
Feedback on doctors’ performance from parents and carers of children: a national pilot study
McGraw, M. E., Fellows, S., Long, A., Millar, H., Muir, G., Thomson, A., Uddin, S., Watt, J., & Williams, S. (2011) · Archives of Disease in Childhood · doi:10.1136/adc.2010.203174 · Cited by 14 · see citing works
- Authors
- Sargeant, J., Mann, K., & Ferrier, S.
- Affiliations
- Dalhousie University (CA)
- Corresponding author
- Sargeant, Joan
- ORCID
- Sargeant, Joan
- Venue
- Medical Education
- Publisher
- Wiley
- Published
- 2005-04-04
- Volume
- 39
- Issue
- 5
- Pages
- 497-504
- ISSN
- 0308-0110 (ISSN-L)
- 1365-2923
- OpenAlex
- W2158642891
- Cited by
- 174
- References
- 30
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 3 · 2023: 7 · 2022: 6 · 2021: 7 · 2020: 8 · 2019: 10 · 2018: 10 · 2017: 10 · 2016: 12 · 2015: 4 · 2014: 19 · 2013: 12 · 2012: 13
- FWCI
- 11.75
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Education (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Education › Student Assessment and Feedback
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- multisource feedback, 360-degree feedback, general practitioners, feedback credibility, perceived usefulness, performance evaluation (human resources), peer feedback, patient feedback, practice improvement
- MeSH terms
- Attitude of Health Personnel, Clinical Competence, Female, Humans, Male, Nova Scotia, Perception, Physicians, Family, Pilot Projects, Rural Health, Sensitivity and Specificity, Urban Health, Focus Groups, Feedback, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
