CoChat
Sign Up
Cited by 280Innovations in Medical EducationClinical Reasoning and Diagnostic SkillsStudent Assessment and Feedback

Difficulties in Recognizing Oneʼs Own Incompetence: Novice Physicians Who Are Unskilled and Unaware of It

HODGES, BRIAN, REGEHR, GLENN, MARTIN, DAWN·2001·Academic Medicine
DOI 10.1097/00001888-200110001-00029
Open this paper in CoChat — free account

Abstract

Kruger and Dunning1 published an elegant series of studies illustrating that deficits in the ability to assess one's own competence were common among the subjects in the lowest-scoring quartile on a wide range of tasks such as recognition of humor and logical reasoning. These researchers demonstrated that subjects who had the highest scores underestimated their abilities but were able to recalibrate following exposure to the performances of others. Those scoring in the middle quartiles were generally accurate in assessing their skills, and this remained so after exposure to others' performances. People in the lowest quartile, however, greatly overestimated their abilities and failed almost entirely to correct their self-assessments following exposure to the performances of others. Kruger and Dunning concluded that those who know less also know less about what they know. In their study they raise the question of how to help learners who intrinsically overestimate their abilities and therefore do not perceive their own incompetence. The work of Kruger and Dunning is intriguing for medical educators because, if replicated with doctors, it suggests that those with the least skill may be most at risk of inaccurately assessing their abilities. We set out to determine whether the same problems of self-assessment so consistently replicated in the studies of Kruger and Dunning could be demonstrated in family medicine residents. We reported in a previous study that the calibration of the self-assessment of novices improved when they viewed benchmark performances of others.2 In that study, the self-assessments of residents remained less than perfect even after viewing others' performances. Having reviewed the work of Kruger and Dunning, we decided to see whether it was indeed the lowest performers in our group who were most inaccurate in their self-assessments. We also wondered whether those in the highest-performing group would underestimate their skills but recalibrate following exposure to the performance of others. With these questions in mind, we reanalyzed the study data following the method of Kruger and Dunning and, indeed, both of these findings were confirmed. Method Twenty-four first-year family medicine residents interviewed a standardized patient (SP) in a difficult “breaking bad news” scenario. Two expert faculty members with extensive experience in teaching communication skills observed the ten-minute interviews. Both the residents and the experts rated the residents' performances on six nine-point rating scales (knowledge, rapport, emotional control, flexibility, questioning skills, and overall). The residents were then shown four videotaped interviews of the same SP scenario in which interviewers demonstrated a range of ability from highly skilled to quite incompetent. The videos were designed to represent a spectrum of performances from incompetence to advanced competence. The four videos were chosen from an initial pool of ten videos that were evaluated independently by two communication experts and four academic family physicians. Reliabilities of the videos and the scales had been established and are described elsewhere.2 Inter-rater reliability between the expert raters was .93 and Cronbach's alpha was .92, suggesting that it was reasonable to collapse the six rating scales into one score. It had also been shown in the previous research that first-year residents clearly performed differently than did more experienced residents, the former group showing statistically significantly lower initial correlation with expert observers' ratings. In an effort to replicate the work of Kruger and Dunning, only these novices were selected for the current analysis. Following their own interviews, the residents were asked to assess the videotaped performances and were then given an opportunity to rescore their own performances. The residents' ratings of their own performances were then compared with scores of experts in two ways. First, the experts' ratings of the residents were sorted in tertiles and plotted against both the pre- and the post-video self-assessed ratings of the residents. Second, in an effort to standardize the use of scales across the residents, each resident's score was rescaled relative to his or her mean assessment of the four benchmark videos. A z score was calculated for each resident's performance by subtracting the resident's mean score of the videos from his or her actual self-assessment and dividing by the standard deviation of the video ratings. The use of z scores requires some explanation. The example of one resident illustrates how analysis of raw scores alone could have obscured the fact that some residents became significantly more accurate in their self-assessments following the videos. Resident 21 gave himself a slightly higher raw score than the experts did prior to his viewing the benchmark videos (6.0 versus 5.9, respectively). He rated himself still higher (6.9) after he h

Integrity signals

Registry

Crossref · 10.1097/00001888-200110001-00029

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
HODGES, B., REGEHR, G., & MARTIN, D.
Affiliations
Toronto General Hospital (CA)
ORCID
Venue
Academic Medicine
Publisher
Oxford University Press (OUP)
Published
2001-10
Volume
76
Issue
Supplement
Pages
S87-S89
ISSN
1040-2446 (ISSN-L)
DOI
10.1097/00001888-200110001-00029
OpenAlex
W1978974820
Cited by
280
References
3
Citations by year
2026: 5 · 2025: 11 · 2024: 13 · 2023: 13 · 2022: 12 · 2021: 15 · 2020: 8 · 2019: 10 · 2018: 14 · 2017: 21 · 2016: 13 · 2015: 10 · 2014: 16 · 2013: 16 · 2012: 16
FWCI
2.71
Type
journal article
Language
en
Open access
closed
Locations
  • Academic Medicine (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
  • Social Sciences › Education › Student Assessment and Feedback
Keywords
self-assessment, overconfidence, Dunning-Kruger effect, family medicine residents, communication skills, breaking bad news, benchmarking, standardized patients, physician self-awareness, medical education, physician-patient communication
MeSH terms
Awareness, Clinical Competence, Internship and Residency, Physicians, Self Efficacy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review