The Diagnostic Performance Feedback “Calibration Gap”: Why Clinical Experience Alone Is Not Enough to Prevent Serious Diagnostic Errors
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Medical diagnostic errors can be thought of as the bottom of the iceberg of patient safety—a hidden yet vast source of morbidity and mortality. According to the U.S. National Academy of Medicine, diagnostic errors represent a major public health problem likely to affect each of us in our lifetime. Diagnostic errors contribute to approximately 10% of deaths and 6% to 17% of hospital adverse events and are the leading cause of medical malpractice claims. Although typically multifactorial, the majority of diagnostic errors can be traced back to failures in bedside examination skills and clinical reasoning; knowledge and skill gaps appear to play important roles that have been underestimated in the context of an overemphasis on cognitive bias as a cause. A critical unanswered question for educational strategies to improve diagnosis is how diagnostic errors could remain so common, even with clinical presentations seen daily in clinical practice. In theory, accumulated clinical experience gained over time should be an “antidote” that gradually eliminates misdiagnosis, but experimental studies suggest that more years of experience does not necessarily confer greater diagnostic accuracy. Part of the problem is that feedback is essential for improved diagnostic performance, but is often lacking. For example, it has been shown that some short-term deaths after discharge from the emergency department (ED) likely reflect missed diagnoses of life-threatening illnesses. Unfortunately, such feedback rarely returns to individual ED clinicians, which prevents “recalibration” (i.e., adjusting one’s own mental models for diagnosis based on real-world accuracy in prior similar cases) that would otherwise improve diagnostic performance. In this article we argue that, absent systematic feedback, even years of sustained clinical practice may not produce the necessary experiential learning to prevent critical diagnostic errors, particularly for high-risk, low-frequency conditions. As a representative case, take the known public health problem of missed stroke in patients presenting with acute dizziness to the ED, where 45,000 to 75,000 strokes are missed at first contact each year and an estimated 10,000 to 25,000 serious preventable harms result from missed opportunities for early treatment. Dizziness and vertigo are common problems, accounting for approximately 3% of all ED
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
McGaghie, W. C., Issenberg, S. B., Cohen, E. R., Barsuk, J. H., & Wayne, D. B. (2011) · Academic Medicine · doi:10.1097/acm.0b013e318217e119 · Cited by 1,866 · cited 1×
“Simulationbased medical education with deliberate practice has been demonstrated to be superior to traditional clinical education.(21) Case-based computer simulations (often known as “virtual patients”(22)) are increasingly being used to enhance clinical education, and libraries of such patients could be used as a res…”
MethodImproving Diagnosis in Health Care
Committee on Diagnostic Error in Health Care, Board on Health Care Services, Institute of Medicine, & The National Academies of Sciences, E. (2015) · doi:10.17226/21794 · Cited by 1,376
Diagnostic Error in Medicine: Analysis of 583 Physician-Reported Errors
Schiff, G. D. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.333 · Cited by 638
The role of deliberate practice in the acquisition of clinical skills
Duvivier, R. J., van Dalen, J., Muijtjens, A. M., Moulaert, V. R., van der Vleuten, C. P., & Scherpbier, A. J. (2011) · BMC Medical Education · doi:10.1186/1472-6920-11-101 · Cited by 223 · cited 1×
“An alternative would be to leverage novel simulation-based educational approaches in a blended learning format to achieve expertise through deliberate (topic-focused) and mixed (epidemiologically valid) practice.(18,19) The theory of deliberate practice states that improved performance emerges when one engages with th…”
BackgroundKononowicz, A. A., Zary, N., Edelbring, S., Corral, J., & Hege, I. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0296-3 · Cited by 203
Cited by
Newman-Toker, D. E., Schaffer, A. C., Yu-Moe, C. W., Nassery, N., Saber Tehrani, A. S., Clemens, G. D., Wang, Z., Zhu, Y., Fanai, M., & Siegal, D. (2019) · Diagnosis · doi:10.1515/dx-2019-0019 · Cited by 178
Self-citationBridging the gap between uncertainty, confidence and diagnostic accuracy: calibration is key
Zwaan, L., & Hautz, W. E. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-009078 · Cited by 63
Improving Medical Residents’ Self-Assessment of Their Diagnostic Accuracy: Does Feedback Help?
Kuhn, J., van den Berg, P., Mamede, S., Zwaan, L., Bindels, P., & van Gog, T. (2022) · Advances in Health Sciences Education · doi:10.1007/s10459-021-10080-9 · Cited by 21
Metadata
- Authors
- Omron, R., Kotwal, S., Garibaldi, B. T., & Newman‐Toker, D. E.
- Venue
- AEM Education and Training
- Publisher
- Wiley
- Published
- 2018-09-17
- Volume
- 2
- Issue
- 4
- Pages
- 339-342
- ISSN
- 2472-5390 (ISSN-L)
- OpenAlex
- W2885214400
- Cited by
- 43
- References
- 5
- FWCI
- 3.72
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.