Identifying Diagnostic Errors in Primary Care Using an Electronic Screening Algorithm
Abstract
BACKGROUND: Diagnostic errors are the leading basis for malpractice claims in primary care, yet these errors are underidentified and understudied. Computerized methods used to screen for other types of errors (eg, medication related) have not been applied to diagnostic errors. Our objectives were to assess the feasibility of computerized screening to identify diagnostic errors in primary care and to categorize diagnostic breakdowns using a recently published taxonomy. METHODS: We used an algorithm to screen the electronic medical records of patients at a single hospital that is part of a closed health care system. A Structured Query Language-based program detected the presence of 1 of 2 mutually exclusive electronic screening criteria: screen 1, a primary care visit (index visit) followed by a hospitalization in the next 10 days; or screen 2, an index visit followed by 1 or more primary care, urgent care, or emergency department visits within 10 days. Two independent, blinded reviewers determined the presence or absence of diagnostic error through medical record review of visits with positive and negative screening results. RESULTS: Among screen 1 and 2 positive visits, 16.1% and 9.4%, respectively, were associated with a diagnostic error. The error rate was 4% in control cases that met neither screening criterion. The most common primary errors in the diagnostic process were failure or delay in eliciting information and misinterpretation or suboptimal weighing of critical pieces of data from the history and physical examination. The most common secondary errors were suboptimal weighing or prioritizing of diagnostic probabilities and failure to recognize urgency of illness or its complications. CONCLUSIONS: Electronic screening has potential to identify records that may contain diagnostic errors in primary care, and its performance is comparable to screening tools for other types of errors. Future studies that validate these findings in other settings could provide improvement initiatives in this area.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Diagnostic errors appear to be burdensome in the training environment.
“Such errors also appear to be burdensome in the training environment.4–6”
From Reducing Diagnostic Errors through Effective Communication: Harnessing the Power of Information Technology (2008) · cites this paper as [4]
Data gathering problems may be the most common reasons for diagnostic errors, at least among trainees.
“Data gathering problems may be the most common reasons for diagnostic errors, at least among trainees.4”
From Reducing Diagnostic Errors through Effective Communication: Harnessing the Power of Information Technology (2008) · cites this paper as [4]
Cites
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
DEYO, R. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90133-8 · Cited by 10,452 · see citing works
Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty.
Fischhoff, B. (1975) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.1.3.288 · Cited by 1,913 · see citing works
Diagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549 · see citing works
Weingart, S. N. (2000) · BMJ · doi:10.1136/bmj.320.7237.774 · Cited by 658 · see citing works
Jha, A. K., Kuperman, G. J., Teich, J. M., Leape, L. L., Shea, B. S., Rittenberg, E., Burdick, E., Seger, D. L., Vliet, M. V., & Bates, D. W. (1998) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1998.0050305 · Cited by 514 · see citing works
Detecting Adverse Events Using Information Technology
Bates, D. W., Evans, R. S., Murff, H. J., Stetson, P. D., Pizziferri, L., & Hripcsak, G. M. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1074 · Cited by 467 · see citing works
Measuring errors and adverse events in health care
Thomas, E. J., & Petersen, L. A. (2003) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2003.20147.x · Cited by 456 · see citing works
Costs of medical injuries in Utah and Colorado.
Thomas, E. J., Studdert, D. M., Newhouse, J. P., Zbar, B. I. W., Howard, K., WILLIAMS, E. J., & Brennan, T. A. (1999) · PubMed · Cited by 395 · see citing works
Detecting adverse events for patient safety research: a review of current methodologies
Murff, H. J., Patel, V. L., Hripcsak, G. M., & Bates, D. W. (2003) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2003.08.003 · Cited by 357 · see citing works
Methodology and rationale for the measurement of harm with trigger tools
Resar, R. K. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.suppl_2.ii39 · Cited by 312 · see citing works
Hindsight != foresight: the effect of outcome knowledge on judgment under uncertainty
Fischhoff, B. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.4.304 · Cited by 298 · see citing works
Sandars, J. E., & Esmail, A. (2003) · Family Practice · doi:10.1093/fampra/cmg301 · Cited by 298 · see citing works
Automated Detection of Adverse Events Using Natural Language Processing of Discharge Summaries
Melton, G. B., & Hripcsak, G. M. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1794 · Cited by 298 · see citing works
The Reliability of Medical Record Review for Estimating Adverse Event Rates
Thomas, E. J., Lipsitz, S. R., Studdert, D. M., & Brennan, T. A. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-136-11-200206040-00009 · Cited by 259 · see citing works
Diagnostic Errors in Medicine: A Case of Neglect
Graber, M. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31015-4 · Cited by 259 · see citing works
Phillips, R. L. (2004) · Quality and Safety in Health Care · doi:10.1136/qshc.2003.008029 · Cited by 233 · see citing works
The moral limits of Feinberg's liberalism
Doppelt, G. (1993) · Inquiry · doi:10.1080/00201749308602322 · Cited by 231 · see citing works
Using Computerized Data to Identify Adverse Drug Events in Outpatients
Honigman, B., Lee, J. D., Rothschild, J., Light, P., Pulling, R. M., Yu, T., & Bates, D. W. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080254 · Cited by 227 · see citing works
Computer surveillance of hospital acquired infections and antibiotic use
Evans, R. S. (1987) · American Journal of Infection Control · doi:10.1016/0196-6553(87)90148-9 · Cited by 216 · see citing works
Why did I miss the diagnosis? Some cognitive explanations and educational implications
Bordage, G. (1999) · Academic Medicine · doi:10.1097/00001888-199910000-00065 · Cited by 186 · see citing works
Diagnosing Diagnosis Errors: Lessons from a Multi-institutional Collaborative Project
Schiff, G. D., Kim, S., Abrams, R. I., Cosby, K. S., Lambert, B. L., Elstein, A. S., Hasler, S., Krosnjar, N., Odwazny, R., Wisniewski, M. F., et al. (2005) · Cited by 167 · see citing works
Unscheduled returns to the emergency department: an outcome of medical errors?
Núñez, S. (2006) · BMJ Quality & Safety · doi:10.1136/qshc.2005.016618 · Cited by 157 · see citing works
Chandra, A., Nundy, S., & Seabury, S. A. (2005) · Health Affairs · doi:10.1377/hlthaff.w5.240 · Cited by 134 · see citing works
Electronically Screening Discharge Summaries for Adverse Medical Events
Murff, H. J. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1201 · Cited by 116 · see citing works
Strategies for Detecting Adverse Drug Events among Older Persons in the Ambulatory Setting
Field, T. S., Gurwitz, J. H., Harrold, L. R., Rothschild, J. M., DeBellis, K. R., Seger, A. C., Fish, L. S., Garber, L., Kelleher, M., & Bates, D. W. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1586 · Cited by 97 · see citing works
Active surveillance using electronic triggers to detect adverse events in hospitalized patients
Szekendi, M. K. (2006) · BMJ Quality & Safety · doi:10.1136/qshc.2005.014589 · Cited by 96 · see citing works
Evaluation of Screening Criteria for Adverse Events in Medical Patients
Bates, D. W., Neil, A. C. O., Petersen, L. A., Lee, T. H.-Y., & Brennan, T. A. (1995) · Medical Care · doi:10.1097/00005650-199505000-00002 · Cited by 80 · see citing works
Hammons, T., Piland, N. F., Small, S., Hatlie, M. J., & Burstin, H. R. (2003) · Journal of Ambulatory Care Management · doi:10.1097/00004479-200301000-00007 · Cited by 58 · see citing works
Audit of unexpected return visits to an accident and emergency department.
Wilkins, P., & Beckett, M. W. (1992) · Emergency Medicine Journal · doi:10.1136/emj.9.4.352 · Cited by 43 · see citing works
Unscheduled return visits by patients to the accident and emergency department.
O’DWYER, F. G., & Bodiwala, G. G. (1991) · Emergency Medicine Journal · doi:10.1136/emj.8.3.196 · Cited by 42 · see citing works
Holohan, T. V., Colestro, J., Grippi, J., et al. (2005) · Southern Medical Journal · doi:10.1097/01.smj.0000170729.51651.f7 · Cited by 27 · see citing works
Automated surveillance for adverse events in hospitalized patients: back to the future
Kilbridge, P. M. (2006) · BMJ Quality & Safety · doi:10.1136/qshc.2006.018218 · Cited by 24 · see citing works
An Initial Assessment of the Veterans Affairs Occurrence Screening Program
Goldman, R. L., & Walder, D. J. (1992) · QRB - Quality Review Bulletin · doi:10.1016/s0097-5990(16)30552-8 · Cited by 7 · see citing works
Risk Screening for Adverse Outcomes in Subacute Care
Evans, R. L., & Connis, R. T. (1996) · Psychological Reports · doi:10.2466/pr0.1996.78.3.1043 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Singh, H., Meyer, A. N. D., & Thomas, E. J. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-002627 · Cited by 677 · see citing works
Self-citationDiagnostic Error in Medicine: Analysis of 583 Physician-Reported Errors
Schiff, G. D. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.333 · Cited by 638 · see citing works
Medical Errors Involving Trainees: A Study of Closed Malpractice Claims From 5 Insurers
Singh, H., Thomas, E. J., Petersen, L. A., & Studdert, D. M. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.19.2030 · Cited by 407 · see citing works
Self-citationThe Global Burden of Diagnostic Errors in Primary Care
Singh, H., Schiff, G. D., Graber, M. L., Onakpoya, I., & Thompson, M. J. (2017) · BMJ Quality & Safety · doi:10.1136/bmjqs-2016-005401 · Cited by 395 · see citing works
Self-citationSingh, H., Naik, A. D., Rao, R., & Petersen, L. A. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0393-z · Cited by 162 · see citing works
Self-citationPaid Malpractice Claims for Adverse Events in Inpatient and Outpatient Settings
Bishop, T. F. (2011) · JAMA · doi:10.1001/jama.2011.813 · Cited by 147 · see citing works
Communication outcomes of critical imaging results in a computerized notification system
Singh, H., Arora, H. S., Vij, M. S., Rao, R., Khan, M. M., & Petersen, L. A. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2280 · Cited by 140 · see citing works
Self-citationElectronic Health Record-Based Surveillance of Diagnostic Errors in Primary Care
Singh, H., Giardina, T. D., Forjuoh, S. N., Reis, M. D., Kosmach, S., Khan, M. M., & Thomas, E. J. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000304 · Cited by 131 · see citing works
Self-citationErrors in cancer diagnosis: current understanding and future directions
Singh, H., Sethi, S., Raber, M., & Petersen, L. A. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2007.13.2142 · Cited by 121 · see citing works
Self-citationNurek, M., Kostopoulou, O., Delaney, B. C., & Esmail, A. (2015) · European Journal of General Practice · doi:10.3109/13814788.2015.1043123 · Cited by 60 · see citing works
- Authors
- Singh, H.
- Affiliations
- Michael E. DeBakey VA Medical Center (US)
- Corresponding author
- Singh, Hardeep
- ORCID
- Singh, Hardeep
- Venue
- Archives of Internal Medicine
- Publisher
- American Medical Association (AMA)
- Published
- 2007-02-12
- Volume
- 167
- Issue
- 3
- Pages
- 302
- ISSN
- 0003-9926 (ISSN-L)
- OpenAlex
- W2009728104
- Cited by
- 107
- References
- 34
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 3 · 2023: 5 · 2022: 2 · 2021: 3 · 2020: 3 · 2019: 7 · 2018: 7 · 2017: 7 · 2016: 6 · 2015: 7 · 2014: 7 · 2013: 9 · 2012: 14
- FWCI
- 4.50
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Archives of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (1P01 HS 11544-01)
- National Center for Research Resources (K12 RR 17665, K12RR017665)
- Topics
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- diagnostic errors, primary care, electronic health records, delayed diagnosis, malpractice claims, screening feasibility
- MeSH terms
- Aged, Algorithms, Diagnostic Errors, Feasibility Studies, Female, Hospitalization, Humans, Male, Middle Aged, Predictive Value of Tests, Primary Health Care, Software, Medical Records Systems, Computerized
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
