The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Electronic health record-based surveillance of diagnostic errors in primary care
Open accessAbstract
Background: Diagnostic errors in primary care are harmful but difficult to detect. The authors tested an electronic health record (EHR)-based method to detect diagnostic errors in routine primary care practice. Methods: The authors conducted a retrospective study of primary care visit records ‘triggered’ through electronic queries for possible evidence of diagnostic errors: Trigger 1: A primary care index visit followed by unplanned hospitalisation within 14 days and Trigger 2: A primary care index visit followed by ≥1 unscheduled visit(s) within 14 days. Control visits met neither criterion. Electronic trigger queries were applied to EHR repositories at two large healthcare systems between 1 October 2006 and 30 September 2007. Blinded physician–reviewers independently determined presence or absence of diagnostic errors in selected triggered and control visits. An error was defined as a missed opportunity to make or pursue the correct diagnosis when adequate data were available at the index visit. Disagreements were resolved by an independent third reviewer. Results: Queries were applied to 212 165 visits. On record review, the authors found diagnostic errors in 141 of 674 Trigger 1-positive records (positive predictive value (PPV)=20.9%, 95% CI 17.9% to 24.0%) and 36 of 669 Trigger 2-positive records (PPV=5.4%, 95% CI 3.7% to 7.1%). The control PPV of 2.1% (95% CI 0.1% to 3.3%) was significantly lower than that of both triggers (p≤0.002). Inter-reviewer reliability was modest, though higher than in comparable previous studies (к=0.37 (95% CI 0.31 to 0.44)). Conclusions: While physician agreement on diagnostic error remains low, an EHR-facilitated surveillance methodology could be useful for gaining insight into the origin of these errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Diagnostic error research has primarily focused on capturing disease-related factors [18,19,20,21], including laboratory results, imaging, and treatments, which are easier to extract from medical records.”
From Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors (2026) · cites this paper as [18]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty.
Fischhoff, B. (1975) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.1.3.288 · Cited by 1,970
Incidence and Types of Adverse Events and Negligent Care in Utah and Colorado
Thomas, E. J., Studdert, D. M., Burstin, H. R., Orav, E. J., Zeena, T., WILLIAMS, E. J., Howard, K., Weiler, P. C., & Brennan, T. A. (2000) · Medical Care · doi:10.1097/00005650-200003000-00003 · Cited by 1,673
Self-citationDiagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549
Classen, D. C., Resar, R., Griffin, F., Federico, F., Frankel, T., Kimmel, N., Whittington, J. C., Frankel, A., Seger, A., & James, B. C. (2011) · Health Affairs · doi:10.1377/hlthaff.2011.0190 · Cited by 1,027
Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims
Gandhi, T. K., Kachalia, A., Thomas, E. J., Puopolo, A. L., Yoon, C., Brennan, T. A., & Studdert, D. M. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-7-200610030-00006 · Cited by 662
Self-citationComputerized Surveillance of Adverse Drug Events in Hospital Patients
Classen, D. C. (1991) · JAMA · doi:10.1001/jama.1991.03470200059035 · Cited by 643
Diagnostic Error in Medicine: Analysis of 583 Physician-Reported Errors
Schiff, G. D. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.333 · Cited by 638
Jha, A. K., Kuperman, G. J., Teich, J. M., Leape, L. L., Shea, B. S., Rittenberg, E., Burdick, E., Seger, D. L., Vliet, M. V., & Bates, D. W. (1998) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1998.0050305 · Cited by 514
Detecting Adverse Events Using Information Technology
Bates, D. W., Evans, R. S., Murff, H. J., Stetson, P. D., Pizziferri, L., & Hripcsak, G. M. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1074 · Cited by 467
Measuring errors and adverse events in health care
Thomas, E. J., & Petersen, L. A. (2003) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2003.20147.x · Cited by 456
Self-citationDiagnostic Errors—The Next Frontier for Patient Safety
Newman‐Toker, D. E. (2009) · JAMA · doi:10.1001/jama.2009.249 · Cited by 434
Hindsight != foresight: the effect of outcome knowledge on judgment under uncertainty
Fischhoff, B. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.4.304 · Cited by 317
The Vanishing Nonforensic Autopsy
Shojania, K. G., & Burton, E. C. (2008) · New England Journal of Medicine · doi:10.1056/nejmp0707996 · Cited by 295
Computerized surveillance of adverse drug events in hospital patients.
Classen, D. C., Pestotnik, S. L., Evans, R. S., & Burke, J. P. (1991) · PubMed · Cited by 281
The Reliability of Medical Record Review for Estimating Adverse Event Rates
Thomas, E. J., Lipsitz, S. R., Studdert, D. M., & Brennan, T. A. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-136-11-200206040-00009 · Cited by 259
Self-citationDiagnostic Errors in Medicine: A Case of Neglect
Graber, M. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31015-4 · Cited by 259
Phillips, R. L. (2004) · Quality and Safety in Health Care · doi:10.1136/qshc.2003.008029 · Cited by 233
Analysing potential harm in Australian general practice: an incident‐monitoring study
Bhasale, A. L., Miller, G. C., Reid, S. E., & Britt, H. C. (1998) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.1998.tb140186.x · Cited by 227
Using Computerized Data to Identify Adverse Drug Events in Outpatients
Honigman, B., Lee, J. D., Rothschild, J., Light, P., Pulling, R. M., Yu, T., & Bates, D. W. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080254 · Cited by 227
Singh, H., Thomas, E. J., Mani, S., Sittig, D., Arora, H., Espadas, D., Khan, M. M., & Petersen, L. A. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.263 · Cited by 212
Self-citationComputerized surveillance of adverse drug events in hospital patients
Classen, D. C. (2005) · BMJ Quality & Safety · doi:10.1136/qshc.2002.002972 · Cited by 211
Elder, N. C., & Dovey, S. M. (2002) · PubMed · Cited by 196
Patient Record Review of the Incidence, Consequences, and Causes of Diagnostic Adverse Events
Zwaan, L. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2010.146 · Cited by 194
Diagnosing Diagnosis Errors: Lessons from a Multi-institutional Collaborative Project
Schiff, G. D., Kim, S., Abrams, R. I., Cosby, K. S., Lambert, B. L., Elstein, A. S., Hasler, S., Krosnjar, N., Odwazny, R., Wisniewski, M. F., et al. (2005) · Cited by 167
Singh, H., Thomas, E. J., Sittig, D. F., Wilson, L., Espadas, D., Khan, M. M., & Petersen, L. A. (2010) · The American Journal of Medicine · doi:10.1016/j.amjmed.2009.07.027 · Cited by 160
Self-citationFrequency of Failure to Inform Patients of Clinically Significant Outpatient Test Results
Casalino, L. P., Dunham, D., Chin, M. H., Bielang, R., Kistner, E. O., Karrison, T. G., Ong, M. K., Sarkar, U., McLaughlin, M. A., & Meltzer, D. O. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.130 · Cited by 159
Understanding diagnostic errors in medicine: a lesson from aviation
Singh, H. (2006) · Quality and Safety in Health Care · doi:10.1136/qshc.2005.016444 · Cited by 156
Self-citationLocalio, A. R., Weaver, S. L., Landis, J. R., Lawthers, A. G., Brennan, T. A., Hebert, L., & Sharp, T. J. (1996) · Annals of Internal Medicine · doi:10.7326/0003-4819-125-6-199609150-00005 · Cited by 152
Paid Malpractice Claims for Adverse Events in Inpatient and Outpatient Settings
Bishop, T. F. (2011) · JAMA · doi:10.1001/jama.2011.813 · Cited by 147
Errors of Diagnosis in Pediatric Practice: A Multisite Survey
Singh, H., Thomas, E. J., Wilson, L., Kelly, P. A., Pietz, K., Elkeeb, D., & Singhal, G. (2010) · PEDIATRICS · doi:10.1542/peds.2009-3218 · Cited by 137
Self-citationSingh, H., Hirani, K., Kadiyala, H., Rudomiotov, O., Giardina, T. D., Khan, M. M., & Wahls, T. L. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2009.25.6636 · Cited by 136
Self-citationWhy Diagnostic Errors Don’t Get Any Respect—And What Can Be Done About Them
Wachter, R. M. (2010) · Health Affairs · doi:10.1377/hlthaff.2009.0513 · Cited by 134
Chandra, A., Nundy, S., & Seabury, S. A. (2005) · Health Affairs · doi:10.1377/hlthaff.w5.240 · Cited by 134
Errors in cancer diagnosis: current understanding and future directions
Singh, H., Sethi, S., Raber, M., & Petersen, L. A. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2007.13.2142 · Cited by 121
Self-citationMissed Opportunities to Initiate Endoscopic Evaluation for Colorectal Cancer Diagnosis
Singh, H., Daci, K., Petersen, L. A., Collins, C., Petersen, N. J., Shethia, A., & El‐Serag, H. B. (2009) · The American Journal of Gastroenterology · doi:10.1038/ajg.2009.324 · Cited by 118
Self-citationPatient Safety beyond the Hospital
Gandhi, T. K., & Lee, T. H.-Y. (2010) · New England Journal of Medicine · doi:10.1056/nejmp1003294 · Cited by 110
Identifying Diagnostic Errors in Primary Care Using an Electronic Screening Algorithm
Singh, H. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.3.302 · Cited by 107
Self-citationAmbulatory care adverse events and preventable adverse events leading to a hospital admission
Woods, D. M., Thomas, E. J., Holl, J. L., Weiss, K. B., & Brennan, T. (2007) · BMJ Quality & Safety · doi:10.1136/qshc.2006.021147 · Cited by 103
Self-citationStrategies for Detecting Adverse Drug Events among Older Persons in the Ambulatory Setting
Field, T. S., Gurwitz, J. H., Harrold, L. R., Rothschild, J. M., DeBellis, K. R., Seger, A. C., Fish, L. S., Garber, L., Kelleher, M., & Bates, D. W. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1586 · Cited by 97
Active surveillance using electronic triggers to detect adverse events in hospitalized patients
Szekendi, M. K. (2006) · BMJ Quality & Safety · doi:10.1136/qshc.2005.014589 · Cited by 96
The Elephant of Patient Safety: What You See Depends on How You Look
Shojania, K. G. (2010) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(10)36058-2 · Cited by 76
Bhasale, A. L. (1998) · Family Practice · doi:10.1093/fampra/15.4.308 · Cited by 68
Medical Diagnoses Commonly Associated With Pediatric Malpractice Lawsuits in the United States
McAbee, G. N., Donn, S. M., Mendelson, R. A., McDonnell, W. M., Gonzalez, J. L., & Ake, J. K. (2008) · PEDIATRICS · doi:10.1542/peds.2008-1594 · Cited by 59
Reducing Diagnostic Error Through Medical Home-Based Primary Care Reform
Singh, H., & Graber, M. L. (2010) · JAMA · doi:10.1001/jama.2010.1035 · Cited by 52
Self-citationDiagnostic error in a national incident reporting system in the UK
Sevdalis, N., Jacklin, R., Arora, S., Vincent, C. A., & Thomson, R. G. (2010) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2009.01328.x · Cited by 51
Diagnostic errors in ambulatory care: dimensions and preventive strategies
Singh, H., & Weingart, S. N. (2009) · Advances in Health Sciences Education · doi:10.1007/s10459-009-9177-z · Cited by 51
Self-citationIs Ambulatory Patient Safety Just Like Hospital Safety, Only without the “Stat”?
Wachter, R. M. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-7-200610030-00014 · Cited by 44
Development of trigger tools for surveillance of adverse events in ambulatory surgery
Kaafarani, H. M. A., Rosen, A. K., Nebeker, J. R., Shimada, S. L., Mull, H. J., Rivard, P. E., Savitz, L. A., Helwig, A., Shin, M. H., & Itani, K. M. F. (2010) · BMJ Quality & Safety · doi:10.1136/qshc.2008.031591 · Cited by 38
Forster, A. J., O’Rourke, K., Shojania, K. G., & van Walraven, C. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.11.019 · Cited by 32
Detecting Adverse Drug Events Using a Nursing Home Specific Trigger Tool.
Handler, S. M., & Hanlon, J. T. (2010) · PubMed · Cited by 27
Paid Malpractice Claims for Adverse Events in Inpatient and Outpatient Settings
Bishop, T. F., Ryan, A. K., & Casalino, L. P. (2012) · Survey of Anesthesiology · doi:10.1097/01.sa.0000418901.89933.74 · Cited by 22
Cited by
Sorted by citations
Singh, H., Meyer, A. N. D., & Thomas, E. J. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-002627 · Cited by 677
Self-citationThe incidence of diagnostic error in medicine
Graber, M. L. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001615 · Cited by 626
Types and Origins of Diagnostic Errors in Primary Care Settings
Singh, H., Giardina, T. D., Meyer, A. N. D., Forjuoh, S. N., Reis, M. D., & Thomas, E. J. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.2777 · Cited by 559
Self-citationThe Global Burden of Diagnostic Errors in Primary Care
Singh, H., Schiff, G. D., Graber, M. L., Onakpoya, I., & Thompson, M. J. (2017) · BMJ Quality & Safety · doi:10.1136/bmjqs-2016-005401 · Cited by 395
Self-citationNewman‐Toker, D. E., Moy, E. M., Valente, E., Coffey, R. M., & Hines, A. L. (2014) · Diagnosis · doi:10.1515/dx-2013-0038 · Cited by 263
Metadata
- Authors
- Singh, H., Giardina, T. D., Forjuoh, S. N., Reis, M. D., Kosmach, S., Khan, M. M., & Thomas, E. J.
- Affiliations
- Michael E. DeBakey VA Medical Center (US)
- Texas A&M Health Science Center (US)
- The University of Texas Health Science Center at Houston (US)
- Memorial Hermann (US)
- Corresponding author
- Singh, Hardeep
- ORCID
- Venue
- BMJ Quality & Safety
- Publisher
- BMJ
- Published
- 2011-10-13
- Volume
- 21
- Issue
- 2
- Pages
- 93-100
- ISSN
- 1475-3898 (ISSN-L)
- 2044-5415
- 2044-5423
- PMC
- PMC3680372
- OpenAlex
- W2169041559
- Cited by
- 131
- References
- 52
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 5 · 2023: 20 · 2022: 10 · 2021: 11 · 2020: 6 · 2019: 7 · 2018: 12 · 2017: 10 · 2016: 6 · 2015: 16 · 2014: 14 · 2013: 5
- FWCI
- 5.89
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- BMJ Quality & Safety (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- U.S. Department of Veterans Affairs
- National Institutes of Health (HFP90-020)
- Agency for Healthcare Research and Quality (R18 HS17244-02, R18HS017244)
- National Cancer Institute (K23CA125585)
- Topics
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- diagnostic errors, electronic health records, primary care, emergency admissions, unscheduled visits, positive predictive value, missed diagnosis
- MeSH terms
- Diagnostic Errors, Humans, Medical Audit, Primary Health Care, Retrospective Studies, Texas, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
