Checklists to Prevent Diagnostic Errors: A Pilot Randomized Controlled Trial
Open accessAbstract
Background: Many diagnostic errors are caused by premature closure of the diagnostic process. To help prevent premature closure, we developed checklists that prompt physicians to consider all reasonable diagnoses for symptoms that commonly present in primary care. Methods: We enrolled 14 primary care physicians and 100 patients in a randomized clinical trial. The study took place in an emergency department (5 physicians) and a same-day access clinic (9 physicians). The physicians were randomized to usual care vs. diagnostic checklist. After completing the history and physical exam, checklist physicians read aloud a differential diagnosis checklist for the chief complaint. The primary outcome was diagnostic error, which was defined as a discrepancy between the diagnosis documented at the acute visit and the diagnosis based on a 1-month follow-up phone call and record review. Results: There were 17 diagnostic errors. The mean error rate among the seven checklist physicians was not significantly different from the rate among the seven usual-care physicians (11.2% vs. 17.8%; p=0.46). In a post-hoc subgroup analysis, emergency physicians in the checklist group had a lower mean error rate than emergency physicians in the usual-care group (19.1% vs. 45.0%; p=0.04). Checklist physicians considered more diagnoses than usual-care physicians during the patient encounters (6.5 diagnoses [SD 4.2] vs. 3.4 diagnoses [SD 2.0], p<0.001). Conclusions: Checklists did not improve the diagnostic error rate in this study. However further development and testing of checklists in larger studies may be warranted.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Checklists assist clinicians in considering a broader range of potential factors and encourage a more comprehensive review of important items and variables.
“They also assist clinicians in considering a broader range of potential factors and encourage a more comprehensive review of important items and variables [204,205].”
From Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors (2026) · cites this paper as [204]
The Main findings report that physicians using a checklist generated a broader differential diagnosis, and the Intervention describes a review of possible diagnoses, including commonly missed and serious ones. This supports broader consideration and review of diagnostic possibilities, but the paper does not establish the broader claim about reviewing important items and variables generally. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Diagnostic error in internal medicine
Graber, M. L., Franklin, N., & Gordon, R. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.13.1493 · Cited by 1,549 · see citing works
Self-citationDesign and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236 · see citing works
Using the Student's t-test with extremely small sample sizes
de Winter, J. C. F. (2013) · University of Massachusetts Amherst · doi:10.7275/e4r6-dj05 · Cited by 992 · see citing works
National Ambulatory Medical Care Survey: 2001 summary.
Cherry, D. K., Burt, C. W., & Woodwell, D. A. (2003) · PubMed · Cited by 686 · see citing works
Singh, H., Meyer, A. N. D., & Thomas, E. J. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-002627 · Cited by 677 · see citing works
Diagnostic Error in Medicine: Analysis of 583 Physician-Reported Errors
Schiff, G. D. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.333 · Cited by 638 · see citing works
The incidence of diagnostic error in medicine
Graber, M. L. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001615 · Cited by 626 · see citing works
Self-citationTypes and Origins of Diagnostic Errors in Primary Care Settings
Singh, H., Giardina, T. D., Meyer, A. N. D., Forjuoh, S. N., Reis, M. D., & Thomas, E. J. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.2777 · Cited by 559 · see citing works
Cognitive interventions to reduce diagnostic error: a narrative review
Graber, M. L., KISSAM, S. M., Payne, V. L., Meyer, A. N. D., Sorensen, A. V., Lenfestey, N. F., Tant, E. M., Henriksen, K., LaBresh, K. A., & Singh, H. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000149 · Cited by 448 · see citing works
Self-citationChecklists to Reduce Diagnostic Errors
Ely, J. W., Graber, M. L., & Croskerry, P. (2011) · Academic Medicine · doi:10.1097/acm.0b013e31820824cd · Cited by 366 · see citing works
Self-citationNational Ambulatory Medical Care Survey: 2000 summary.
Cherry, D. K., & Woodwell, D. A. (2002) · PubMed · Cited by 294 · see citing works
Physicians’ Diagnostic Accuracy, Confidence, and Resource Requests: A Vignette Study
Meyer, A. N. D., Payne, V. L., Meeks, D. W., Rao, R., & Singh, H. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.10081 · Cited by 268 · see citing works
Can Electronic Clinical Documentation Help Prevent Diagnostic Errors?
Schiff, G. D., & Bates, D. W. (2010) · New England Journal of Medicine · doi:10.1056/nejmp0911734 · Cited by 239 · see citing works
Phillips, R. L. (2004) · Quality and Safety in Health Care · doi:10.1136/qshc.2003.008029 · Cited by 233 · see citing works
Statistical considerations in the design and analysis of community intervention trials
Donner, A., & Klar, N. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00511-0 · Cited by 230 · see citing works
Patient Record Review of the Incidence, Consequences, and Causes of Diagnostic Adverse Events
Zwaan, L. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2010.146 · Cited by 194 · see citing works
Robustness of the two independent samplest‐test when applied to ordinal scaled data
Heeren, T., & D’Agostino, R. B. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060110 · Cited by 192 · see citing works
Friedman, C. P., Gatti, G. G., Franz, T. M., Murphy, G. C., Wolf, F. M., Heckerling, P. S., Fine, P. L., Miller, T. M., & Elstein, A. S. (2005) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2005.30145.x · Cited by 187 · see citing works
DONNER, A. P., & Klar, N. S. (2000) · Statistical Methods in Medical Research · doi:10.1191/096228000669355658 · Cited by 149 · see citing works
Patient Safety Strategies Targeted at Diagnostic Errors
McDonald, K. M., Matesic, B., Contopoulos‐Ioannidis, D. G., Lonhart, J. A., Schmidt, E. M., Pineda, N., & Ioannidis, J. P. A. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-5-201303051-00004 · Cited by 140 · see citing works
System-related interventions to reduce diagnostic errors: a narrative review
Singh, H., Graber, M. L., KISSAM, S. M., Sorensen, A. V., Lenfestey, N. F., Tant, E. M., Henriksen, K., & LaBresh, K. A. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000150 · Cited by 134 · see citing works
Self-citationDifferential Diagnosis Generators: an Evaluation of Currently Available Computer Programs
Bond, W. F., Schwartz, L. M., Weaver, K., Levick, D., Giuliano, M., & Graber, M. L. (2011) · Journal of General Internal Medicine · doi:10.1007/s11606-011-1804-8 · Cited by 132 · see citing works
Self-citationBringing Diagnosis into the Quality and Safety Equations
Graber, M. L., Wachter, R. M., & Cassel, C. K. (2012) · JAMA · doi:10.1001/2012.jama.11913 · Cited by 127 · see citing works
Self-citationTwelve tips for teaching avoidance of diagnostic errors
Trowbridge, R. L. (2008) · Medical Teacher · doi:10.1080/01421590801965137 · Cited by 126 · see citing works
Performance of a Web-Based Clinical Diagnosis Support System for Internists
Graber, M. L., & Mathew, A. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0271-8 · Cited by 123 · see citing works
Self-citationSherbino, J., Kulasegaram, K., Howey, E., & Norman, G. (2014) · CJEM · doi:10.2310/8000.2013.130860 · Cited by 108 · see citing works
Checklists Improve Experts’ Diagnostic Decisions
Sibbald, M., de Bruin, A. B. H., & van Merrienboer, J. J. G. (2013) · Medical Education · doi:10.1111/medu.12080 · Cited by 92 · see citing works
Patient safety in emergency medicine
Croskerry, P. (2009) · Cited by 84 · see citing works
The pursuit of better diagnostic performance: a human factors perspective
Henriksen, K., & Brady, J. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-001827 · Cited by 73 · see citing works
Ramnarayan, P., Roberts, G. C., Coren, M., Nanduri, V., Tomlinson, A., Taylor, P. M., Wyatt, J. C., & Britto, J. F. (2006) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-6-22 · Cited by 60 · see citing works
Ramnarayan, P., Kapoor, R. R., Coren, M., Nanduri, Tomlinson, A. L., Taylor, P. M., Wyatt, J. C., & Britto, J. F. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1338 · Cited by 56 · see citing works
Association Between Angina and Treatment Satisfaction after Myocardial Infarction
Plomondon, M. E., Magid, D. J., Masoudi, F. A., Jones, P. G., Barry, L. C., Havranek, E. P., Peterson, E. D., Krumholz, H. M., Spertus, J. A., Rumsfeld, J. S., et al. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0430-y · Cited by 24 · see citing works
Schiff, G. D., & Leape, L. L. (2012) · Academic Medicine · doi:10.1097/acm.0b013e31823f711c · Cited by 22 · see citing works
Commentary: Reducing Diagnostic Errors: Another Role for Checklists?
Winters, B. D., Aswani, M. S., & Pronovost, P. J. (2011) · Academic Medicine · doi:10.1097/acm.0b013e3182082692 · Cited by 19 · see citing works
Carlson, J., Abel, M. S., Bridges, D. R., & Tomkowiak, J. M. (2011) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0b013e3181f24acd · Cited by 16 · see citing works
DIAGNOSES COMMONLY MISSED IN CHILDHOOD
Burger, F. L., & Lang, C. M. (1998) · Psychiatric Clinics of North America · doi:10.1016/s0193-953x(05)70050-2 · Cited by 13 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Schiff GD, Leape LL. Commentary: how can we make diagnosis safer? Acad Med 2012;87:135-8.
- Meyer AN, Payne VL, Meeks DW, Rao R, Singh H. Physicians' diagnostic accuracy, confidence, and resource requests. J Am Med Assoc Intern Med 2013;173:1952-8.
Cited by
Sorted by citations
Strategies to reduce diagnostic errors: a systematic review
Abimanyi-Ochom, J., Mudiyanselage, S. B., Catchpool, M., Firipis, M., Dona, S. W. A., & Watts, J. J. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0901-1 · Cited by 92 · see citing works
Tomás Pantoja, C., Grimshaw, J., Colomer, N., Castañón, C., & Léniz, J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001174.pub4 · Cited by 53 · see citing works
Interventions targeted at reducing diagnostic error: systematic review
Dave, N., Bui, S., Morgan, C., Hickey, S., & Paul, C. L. (2021) · BMJ Quality & Safety · doi:10.1136/bmjqs-2020-012704 · Cited by 51 · see citing works
Barwise, A., Leppin, A., Dong, Y., Huang, C., Pinevich, Y., Herasevich, S., Soleimani, J., Gajic, O., Pickering, B., & Kumbamu, A. (2021) · Journal of Patient Safety · doi:10.1097/pts.0000000000000817 · Cited by 43 · see citing works
Diagnostic Error in Pediatrics: A Narrative Review
Marshall, T. L., Rinke, M. L., Olson, A. P. J., & Brady, P. W. (2022) · PEDIATRICS · doi:10.1542/peds.2020-045948d · Cited by 40 · see citing works
A Scoping Review of Physicians’ Clinical Reasoning in Emergency Departments
Pelaccia, T., Plotnick, L. H., Audétat, M., NENDAZ, M. R., Lubarsky, S., Torabi, N., Thomas, A., Young, M. E., & Dory, V. (2019) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2019.06.023 · Cited by 39 · see citing works
Teaching heuristics and mnemonics to improve generation of differential diagnoses
Leeds, F. S., Atwa, K. M., Cook, A. M., Conway, K., & Crawford, T. N. (2020) · Medical Education Online · doi:10.1080/10872981.2020.1742967 · Cited by 36 · see citing works
Staal, J., Hooftman, J., Gunput, S. T. G., Mamede, S., Frens, M. A., Van den Broek, W. W., Alsma, J., & Zwaan, L. (2022) · BMJ Quality & Safety · doi:10.1136/bmjqs-2022-014865 · Cited by 32 · see citing works
Diagnostic reasoning in cardiovascular medicine
Brush, J. E., Sherbino, J., & Norman, G. R. (2022) · BMJ · doi:10.1136/bmj-2021-064389 · Cited by 27 · see citing works
Al-Khafaji, J., Townshend, R. F., Townsend, W., Chopra, V., & Gupta, A. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058219 · Cited by 19 · see citing works
Lockhart, J. J., & Satya‐Murti, S. (2017) · Journal of Forensic Sciences · doi:10.1111/1556-4029.13453 · Cited by 20 · see citing works
Differential diagnosis checklists reduce diagnostic error differentially: A randomised experiment
Kämmer, J. E., Schauber, S. K., Hautz, S. C., Stroben, F., & Hautz, W. E. (2021) · Medical Education · doi:10.1111/medu.14596 · Cited by 20 · see citing works
Diagnostic Schemas: Form and Function
Cammarata, M. J., & Dhaliwal, G. (2022) · Journal of General Internal Medicine · doi:10.1007/s11606-022-07935-1 · Cited by 19 · see citing works
Diagnostic Challenges in Outpatient Stroke: Stroke Chameleons and Atypical Stroke Syndromes
Wallace, E., & Liberman, A. L. (2021) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s275750 · Cited by 18 · see citing works
Yamamoto, S., Gu, Y., Fujitomo, Y., Kanai, N., Yamahata, Y., Saito, H., Hashimoto, T., & Ohmagari, N. (2018) · Journal of General and Family Medicine · doi:10.1002/jgf2.183 · Cited by 15 · see citing works
Garber, A., Garabedian, P., Wu, L., Lam, A., Malik, M., Fraser, H., Bersani, K., Piniella, N., Motta-Calderon, D., Rozenblum, R., et al. (2023) · JAMIA Open · doi:10.1093/jamiaopen/ooad031 · Cited by 10 · see citing works
To Err is Human: A Case-Based Review of Cognitive Bias and its Role in Clinical Decision Making
Mithoowani, S., Toma, A., Mulloy, A. J., & Patel, A. (2017) · Canadian Journal of General Internal Medicine · doi:10.22374/cjgim.v12i2.166 · Cited by 9 · see citing works
Yousef, E., Sutcliffe, K., McDonald, K., & Newman-Toker, D. (2021) · Hum. Factors · doi:10.1177/0018720821996187 · Cited by 6 · see citing works
Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors
Yen, C., Epling, J. W., Rockwell, M., & Vaughn-Cooke, M. (2026) · Diagnostics · doi:10.3390/diagnostics16020347 · Cited by 6 · see citing works
Mejorar en seguridad diagnóstica: la asignatura pendiente
Tudela, P., Forcada, C. S., Molas, A. C., & Ballester, M. (2019) · Medicina Clínica · doi:10.1016/j.medcli.2019.06.001 · Cited by 6 · see citing works
Impact of diagnostic checklists on the interpretation of normal and abnormal electrocardiograms
Staal, J., Zegers, R., Caljouw-Vos, J., Mamede, S., & Zwaan, L. (2022) · Diagnosis · doi:10.1515/dx-2022-0092 · Cited by 5 · see citing works
Learning from Missed Opportunities Through Reflective Practice
Astik, G. J., & Olson, A. P. J. (2021) · Critical Care Clinics · doi:10.1016/j.ccc.2021.09.003 · Cited by 5 · see citing works
Emergency medicine physicians’ perspectives on diagnostic accuracy in neurology: a qualitative study
Liberman, A. L., Cheng, N. T., Friedman, B. W., Gerstein, M. T., Moncrieffe, K., Labovitz, D., & Lipton, R. B. (2021) · Diagnosis · doi:10.1515/dx-2021-0125 · Cited by 4 · see citing works
Isolating red flags to enhance diagnosis (I-RED): An experimental vignette study
Chartan, C. A., Singh, H., Krishnamurthy, P., Sur, M., Meyer, A. N. D., Lutfi, R., Stark, J. E., & Thammasitboon, S. (2019) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzz082 · Cited by 3 · see citing works
Khan, H., Mahboob, U., & Ahmad, T. (2022) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.38.8.5392 · Cited by 3 · see citing works
A checklist manifesto: Can a checklist of common diagnoses improve accuracy in ECG interpretation?
Nickerson, J. E., Taub, E. S., & Shah, K. H. (2019) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2019.03.048 · Cited by 3 · see citing works
Hill, M. A., Haskell, H. W., Dainty, K. N., Kuluski, K., Shea, C., Heimlich, L., Kazi, S., Sochaniwskyj, M., Priore, A., Mason, T. L. M., et al. (2025) · BMJ Quality & Safety · doi:10.1136/bmjqs-2025-019145 · Cited by 2 · see citing works
Checklist of cognitive contributions to diagnostic errors: A tool for clinician-educators
Naude, J. M., & Burch, V. C. (2018) · African Journal of Health Professions Education · doi:10.7196/ajhpe.2018.v10i3.1059 · Cited by 2 · see citing works
Heuristics and medical errors. Part 2: How to make better medical decisions
Graber, M. A. (2021) · Russian Family Doctor · doi:10.17816/rfd62009 · Cited by 2 · see citing works
Self-citationFeasibility and Perceived Value of a Team Diagnostic Timeout: A Multi-Institutional Pilot Study
Sawicki, J. G., Buckwalter, E., Tooley, E., Hall, M., Montgomery, V., Capouya, J. D., pardue, J. P., & Godambe, S. A. (2026) · Hospital Pediatrics · doi:10.1542/hpeds.2025-008711 · Cited by 1 · see citing works
Stroke mimics in a Brazilian stroke center: a retrospective cohort in a Brazilian stroke unit
Ribas, M. Z., Fernandes, A. C., de Mendonça, L. L. P., Filho, P. C. D. S., Santiago, R. B., Gomes, G. F., dos Santos, N. L., da Costa, R. I., Galvão, S. L., Filho, L. B. D. A., et al. (2023) · Brazilian Journal of Clinical Medicine and Review · doi:10.52600/2965-0968.bjcmr.2023.1.3.41-48 · Cited by 1 · see citing works
To improve in diagnostic safety: The pending subject
Tudela, P., Forcada, C., Molas, A. C., & Ballester, M. (2019) · Medicina Clínica (English Edition) · doi:10.1016/j.medcle.2019.06.012 · Cited by 1 · see citing works
The role of cognitive science and artificial intelligence in supporting clinical diagnosis
Lucchiari, C., Vanutelli, M. E., & Folgieri, R. (2020) · Communication & Medicine · doi:10.1558/cam.36184 · Cited by 1 · see citing works
Patient safety in primary and emergency care
Fernholm, R. (2020) · Figshare
Development of a Prototype System for Milling Tool Condition Monitoring
Mushtaq, H., & Zahid, M. (2011) · Chalmers Publication Library (Chalmers University of Technology)
Stroke Mimics in a Brazilian Stroke Center: a Retrospective Cohort in a Brazilian Stroke UnitPreprint
dos Santos, J. C. C., Ribas, M. Z., Fernandes, A. C., de Mendonça, L. L. P., Filho, P. C. D. S., Santiago, R. B., Gomes, G. F., dos Santos, N. L., da Costa, R. I., Galvão, S. L., et al. (2023) · Research Square · doi:10.21203/rs.3.rs-2382962/v1
Strategies to Avoid a Misdiagnosis
Tohid, H. (2023) · Springer eBooks · doi:10.1007/978-3-031-28296-6_70
- Authors
- Ely, J. W., & Graber, M. A.
- Affiliations
- University of Iowa (US)
- Corresponding author
- Ely, John W.
- Venue
- Diagnosis
- Publisher
- Walter de Gruyter GmbH
- Published
- 2015-05-13
- Volume
- 2
- Issue
- 3
- Pages
- 163-169
- ISSN
- 2194-8011 (ISSN-L)
- 2194-802X
- OpenAlex
- W2214474504
- Cited by
- 37
- References
- 36
- Citations by year
- 2026: 2 · 2025: 1 · 2023: 4 · 2022: 7 · 2021: 8 · 2020: 3 · 2019: 7 · 2018: 2 · 2017: 2
- FWCI
- 1.56
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Diagnosis (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- diagnostic errors, checklists, premature closure, differential diagnosis, primary care, emergency department, diagnostic error prevention, physician decision-making, telephone follow-up
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review