Should electronic differential diagnosis support be used early or late in the diagnostic process? A multicentre experimental study of Isabel
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Background: Diagnostic errors unfortunately remain common. Electronic differential diagnostic support (EDS) systems may help, but it is unclear when and how they ought to be integrated into the diagnostic process. Objective: To explore how much EDS improves diagnostic accuracy, and whether EDS should be used early or late in the diagnostic process. Setting: 6 Canadian medical schools. A volunteer sample of 67 medical students, 62 residents in internal medicine or emergency medicine, and 61 practising internists or emergency medicine physicians were recruited in May through June 2020. Intervention: Participants were randomised to make use of EDS either early (after the chief complaint) or late (after the complete history and physical is available) in the diagnostic process while solving each of 16 written cases. For each case, we measured the number of diagnoses proposed in the differential diagnosis and how often the correct diagnosis was present within the differential. Results: EDS increased the number of diagnostic hypotheses by 2.32 (95% CI 2.10 to 2.49) when used early in the process and 0.89 (95% CI 0.69 to 1.10) when used late in the process (both p<0.001). Both early and late use of EDS increased the likelihood of the correct diagnosis being present in the differential (7% and 8%, respectively, both p<0.001). Whereas early use increased the number of diagnostic hypotheses (most notably for students and residents), late use increased the likelihood of the correct diagnosis being present in the differential regardless of one’s experience level. Conclusions and relevance: EDS increased the number of diagnostic hypotheses and the likelihood of the correct diagnosis appearing in the differential, and these effects persisted irrespective of whether EDS was used early or late in the diagnostic process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
When to use the Bonferroni correction
Armstrong, R. A. (2014) · Ophthalmic and Physiological Optics · doi:10.1111/opo.12131 · Cited by 3,031
A Universal Model of Diagnostic Reasoning
Croskerry, P. (2009) · Academic Medicine · doi:10.1097/acm.0b013e3181ace703 · Cited by 1,016
The incidence of diagnostic error in medicine
Graber, M. L. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001615 · Cited by 626
Types and Origins of Diagnostic Errors in Primary Care Settings
Singh, H., Giardina, T. D., Meyer, A. N. D., Forjuoh, S. N., Reis, M. D., & Thomas, E. J. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.2777 · Cited by 559
Checklists to Reduce Diagnostic Errors
Ely, J. W., Graber, M. L., & Croskerry, P. (2011) · Academic Medicine · doi:10.1097/acm.0b013e31820824cd · Cited by 366
Assessing the level of healthcare information technology adoption in the United States: a snapshot
Poon, E. G., Jha, A. K., Christino, M., Honour, M. M., Fernandopulle, R., Middleton, B., Newhouse, J., Leape, L., Bates, D. W., Blumenthal, D., et al. (2006) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-6-1 · Cited by 318
Why did I miss the diagnosis? Some cognitive explanations and educational implications
Bordage, G. (1999) · Academic Medicine · doi:10.1097/00001888-199910000-00065 · Cited by 186
Gunderson, C. G., Bilan, V. P., Holleck, J. L., Nickerson, P., Cherry, B. M., Chui, P., Bastian, L. A., Grimshaw, A. A., & Rodwin, B. A. (2020) · BMJ Quality & Safety · doi:10.1136/bmjqs-2019-010822 · Cited by 130
Cited by
Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial
Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J. A., Kanjee, Z., Parsons, A. S., Ahuja, N., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.40969 · Cited by 724
Data Integration Challenges for Machine Learning in Precision Medicine
Martínez-García, M., & Hernández-Lemus, E. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2021.784455 · Cited by 185
Yanagita, Y., Shikino, K., Ishizuka, K., Uchida, S., Li, Y., Yokokawa, D., Tsukamoto, T., Noda, K., Uehara, T., & Ikusaka, M. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04370-6 · Cited by 13
“One study evaluated the diagnostic accuracy of the CDSS at the time the patient presents with the original complaint and at the time all test results are available, including the interview, physical examination, blood tests, and imaging studies [16].”
Background
Metadata
- Authors
- Sibbald, M., Monteiro, S., Sherbino, J., LoGiudice, A., Friedman, C., & Norman, G.
- Venue
- BMJ Quality & Safety
- Publisher
- BMJ
- Published
- 2021-10-05
- Volume
- 31
- Issue
- 6
- Pages
- 426-433
- ISSN
- 1475-3898 (ISSN-L)
- 2044-5415
- 2044-5423
- OpenAlex
- W3202293436
- Cited by
- 51
- References
- 8
- FWCI
- 9.14
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Should electronic differential diagnosis support be used early or late in the diagnostic process? A multicentre experimental study of Isabel. https://app.cochat.ai/papers/10.1136/bmjqs-2021-013493
- The paper
- Sibbald, M., Monteiro, S., Sherbino, J., LoGiudice, A., Friedman, C., & Norman, G. (2022). Should electronic differential diagnosis support be used early or late in the diagnostic process? A multicentre experimental study of Isabel. BMJ Quality & Safety. https://doi.org/10.1136/bmjqs-2021-013493
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.