Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: A systematic review
Open accessAbstract
BACKGROUND: Although timely treatment of sepsis improves outcomes, delays in administering evidence‐based therapies are common. PURPOSE: To determine whether automated real‐time electronic sepsis alerts can: (1) accurately identify sepsis and (2) improve process measures and outcomes. DATA SOURCES: We systematically searched MEDLINE, Embase, The Cochrane Library, and Cumulative Index to Nursing and Allied Health Literature from database inception through June 27, 2014. STUDY SELECTION: Included studies that empirically evaluated 1 or both of the prespecified objectives. DATA EXTRACTION: Two independent reviewers extracted data and assessed the risk of bias. Diagnostic accuracy of sepsis identification was measured by sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratio (LR). Effectiveness was assessed by changes in sepsis care process measures and outcomes. DATA SYNTHESIS: Of 1293 citations, 8 studies met inclusion criteria, 5 for the identification of sepsis (n = 35,423) and 5 for the effectiveness of sepsis alerts (n = 6894). Though definition of sepsis alert thresholds varied, most included systemic inflammatory response syndrome criteria ± evidence of shock. Diagnostic accuracy varied greatly, with PPV ranging from 20.5% to 53.8%, NPV 76.5% to 99.7%, LR+ 1.2 to 145.8, and LR− 0.06 to 0.86. There was modest evidence for improvement in process measures (ie, antibiotic escalation), but only among patients in non–critical care settings; there were no corresponding improvements in mortality or length of stay. Minimal data were reported on potential harms due to false positive alerts. CONCLUSIONS: Automated sepsis alerts derived from electronic health data may improve care processes but tend to have poor PPV and do not improve mortality or length of stay. Journal of Hospital Medicine 2015;10:396–402. © 2015 Society of Hospital Medicine
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Current approaches to identify septic patients have centered around biomarkers and (automated) clinical decision rules such as the SIRS and (q)SOFA criteria.
“Current approaches to identify septic patients have centered around biomarkers and (automated) clinical decision rules such as the SIRS and (q)SOFA criteria [38, 39].”
From Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy (2020) · cites this paper as [38]
The Results (Description of Studies; page unavailable) say that automated sepsis alerts commonly used two or more SIRS criteria to identify sepsis. The paper does not support the sentence’s additional references to biomarkers or SOFA/qSOFA criteria; searches for those terms found no relevant passage in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThe implementation gap has largely been filled by commercial electronic health record vendors, who integrated early warning systems into the EHR.
“This gap has largely been filled by commercial electronic health record (EHR) vendors, who have integrated early warning systems into the EHR where they can be readily accessed by clinicians and linked to clinical interventions.12,13”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [13]
The Introduction says automated sepsis alerts can notify clinicians through an EHR, and the Results describe studies using EHR alerts, some with clinical management suggestions. Neither passage says commercial EHR vendors supplied these systems or that they largely filled an implementation gap. (LLM, conf=0.85)
Experimental — accuracy not yet measuredSepsis screening tools vary widely in diagnostic accuracy, with most having poor predictive values, although use of some is associated with improvements in care processes.
“There is wide variation in diagnostic accuracy of these tools with most having poor predictive values, although the use of some was associated with improvements in care processes [28–31].”
From Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021 (2021) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 27 · Cited by 99
- Authors
- Makam, A. N., Nguyen, O. K., & Auerbach, A. D.
- Affiliations
- The University of Texas Southwestern Medical Center (US)
- University of California, San Francisco (US)
- Venue
- Journal of Hospital Medicine
- Publisher
- Wiley
- Published
- 2015-03-11
- Volume
- 10
- Issue
- 6
- Pages
- 396-402
- ISSN
- 1553-5592 (ISSN-L)
- 1553-5606
- PMC
- PMC4477829
- OpenAlex
- W2140410010
- Cited by
- 99
- References
- 27
- Citations by year
- 2026: 10 · 2025: 9 · 2024: 8 · 2023: 4 · 2022: 9 · 2021: 15 · 2020: 11 · 2019: 11 · 2018: 8 · 2017: 8 · 2016: 4 · 2015: 2
- FWCI
- 0.95
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Hospital Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Center for Advancing Translational Sciences (KL2TR001103)
- Agency for Healthcare Research and Quality (R24HS022428-01)
- National Institutes of Health (T32 HP19025-07-00)
- National Heart, Lung, and Blood Institute (K24HL098372)
- Topics
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- sepsis, electronic alerts, automated clinical decision support, diagnostic accuracy, positive predictive value, systemic inflammatory response syndrome, antibiotic escalation, sepsis mortality, length of hospital stay, false positive alerts, electronic health records
- MeSH terms
- Diagnosis, Differential, Humans, Predictive Value of Tests, Reproducibility of Results, Databases, Bibliographic, Outcome Assessment, Health Care, Sepsis, Decision Support Systems, Clinical, Early Diagnosis, Medical Order Entry Systems, Clinical Alarms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
