CoChat
Sign Up
PreprintCited by 20Sepsis Diagnosis and TreatmentMachine Learning in HealthcareArtificial Intelligence in Healthcare

Evaluating a sepsis prediction machine learning algorithm in the emergency department and intensive care unit: a before and after comparative study

Open access
Burdick, Hoyt, Pino, Eduardo, Gabel-Comeau, Denise, Gu, Carol, Huang, Heidi, Lynn-Palevsky, Anna, Das, Ritankar·2017·bioRxiv (Cold Spring Harbor Laboratory)
DOI 10.1101/224014
Open this paper in CoChat — free accountOpen PDF

Abstract

Introduction: Sepsis is a major health crisis in US hospitals, and several clinical identification systems have been designed to help care providers with early diagnosis of sepsis. However, many of these systems demonstrate low specificity or sensitivity, which limits their clinical utility. We evaluate the effects of a machine learning algodiagnostic (MLA) sepsis prediction and detection system using a before-and-after clinical study performed at Cabell Huntington Hospital (CHH) in Huntington, West Virginia. Prior to this study, CHH utilized the St. John’s Sepsis Agent (SJSA) as a rules-based sepsis detection system. Methods: The Predictive algoRithm for EValuation and Intervention in SEpsis (PREVISE) study was carried out between July 1, 2017 and August 30, 2017. All patients over the age of 18 who were admitted to the emergency department or intensive care units at CHH were monitored during the study. We assessed pre-implementation baseline metrics during the month of July, 2017, when the SJSA was active. During implementation in the month of August, 2017, SJSA and the MLA concurrently monitored patients for sepsis risk. At the conclusion of the study period, the primary outcome of sepsis-related in-hospital mortality and secondary outcome of sepsis-related hospital length of stay were compared between the two groups. Results: Sepsis-related in-hospital mortality decreased from 3.97% to 2.64%, a 33.5% relative decrease (P = 0.038), and sepsis-related length of stay decreased from 2.99 days in the pre-implementation phase to 2.48 days in the post-implementation phase, a 17.1% relative reduction (P < 0.001). Conclusion: Reductions in patient mortality and length-of-stay were observed with use of a machine learning algorithm for early sepsis detection in the emergency department and intensive care units at Cabell Huntington Hospital, and may present a method for improving patient outcomes. Trial Registration: ClinicalTrials.gov, NCT03235193 , retrospectively registered on July 27th 2017.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/224014

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 23

  1. Fleischmann C, Scherag A, Adhikari NK, et al. Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations. Am. J. Respir. Crit. Care Med. 2016;193:259-72.
  2. Chalupka AN, Talmor D. The economics of sepsis. Crit. Care Clin. 2012;28(2):57-76.
  3. Angus DC, Linde-Zwirble WT, Lidicker, et al. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit. Care Med 2001;29:1303-1310.
  4. Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 2016;315:801-10.
  5. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:762-74.
  6. Nguyen HB, Corbett SW, Steele R, et al. Implementation of a bundle of quality indicators for the early management of severe sepsis and septic shock is associated with decreased mortality. Crit. Care Med. 2007;35:1105-1112.
  7. Hooper MH, Weavind L, Wheeler AP, Martin JB, Gowda SS, Semler MW, Hayes RM, Albert DW, Deane NB, Nian H, Mathe JL. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Crit. Care Med. 2012;40:2096-101.
  8. Herasevich V, Afessa B, Chute CG, Gajic O. Designing and testing computer based screening engine for severe sepsis/septic shock. AMIA Annu. Symp. Proc. 2007;2007: 966.
  9. Umscheid CA, Betesh J, Van Zandbergen C, et al. Development, implementation, and impact of an automated early warning and response system for sepsis. J. Hosp. Med. 2015;10:26-31
  10. Henry KE, Hager DN, Pronovost PJ, Saria S. A targeted real-time early warning score (TREWScore) for septic shock. Sci. Transl. Med. 2015;7:299ra122.
  11. Nachimuthu SK, Haug PJ. Early detection of sepsis in the emergency department using Dynamic Bayesian Networks. AMIA Annu. Symp. Proc. 2012;2012:653.
  12. Sawyer AM, Deal EN, Labelle AJ, et al. Implementation of a real-time computerized sepsis alert in nonintensive care unit patients. Crit. Care Med. 2011;39:469-73.
  13. Desautels T, Calvert J, Hoffman J, et al. Prediction of Sepsis in the Intensive Care Unit With Minimal Electronic Health Record Data: A Machine Learning Approach. JMIR Medical Informatics 2016;4(3):28. PMID: 27694098
  14. Calvert JS, Price DA, Chettipally UK, et al. High-performance detection and early prediction of septic shock for alcohol-use disorder patients. Ann. Med. Surg. 2016;8:50- 55.
  15. Mao Q, Jay M, et al. Multicenter validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU. BMJ Open 2017, In Press.
  16. Shimabukuro D, Barton CW, Feldman MD, et al. Effect of a Machine Learning-Based Severe Sepsis Prediction Algorithm on Patient Survival and Hospital Length of Stay: A Randomized Clinical Trial. BMJ Open Respiratory Research 2017;4:e000234.
  17. Rangel-Frausto MS, Pittet D, Costigan M, Hwang T, Davis CS, Wenzel RP. The natural history of the systemic inflammatory response syndrome (SIRS): a prospective study. JAMA. 1995;273:117-23.
  18. Jaimes F, Garcés J, Cuervo J, et al. The systemic inflammatory response syndrome (SIRS) to identify infected patients in the emergency room. Intensive Care Med. 2003;29:1368-1371.
  19. Ruskin KJ, Hueske-Kraus D. Alarm fatigue: impacts on patient safety. Current Opinion in Anesthesiology. 2015 Dec 1;28(6):685-90.
  20. Cvach M. Monitor alarm fatigue: an integrative review. Biomedical Instrumentation & Technology. 2012 Jul;46(4):268-77. PMID: 22839984
  21. Subbe CP, Slater A, Menon D, Gemmell L. Validation of physiological scoring systems in the accident and emergency department. Emergency Medicine Journal 2006 Nov;23(11):841-845. PMID: 17057134
  22. Rivers E, Nguyen B, Havstad S, Ressler J, Muzzin A, Knoblich B, Peterson E, Tomlanovich M. Early goal-directed therapy in the treatment of severe sepsis and septic shock. New England Journal of Medicine. 2001 Nov 8;345(19):1368-77. PMID: 11794169
  23. Cvach M. Monitor alarm fatigue: an integrative review. Biomedical Instrumentation & Technology. 2012 Jul;46(4):268-77. PMID: 22839984

Cited by

Sorted by citations

Authors
Burdick, H., Pino, E., Gabel-Comeau, D., Gu, C., Huang, H., Lynn-Palevsky, A., & Das, R.
Affiliations
  • Cabell Huntington Hospital (US)
  • Dascena (United States) (US)
Corresponding author
Lynn-Palevsky, Anna
ORCID
Venue
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Published
2017-11-28
DOI
10.1101/224014
OpenAlex
W2771699609
Cited by
20
References
1
Citations by year
2026: 1 · 2025: 2 · 2024: 1 · 2023: 5 · 2022: 2 · 2021: 3 · 2020: 3 · 2019: 2 · 2018: 1
Type
posted content
Language
en
Open access
Locations
bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
Topics
  • Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
sepsis, sepsis prediction, machine learning algorithms, early sepsis detection, in-hospital mortality, length of hospital stay, emergency department, intensive care unit, clinical decision support, pre-post test comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review