Accuracy of the Epic Sepsis Prediction Model in a Regional Health System
Open accessAbstract
Interest in an electronic health record-based computational model that can accurately predict a patient's risk of sepsis at a given point in time has grown rapidly in the last several years. Like other EHR vendors, the Epic Systems Corporation has developed a proprietary sepsis prediction model (ESPM). Epic developed the model using data from three health systems and penalized logistic regression. Demographic, comorbidity, vital sign, laboratory, medication, and procedural variables contribute to the model. The objective of this project was to compare the predictive performance of the ESPM with a regional health system's current Early Warning Score-based sepsis detection program.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ESM-driven alerts reflect the presence of sepsis already apparent to clinicians.
“suggesting that ESM-driven alerts reflect the presence of sepsis already apparent to clinicians.27”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [27]
The Results (p. 2) report that the ESPM reached its threshold a median of seven hours after the first lactate draw and that some patients may already have come to the attention of the existing sepsis detection program. This supports the suggested timing concern, but the paper does not establish that sepsis was already apparent to clinicians when ESM-driven alerts occurred. (LLM, conf=0.84)
Experimental — accuracy not yet measuredA prior study found that the ESM produced an alert at a median of 7 hours (IQR, 4-22 hours) after the first lactate level was measured.
“A prior study that did not exclude predictions made after development of sepsis found that the ESM produced an alert at a median of 7 hours (interquartile range, 4-22 hours) after the first lactate level was measured”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [27]
The observed hospitalization-level AUC of 0.63 is substantially worse than the performance reported by Epic Systems (AUC, 0.76-0.83) and in a prior conference proceeding coauthored with Epic Systems (AUC, 0.73).
“Our observed hospitalization-level model performance (AUC, 0.63) was substantially worse than that reported by Epic Systems (AUC, 0.76-0.83) in internal documentation (shared with permission) and in a prior conference proceeding coauthored with Epic Systems (AUC, 0.73).27”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 5
- [1][1] Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016 Feb;315(8):801-10. -
- [2][2] Rivers E, Nguyen B, Havstad S, Ressler J, Muzzin A, Knoblich B, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001 Nov;345(19):1368-77. -
- [3][3] Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, et al. Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside the Intensive Care Unit. Am J Respir Crit Care Med. 2017 Apr;195(7):906-911. -
- [4][4] Desautels T, Calvert J, Hoffman J, Jay M, Kerem Y, Shieh L, et al. Prediction of Sepsis in the Intensive Care Unit With Minimal Electronic Health Record Data: A Machine Learning Approach. JMIR Med Inform. 2016 Sep;4(3):e28. -
- [5][5] Amland RC, Hahn-Cover KE. Clinical Decision Support for Early Recognition of Sepsis. Am J Med Qual. 2016;31(2):103-10.
Cited by
Sorted by citations
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
The epic sepsis model falls short-the importance of external validation
Habib, A. R., Lin, A. L., & Grant, R. W. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.3333 · Cited by 101 · see citing works
Sepsis prediction model for determining sepsis vs SIRS, qSOFA, and SOFA
Schertz, A. R., Lenoir, K. M., Bertoni, A. G., Levine, B. J., Mongraw-Chaffin, M., & Thomas, K. W. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.29729 · Cited by 78 · see citing works
Epic Sepsis Model Inpatient Predictive Analytic Tool: A Validation Study
Cull, J., Brevetta, R., Gerac, J., Kothari, S., & Blackhurst, D. (2023) · Critical Care Explorations · doi:10.1097/cce.0000000000000941 · Cited by 37 · see citing works
- Authors
- Bennett, T., Russell, S., King, J., Schilling, L., Voong, C., Rogers, N., Adrian, B., Bruce, N., & Ghosh, D.
- Affiliations
- University of Colorado Anschutz (US)
- University of Colorado Health (US)
- Epic Systems (United States) (US)
- Colorado School of Public Health (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1902.07276
- OpenAlex
- W2915740318
- Cited by
- 19
- References
- 0
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 3 · 2023: 4 · 2022: 4 · 2021: 4 · 2019: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- AMIA
- Topics
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- sepsis prediction, early warning scores, sepsis detection, electronic health records, risk prediction model, model validation, regional health systems, clinical decision support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
