External validation of the Epic sepsis predictive model in 2 county emergency departments.

Abstract

Objective: To describe the diagnostic characteristics of the proprietary Epic sepsis predictive model best practice advisory (BPA) alert for physicians in the emergency department (ED). Materials and Methods: The Epic Sepsis Predictive Model v1.0 (ESPMv1), a proprietary algorithm, is intended to improve provider alerting of patients with a likelihood of developing sepsis. This retrospective cohort study conducted at 2 county EDs from January 1, 2023 to December 31, 2023 evaluated the predictive characteristics of the ESPMv1 for 145 885 encounters. Sepsis was defined according to the Sepsis-3 definition with the onset of sepsis defined as an increase in 2 points on the Sequential Organ Function Assessment (SOFA) score in patients with the ordering of at least one blood culture and antibiotic. Alerting occurred at an Epic recommended model threshold of 6. Results: The ESPMv1 BPA alert was present in 7183 (4.9%) encounters of which 2253 had sepsis, and not present in 138 702 encounters of which 3180 had sepsis. Within a 6-hour time window for sepsis, the ESPMv1 had a sensitivity of 14.7%, specificity of 95.3%, positive predictive value of 7.6%, and negative predictive value of 97.7%. Providers were alerted with a median lead time of 0 minutes (80% CI, −6 hours and 42 minutes to 12 hours and 0 minutes). Discussion: In our population, the ESPMv1 alerted providers with a median lead time of 0 minutes (80% CI, −6 hours and 42 minutes to 12 hours and 0 minutes) and only alerted providers in half of the cases prior to sepsis occurrence. This suggests that the ESPMv1 alert is adding little assistance to physicians identifying sepsis. With clinicians treating sepsis 50% of the time without an alert, pop-ups can only marginally assist in disease identification. Conclusions: The ESPMv1 provides suboptimal diagnostic characteristics for undifferentiated patients in a county ED.