SEP-1—Taking the Measure of a Measure

Abstract

One wonders if the controversy regarding the use of the severe sepsis/septic shock bundle performance measure (SEP-1) by the Center for Medicare and Medicaid Services (CMS) will ever be reconciled or concluded.SEP-1 aside, the global burden of illness from sepsis as a major cause of death, disability, and health care expense is beyond debate.1 Variation in the quality of care delivery among health care organizations, including care for sepsis, has been repeatedly demonstrated and is indisputable.As a result, efforts to ensure that quality measurement efforts and our finite resources are focused on improving care and outcomes for patients with sepsis must represent common ground for collective action regardless of which "SEP-1 side" you are on.To that end, Rhee et al 2 make another important contribution to our understanding of the complex association between SEP-1 implementation by CMS and outcomes for patients with sepsis.Making novel use of electronic health data to define the cohort, populate the SEP-1 metric, and evaluate mortality, they find no association between short-term mortality and SEP-1 measurement implementation in this cohort study across 114 hospitals.Whether these hospitals are representative of all hospitals in the US engaging in SEP-1 implementation is not entirely clear.Given that most hospitals are not able to make use of their electronic health data to generate SEP-1 reliably, this methodology, while having some advantages over manual abstraction in standardization and consistency of data, may limit comparison to the actual metric in use by most hospitals.Furthermore, given the usual time frame for new and complex quality metrics to be successfully deployed and productively used for quality improvement, generalized conclusions regarding the role of SEP-1 implementation within 2 years of introduction may be premature.A lack of association between SEP-1 implementation and short-term mortality in these 114 hospitals is not equivalent to a demonstration of a failed relationship between improvement in SEP-1 performance and reduced mortality.To that end, a recent study by Townsend et al 3 using a propensity score-matched cohort study of patient-level data reported to CMS by 3241 hospitals concludes that compliance with SEP-1 is associated with a lower 30-day mortality.This suggests that SEP-1-compliant care may, in fact, reduce preventable deaths.How do we make sense of these disparate findings and, importantly, how do we move forward to ensure that efforts to reduce sepsis mortality and morbidity are achieving their goals?A limited association between a quality metric such as SEP-1, representing processes of care, and an outcome such as mortality raises more questions than it answers.There are multiple potential explanations to consider, and as with many complex processes, multiple explanatory factors may exist simultaneously: the underlying science behind the interventions, the technical construction of the measure, the way in which the measure is implemented, the chosen outcomes of interest, and the population and the methodology used to evaluate the measure.We know that all hospitals are not helped equally by quality initiatives.This raises particular challenges for broad improvement efforts across states, regions, or the nation.Perhaps even more fundamental is to question if any quality metric, on its own, is able to deliver improved patient outcomes.SEP-1 was introduced by CMS in October 2015, having been reviewed, passionately debated, revised, and ultimately endorsed (and recently re-endorsed) by a diverse panel of experts within the National Quality Forum (NQF).Public reporting of hospital performance on SEP-1 did not begin, however, until 2018 and there is currently no financial incentive or disincentive based on performance.As expected, there is wide variation in SEP-1 performance across hospitals.