Inter-rater Agreement for Abstraction of the Early Management Bundle, Severe Sepsis/Septic Shock (SEP-1) Quality Measure in a Multi-Hospital Health System
The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
BACKGROUND The Early Management Bundle, Severe Sepsis/Septic Shock (SEP-1) quality measure is complex to abstract, which may lead to discrepancies between abstractors. This study was designed to evaluate inter-rater agreement between abstractors at individual hospitals in a health system and a lead abstractor on abstraction elements and measure compliance for SEP-1. METHODS Patient cases qualifying for abstraction for SEP-1 over a four-month period in 2016 were initially abstracted at a local hospital and then centrally by a lead abstractor. Abstraction results were retrospectively compared to determine inter-rater agreement. RESULTS A total of 580 SEP-1 cases were abstracted locally and centrally. Each site contributed a median (interquartile range) of 63 (49, 86) cases. There was complete concordance of measure-related elements in 391 cases (67%) (inter-rater agreement: κ = 0.40, p < 0.01). The most common discrepancy (60 cases) was severe sepsis presentation time. There was a weak correlation between SEP-1 compliance adjudicated locally and centrally (r2 = 0.41, p < 0.01). The average change in monthly SEP-1 measure compliance at each site after central adjudication was a 1% increase but ranged from a 49% decrease to a 40% increase. CONCLUSIONS Concordance on SEP-1 abstraction elements between local and expert adjudicators was fair, and SEP-1 performance varied considerably from initial site-reported performance. The detailed nature of SEP-1 can lead to unreliable abstraction, which may lead to inaccurate reporting of compliance with the measure and affect comparability of performance between hospitals. Abstraction by a dedicated team for SEP-1 can reduce variability and improve efficiency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Time to treatment and mortality during mandated emergency care for sepsis.
Seymour, C. W., Gesten, F., Prescott, H. C., Friedrich, M. E., Iwashyna, T. J., Phillips, G. S., Lemeshow, S., Osborn, T., Terry, K. M., & Levy, M. M. (2017) · New England Journal of Medicine · doi:10.1056/nejmoa1703058 · Cited by 2,225
Cited by
Rhee, C., Chiotos, K., Cosgrove, S. E., Heil, E. L., Kadri, S. S., Kalil, A. C., Gilbert, D. N., Masur, H., Septimus, E. J., Sweeney, D. A., et al. (2021) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa059 · Cited by 191
“Given these challenges, it is unsurprising that abstractors frequently disagree on time zero [71, 72].”
BackgroundTownsend, S. R., Phillips, G. S., Duseja, R., Tefera, L., Cruikshank, D., Dickerson, R., Nguyen, H. B., Schorr, C. A., Levy, M. M., Dellinger, R. P., et al. (2022) · CHEST · doi:10.1016/j.chest.2021.07.2167 · Cited by 69
“Abstractors spend 30 to 120 min abstracting each chart.(55,56) In the unusual circumstance that a hospital accrued 300 sepsis”
BackgroundComparison of Sepsis Definitions as Automated Criteria
Yu, S. C., Betthauser, K. D., Gupta, A., Lyons, P. G., Lai, A. M., Kollef, M. H., Payne, P. R. O., & Michelson, A. P. (2021) · Critical Care Medicine · doi:10.1097/ccm.0000000000004875 · Cited by 29
Antimicrobials in Sepsis: Time to Pay Attention to When Delays Happen
Lyons, P. G., & Hough, C. L. (2023) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202306-519ed · Cited by 1
Grooms, E., Biesack, K., Abban, B., & Kramer, J. (2025) · Journal for Healthcare Quality · doi:10.1097/jhq.0000000000000493 · Cited by 1
Metadata
- Authors
- Bauer, S. R., Gonet, J. A., Rosario, R. F., Griffiths, L. A., Kingery, T., & Reddy, A. J.
- Venue
- The Joint Commission Journal on Quality and Patient Safety
- Publisher
- Elsevier BV
- Published
- 2019-02
- Volume
- 45
- Issue
- 2
- Pages
- 108-111
- ISSN
- 1553-7250 (ISSN-L)
- OpenAlex
- W2902599520
- Cited by
- 12
- References
- 1
- FWCI
- 0.85
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Inter-rater Agreement for Abstraction of the Early Management Bundle, Severe Sepsis/Septic Shock (SEP-1) Quality Measure in a Multi-Hospital Health System. https://app.cochat.ai/papers/10.1016/j.jcjq.2018.10.002
- The paper
- Bauer, S. R., Gonet, J. A., Rosario, R. F., Griffiths, L. A., Kingery, T., & Reddy, A. J. (2019). Inter-rater Agreement for Abstraction of the Early Management Bundle, Severe Sepsis/Septic Shock (SEP-1) Quality Measure in a Multi-Hospital Health System. The Joint Commission Journal on Quality and Patient Safety. https://doi.org/10.1016/j.jcjq.2018.10.002
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.