Identifying Patients With Severe Sepsis Using Administrative Claims: Patient-Level Validation of the Angus Implementation of the International Consensus Conference Definition of Severe Sepsis
Open accessAbstract
BACKGROUND: Severe sepsis is a common and costly problem. Although consistently defined clinically by consensus conference since 1991, there have been several different implementations of the severe sepsis definition using ICD-9-CM codes for research. We conducted a single center, patient-level validation of 1 common implementation of the severe sepsis definition, the so-called "Angus" implementation. METHODS: Administrative claims for all hospitalizations for patients initially admitted to general medical services from an academic medical center in 2009-2010 were reviewed. On the basis of ICD-9-CM codes, hospitalizations were sampled for review by 3 internal medicine-trained hospitalists. Chart reviews were conducted with a structured instrument, and the gold standard was the hospitalists' summary clinical judgment on whether the patient had severe sepsis. RESULTS: Three thousand one hundred forty-six (13.5%) hospitalizations met ICD-9-CM criteria for severe sepsis by the Angus implementation (Angus-positive) and 20,142 (86.5%) were Angus-negative. Chart reviews were performed for 92 randomly selected Angus-positive and 19 randomly-selected Angus-negative hospitalizations. Reviewers had a κ of 0.70. The Angus implementation's positive predictive value was 70.7% [95% confidence interval (CI): 51.2%, 90.5%]. The negative predictive value was 91.5% (95% CI: 79.0%, 100%). The sensitivity was 50.4% (95% CI: 14.8%, 85.7%). Specificity was 96.3% (95% CI: 92.4%, 100%). Two alternative ICD-9-CM implementations had high positive predictive values but sensitivities of <20%. CONCLUSIONS: The Angus implementation of the international consensus conference definition of severe sepsis offers a reasonable but imperfect approach to identifying patients with severe sepsis when compared with a gold standard of structured review of the medical chart by trained hospitalists.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Bone, R. C., BALK, R. A., Cerra, F. B., Dellinger, R. P., Fein, A. M., Knaus, W. A., Schein, R. M. H., & Sibbald, W. J. (1992) · CHEST Journal · doi:10.1378/chest.101.6.1644 · Cited by 13,192 · see citing works
Early goal-directed therapy in the treatment of severe sepsis and septic shock
Rivers, E., Nguyen, B., Havstad, S., Ressler, J., Muzzin, A., Knoblich, B., Peterson, E., & Tomlanovich, M. (2001) · New England Journal of Medicine · doi:10.1056/nejmoa010307 · Cited by 10,743 · see citing works
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
DEYO, R. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90133-8 · Cited by 10,452 · see citing works
Angus, D. C., Linde-Zwirble, W. T., Lidicker, J., Clermont, G., Carcillo, J., & Pinsky, M. R. (2001) · Critical Care Medicine · doi:10.1097/00003246-200107000-00002 · Cited by 8,503 · see citing works
Efficacy and Safety of Recombinant Human Activated Protein C for Severe Sepsis
Bernard, G. R., Vincent, J., Laterre, P., LaRosa, S. P., Dhainaut, J.-F., Lopez-Rodriguez, A., Steingrub, J. S., Garber, G. E., Helterbrand, J. D., Ely, E. W., et al. (2001) · New England Journal of Medicine · doi:10.1056/nejm200103083441001 · Cited by 6,124 · see citing works
2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference
Levy, M. M., Fink, M. P., Marshall, J. C., Abraham, E. H., Angus, D. C., Cook, D., Cohen, J., Opal, S. M., Vincent, J., & Ramsay, G. (2003) · Critical Care Medicine · doi:10.1097/01.ccm.0000050454.01978.3b · Cited by 5,976 · see citing works
The epidemiology of sepsis in the United States from 1979 through 2000
Martin, G. S., Mannino, D. M., Eaton, S., & Moss, M. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa022139 · Cited by 5,841 · see citing works
2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference
Conference, F. T. I. S. D., Levy, M. M., Fink, M. P., Marshall, J. C., Abraham, E. H., Angus, D. C., Cook, D., Cohen, J., Opal, S. M., Vincent, J., et al. (2003) · Intensive Care Medicine · doi:10.1007/s00134-003-1662-x · Cited by 2,877 · see citing works
Sepsis in European intensive care units: Results of the SOAP study*
Vincent, J., Sakr, Y. L., Sprung, C. L., Ranieri, V. M., Reinhart, K., Gerlach, H., Moreno, R. P., Carlet, J. M., Gall, J. L., & Payen, D. M. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000194725.48928.3a · Cited by 2,841 · see citing works
Long-term cognitive impairment and functional disability among survivors of severe sepsis.
Iwashyna, T. J., Ely, E. W., Smith, D. M., & Langa, K. M. (2010) · JAMA · doi:10.1001/jama.2010.1553 · Cited by 2,582 · see citing works
Self-citationPopulation Trends in the Incidence and Outcomes of Acute Myocardial Infarction
Yeh, R. W., Sidney, S., Chandra, M., Sorel, M. E., Selby, J. V., & Go, A. S. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa0908610 · Cited by 1,732 · see citing works
Romano, P. S., Roos, L. L., & Jollis, J. G. (1993) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(93)90103-8 · Cited by 1,717 · see citing works
Annane, D., Bellissant, É., & Cavaillon, J. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17667-8 · Cited by 1,207 · see citing works
Epidemiology of sepsis: an update
Angus, D. C., & Wax, R. S. (2001) · Critical Care Medicine · doi:10.1097/00003246-200107001-00035 · Cited by 813 · see citing works
Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to 2007
Lagu, T., Rothberg, M. B., Shieh, M.-S., Pekow, P. S., Steingrub, J. S., & Lindenauer, P. K. (2012) · Critical Care Medicine · doi:10.1097/ccm.0b013e318232db65 · Cited by 679 · see citing works
Epidemiology of Sepsis Syndrome in 8 Academic Medical Centers
Sands, K. (1997) · JAMA · doi:10.1001/jama.1997.03550030074038 · Cited by 668 · see citing works
Long-term mortality and quality of life in sepsis: A systematic review*
Winters, B. D., Eberlein, M. H., Leung, J. M., Needham, D. M., Pronovost, P. J., & Sevransky, J. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181d8cc1d · Cited by 619 · see citing works
Walkey, A. J., Wiener, R. S., Ghobrial, J. M., Curtis, L. H., & Benjamin, E. J. (2011) · JAMA · doi:10.1001/jama.2011.1615 · Cited by 490 · see citing works
The Immunopathogenesis of Sepsis in Elderly Patients
Opal, S. M., Girard, T. D., & Ely, E. W. (2005) · Clinical Infectious Diseases · doi:10.1086/432007 · Cited by 461 · see citing works
Magnitude and Duration of the Effect of Sepsis on Survival
Quartin, A. A. (1997) · JAMA · doi:10.1001/jama.1997.03540370048035 · Cited by 418 · see citing works
Sepsis incidence and outcome: Contrasting the intensive care unit with the hospital ward*
Esteban, A., Frutos–Vivar, F., Ferguson, N. D., Peñuelas, Ó., Lorente, J. A., Gordo, F., Honrubia, T., Algora, A., Bustos, A., García, G. V., et al. (2007) · Critical Care Medicine · doi:10.1097/01.ccm.0000260960.94300.de · Cited by 373 · see citing works
Mayr, F. (2010) · JAMA · doi:10.1001/jama.2010.851 · Cited by 321 · see citing works
Racial Variation in the Incidence, Care, and Outcomes of Severe Sepsis
Barnato, A. E., Alexander, S. L., Linde-Zwirble, W. T., & Angus, D. C. (2008) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200703-480oc · Cited by 284 · see citing works
The Search for Effective Therapy for Sepsis
Angus, D. C. (2011) · JAMA · doi:10.1001/jama.2011.1853 · Cited by 264 · see citing works
Long-term mortality and medical care charges in patients with severe sepsis
Weycker, D. A., Akhras, K. S., Edelsberg, J., Angus, D. C., & Oster, G. (2003) · Critical Care Medicine · doi:10.1097/01.ccm.0000085178.80226.0b · Cited by 250 · see citing works
Long-term outcome and quality-adjusted life years after severe sepsis*
Karlsson, S., Ruokonen, E., Varpula, T., Ala-Kokko, T. I., & Pettilä, V. (2009) · Critical Care Medicine · doi:10.1097/ccm.0b013e31819c13ac · Cited by 174 · see citing works
Romano, P. S., Roos, L. L., & Jollis, J. G. (1993) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(93)90106-b · Cited by 143 · see citing works
The Lingering Consequences of Sepsis
Angus, D. C. (2010) · JAMA · doi:10.1001/jama.2010.1546 · Cited by 136 · see citing works
Novel Therapies for Septic Shock Over the Past 4 Decades
Suffredini, A. F., & Munford, R. S. (2011) · JAMA · doi:10.1001/jama.2011.909 · Cited by 75 · see citing works
Detailed cost analysis of care for survivors of severe sepsis
Lee, H., Doig, C. J., Ghali, W. A., Donaldson, C., Johnson, D. H., & Manns, B. (2004) · Critical Care Medicine · doi:10.1097/01.ccm.0000120053.98734.2c · Cited by 70 · see citing works
Toward an Integrated Research Agenda for Critical Illness in Aging
Milbrandt, E. B., Eldadah, B. A., Nayfield, S., Hadley, E., & Angus, D. C. (2010) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200904-0630cp · Cited by 65 · see citing works
The Relationship Between Hospital Spending and Mortality in Patients With Sepsis
Lagu, T. C., Rothberg, M. B., Nathanson, B. H., Pekow, P. S., Steingrub, J. S., & Lindenauer, P. K. (2011) · Archives of Internal Medicine · doi:10.1001/archinternmed.2011.12 · Cited by 59 · see citing works
Lagu, T. C., Lindenauer, P. K., Rothberg, M. B., et al. (2011) · Critical Care Medicine · doi:10.1097/ccm.0b013e31822572e3 · Cited by 55 · see citing works
Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: A response
Deyo, R. A. (1993) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(93)90104-9 · Cited by 41 · see citing works
Angus, D. C. (2011) · JAMA · doi:10.1001/jama.2011.438 · Cited by 25 · see citing works
Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: A response
Charlson, M. E. (1993) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(93)90105-a · Cited by 23 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [33]Rothman KJ Greenland S Modern Epidemiology 2 Philadelphia Lippincott, Williams & Wilkins 1998
- [39]Poulose JT Cartin-Ceba R Shoja A Comparison of International Classification of Disease–Ninth Revision (ICD–9) Coding with Retrospective Case Review for the Diagnosis of Septic Shock Am J Respir Crit Care Med 2009 179 A4361
Cited by
Sorted by citations
Rudd, K. E., Johnson, S. C., Agesa, K. M., Shackelford, K. A., Tsoi, D., Kievlan, D. R., Colombara, D. V., Ikuta, K. S., Kissoon, N., Finfer, S., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(19)32989-7 · Cited by 7,971 · see citing works
Seymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., Rubenfeld, G., Kahn, J. M., Shankar-Hari, M., Singer, M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0288 · Cited by 3,939 · see citing works
Self-citationIncidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014.
Rhee, C., Dantes, R., Epstein, L., Murphy, D. J., Seymour, C. W., Iwashyna, T. J., Kadri, S. S., Angus, D. C., Danner, R. L., Fiore, A. E., et al. (2017) · JAMA · doi:10.1001/jama.2017.13836 · Cited by 1,884 · see citing works
Self-citationThe MIMIC Code Repository: enabling reproducibility in critical care research
Johnson, A. E. W., Stone, D. J., Celi, L. A., & Pollard, T. J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx084 · Cited by 442 · see citing works
The Epidemiology of Chronic Critical Illness in the United States*
Kahn, J. M., Le, T., Angus, D. C., Cox, C. E., Hough, C. L., White, D. B., Yende, S., & Carson, S. S. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000000710 · Cited by 420 · see citing works
Readmission diagnoses after hospitalization for severe sepsis and other acute medical conditions
Prescott, H. C., Langa, K. M., & Iwashyna, T. J. (2015) · JAMA · doi:10.1001/jama.2015.1410 · Cited by 286 · see citing works
Self-citationED door-to-antibiotic time and long-term mortality in sepsis.
Peltan, I. D., Brown, S. M., Bledsoe, J. R., Sorensen, J., Samore, M. H., Allen, T. L., & Hough, C. L. (2019) · Chest · doi:10.1016/j.chest.2019.02.008 · Cited by 243 · see citing works
Diagnosing sepsis is subjective and highly variable: A survey of intensivists using case vignettes.
Rhee, C., Kadri, S. S., Danner, R. L., Suffredini, A. F., Massaro, A. F., Kitch, B. T., Lee, G., & Klompas, M. (2016) · Critical Care · doi:10.1186/s13054-016-1266-9 · Cited by 187 · see citing works
Hospitalization type and subsequent severe sepsis
Prescott, H. C., Dickson, R. P., Rogers, M. A. M., Langa, K. M., & Iwashyna, T. J. (2015) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201503-0483oc · Cited by 181 · see citing works
Self-citationMcCoy, A., & Das, R. (2017) · BMJ Open Quality · doi:10.1136/bmjoq-2017-000158 · Cited by 148 · see citing works
Kahn, J. M., Davis, B. S., Yabes, J. G., Chang, C.-C. H., Chong, D. H., Hershey, T. B., Martsolf, G. R., & Angus, D. C. (2019) · JAMA · doi:10.1001/jama.2019.9021 · Cited by 149 · see citing works
Post-acute care use and hospital readmission after sepsis.
Jones, T. K., Fuchs, B. D., Small, D. S., Halpern, S. D., Hanish, A., Umscheid, C. A., Baillie, C. A., Kerlin, M. P., Gaieski, D. F., & Mikkelsen, M. E. (2015) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201411-504oc · Cited by 140 · see citing works
Rhee, C., Wang, R., Zhang, Z., Fram, D., Kadri, S. S., & Klompas, M. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003817 · Cited by 122 · see citing works
Fleischmann-Struzek, C., Thomas-Rüddel, D. O., Schettler, A., Schwarzkopf, D., Stacke, A., Seymour, C. W., Haas, C., Dennler, U., & Reinhart, K. (2018) · PLOS ONE · doi:10.1371/journal.pone.0198847 · Cited by 95 · see citing works
Rhee, C., Jentzsch, M. S., Kadri, S. S., Seymour, C. W., Angus, D. C., Murphy, D. J., Martin, G. S., Dantes, R. B., Epstein, L., Fiore, A. E., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003554 · Cited by 85 · see citing works
Trends in empiric broad-Spectrum antibiotic use for suspected community-onset Sepsis in US hospitals
Rhee, C., Chen, T., Kadri, S. S., Lawandi, A., Yek, C., Walker, M., Warner, S., Fram, D., Chen, H.-C., Shappell, C. N., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.18923 · Cited by 58 · see citing works
Dietz, B. W., Jones, T. K., Small, D. S., Gaieski, D. F., & Mikkelsen, M. E. (2017) · Medical Care · doi:10.1097/mlr.0000000000000669 · Cited by 18 · see citing works
- Authors
- Iwashyna, T. J., Odden, A., Rohde, J., Bonham, C., Kuhn, L., Malani, P., Chen, L., & Flanders, S.
- Affiliations
- University of Michigan (US)
- VA Center for Clinical Management Research (US)
- Corresponding author
- Iwashyna, Theodore J.
- Odden, Andrew
- Rohde, Jeffrey
- Bonham, Catherine
- Malani, Preeti
- Chen, Lena
- Flanders, Scott
- Venue
- Medical Care
- Published
- 2014-06
- Volume
- 52
- Issue
- 6
- Pages
- e39-e43
- ISSN
- 0025-7079 (ISSN-L)
- PMC
- PMC3568444
- OpenAlex
- W2331591468
- Cited by
- 358
- References
- 36
- Citations by year
- 2026: 12 · 2025: 12 · 2024: 18 · 2023: 29 · 2022: 21 · 2021: 25 · 2020: 37 · 2019: 33 · 2018: 37 · 2017: 33 · 2016: 32 · 2015: 40 · 2014: 20 · 2013: 8 · 2012: 1
- FWCI
- 16.52
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Medical Care (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- University of Michigan
- National Heart, Lung, and Blood Institute (K08HL091249, HL091249)
- Topics
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- severe sepsis, sepsis definitions, ICD-9-CM codes, administrative claims, positive predictive value, sensitivity and specificity, hospitalization, medical records review, hospitalists
- MeSH terms
- Academic Medical Centers, Adult, Aged, Algorithms, Diagnosis, Differential, Female, Medicare, Health Plan Implementation, Health Status Indicators, Hospitalization, Humans, Insurance Claim Review, Male, Middle Aged, Multiple Organ Failure, Shock, Septic, United States, Sepsis, Guideline Adherence, International Classification of Diseases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
