CoChat

FDA-Authorized AI/ML Tool for Sepsis Prediction: Development and Validation

Bhargava, Akhil, López-Espina, Carlos, Schmalz, Lee, Khan, Shah, Watson, Gregory L., Urdiales, Dennys, Updike, Lincoln, Kurtzman, Niko, Dagan, Alon, Doodlesack, Amanda, Stenson, Bryan A., Sarma, Deesha, Reseland, Eric, Lee, John H., Kravitz, Max S., Antkowiak, Peter S., Shvilkina, Tatyana, Espinosa, Aimee, Halalau, Alexandra, Demarco, Carmen, Davila, Francisco, Davila, Hugo, Sims, Matthew, Maddens, Nicholas, Berghea, Ramona, Smith, Scott, Palagiri, Ashok V., Ezekiel, Clinton, Sadaka, Farid, Iyer, Karthik, Crisp, Matthew, Azad, Saleem, Oke, Vikram, Friederich, Andrew, Syed, Anwaruddin, Gosai, Falgun, Chawla, Lavneet, Evans, Neil, Thomas, Kurian, Malkani, Roneil, Patel, Roshni, Mayer, Stockton, Ali, Farhan, Raghavakurup, Lekshminarayan, Tafa, Muleta, Singh, Sahib, Raouf, Samuel, Zhao, Sihai Dave, Zhu, Ruoqing, Bashir, Rashid, Reddy, Bobby, Shapiro, Nathan I. · 2024 · NEJM AI
Cited by 72

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Abstract

BACKGROUND Sepsis is a life-threatening condition that demands prompt treatment for improved patient outcomes. Its heterogenous presentation makes early detection challenging, highlighting the need for effective risk assessment tools. Artificial intelligence (AI) models could potentially identify patients with sepsis, but none have previously been authorized by the U.S. Food and Drug Administration (FDA) for commercial use. This study outlines the development and validation of the Sepsis ImmunoScore, the first FDA-authorized AI-based software designed to identify patients at risk of sepsis. METHODS In this prospective study, we enrolled adult patients (18+ years of age) suspected of infection, as indicated by a blood culture order, from five U.S. institutions between April 2017 and July 2022. The participants were divided into an algorithm development cohort (n=2366), an internal validation cohort (n=393), and an external validation cohort (n=698). The primary end point was sepsis presence (as defined by Sepsis-3) within 24 hours of test initiation. Secondary end points included length of hospital stay, intensive care unit (ICU) admission within 24 hours, mechanical ventilation use within 24 hours, vasopressor use within 24 hours, and in-hospital mortality. RESULTS The Sepsis ImmunoScore demonstrated high diagnostic accuracy, with an area under the curve of 0.85 (0.83 to 0.87) in the derivation cohort, 0.80 (0.74 to 0.86) in internal validation, and 0.81 (0.77 to 0.86) in external validation. The scores were categorized into four sepsis risk levels with corresponding likelihood ratios: low (0.1), medium (0.5), high (2.1), and very high (8.3). These risk categories also predicted in-hospital mortality: low (0.0%), medium (1.9%), high (8.7%), and very high (18.2%) in the external validation cohort. Similar trends were observed for other metrics, such as length of hospital stay, ICU utilization, mechanical ventilation, and vasopressor use. CONCLUSIONS The Sepsis ImmunoScore demonstrated high accuracy for the identification and prediction of sepsis and critical illness metrics that could enable prompt identification of patients at high risk of sepsis and adverse outcomes, potentially improving clinical decision-making and patient outcomes. (Funded by the Defense Threat Reduction Agency and others.).

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1056/aioa2400867

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 72

Metadata

Authors
Bhargava, A., López-Espina, C., Schmalz, L., Khan, S., Watson, G. L., Urdiales, D., Updike, L., Kurtzman, N., Dagan, A., Doodlesack, A., Stenson, B. A., Sarma, D., Reseland, E., Lee, J. H., Kravitz, M. S., Antkowiak, P. S., Shvilkina, T., Espinosa, A., Halalau, A., Demarco, C., Davila, F., Davila, H., Sims, M., Maddens, N., Berghea, R., Smith, S., Palagiri, A. V., Ezekiel, C., Sadaka, F., Iyer, K., Crisp, M., Azad, S., Oke, V., Friederich, A., Syed, A., Gosai, F., Chawla, L., Evans, N., Thomas, K., Malkani, R., Patel, R., Mayer, S., Ali, F., Raghavakurup, L., Tafa, M., Singh, S., Raouf, S., Zhao, S. D., Zhu, R., Bashir, R., Reddy, B., & Shapiro, N. I.
Venue
NEJM AI
Published
2024-11-27
Volume
1
Issue
12
ISSN
2836-9386 (ISSN-L)
OpenAlex
W4404756513
Cited by
72
References
12
FWCI
18.83
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.