CoChat

Developing an Interpretable Machine Learning Model to Predict in-Hospital Mortality in Sepsis Patients: A Retrospective Temporal Validation Study

Open access

The DOI matches its Crossref record; 44 of the 45 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

Background: Risk stratification plays an essential role in the decision making for sepsis management, as existing approaches can hardly satisfy the need to assess this heterogeneous population. We aimed to develop and validate a machine learning model to predict in-hospital mortality in critically ill patients with sepsis. Methods: Adult patients fulfilling the definition of Sepsis-3 were included at a large tertiary medical center. Relevant clinical features were extracted within the first 24 h in ICU, re-classified into different genres, and utilized for model development under three strategies: “Basic + Lab”, “Basic + Intervention”, and “Whole” feature sets. Extreme gradient boosting (XGBoost) was compared with logistic regression (LR) and established severity scores. Temporal validation was conducted using admissions from 2017 to 2019. Results: The final cohort included 24,272 patients, of which 4013 patients formed the test cohort for temporal validation. The trained and fine-tuned XGBoost model with the whole feature set showed the best discriminatory ability in the test cohort with AUROC as 0.85, significantly higher than the XGBoost “Basic + Lab” model (0.83), the LR “Whole” model (0.82), SOFA (0.63), SAPS-II (0.73), and LODS score (0.74). The performance in varying subgroups remained robust, and predictors, such as increased urine output and supplemental oxygen therapy, were crucially correlated with improved survival when interpretability was explored. Conclusions: We developed and validated a novel XGBoost-based model and demonstrated significantly improved performance to LR and other scores in predicting the mortality risks of sepsis patients in the hospital using features in the first 24 h.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/jcm12030915

Cites

44 of 45 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Flowchart of cohort selection.
Figure 1 Flowchart of cohort selection.CC BY · PubMed Central
Figure 2 The distribution of hospital stays and sepsis onset after ICU admission according to hospital mortality in all patients (N = 24,272). Panel (A) illustrates the distribution of hospital length of stays over 30 days after ICU admission stratified by patients’ in-hospital outcomes; (B) shows the distribution of sepsis onset before 24 h and after 48 h of ICU admission among survivors and non-survivors. Time intervals were calculated using the timepoint of ICU admission as point zero. Blue bins represented survivors and yellow bins represented non-survivors.
Figure 2 The distribution of hospital stays and sepsis onset after ICU admission according to hospital mortality in all patients (N = 24,272). Panel (A) illustrates the distribution of hospital length of stays over 30 days after ICU admission stratified by patients’ in-hospital outcomes; (B) shows the distribution of sepsis onset before 24 h and after 48 h of ICU admission among survivors and non-survivors. Time intervals were calculated using the timepoint of ICU admission as point zero. Blue bins represented survivors and yellow bins represented non-survivors.CC BY · PubMed Central
Figure 3 ROC curves presenting the performance of logistic regression, XGBoost models and single scores for predicting in-hospital mortality in the test cohort (N = 4013). ROC, receiver operating characteristics; SOFA, sequential organ failure assessment; SAPS, simplified acute physiology score; LODS, logistic organ dysfunction score; OASIS, oxford acute severity of illness score; XGBoost, extreme gradient boosting; LR, logistic regression.
Figure 3 ROC curves presenting the performance of logistic regression, XGBoost models and single scores for predicting in-hospital mortality in the test cohort (N = 4013). ROC, receiver operating characteristics; SOFA, sequential organ failure assessment; SAPS, simplified acute physiology score; LODS, logistic organ dysfunction score; OASIS, oxford acute severity of illness score; XGBoost, extreme gradient boosting; LR, logistic regression.CC BY · PubMed Central
Figure 4 The SHAP summary plot of the XGBoost “Whole” model for predicting in-hospital mortality of sepsis in the test cohort (N = 4013) (top 10 features included). SHAP, Shapley Additive explanations; XGBoost, extreme gradient boosting; CVD, cerebrovascular disorder; RDW, red blood cell distribution width; BUN, blood urea nitrogen; SpO2, peripheral oxygen saturation; WBC, white blood cell.
Figure 4 The SHAP summary plot of the XGBoost “Whole” model for predicting in-hospital mortality of sepsis in the test cohort (N = 4013) (top 10 features included). SHAP, Shapley Additive explanations; XGBoost, extreme gradient boosting; CVD, cerebrovascular disorder; RDW, red blood cell distribution width; BUN, blood urea nitrogen; SpO2, peripheral oxygen saturation; WBC, white blood cell.CC BY · PubMed Central
Figure 5 Calibration plot of the XGBoost “Whole” model in the test cohort (N = 4013). The black solid line at 45 degrees referred to a benchmark line of perfect calibration. The 10 black dots were average estimated and observed probabilities within 10% of the population divided based on predicted probability, and a black dashed line was straight through estimates. The red solid line represented a smoothed loess line through predicted outcomes.
Figure 5 Calibration plot of the XGBoost “Whole” model in the test cohort (N = 4013). The black solid line at 45 degrees referred to a benchmark line of perfect calibration. The 10 black dots were average estimated and observed probabilities within 10% of the population divided based on predicted probability, and a black dashed line was straight through estimates. The red solid line represented a smoothed loess line through predicted outcomes.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 45 references

Research works printed but not linked: 1

  1. 2016, Imputation with the R Package VIM, J. Stat. Softw., 74, 1

Cited by

See all 25

Metadata

Authors
Li, S., Dou, R., Song, X., Lui, K., Xu, J., Guo, Z., Hu, X., Guan, X., & Cai, C.
Affiliations
  • The First Affiliated Hospital, Sun Yat-sen University (CN)
  • Rice University (US)
Corresponding author
Cai, Changjie
Venue
Journal of Clinical Medicine
Publisher
MDPI AG
Published
2023-01-24
Volume
12
Issue
3
Pages
915
ISSN
2077-0383 (ISSN-L)
OpenAlex
W4317933350
Cited by
25
References
43
Citations by year
2026: 9 · 2025: 7 · 2024: 5 · 2023: 4
FWCI
4.99
Type
journal article
Language
en
Open access
Locations
  • Journal of Clinical Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
Massachusetts Institute of Technology
Topics
  • Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Hyperglycemia and glycemic control in critically ill and hospitalized patients
Keywords
sepsis, in-hospital mortality prediction, model interpretability, XGBoost, risk stratification, Sepsis-3, severity scoring, SOFA score, SAPS II, urine output, supplemental oxygen, critically ill patients, temporal validation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Developing an Interpretable Machine Learning Model to Predict in-Hospital Mortality in Sepsis Patients: A Retrospective Temporal Validation Study. https://app.cochat.ai/papers/10.3390/jcm12030915
The paper
Li, S., Dou, R., Song, X., Lui, K., Xu, J., Guo, Z., Hu, X., Guan, X., & Cai, C. (2023). Developing an Interpretable Machine Learning Model to Predict in-Hospital Mortality in Sepsis Patients: A Retrospective Temporal Validation Study. Journal of Clinical Medicine. https://doi.org/10.3390/jcm12030915

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.