CoChat

DeepSOFA: a continuous acuity score for critically ill patients using clinically interpretable deep learning.

Open access
Shickel, Benjamin, Loftus, Tyler J., Adhikari, Lasith, Ozrazgat-Baslanti, Tezcan, Bihorac, Azra, Rashidi, Parisa · 2019 · Scientific Reports

The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.

Abstract

Traditional methods for assessing illness severity and predicting in-hospital mortality among critically ill patients require time-consuming, error-prone calculations using static variable thresholds. These methods do not capitalize on the emerging availability of streaming electronic health record data or capture time-sensitive individual physiological patterns, a critical task in the intensive care unit. We propose a novel acuity score framework (DeepSOFA) that leverages temporal measurements and interpretable deep learning models to assess illness severity at any point during an ICU stay. We compare DeepSOFA with SOFA (Sequential Organ Failure Assessment) baseline models using the same model inputs and find that at any point during an ICU admission, DeepSOFA yields significantly more accurate predictions of in-hospital mortality. A DeepSOFA model developed in a public database and validated in a single institutional cohort had a mean AUC for the entire ICU stay of 0.90 (95% CI 0.90–0.91) compared with baseline SOFA models with mean AUC 0.79 (95% CI 0.79–0.80) and 0.85 (95% CI 0.85–0.86). Deep models are well-suited to identify ICU patients in need of life-saving interventions prior to the occurrence of an unexpected adverse event and inform shared decision-making processes among patients, providers, and families regarding goals of care and optimal resource utilization.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-019-38491-0

Cites

25 of 25 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 DeepSOFA performance in two external validation cohorts. (A,B) Externally validated DeepSOFA, Bedside SOFA, and Traditional SOFA score accuracy in predicting in-hospital mortality, expressed as area under the receiver operating characteristic curve (AUC) for the first 100 hours following ICU admission. (C,D) Externally validated DeepSOFA accuracy for individual models corresponding to variable sets derived from SOFA organ system classification for the first 100 hours following ICU admission. Shaded regions represent 95% confidence intervals based on 100 bootstrapped iterations. Columns specify the validation cohort. DeepSOFA model validated in UFHealth (A,C) was trained using MIMIC, and DeepSOFA model validated in MIMIC (B,D) was trained using UFHealth. SOFA: Sequential Organ Failure Assessment.
Figure 1 DeepSOFA performance in two external validation cohorts. (A,B) Externally validated DeepSOFA, Bedside SOFA, and Traditional SOFA score accuracy in predicting in-hospital mortality, expressed as area under the receiver operating characteristic curve (AUC) for the first 100 hours following ICU admission. (C,D) Externally validated DeepSOFA accuracy for individual models corresponding to variable sets derived from SOFA organ system classification for the first 100 hours following ICU admission. Shaded regions represent 95% confidence intervals based on 100 bootstrapped iterations. Columns specify the validation cohort. DeepSOFA model validated in UFHealth (A,C) was trained using MIMIC, and DeepSOFA model validated in MIMIC (B,D) was trained using UFHealth. SOFA: Sequential Organ Failure Assessment.CC BY · PubMed Central
Figure 2 Model accuracy for prediction windows of increasing time from hospital discharge or death. (A,B) Externally validated DeepSOFA, Bedside SOFA, and Traditional SOFA score accuracy in predicting in-hospital mortality, expressed as area under the receiver operating characteristic curve (AUC) for 100 hours preceding death or hospital discharge. (C,D) Externally validated DeepSOFA accuracy for individual models corresponding to variable sets derived from SOFA organ system classification for the 100 hours preceding death or hospital discharge. Shaded regions represent 95% confidence intervals based on 100 bootstrapped iterations. Columns specify the validation cohort. DeepSOFA model validated in UFHealth (A,C) was trained using MIMIC, and DeepSOFA model validated in MIMIC (B,D) was trained using UFHealth. SOFA: Sequential Organ Failure Assessment.
Figure 2 Model accuracy for prediction windows of increasing time from hospital discharge or death. (A,B) Externally validated DeepSOFA, Bedside SOFA, and Traditional SOFA score accuracy in predicting in-hospital mortality, expressed as area under the receiver operating characteristic curve (AUC) for 100 hours preceding death or hospital discharge. (C,D) Externally validated DeepSOFA accuracy for individual models corresponding to variable sets derived from SOFA organ system classification for the 100 hours preceding death or hospital discharge. Shaded regions represent 95% confidence intervals based on 100 bootstrapped iterations. Columns specify the validation cohort. DeepSOFA model validated in UFHealth (A,C) was trained using MIMIC, and DeepSOFA model validated in MIMIC (B,D) was trained using UFHealth. SOFA: Sequential Organ Failure Assessment.CC BY · PubMed Central
Figure 3 Externally validated DeepSOFA and Bedside SOFA score predicted mortality (A) for a single patient from the UFHealth cohort, correlated with vital signs (B), clinical events (D), and clinical interventions (E). Shown also are model self-attention weights (C) for each hour after ICU admission, where darker bars indicate increased model focus. Attention weights taken from the diagonal of full self-attention matrix and indicate how important the model believes each hour’s data is, as it is encountered in real-time. SOFA: Sequential Organ Failure Assessment, SpO2: oxygen saturation, MICU: Medical Intensive Care Unit, CT: computed tomography, ACLS: Advanced Cardiac Life Support, ROSC: return of spontaneous circulation.
Figure 3 Externally validated DeepSOFA and Bedside SOFA score predicted mortality (A) for a single patient from the UFHealth cohort, correlated with vital signs (B), clinical events (D), and clinical interventions (E). Shown also are model self-attention weights (C) for each hour after ICU admission, where darker bars indicate increased model focus. Attention weights taken from the diagonal of full self-attention matrix and indicate how important the model believes each hour’s data is, as it is encountered in real-time. SOFA: Sequential Organ Failure Assessment, SpO2: oxygen saturation, MICU: Medical Intensive Care Unit, CT: computed tomography, ACLS: Advanced Cardiac Life Support, ROSC: return of spontaneous circulation.CC BY · PubMed Central
Figure 4 Mean predicted mortality probabilities for externally validated DeepSOFA and Bedside SOFA models stratified by outcome. Probabilities shown both for first 100 hours after ICU admission (A,B) and final 100 hours before hospital discharge or death (C,D). Number of ICU encounters at each time point shown below each panel. Shaded regions around each line represent 95% confidence intervals based on 100 bootstrapped iterations. Gray shared area denotes hourly mortality rate for active ICU encounters. Columns specify the validation cohort. DeepSOFA model validated in UFHealth (A,C) was trained using MIMIC, and DeepSOFA model validated in MIMIC (B,D) was trained using UFHealth. SOFA: Sequential Organ Failure Assessment.
Figure 4 Mean predicted mortality probabilities for externally validated DeepSOFA and Bedside SOFA models stratified by outcome. Probabilities shown both for first 100 hours after ICU admission (A,B) and final 100 hours before hospital discharge or death (C,D). Number of ICU encounters at each time point shown below each panel. Shaded regions around each line represent 95% confidence intervals based on 100 bootstrapped iterations. Gray shared area denotes hourly mortality rate for active ICU encounters. Columns specify the validation cohort. DeepSOFA model validated in UFHealth (A,C) was trained using MIMIC, and DeepSOFA model validated in MIMIC (B,D) was trained using UFHealth. SOFA: Sequential Organ Failure Assessment.CC BY · PubMed Central
Figure 5 Visualized self-attention distributions for an example survivor and non-survivor from the UFHealth cohort, using DeepSOFA trained on the MIMIC cohort. Darker squares indicate increased model focus as a function of the passage of time (x-axis). Shown also are variable time series at each hour of the ICU stay, with initial and final measurement values shown on the left and right, respectively. MAP: mean arterial pressure, FiO2: fraction of inspired oxygen, PaO2: partial pressure of oxygen, SpO2: oxygen saturation, GCS: Glasgow Coma Scale.
Figure 5 Visualized self-attention distributions for an example survivor and non-survivor from the UFHealth cohort, using DeepSOFA trained on the MIMIC cohort. Darker squares indicate increased model focus as a function of the passage of time (x-axis). Shown also are variable time series at each hour of the ICU stay, with initial and final measurement values shown on the left and right, respectively. MAP: mean arterial pressure, FiO2: fraction of inspired oxygen, PaO2: partial pressure of oxygen, SpO2: oxygen saturation, GCS: Glasgow Coma Scale.CC BY · PubMed Central
Figure 6 Three identical views of our DeepSOFA model at increasing levels of abstraction, from most technical (A) to highest level operation (C). Panel A illustrates mortality prediction calculation for an example data segment x of five hours in the ICU and corresponding hourly acuity assessments p, where at each time point only the current and previous time steps are used in attention calculations and mortality predictions. Panel B displays a more general and compact form for the same stages of data transformation. Panel C describes the high-level inputs and outputs for DeepSOFA, where along with overall acuity assessment, interpretable prediction rationale by way of salient sequence patterns are visualized by a heatmap of self-attention weights. A more technical description of each stage of DeepSOFA can be found in the Supplemental Section Model Details.
Figure 6 Three identical views of our DeepSOFA model at increasing levels of abstraction, from most technical (A) to highest level operation (C). Panel A illustrates mortality prediction calculation for an example data segment x of five hours in the ICU and corresponding hourly acuity assessments p, where at each time point only the current and previous time steps are used in attention calculations and mortality predictions. Panel B displays a more general and compact form for the same stages of data transformation. Panel C describes the high-level inputs and outputs for DeepSOFA, where along with overall acuity assessment, interpretable prediction rationale by way of salient sequence patterns are visualized by a heatmap of self-attention weights. A more technical description of each stage of DeepSOFA can be found in the Supplemental Section Model Details.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 25 references

Research works printed but not linked: 1

  1. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning. (MIT Press, 2016).

Cited by

See all 165

Metadata

Authors
Shickel, B., Loftus, T. J., Adhikari, L., Ozrazgat-Baslanti, T., Bihorac, A., & Rashidi, P.
Affiliations
University of Florida (US)
Corresponding author
Bihorac, Azra
Venue
Scientific Reports
Published
2019-02-12
Volume
9
Issue
1
Pages
1879-1879
ISSN
2045-2322 (ISSN-L)
OpenAlex
W2889037799
Cited by
165
References
22
Citations by year
2026: 12 · 2025: 22 · 2024: 16 · 2023: 26 · 2022: 28 · 2021: 29 · 2020: 23 · 2019: 9
FWCI
10.15
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Internal funds at J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida.
  • U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (T32 GM-008721, R01 GM110240, P50 GM-111152)
  • National Science Foundation (NSF-IIS 1750192)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
in-hospital mortality prediction, critically ill patients, intensive care unit, explainable deep learning, SOFA score, electronic health record data, goals of care, shared decision-making
MeSH terms
Deep Learning, Adolescent, Adult, Aged, Critical Care, Decision Making, Female, Humans, Intensive Care Units, Length of Stay, Longitudinal Studies, Male, Middle Aged, Probability, Retrospective Studies, Severity of Illness Index, Databases, Factual, Critical Illness, Hospital Mortality, Area Under Curve, Young Adult, Organ Dysfunction Scores

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: DeepSOFA: a continuous acuity score for critically ill patients using clinically interpretable deep learning. https://app.cochat.ai/papers/10.1038/s41598-019-38491-0
The paper
Shickel, B., Loftus, T. J., Adhikari, L., Ozrazgat-Baslanti, T., Bihorac, A., & Rashidi, P. (2019). DeepSOFA: a continuous acuity score for critically ill patients using clinically interpretable deep learning. Scientific Reports. https://doi.org/10.1038/s41598-019-38491-0

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.