Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index; 63 of the 87 sources its claims cite were checked for support.

Explainable AI for critical care: a systematic review of interpretable models for sepsis and ICU mortality prediction

Open access
Athukorala, V. S., Ilmini, W. M. K. S. · 2026 · BMC Medical Informatics and Decision Making

Abstract

PURPOSE: BACKGROUND: Sepsis is a leading cause of mortality in intensive care units (ICUs), and its rapid progression poses significant challenges for early detection. Traditional scoring systems, such as SOFA and APACHE II, provide clinical benchmarks but often fail to capture subtle early signs of deterioration. Machine learning (ML) and deep learning (DL) models have demonstrated strong predictive performance; however, their “black-box” nature limits transparency, clinician trust, and adoption in real-world ICU settings. Explainable artificial intelligence (XAI) clarifies how predictions are derived. METHODOLOGY: This systematic review examines studies published between 2020 and 2025 that applied XAI methods, including SHAP, LIME, Grad-CAM, and sensitivity analysis, to predict sepsis onset and ICU mortality. We analyzed the datasets used (e.g., MIMIC-III/IV, Emory University Hospital, Ruijin Hospital), model architectures, interpretability strategies, and the clinical features most strongly associated with predictions. RESULTS: Findings indicate that XAI-enhanced models not only maintain high predictive accuracy but also highlight clinically meaningful indicators such as respiratory rate, blood urea nitrogen (BUN), urine output, and the Glasgow Coma Scale (GCS), thereby improving clinician confidence and facilitating adoption. CONCLUSION: Despite these advances, challenges remain, including limited prospective evaluation, inconsistent interpretability metrics, and variable integration into clinical workflows. We conclude by recommending future research priorities, including real-world validation, user-centered design, and multimodal data integration, to ensure that XAI can reliably support timely and informed decision-making in critical care environments.

Overview

CoChat Summary

## TL;DR

This systematic review synthesized 43 studies (2020–2025) on explainable AI for sepsis onset and ICU mortality/readmission prediction in adult ICUs. It finds that XAI methods, especially SHAP, can preserve strong predictive performance while highlighting clinically meaningful features, but prospective validation and standardized interpretability evaluation remain limited.

## Key findings

  • The review included 43 studies; 15 were summarized as representative studies in Table 1.
  • Across the reviewed literature, XAI-enhanced models often reported strong discrimination, with AUC values above 0.84 and some reaching up to 0.94 (e.g., Random Forest on MIMIC-IV) and 0.89 for early sepsis prediction.
  • Commonly highlighted predictive features were respiratory rate, Glasgow Coma Scale (GCS), blood urea nitrogen (BUN), urine output, and mean arterial pressure (MAP).
  • The most commonly used interpretability methods were SHAP, LIME, and sensitivity analysis; Grad-CAM was also used in some studies.

## Methods

  • Design: Systematic review conducted according to PRISMA 2020 guidelines.
  • Data/sample: Studies published between January 2020 and January 2025 on adult ICU sepsis onset, ICU readmission, or in-hospital mortality; sources included ResearchGate, IEEE Xplore, Scopus, Web of Science, and Google Scholar.
  • Analysis: Two reviewers independently screened studies and extracted data; study quality/risk of bias was assessed with PROBAST, and evidence was synthesized by dataset, model type, validation approach, performance metrics, XAI method, and top features.

## Limitations

  • The paper states that many included studies relied on retrospective data, often from single-center datasets, which limits generalizability.
  • It also states that external/temporal validation, prospective studies, and clinician-centered evaluations were limited, and that interpretability metrics were inconsistent across studies.
  • Evident from the text: the review’s conclusions are based on a literature synthesis rather than new patient-level experimentation, so its claims depend on the quality and reporting of the included studies.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12911-026-03344-0

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Illustration of explainable AI (XAI) in ICU settings, showing how model transparency supports clinical decision-making (adapted from Chaddad et al. [3])
Fig. 1 Illustration of explainable AI (XAI) in ICU settings, showing how model transparency supports clinical decision-making (adapted from Chaddad et al. [3])CC BY-NC-ND · PubMed Central
Fig. 2 Traditional scoring systems versus AI/XAI pipelines for sepsis and ICU mortality prediction
Fig. 2 Traditional scoring systems versus AI/XAI pipelines for sepsis and ICU mortality predictionCC BY-NC-ND · PubMed Central
Fig. 3 PRISMA 2020 flow diagram of study selection
Fig. 3 PRISMA 2020 flow diagram of study selectionCC BY-NC-ND · PubMed Central
Fig. 4 Overview of interpretable machine learning models for sepsis prediction. The diagram shows the models, their predictive tasks (e.g., ICU mortality, early sepsis onset), and the associated references
Fig. 4 Overview of interpretable machine learning models for sepsis prediction. The diagram shows the models, their predictive tasks (e.g., ICU mortality, early sepsis onset), and the associated referencesCC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

19 Valid · 6 Not valid · 7 Not checked · 45 Own results and interpretations

30 more claims · 4 with their cited sources checked

Create a free account to see all 77 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 35
  • Probably not a substantial claim. 14

Create a free account to see these claims.

Citations

Cites

Show all 35 references

Research works printed but not linked: 4

  1. Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. In: Advances in neural information processing systems (NeurIPS). Vol. 30. 2017. arXiv:1705.07874. · arXiv
  2. Nicolaou A, Panayides A, Pattichis C, Kakas A, Stylianides C. An overview of explainable AI studies in the prediction of sepsis onset and sepsis mortality. In: Digital health and informatics innovations for sustainable health. Elsevier; 2024.
  3. Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. In: Advances in neural information processing systems (NeurIPS). Vol. 30. 2017. arXiv:1705.07874. · arXiv
  4. Zhang TY, Zhong M, Cheng YZ, Zhang MW. An interpretable machine learning model for real-time sepsis prediction based on basic physiological indicators. Eur Rev Med Pharmacological Sci. 2023;27(10):4348–56.

Cited by

See all 11

Metadata

Authors
Athukorala, V. S., & Ilmini, W. M. K. S.
Affiliations
University of Sri Jayewardenepura (LK)
Corresponding author
  • Athukorala, V. S.
  • Ilmini, W. M. K. S.
Venue
BMC Medical Informatics and Decision Making
Published
2026-02-03
Volume
26
Issue
1
ISSN
1472-6947 (ISSN-L)
OpenAlex
W7127278855
Cited by
10
References
35
Citations by year
2026: 10
FWCI
11.53
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Informatics and Decision Making (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
explainable AI, sepsis prediction, ICU mortality prediction, model interpretability, SHAP, lime, Grad-CAM, sensitivity analysis, intensive care, clinical decision support, MIMIC-III, MIMIC-IV, early sepsis detection
MeSH terms
Predictive Learning Models, Artificial Intelligence, Critical Care, Humans, Intensive Care Units, Hospital Mortality, Sepsis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Explainable AI for critical care: a systematic review of interpretable models for sepsis and ICU mortality prediction. https://app.cochat.ai/papers/10.1186/s12911-026-03344-0
The paper
Athukorala, V. S., & Ilmini, W. M. K. S. (2026). Explainable AI for critical care: a systematic review of interpretable models for sepsis and ICU mortality prediction. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-026-03344-0

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.