Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework
Open accessThe DOI matches its Crossref record; 67 of the 75 references it cites were found in a registry or an index; 9 of the 13 sources its claims cite were checked for support.
Abstract
OBJECTIVE: To retrieve and appraise studies of deployed artificial intelligence (AI)-based sepsis prediction algorithms using systematic methods, identify implementation barriers, enablers, and key decisions and then map these to a novel end-to-end clinical AI implementation framework. MATERIALS AND METHODS: Systematically review studies of clinically applied AI-based sepsis prediction algorithms in regard to methodological quality, deployment and evaluation methods, and outcomes. Identify contextual factors that influence implementation and map these factors to the SALIENT implementation framework. RESULTS: The review identified 30 articles of algorithms applied in adult hospital settings, with 5 studies reporting significantly decreased mortality post-implementation. Eight groups of algorithms were identified, each sharing a common algorithm. We identified 14 barriers, 26 enablers, and 22 decision points which were able to be mapped to the 5 stages of the SALIENT implementation framework. DISCUSSION: Empirical studies of deployed sepsis prediction algorithms demonstrate their potential for improving care and reducing mortality but reveal persisting gaps in existing implementation guidance. In the examined publications, key decision points reflecting real-word implementation experience could be mapped to the SALIENT framework and, as these decision points appear to be AI-task agnostic, this framework may also be applicable to non-sepsis algorithms. The mapping clarified where and when barriers, enablers, and key decisions arise within the end-to-end AI implementation process. CONCLUSIONS: A systematic review of real-world implementation studies of sepsis prediction algorithms was used to validate an end-to-end staged implementation framework that has the ability to account for key factors that warrant attention in ensuring successful deployment, and which extends on previous AI implementation frameworks.
Overview
CoChat Summary
TL;DR
This systematic review of real-world sepsis prediction MLAs found 30 articles on adult hospital deployments and concluded that these systems may reduce mortality, but a definitive causal link was not demonstrated. The authors also mapped 14 barriers, 26 enablers, and 22 decision points onto the SALIENT implementation framework to show how end-to-end AI deployment factors fit into a staged model.
Key findings
- The review included 30 articles (19 primary articles plus 11 linked papers) on adult hospital settings; the studies were published between 2015 and 2022.
- Across the included studies, the authors identified 14 unique barriers, 26 unique enablers, and 22 decision points; 17 of the decision points were identified by at least 2 groups.
- 5 studies reported significantly decreased mortality post-implementation; all 9 papers reporting mortality found decreased mortality, but significance was limited to those 5 studies.
- The most common barriers were lack of clinician trust, alert fatigue, and dismissal of alerts; the most common enabler was frequent communication to raise awareness of the MLA.
Methods
- Systematic review conducted according to PRISMA; searched PubMed/Medline, EMBASE, Scopus, Web of Science, and clinicaltrials.gov for English-language studies from January 1, 2012 to June 23, 2022.
- Included studies had to deploy MLAs to adult hospitalized patients with formally defined sepsis, use live or near-live data, and report at least one performance metric.
- Extracted implementation barriers, enablers, and decision points were mapped to the SALIENT framework; mortality studies also underwent risk-of-bias assessment using ROBINS-I or RoB 2.
Limitations
- The paper states limitations of the evidence base: small number of empirical deployed studies, under-reporting of post-implementation performance metrics, focus on adult hospital settings, and potential publication bias from under-reporting of other implementation studies.
- The authors also note that many studies were non-randomized and subject to confounding from co-interventions such as sepsis awareness programs, and that post-implementation performance metrics were infrequently reported and non-standardized.
- Evident from the text: most studies were from the US, so generalizability beyond that setting is limited (evident in the study characteristics).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2. Evaluation results for algorithm performance, adoption, clinical process, and mortality improvement for each MLA group. Figure includes risk of bias (RoB) assessment for studies reporting mortality assessment (M = moderate [some concerns for RoB-2], S = serious, C = critical). Black numbered squares denote the cited paper for that result. AUC: area under the receiver operating curve; Sens: sensitivity; Spec: specificity; PPV/NPV: positive/negative predictive value. Solid shaded up arrow: significant improvement, whereas hollow up arrow: non-significant improvement.](https://figures.cochat.ai/paper-figures/335a66efe8784a43b0da8b8f017ba02a/deployment-of-machine-learning-algorithms-to-predict-sepsis-systematic-review-figure-2.jpg)

Verification
Claims the paper makes
3 Valid · 0 Not valid · 0 Not checked · 12 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Eight groups of algorithms were identified, each sharing a common algorithm.”
§ Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All barriers and enablers could be mapped to the SALIENT AI implantation framework (see Figure 3).”
§ Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework > RESULTS > Identification of implementation barriers, enablers, and decision points and mapping to SALIENT AI implementation framework
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A systematic review of real-world implementation studies of sepsis prediction algorithms was used to validate an end-to-end staged implementation framework”
§ Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework > Abstract > Conclusions
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 12
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“More complex systems have incorporated gradient boosting, neural networks, and ensemble methods to identify sepsis risk hours before clinical recognition, providing a larger window for interventions [46].”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Across studies, post-implementation performance was inconsistently reported and often declined compared to pre-deployment metrics, reflecting the impact of data and model drift due to evolving clinical practices and data variability, underscoring the need for continuous monitoring, recalibration, and lifecycle evaluation to maintain model reliability in real-world settings.”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Implementation barriers for sepsis AI cluster around trust and interpretability, workflow integration, technical readiness and generalizability, governance and safety, leadership alignment, and the resources required for sustainment (Table 4) [10,33,34,46,48,77,78].”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [46]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A systematic review of machine learning sepsis prediction implementations found that only a minority demonstrated statistically significant reductions in mortality, and many reported primarily algorithm performance and process improvements rather than patient-centered outcomes [46].”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [46]
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496
Bone, R. C., Balk, R. A., Cerra, F. B., Dellinger, R. P., Fein, A. M., Knaus, W. A., Schein, R. M. H., & Sibbald, W. J. (1992) · Chest · doi:10.1378/chest.101.6.1644 · Cited by 13,198 · cited 1×
“Sepsis has been defined as a “life-threatening organ dysfunction caused by a dysregulated host response to infection”.(2,3) Early recognition and treatment of sepsis can reduce mortality, and rule-based surveillance systems for detecting sepsis in hospital settings can improve outcomes.”
BackgroundRudd, K. E., Johnson, S. C., Agesa, K. M., Shackelford, K. A., Tsoi, D., Kievlan, D. R., Colombara, D. V., Ikuta, K. S., Kissoon, N., Finfer, S., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(19)32989-7 · Cited by 7,971 · cited 1×
“Sepsis accounts for nearly 20% of deaths worldwide, killing over 11 million people in 2017.(1) Sepsis has been defined as a “life-threatening organ dysfunction caused by a dysregulated host response to infection”.”
BackgroundBioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Seymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., Rubenfeld, G., Kahn, J. M., Shankar-Hari, M., Singer, M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0288 · Cited by 3,939
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887
Cited by
Nair, M., Svedberg, P., Larsson, I., & Nygren, J. M. (2024) · PLOS ONE · doi:10.1371/journal.pone.0305949 · Cited by 179
A Nationwide Network of Health AI Assurance Laboratories
Shah, N. H., Halamka, J. D., Saria, S., Pencina, M., Tazbaz, T., Tripathi, M., Callahan, A., Hildahl, H., & Anderson, B. (2024) · JAMA · doi:10.1001/jama.2023.26930 · Cited by 114
Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework
van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad088 · Cited by 109
Self-citationLi, F., Wang, S., Gao, Z., Qing, M., Pan, S. L., Liu, Y., & Hu, C. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1510792 · Cited by 119
Achieving large-scale clinician adoption of AI-enabled decision support
Scott, I. A., van der Vegt, A., Lane, P., McPhail, S., & Magrabi, F. (2024) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2023-100971 · Cited by 65
Self-citation
Metadata
- Authors
- van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J.
- Affiliations
- Queensland Digital Health Centre, The University of Queensland , Brisbane, Queensland, Australia
- Princess Alexandra Hospital (AU)
- The University of Queensland (AU)
- Queensland Health (AU)
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-05-12
- Volume
- 30
- Issue
- 7
- Pages
- 1349-1361
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC10280361
- OpenAlex
- W4376270103
- Cited by
- 72
- References
- 75
- Citations by year
- 2026: 24 · 2025: 25 · 2024: 11 · 2023: 12
- FWCI
- 2.80
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Queensland Government
- Advanced Queensland Industry Research Fellowship
- Topics
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- sepsis prediction, machine learning algorithms, implementation barriers and facilitators, sepsis mortality, algorithm deployment
- MeSH terms
- Machine Learning, Adult, Algorithms, Artificial Intelligence, Humans, Sepsis, Empirical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework. https://app.cochat.ai/papers/10.1093/jamia/ocad075
- The paper
- van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J. (2023). Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocad075
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.