Real-World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Background: Successful integrations of machine learning into routine clinical care are exceedingly rare, and barriers to its adoption are poorly characterized in the literature. Objective: This study aims to report a quality improvement effort to integrate a deep learning sepsis detection and management platform, Sepsis Watch, into routine clinical care. Methods: In 2016, a multidisciplinary team consisting of statisticians, data scientists, data engineers, and clinicians was assembled by the leadership of an academic health system to radically improve the detection and treatment of sepsis. This report of the quality improvement effort follows the learning health system framework to describe the problem assessment, design, development, implementation, and evaluation plan of Sepsis Watch. Results: Sepsis Watch was successfully integrated into routine clinical care and reshaped how local machine learning projects are executed. Frontline clinical staff were highly engaged in the design and development of the workflow, machine learning model, and application. Novel machine learning methods were developed to detect sepsis early, and implementation of the model required robust infrastructure. Significant investment was required to align stakeholders, develop trusting relationships, define roles and responsibilities, and to train frontline staff, leading to the establishment of 3 partnerships with internal and external research groups to evaluate Sepsis Watch. Conclusions: Machine learning models are commonly developed to enhance clinical decision making, but successful integrations of machine learning into routine clinical care are rare. Although there is no playbook for integrating deep learning into clinical care, learnings from the Sepsis Watch integration can inform efforts to develop machine learning technologies at other health care delivery systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Conversely, alerts that are poorly integrated into workflow can increase interruptions, contribute to unnecessary antibiotics, and shift attention away from other competing tasks.”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Sendak et al. performed a single-center implementation study integrating a deep learning-based sepsis detection platform into the emergency department of an academic health system. They used EHR time-series data from ~39,918 adult ED encounters to develop and deploy a multitask Gaussian processes (MGPs)–recurrent neural network (RNN) model within a real-time clinical workflow.”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studies of sepsis alerts frequently report improvements in process measures such as time-to-antibiotics or completion of key bundle elements, particularly when these alerts are coupled to structured triage, escalation, and rapid clinician response.”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Implementation barriers for sepsis AI cluster around trust and interpretability, workflow integration, technical readiness and generalizability, governance and safety, leadership alignment, and the resources required for sustainment (Table 4) [10,33,34,46,48,77,78].”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [10]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The system was successfully implemented with high clinician engagement and continuous risk prediction, demonstrating feasibility of real-world integration but requiring substantial infrastructure and organizational coordination [10].”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [10]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 1×
“The next goal to drive adoption was to broadly communicate the change vision and empower action [43,44].”
BackgroundHigh-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · cited 1×
“Machine learning, a set of statistical methods optimized for prediction on new observations, is central to this transformation [1].”
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141
Surviving Sepsis Campaign: International guidelines for management of sepsis and septic shock: 2016.
Rhodes, A., Evans, L. E., Alhazzani, W., Levy, M. M., Antonelli, M., Ferrer, R., Kumar, A., Sevransky, J. E., Sprung, C. L., Nunnally, M. E., et al. (2017) · Intensive Care Medicine · doi:10.1007/s00134-017-4683-6 · Cited by 6,621
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“All new machine learning implementations have the potential for introducing inequality and bias that is not always clearly visible in numeric data [45-48].”
BackgroundRecurrent Neural Networks for Multivariate Time Series with Missing Values
Che, Z., Purushotham, S., Cho, K., Sontag, D., & Liu, Y. (2018) · Scientific Reports · doi:10.1038/s41598-018-24271-9 · Cited by 2,370 · cited 1×
“RNNs are a type of deep learning configured to ingest time series data and are ideally suited to combine static and dynamic features of hospital encounters that vary in length [39,40].”
BackgroundAbràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0040-6 · Cited by 1,638 · cited 1×
“Isolated efforts demonstrating the clinical impact of previously validated technologies show the potential of machine learning in health care [3,4].”
BackgroundPrognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508
Research works printed but not linked: 5
- SculleyDHoltGGolovinDHidden Technical Debt in Machine Learning SystemsNIPS Proceedings20142020-06-16https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf
- SariaSSubbaswamyATutorial: Safe and Reliable Machine LearningACM Conference on Fairness, Accountability, and Transparency201901FAT'19January 29-31, 2019Atlanta, USA
- 10.5555/3305381.3305503 · doi:10.5555/3305381.3305503
- Barocas, S, 2016, Cal L Rev, 104, 671
- Moore, G, 2006, Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers
Cited by
A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare
Ahmed, M. I., Spooner, B., Isherwood, J., Lane, M., Orrock, E., & Dennison, A. (2023) · Cureus · doi:10.7759/cureus.46454 · Cited by 364
Power to the People? Opportunities and Challenges for Participatory AI
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S. (2022) · Equity and Access in Algorithms Mechanisms and Optimization · doi:10.1145/3551624.3555290 · Cited by 330
“The recent upswing in participatory tools have varied in approach and point in the machine learning product lifecycle, including: auditing methods for machine learning based interventions [48, 75], public consultation methods such as citizen juries [6, 83] or joint problem formulation [61], information disclosures suc…”
BackgroundSelf-citationThe Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623261 · Cited by 256
Artificial Intelligence Applications in Health Care Practice: Scoping Review
Sharma, M., Savage, C., Nair, M., Larsson, I., Svedberg, P., & Nygren, J. M. (2022) · Journal of Medical Internet Research · doi:10.2196/40238 · Cited by 198
“However, the study by Sendak et al [46] encouraged the separation of developers and clinicians and made conscious efforts to shift focus away from the technical aspects of AI.”
InfluentialNair, M., Svedberg, P., Larsson, I., & Nygren, J. M. (2024) · PLOS ONE · doi:10.1371/journal.pone.0305949 · Cited by 179
Metadata
- Authors
- Sendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Miller, C., Kester, K., Sandhu, S., Corey, K., Brajer, N., Tan, C., Lin, A., Brown, T., Engelbosch, S., Anstrom, K., Elish, M. C., Heller, K., Donohoe, R., Theiling, J., Poon, E., Balu, S., Bedoya, A., & O'Brien, C.
- Affiliations
- Duke University (US)
- Duke University Hospital (US)
- Harvard University (US)
- Duke Clinical Research Institute (US)
- Data & Society Research Institute (US)
- Google (United States) (US)
- Corresponding author
- Sendak, Mark P
- ORCID
- Sendak, Mark P
- Ratliff, William
- Sarro, Dina
- Alderton, Elizabeth
- Futoma, Joseph
- Gao, Michael
- Nichols, Marshall
- Revoir, Mike
- Yashar, Faraz
- Miller, Corinne
- Kester, Kelly
- Sandhu, Sahil
- Corey, Kristin
- Brajer, Nathan
- Tan, Christelle
- Lin, Anthony
- Brown, Tres
- Engelbosch, Susan
- Anstrom, Kevin
- Elish, Madeleine Clare
- Heller, Katherine
- Donohoe, Rebecca
- Theiling, Jason
- Poon, Eric
- Balu, Suresh
- Bedoya, Armando
- O'Brien, Cara
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2020-07-15
- Volume
- 8
- Issue
- 7
- Pages
- e15182
- ISSN
- 2291-9694 (ISSN-L)
- PMC
- PMC7391165
- OpenAlex
- W2998524378
- Cited by
- 287
- References
- 48
- Citations by year
- 2026: 102 · 2025: 59 · 2024: 36 · 2023: 32 · 2022: 38 · 2021: 18 · 2020: 2
- FWCI
- 11.04
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Informatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- John D. and Catherine T. MacArthur Foundation
- Duke Institute for Health Innovation
- Topics
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- sepsis, deep learning, clinical implementation, early sepsis detection, learning health systems, quality improvement, clinical decision support, stakeholder engagement, clinical workflow integration, health care delivery systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


