Clinician perception of a machine learning–based early warning system designed to predict severe sepsis and septic shock

Ginestra, Jennifer C.; Giannini, Heather M.; Schweickert, William D.; Meadows, Laurie; Lynch, Michael J.; Pavan, Kimberly; Chivers, Corey J.; Draugelis, Michael; Donnelly, Patrick J.; Fuchs, Barry D.; Umscheid, Craig A.

doi:10.1097/ccm.0000000000003803

Cited by 159Sepsis Diagnosis and TreatmentHealthcare Technology and Patient MonitoringMachine Learning in Healthcare

Clinician perception of a machine learning–based early warning system designed to predict severe sepsis and septic shock

Open access

Ginestra, Jennifer C., Giannini, Heather M., Schweickert, William D., Meadows, Laurie, Lynch, Michael J., Pavan, Kimberly, Chivers, Corey J., Draugelis, Michael +3 more·2019·Critical Care Medicine

DOI 10.1097/ccm.0000000000003803PMC PMC6791738

Open this paper in CoChat — free account

Abstract

Objective: To assess clinician perceptions of a machine learning–based early warning system to predict severe sepsis and septic shock (Early Warning System 2.0). Design: Prospective observational study. Setting: Tertiary teaching hospital in Philadelphia, PA. Patients: Non-ICU admissions November–December 2016. Interventions: During a 6-week study period conducted 5 months after Early Warning System 2.0 alert implementation, nurses and providers were surveyed twice about their perceptions of the alert’s helpfulness and impact on care, first within 6 hours of the alert, and again 48 hours after the alert. Measurements and Main Results: For the 362 alerts triggered, 180 nurses (50% response rate) and 107 providers (30% response rate) completed the first survey. Of these, 43 nurses (24% response rate) and 44 providers (41% response rate) completed the second survey. Few (24% nurses, 13% providers) identified new clinical findings after responding to the alert. Perceptions of the presence of sepsis at the time of alert were discrepant between nurses (13%) and providers (40%). The majority of clinicians reported no change in perception of the patient’s risk for sepsis (55% nurses, 62% providers). A third of nurses (30%) but few providers (9%) reported the alert changed management. Almost half of nurses (42%) but less than a fifth of providers (16%) found the alert helpful at 6 hours. Conclusions: In general, clinical perceptions of Early Warning System 2.0 were poor. Nurses and providers differed in their perceptions of sepsis and alert benefits. These findings highlight the challenges of achieving acceptance of predictive and machine learning–based sepsis alerts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1097/ccm.0000000000003803

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

All 9 papers reported decreased mortality, be that all-cause, sepsis-related or both, although this was statistically significant only for 5 studies.
“All 9 papers34,44,46,50,52,58–61 reported decreased mortality, be that all-cause, sepsis-related or both, although this was statistically significant only for 5 studies”
From Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework (2023) · cites this paper as [44]
The Introduction reports a nonsignificant trend toward reduced mortality for the authors’ earlier EWS 1.0 system, but attributes that finding to a previous study. This paper surveys clinician perceptions of EWS 2.0; its Results do not report a mortality decrease for the system studied here. It also does not establish the sentence’s count of nine papers or five statistically significant findings. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Many models have been developed to improve timely identification of sepsis, but their lack of adoption has led to an implementation gap in early warning systems for sepsis.
“Many models have been developed to improve timely identification of sepsis,4,5,6,7,8,9 but their lack of adoption has led to an implementation gap in early warning systems for sepsis.10,11”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [11]
The Introduction (page not available in our copy) says clinician acceptance is crucial and reports that only a minority of providers found the authors’ earlier sepsis alert helpful. The Discussion > Challenges describes poor acceptance of their newer system. These passages support an adoption barrier, but do not establish that widespread lack of adoption has caused an implementation gap across sepsis early warning systems. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Some implementation studies have found that AI-based alerts are relatively more salient to nurses than physicians in this regard.
“Some implementation studies have found that AI-based alerts are relatively more salient to nurses than physicians in this regard.79”
From Potential uses of AI for perioperative nursing handoffs: a qualitative study (2023) · cites this paper as [79]
It has been difficult for predictive CDSSs to achieve impactful adoption.
“It has been difficult for predictive CDSSs to achieve impactful adoption [10,13,14].”
From Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study (2022) · cites this paper as [13]
The same previous studies found that false positives from other nonsepsis etiologies could increase alarm fatigue.
“They also found that false positives from other nonsepsis etiologies could increase alarm fatigue.”
From Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study (2020) · cites this paper as [27]

Two previous studies examining predictive alerts for sepsis suggested that perceived utility decreased when the model frequently identified clinically stable patients.
“Similarly, 2 previous studies examining predictive alerts for sepsis suggested that the perceived utility decreased when the model frequently identified clinically stable patients [27,28].”
From Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study (2020) · cites this paper as [27]
Concerns have arisen recently regarding the potential for sepsis models to cause alert fatigue.
“Concerns have arisen recently regarding the potential for sepsis models to cause alert fatigue.2,3,4”
From Quantification of Sepsis Model Alerts in 24 US Hospitals Before and During the COVID-19 Pandemic (2021) · cites this paper as [3]
Previous studies of deployed systems have used post-hoc surveys or interviews to gauge provider impressions.
“In order to better understand factors impacting the adoption of CDS, previous studies of deployed systems have used post-hoc surveys or interviews to gauge provider impressions 28,44,53 ; however, provider impressions can differ from their actual use 4,18,27 .”
From Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing (2022) · cites this paper as [53]
Ginestra et al. (2019) developed an early warning score system with machine learning algorithms to predict sepsis or septic shock, deployed in a large academic hospital with nurses and physicians, but their perceptions of the AI based tool were poor as most reported no change in their understanding of patient risk and only a handful of alerts led to change in patient management.
“Ginestra et al. (2019) developed an early warning score (EWS) system with machine learning algorithms to predict sepsis or septic shock.This was deployed in a large academic hospital in the United States with nurses and physicians, who received alerts from the EWS system about at-risk patients, but their perceptions of the AI based tool were poor as most reported no change in their understanding”
From The application and use of artificial intelligence in cancer nursing: A systematic review (2024) · cites this paper as Ginestra et al., 2019

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.08.010 · Cited by 51,384 · see citing works
Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock
Kumar, A., Roberts, D., Wood, K. E., Light, B., Parrillo, J. E., Sharma, S., Suppes, R., Feinstein, D., Zanotti, S., Taiberg, L., et al. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000217961.75225.e9 · Cited by 6,109 · see citing works
Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014.
Rhee, C., Dantes, R., Epstein, L., Murphy, D. J., Seymour, C. W., Iwashyna, T. J., Kadri, S. S., Angus, D. C., Danner, R. L., Fiore, A. E., et al. (2017) · JAMA · doi:10.1001/jama.2017.13836 · Cited by 1,884 · see citing works
Unintended consequences of machine learning in medicine
Cabitza, F., Rasoini, R., & Gensini, G. F. (2017) · JAMA · doi:10.1001/jama.2017.7797 · Cited by 1,066 · see citing works
Exploring physician specialist response rates to web-based surveys
Cunningham, C. T., Quan, H., Hemmelgarn, B., Noseworthy, T., Beck, C. A., Dixon, E., Samuel, S., Ghali, W. A., Sykes, L. L., & Jetté, N. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0016-z · Cited by 995 · see citing works
National Performance Benchmarks for Modern Screening Digital Mammography: Update from the Breast Cancer Surveillance Consortium
Lehman, C. D., Arao, R. F., Sprague, B. L., Lee, J. M., Buist, D. S. M., Kerlikowske, K. M., Henderson, L. M., Onega, T., Tosteson, A. N. A., Rauscher, G. H., et al. (2016) · Radiology · doi:10.1148/radiol.2016161174 · Cited by 667 · see citing works
Multicenter Comparison of Machine Learning Methods and Conventional Regression for Predicting Clinical Deterioration on the Wards
Churpek, M. M., Yuen, T. C., Winslow, C., Meltzer, D. O., Kattan, M. W., & Edelson, D. P. (2016) · Critical Care Medicine · doi:10.1097/ccm.0000000000001571 · Cited by 638 · see citing works
A targeted real-time early warning score (TREWScore) for septic shock.
Henry, K. E., Hager, D. N., Pronovost, P. J., & Saria, S. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aab3719 · Cited by 598 · see citing works
Surviving Sepsis Campaign
Levy, M. M., Rhodes, A., Phillips, G. S., Townsend, S. R., Schorr, C., Beale, R. J., Osborn, T. M., Lemeshow, S. A., Chiche, J., Artigas, A., et al. (2014) · Critical Care Medicine · doi:10.1097/ccm.0000000000000723 · Cited by 566 · see citing works
Screening for Lung Cancer With Low-Dose Computed Tomography: A Systematic Review to Update the U.S. Preventive Services Task Force Recommendation
Humphrey, L. L., Deffebach, M. E., Pappas, M., Baumann, C. I., Artis, K. A., Mitchell, J. P., Zakher, B., Fu, R. F., & Slatore, C. G. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-6-201309170-00690 · Cited by 534 · see citing works
Prediction of Sepsis in the Intensive Care Unit With Minimal Electronic Health Record Data: A Machine Learning Approach
Desautels, T., Calvert, J., Hoffman, J., Jay, M., Kerem, Y., Shieh, L., Shimabukuro, D., Chettipally, U., Feldman, M. D., Barton, C., et al. (2016) · JMIR Medical Informatics · doi:10.2196/medinform.5909 · Cited by 509 · see citing works
Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: A randomised clinical trial.
Shimabukuro, D. W., Barton, C. W., Feldman, M. D., Mataraso, S. J., & Das, R. (2017) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2017-000234 · Cited by 382 · see citing works
Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5-year study
Levy, M. M., Rhodes, A., Phillips, G. S., Townsend, S. R., Schorr, C., Beale, R. J., Osborn, T. M., Lemeshow, S. A., Chiche, J., Artigas, A., et al. (2014) · Intensive Care Medicine · doi:10.1007/s00134-014-3496-0 · Cited by 279 · see citing works
A Machine Learning Algorithm to Predict Severe Sepsis and Septic Shock: Development, Implementation, and Impact on Clinical Practice*
Giannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269 · see citing works
Self-citation
A computational approach to early sepsis detection
Calvert, J. S., Price, D. A., Chettipally, U. K., Barton, C. W., Feldman, M. D., Hoffman, J. L., Jay, M., & Das, R. (2016) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2016.05.003 · Cited by 244 · see citing works
Early detection of impending physiologic deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record
Escobar, G. J., LaGuardia, J. C., Turk, B. J., Ragins, A. I., Kipnis, P., & Draper, D. (2012) · Journal of Hospital Medicine · doi:10.1002/jhm.1929 · Cited by 185 · see citing works
Implementation of a real-time computerized sepsis alert in nonintensive care unit patients*
Sawyer, A. M., Deal, E. N., Labelle, A. J., Witt, C., Thiel, S. W., Heard, K., Reichley, R. M., Micek, S. T., & Kollef, M. H. (2011) · Critical Care Medicine · doi:10.1097/ccm.0b013e318205df85 · Cited by 169 · see citing works
Development, implementation, and impact of an automated early warning and response system for sepsis
Umscheid, C. A., Betesh, J., VanZandbergen, C., Hanish, A., Tait, G., Mikkelsen, M. E., French, B. C., & Fuchs, B. D. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2259 · Cited by 167 · see citing works
Self-citation
Reducing patient mortality, length of stay and readmissions through machine learning-based sepsis prediction in the emergency department, intensive care unit and hospital floor units
McCoy, A., & Das, R. (2017) · BMJ Open Quality · doi:10.1136/bmjoq-2017-000158 · Cited by 148 · see citing works
Using Electronic Health Record Data to Develop and Validate a Prediction Model for Adverse Outcomes in the Wards*
Churpek, M. M., Yuen, T. C., Park, S. Y., Gibbons, R. D., & Edelson, D. P. (2013) · Critical Care Medicine · doi:10.1097/ccm.0000000000000038 · Cited by 138 · see citing works
National Performance Benchmarks for Modern Diagnostic Digital Mammography: Update from the Breast Cancer Surveillance Consortium
Sprague, B. L., Arao, R. F., Miglioretti, D. L., Henderson, L. M., Buist, D. S. M., Onega, T., Rauscher, G. H., Lee, J. M., Tosteson, A. N. A., Kerlikowske, K. M., et al. (2017) · Radiology · doi:10.1148/radiol.2017161519 · Cited by 125 · see citing works
A trial of a real‐time Alert for clinical deterioration in Patients hospitalized on general medical wards
Bailey, T. C., Chen, Y., Mao, Y., Lu, C., Hackmann, G., Micek, S. T., Heard, K. M., Faulkner, K. M., & Kollef, M. H. (2013) · Journal of Hospital Medicine · doi:10.1002/jhm.2009 · Cited by 121 · see citing works
Identifying Patients With Sepsis on the Hospital Wards
Bhattacharjee, P., Edelson, D. P., & Churpek, M. M. (2017) · Chest · doi:10.1016/j.chest.2016.06.020 · Cited by 120 · see citing works
Early prediction of septic shock in hospitalized patients
Thiel, S. W., Rosini, J. M., Shannon, W. D., Doherty, J. A., Micek, S. T., & Kollef, M. H. (2010) · Journal of Hospital Medicine · doi:10.1002/jhm.530 · Cited by 103 · see citing works
A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team
Kollef, M. H., Chen, Y., Heard, K. M., LaRossa, G. N., Lu, C., Martin, N. R., Martin, N., Micek, S. T., & Bailey, T. C. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2193 · Cited by 91 · see citing works
Clinical Decision Support for Early Recognition of Sepsis
Amland, R. C., & Hahn-Cover, K. E. (2014) · American Journal of Medical Quality · doi:10.1177/1062860614557636 · Cited by 85 · see citing works
Nonelective Rehospitalizations and Postdischarge Mortality
Escobar, G. J., Ragins, A. I., Scheirer, P., Liu, V. X., Robles, J. A. M., & Kipnis, P. (2015) · Medical Care · doi:10.1097/mlr.0000000000000435 · Cited by 74 · see citing works
Outcomes and Resource Use of Sepsis-associated Stays by Presence on Admission, Severity, and Hospital Type
Jones, S. L., Ashton, C. M., Kiehne, L., Nicolas, J. C., Rose, A. L., Shirkey, B. A., Masud, F. N., & Wray, N. P. (2016) · Medical Care · doi:10.1097/mlr.0000000000000481 · Cited by 69 · see citing works
Testing the Impact of Mixed‐Mode Designs (Mail and Web) and Multiple Contact Attempts within Mode (Mail or Web) on Clinician Survey Response
Beebe, T. J., Jacobson, R. M., Jenkins, S. M., Lackore, K. A., & Rutten, L. J. F. (2018) · Health Services Research · doi:10.1111/1475-6773.12827 · Cited by 54 · see citing works
Clinician Perception of the Effectiveness of an Automated Early Warning and Response System for Sepsis in an Academic Medical Center
Guidi, J. L., Clark, K., Upton, M. T., Faust, H., Umscheid, C. A., Lane-Fall, M. B., Mikkelsen, M. E., Schweickert, W. D., Vanzandbergen, C. A., Betesh, J., et al. (2015) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201503-129oc · Cited by 43 · see citing works
Self-citation
The impact of an electronic medical record surveillance program on outcomes for patients with sepsis
McRee, L., Thanavaro, J. L., Moore, K. S., Goldsmith, M., & Pasvogel, A. E. (2014) · Heart & Lung · doi:10.1016/j.hrtlng.2014.05.009 · Cited by 35 · see citing works
Toward a Two-Tier Clinical Warning System for Hospitalized Patients
Hackmann, G. (2010) · PubMed · doi:10.7936/k70v8b1g · Cited by 34 · see citing works
Identifying Severe Sepsis via Electronic Surveillance
Brandt, B., Gartner, A., Moncure, M., Cannon, C. M., Carlton, E. F., Cleek, C., Wittkopp, C., & Simpson, S. Q. (2014) · American Journal of Medical Quality · doi:10.1177/1062860614541291 · Cited by 27 · see citing works
Developing an Early Sepsis Alert Program
Buck, K. (2013) · Journal of Nursing Care Quality · doi:10.1097/ncq.0b013e3182a98182 · Cited by 23 · see citing works
An Interprofessional Process to Improve Early Identification and Treatment for Sepsis
Palleschi, M. T., Sirianni, S., O’Connor, N., Dunn, D., & Hasenau, S. M. (2013) · Journal for Healthcare Quality · doi:10.1111/jhq.12006 · Cited by 20 · see citing works
Reduction in Time to First Action as a Result of Electronic Alerts for Early Sepsis Recognition
Kurczewski, L., Sweet, M. A., McKnight, R., & Halbritter, K. (2015) · Critical Care Nursing Quarterly · doi:10.1097/cnq.0000000000000060 · Cited by 15 · see citing works

Flagged first, then by citations

Sorted by citations

External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review
Yin, J., Ngiam, K. Y., & Teo, H. H. (2021) · Journal of Medical Internet Research · doi:10.2196/25759 · Cited by 419 · see citing works
Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis
Adams, R., Henry, K. E., Sridharan, A., Soleimani, H., Zhan, A., Rawat, N., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01894-0 · Cited by 365 · see citing works
Artificial Intelligence -based technologies in nursing: A scoping literature review of the evidence
von Gerich, H., Moen, H., Block, L. J., Chu, C. H., DeForest, H., Hobensack, M., Michalowski, M., Mitchell, J., Nibber, R., Olalia, M. A., et al. (2022) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.104153 · Cited by 360 · see citing works
A Machine Learning Algorithm to Predict Severe Sepsis and Septic Shock: Development, Implementation, and Impact on Clinical Practice*
Giannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269 · see citing works
Self-citation
Predicted Influences of Artificial Intelligence on the Domains of Nursing: Scoping Review
Buchanan, C., Howitt, M. L., Wilson, R., Booth, R. G., Risling, T., & Bamford, M. (2020) · JMIR Nursing · doi:10.2196/23939 · Cited by 238 · see citing works
A Path for Translation of Machine Learning Products into Healthcare Delivery
(2020) · EMJ Innovations · doi:10.33590/emjinnov/19-00172 · Cited by 172 · see citing works
Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study
Sandhu, S., Lin, A. L., Brajer, N., Sperling, J., Ratliff, W., Bedoya, A. D., Balu, S., O'Brien, C., & Sendak, M. P. (2020) · Journal of Medical Internet Research · doi:10.2196/22421 · Cited by 168 · see citing works
Applications of Artificial Intelligence in Nursing Care: A Systematic Review
Martinez‐Ortigosa, A., Martinez-Granados, A., Gil-Hernández, E., Rodríguez-Arrastia, M. J., Ropero‐Padilla, C., & Román, P. (2023) · Journal of Nursing Management · doi:10.1155/2023/3219127 · Cited by 151 · see citing works
Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141 · see citing works
Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing
Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01895-z · Cited by 127 · see citing works
Machine Learning-Based Early Prediction of Sepsis Using Electronic Health Records: A Systematic Review
Islam, K. R., Prithula, J., Kumar, J., Tan, T. L., Reaz, M. B. I., Sumon, M. S. I., & Chowdhury, M. E. H. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12175658 · Cited by 124 · see citing works
An Explainable Artificial Intelligence Predictor for Early Detection of Sepsis
Yang, M., Liu, C., Wang, X., Li, Y., Gao, H., Liu, X., & Li, J. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004550 · Cited by 120 · see citing works
Vasopressor Therapy in the Intensive Care Unit
Russell, J. A., Gordon, A. C., Williams, M. D., Boyd, J. H., Walley, K. R., & Kissoon, N. “. (2020) · Seminars in Respiratory and Critical Care Medicine · doi:10.1055/s-0040-1710320 · Cited by 96 · see citing works
Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study
Schwartz, J. M., George, M., Rossetti, S. C., Dykes, P. C., Minshall, S. R., Lucas, E., & Cato, K. D. (2022) · JMIR Human Factors · doi:10.2196/33960 · Cited by 88 · see citing works
“It has been difficult for predictive CDSSs to achieve impactful adoption [10,13,14].”
Background
Harnessing artificial intelligence in sepsis care: advances in early detection, personalized treatment, and real-time monitoring
Li, F., Wang, S., Gao, Z., Qing, M., Pan, S. L., Liu, Y., & Hu, C. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1510792 · Cited by 87 · see citing works
A Time-Phased Machine Learning Model for Real-Time Prediction of Sepsis in Critical Care
Li, X., Xu, X., Xie, F., Xu, X., Sun, Y., Liu, X., Jia, X., Kang, Y., Xie, L., Wang, F., et al. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004494 · Cited by 82 · see citing works
Development and Verification of a Digital Twin Patient Model to Predict Specific Treatment Response During the First 24 Hours of Sepsis
Lal, A., Li, G., Cubro, E., Chalmers, S. J., Li, H., Herasevich, V., Dong, Y., Pickering, B. W., Kılıçkaya, O., & Gajic, O. (2020) · Critical Care Explorations · doi:10.1097/cce.0000000000000249 · Cited by 80 · see citing works
Clinician involvement in research on machine learning–based predictive clinical decision support for the hospital setting: A scoping review
Schwartz, J., Moy, A. J., Rossetti, S. C., Elhadad, N., & Cato, K. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa296 · Cited by 79 · see citing works
Barriers to and facilitators of clinician acceptance and use of artificial intelligence in healthcare settings: a scoping review
Scipion, C. E. A., Manchester, M., Federman, A. D., Wang, Y., & Arias, J. J. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-092624 · Cited by 78 · see citing works
Clinician-Facing AI in the Wild: Taking Stock of the Sociotechnical Challenges and Opportunities for HCI
Zając, H. D., Li, D., Dai, X., Carlsen, J. F., Kensing, F., & Andersen, T. O. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3582430 · Cited by 76 · see citing works
Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework
van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad075 · Cited by 72 · see citing works
“Predicting onset of sepsis as early as possible involved trade-offs between: (1) alerts that were too early, where clinicians may not have known what to do, and therefore dismissed the alerts(33,41,42,44,47,50,53); and (2) alerts that were too late for patients for whom clinicians already suspected sepsis and had init…”
Background
Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review
Giddings, R., Joseph, A., Callender, T., Janes, S. M., van der Schaar, M., Sheringham, J., & Navani, N. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00241-8 · Cited by 70 · see citing works
Early Detection of Sepsis With Machine Learning Techniques: A Brief Clinical Perspective
Giacobbe, D. R., Signori, A., Puente, F. D., Mora, S., Carmisciano, L., Briano, F., Vena, A., Ball, L., Robba, C., Pelosi, P. P., et al. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.617486 · Cited by 68 · see citing works
Using machine learning to improve the accuracy of patient deterioration predictions: Mayo Clinic Early Warning Score (MC-EWS)
Brufau, S. R., Whitford, D., Johnson, M. G., Hickman, J. A., Morlan, B. W., Therneau, T. M., Naessens, J. M., & Huddleston, J. M. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa347 · Cited by 68 · see citing works
Effects of machine learning-based clinical decision support systems on decision-making, care delivery, and patient outcomes: a scoping review
Susanto, A. P., Lyell, D., Widyantoro, B., Berkovsky, S., & Magrabi, F. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad180 · Cited by 67 · see citing works
Survival prediction of patients with sepsis from age, sex, and septic episode number alone
Chicco, D., & Jurman, G. (2020) · Scientific Reports · doi:10.1038/s41598-020-73558-3 · Cited by 65 · see citing works
Implementation of Machine Learning Applications in Health Care Organizations: Systematic Review of Empirical Studies
Preti, L. M., Ardito, V., Compagni, A., Petracca, F., & Cappellaro, G. (2024) · Journal of Medical Internet Research · doi:10.2196/55897 · Cited by 62 · see citing works
Development and validation of an interpretable neural network for prediction of postoperative in-hospital mortality
Lee, C. K., Samad, M., Hofer, I. S., Cannesson, M. P., & Baldi, P. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00377-1 · Cited by 61 · see citing works
Technology acceptance of a machine learning algorithm predicting delirium in a clinical setting: a mixed-methods study
Jauk, S., Kramer, D., Avian, A., Berghold, A., Leodolter, W., & Schulz, S. (2021) · Journal of Medical Systems · doi:10.1007/s10916-021-01727-6 · Cited by 59 · see citing works
Quantification of Sepsis Model Alerts in 24 US Hospitals Before and During the COVID-19 Pandemic
Wong, A., Cao, J., Lyons, P. G., Dutta, S., Major, V. J., Ötleş, E., & Singh, K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.35286 · Cited by 57 · see citing works
Health Care Professionals’ Experience of Using AI: Systematic Review With Narrative Synthesis
Ayorinde, A., Mensah, D. O., Walsh, J., Ghosh, I., Ibrahim, S. A., Hogg, J., Peek, N., & Griffiths, F. (2024) · Journal of Medical Internet Research · doi:10.2196/55766 · Cited by 55 · see citing works
Optimization of critical care pharmacy clinical services: A gap analysis approach
Sikora, A. N., Murray, B., Smith, S. E., Brothers, T. N., Al-Mamun, M. A., Chase, A. M., Rowe, S., Buckley, M. S., Murphy, D. J., & Devlin, J. W. (2021) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxab237 · Cited by 53 · see citing works
The application and use of artificial intelligence in cancer nursing: A systematic review
O'Connor, S., Vercell, A., Wong, D., Yorke, J., Fallatah, F. A., Cave, L., & Anny Chen, L.-Y. (2024) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2024.102510 · Cited by 51 · see citing works
Early Detection of In-Patient Deterioration: One Prediction Model Does Not Fit All
Blackwell, J., Keim‐Malpass, J., Clark, M., Kowalski, R. L., Najjar, S. N., Bourque, J. M., Lake, D. E., & Moorman, J. R. (2020) · Critical Care Explorations · doi:10.1097/cce.0000000000000116 · Cited by 51 · see citing works
Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: A qualitative study
Joshi, M., Mecklai, K., Rozenblum, R., & Samal, L. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac022 · Cited by 46 · see citing works
A systematic review of clinicians’ acceptance and use of clinical decision support systems over time
Newton, N., Bamgboje‐Ayodele, A., Forsyth, R., Tariq, A., & Baysari, M. T. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01662-7 · Cited by 45 · see citing works
Clinician preimplementation perspectives of a decision-support tool for the prediction of cardiac arrhythmia based on machine learning: near-live feasibility and qualitative study
Matthiesen, S., Diederichsen, S. Z., Hansen, M. K. H., Villumsen, C., Lassen, M. C. H., Jacobsen, P. K., Risum, N., Winkel, B. G., Philbert, B. T., Svendsen, J. H., et al. (2021) · JMIR Human Factors · doi:10.2196/26964 · Cited by 45 · see citing works
The application of artificial intelligence in the management of sepsis
Yang, J., Hao, S., Huang, J., Chen, T., Liu, R., Zhang, P., Feng, M., He, Y., Xiao, W., Hong, Y., et al. (2023) · Medical Review · doi:10.1515/mr-2023-0039 · Cited by 44 · see citing works
Microbiology in the era of artificial intelligence: transforming medical and pharmaceutical microbiology
Tsitou, V.-M., Rallis, D., Cekova, M., & Yanev, N. (2024) · Biotechnology & Biotechnological Equipment · doi:10.1080/13102818.2024.2349587 · Cited by 44 · see citing works
Preventing sepsis; how can artificial intelligence inform the clinical decision-making process? A systematic review
Hassan, N., Slight, R., Weiand, D., Vellinga, A., Morgan, G., Aboushareb, F., & Slight, S. P. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104457 · Cited by 42 · see citing works
Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review
Evans, R. P., Bryant, L. D., Russell, G., & Absolom, K. L. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105342 · Cited by 38 · see citing works
Quality of interaction between clinicians and artificial intelligence systems. A systematic review
Perivolaris, A., Adams-McGavin, C., Madan, Y., Kishibe, T., Antoniou, T., Mamdani, M., & Jung, J. J. (2024) · Future Healthcare Journal · doi:10.1016/j.fhj.2024.100172 · Cited by 38 · see citing works
A Vital Sign-Based Model to Predict Clinical Deterioration in Hospitalized Children*
Mayampurath, A. M., Jani, P., Dai, Y., Gibbons, R. D., Edelson, D. P., & Churpek, M. M. (2020) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000002414 · Cited by 37 · see citing works
Attitudes towards artificial intelligence in emergency medicine
Stewart, J. E., Freeman, S., Eroglu, E., Dumitrascu, N., Lü, J., Goudie, A., Sprivulis, P., Akhlaghi, H., Tran, V., Sanfilippo, F. M., et al. (2023) · Emergency Medicine Australasia · doi:10.1111/1742-6723.14345 · Cited by 36 · see citing works
Bringing the Promise of Artificial Intelligence to Critical Care: What the Experience With Sepsis Analytics Can Teach Us
Wardi, G., Owens, R., Josef, C., Malhotra, A., Longhurst, C., & Nemati, S. (2023) · Critical Care Medicine · doi:10.1097/ccm.0000000000005894 · Cited by 35 · see citing works
Artificial intelligence and computer simulation models in critical illness
Lal, A., Pinevich, Y., Gajic, O., Herasevich, V., & Pickering, B. (2020) · World Journal of Critical Care Medicine · doi:10.5492/wjccm.v9.i2.13 · Cited by 34 · see citing works
Machine Learning Models for Early Prediction of Sepsis on Large Healthcare Datasets
Camacho-Cogollo, J. E., Bonet, I., Gil, B., & Iadanza, E. (2022) · Electronics · doi:10.3390/electronics11091507 · Cited by 35 · see citing works
Intelligent Clinical Decision Support
Pinsky, M. R., Dubrawski, A. W., & Clermont, G. (2022) · Sensors · doi:10.3390/s22041408 · Cited by 34 · see citing works
Development of Machine Learning Models to Validate a Medication Regimen Complexity Scoring Tool for Critically Ill Patients
Al-Mamun, M. A., Brothers, T. N., & Sikora, A. N. (2020) · Annals of Pharmacotherapy · doi:10.1177/1060028020959042 · Cited by 33 · see citing works
Prediction and risk assessment of sepsis-associated encephalopathy in ICU based on interpretable machine learning
Lu, X., Kang, H., Zhou, D., & Li, Q. (2022) · Scientific Reports · doi:10.1038/s41598-022-27134-6 · Cited by 31 · see citing works
Development and validation of a deep-learning-based pediatric early warning system: A single-center study
Park, S. J., Cho, K.-J., Kwon, O., Park, H., Lee, Y., Shim, W. H., Park, C. R., & Jhang, W. K. (2021) · Biomedical Journal · doi:10.1016/j.bj.2021.01.003 · Cited by 30 · see citing works
Assessing Barriers to Implementation of Machine Learning and Artificial Intelligence–Based Tools in Critical Care: Web-Based Survey Study
Mlodzinski, E., Wardi, G., Viglione, C., Nemati, S., Crotty Alexander, L., & Malhotra, A. (2023) · JMIR Perioperative Medicine · doi:10.2196/41056 · Cited by 28 · see citing works
Potential uses of AI for perioperative nursing handoffs: a qualitative study
King, C. R., Shambe, A., & Abraham, J. (2023) · JAMIA Open · doi:10.1093/jamiaopen/ooad015 · Cited by 27 · see citing works
“79 Imperative AI-based CDS has been effective in some direct use cases, supporting this approach, 80 but it runs the risk of automation bias.”
Method
Evaluation of Machine Learning Models for Clinical Prediction Problems*
Sanchez-Pinto, L. N., & Bennett, T. D. (2022) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000002942 · Cited by 25 · see citing works
Nurses' perceptions of the design, implementation, and adoption of machine learning clinical decision support: A descriptive qualitative study
Wieben, A. M., Alreshidi, B. G., Douthit, B. J., Sileo, M., Vyas, P., Steege, L. M. B., & Gilmore‐Bykovskyi, A. L. (2024) · Journal of Nursing Scholarship · doi:10.1111/jnu.13001 · Cited by 24 · see citing works
Comparison of automated sepsis identification methods and electronic health record-based sepsis phenotyping: improving case identification accuracy by accounting for confounding comorbid conditions.
Henry, K. E., Hager, D. N., Osborn, T. M., Wu, A. W., & Saria, S. (2019) · Critical Care Explorations · doi:10.1097/cce.0000000000000053 · Cited by 22 · see citing works
Evaluating sepsis watch generalizability through multisite external validation of a sepsis machine learning model.
Valan, B., Prakash, A., Ratliff, W., Gao, M., Muthya, S., Thomas, A., Eaton, J. L., Gardner, M., Nichols, M., Revoir, M., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01664-5 · Cited by 21 · see citing works
Barriers and facilitators to optimal sepsis care – a systematized review of healthcare professionals’ perspectives
Draeger, L., Fleischmann-Struzek, C., Gehrke-Beck, S., Heintze, C., Thomas-Rüddel, D. O., & Schmidt, K. F. R. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-12777-8 · Cited by 20 · see citing works
Saving Lives With Algorithm-Enabled Process Innovation for Sepsis Care
Adjerid, I., Ayvaci, M., & Özer, Ö. (2019) · Manufacturing & Service Operations Management · doi:10.1287/msom.2023.1226 · Cited by 19 · see citing works
What can a learning healthcare system teach us about improving outcomes?
Casey, J. D., Courtright, K. R., Rice, T. W., & Semler, M. W. (2021) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000000857 · Cited by 17 · see citing works
Machine Learning Models for the Early Real-Time Prediction of Deterioration in Intensive Care Units—A Novel Approach to the Early Identification of High-Risk Patients
Thiele, D., Rodseth, R. N., Friedland, R., Berger, F., Mathew, C., Maslo, C., Moll, V., Leithner, C., Storm, C., Krannich, A., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14020350 · Cited by 16 · see citing works
Clinical Decision Support for Septic Shock in the Emergency Department: A Cluster Randomized Trial
Scott, H. F., Sevick, C. J., Colborn, K. L., Davies, S. J. D., Greer, C. H., Dafoe, A., Holliman, B. D., Bajaj, L., Schmidt, S. K., & Kempe, A. (2025) · PEDIATRICS · doi:10.1542/peds.2024-069478 · Cited by 15 · see citing works
Pharmacophenotype identification of intensive care unit medications using unsupervised cluster analysis of the ICURx common data model
Sikora, A. N., Rafiei, A., Rad, M. G., Keats, K. R., Smith, S. E., Devlin, J. W., Murphy, D. J., Murray, B., Kamaleswaran, R., & MRC-ICU Investigator Team (2023) · Critical Care · doi:10.1186/s13054-023-04437-2 · Cited by 15 · see citing works
"It depends": Configuring AI to Improve Clinical Usefulness Across Contexts
Zając, H. D., Ribeiro, J., Ingala, S., Gentile, S., Wanjohi, R., Gitau, S. N., Carlsen, J. F., Nielsen, M. B., & Andersen, T. O. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660707 · Cited by 14 · see citing works
Crawling robot manipulator tracking based on gaussian mixture model of machine vision
Lou, J. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06063-x · Cited by 14 · see citing works
Artificial Intelligence and Medicine
Vanak, J. (2022) · Science Insights · doi:10.15354/si.22.re068 · Cited by 14 · see citing works
Cluster analysis driven by unsupervised latent feature learning of medications to identify novel pharmacophenotypes of critically ill patients
Sikora, A. N., Jeong, H., Yu, M., Chen, X., Murray, B., & Kamaleswaran, R. (2023) · Scientific Reports · doi:10.1038/s41598-023-42657-2 · Cited by 13 · see citing works
Early Warning Systems for Critical Illness Outside the Intensive Care Unit
Henry, K. E., & Giannini, H. M. (2024) · Critical Care Clinics · doi:10.1016/j.ccc.2024.03.007 · Cited by 13 · see citing works
Self-citation
A novel classical machine learning framework for early sepsis prediction using electronic health record data from ICU patients
Prithula, J., Islam, K. R., Kumar, J., Tan, T. L., Reaz, M. B. I., Rahman, T., Zughaier, S. M., Khan, M. S., Murugappan, M., & Chowdhury, M. E. H. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109284 · Cited by 13 · see citing works
Validation of a Machine Learning Model for Early Shock Detection
Pinevich, Y., Amos‐Binks, A., Burris, C., Rule, G. T., Bogojevic, M., Flint, I., Pickering, B. W., Nemeth, C., & Herasevich, V. (2021) · Military Medicine · doi:10.1093/milmed/usab220 · Cited by 13 · see citing works
Development and validation of a risk model for hospital-acquired venous thrombosis: the Medical Inpatients Thrombosis and Hemostasis study
Zakai, N. A., Wilkinson, K. S., Sparks, A. D., Packer, R. T., Koh, I., Roetker, N. S., Repp, A. B., Thomas, R., Holmes, C. E., Cushman, M., et al. (2023) · Journal of Thrombosis and Haemostasis · doi:10.1016/j.jtha.2023.10.015 · Cited by 13 · see citing works
Influence of nurses in the implementation of artificial intelligence in health care: a scoping review
Sodeau, A., & Fox, A. (2022) · Australian Health Review · doi:10.1071/ah22164 · Cited by 12 · see citing works
A common data model for the standardization of intensive care unit medication features
Sikora, A. N., Keats, K. R., Murphy, D. J., Devlin, J. W., Smith, S. E., Murray, B., Buckley, M. S., Rowe, S., Coppiano, L. S., & Kamaleswaran, R. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae033 · Cited by 12 · see citing works
Challenges for Responsible AI Design and Workflow Integration in Healthcare: A Case Study of Automatic Feeding Tube Qualification in Radiology
Thieme, A., Rajamohan, A. M., Cooper, B., Groombridge, H., Simister, R. J., Wong, B., Woznitza, N., Pinnock, M. A., Wetscherek, M. T., Morrison, C. P., et al. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3716500 · Cited by 12 · see citing works
Toward Knowledge-Driven Speech-Based Models of Depression: Leveraging Spectrotemporal Variations in Speech Vowels
Feng, K. (2022) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi56158.2022.9926939 · Cited by 12 · see citing works
Implementation and evaluation of sepsis surveillance and decision support in medical ICU and emergency department
Lipatov, K., Daniels, C. E., Park, J. G., Elmer, J., Hanson, A. C., Madsen, B. E., Clements, C. M., Gajic, O., Pickering, B. W., & Herasevich, V. (2022) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2021.09.086 · Cited by 11 · see citing works
Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review
Wan, Y.-K. J., Wright, M. C., McFarland, M. M., Dishman, D., Nies, M. A., Rush, A., Madaras-Kelly, K., Jeppesen, A., & Del Fiol, G. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad203 · Cited by 11 · see citing works
Intensive care nurses’ awareness of identification of early sepsis findings
Birge, A. Ö., Aydın, A. K., & Çamdeviren, E. K. (2021) · Journal of Clinical Nursing · doi:10.1111/jocn.16116 · Cited by 11 · see citing works
Towards an early warning system for monitoring of cancer patients using hybrid interactive machine learning
Trojan, A., Laurenzi, E., Jüngling, S., Roth, S., Kiessling, M., Atassi, Z., Kadvany, Y., Mannhart, M., Jackisch, C., Kullak-Ublick, G., et al. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1443987 · Cited by 10 · see citing works
Early detection of sepsis using machine learning algorithms
El-Aziz, R. M. A., & Rayan, A. (2024) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.10.005 · Cited by 10 · see citing works
Beyond prediction: Off‐target uses of artificial intelligence‐based predictive analytics in a learning health system
Keim‐Malpass, J., Moorman, L. P., Monfredi, O., Clark, M., & Bourque, J. M. (2022) · Learning Health Systems · doi:10.1002/lrh2.10323 · Cited by 10 · see citing works
Desired Characteristics of a Clinical Decision Support System for Early Sepsis Recognition: Interview Study Among Hospital-Based Clinicians
Silvestri, J. A., Kmiec, T. E., Bishop, N. S., Regli, S. H., & Weissman, G. E. (2022) · JMIR Human Factors · doi:10.2196/36976 · Cited by 10 · see citing works
Research frontiers and trends in the application of artificial intelligence to sepsis: A bibliometric analysis
Tang, M., Mu, F., Cui, C., Zhao, J., Lin, R., Sun, K., Guan, Y., & Wang, J. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2022.1043589 · Cited by 10 · see citing works
Sepsis Prediction in Hospitalized Children: Model Development and Validation
Stephen, R. J., Carroll, M. S., Hoge, J., Maciorowski, K., Jones, R. C., Lucey, K. E., O’Connell, M., Schwab, C., Rojas, J., & Sanchez-Pinto, L. N. (2023) · Hospital Pediatrics · doi:10.1542/hpeds.2022-006964 · Cited by 10 · see citing works
Predictors of rural hospital closures in the United States: a systematic review and call for AI-driven early warning systems
Balakrishnan, K., Zhi, L., Hinkle, H. E., Weidenbacher-Hoper, V., Reynolds, C., & Keesee, A. O. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13847-7 · Cited by 9 · see citing works
Artificial Intelligence and Mechanical Circulatory Support
Li, S., Hickey, G., Lander, M. M., & Kanwar, M. K. (2022) · Heart Failure Clinics · doi:10.1016/j.hfc.2021.11.005 · Cited by 9 · see citing works
Sepsis Prediction in Hospitalized Children: Clinical Decision Support Design and Deployment
Stephen, R. J., Lucey, K. E., Carroll, M. S., Hoge, J., Maciorowski, K., Jones, R. C., O’Connell, M., Schwab, C., Rojas, J., & Pinto, L. N. S. (2023) · Hospital Pediatrics · doi:10.1542/hpeds.2023-007218 · Cited by 9 · see citing works
Sepsis and acute respiratory failure in patients with cancer: how can we improve care and outcomes even further?
Lyons, P. G., McEvoy, C. A., & Hayes‐Lattin, B. (2023) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000001078 · Cited by 9 · see citing works
Systematic review of the performance evaluation of clinicians with or without the aid of machine learning clinical decision support system
Nuutinen, M., & Leskelä, R. (2023) · Health and Technology · doi:10.1007/s12553-023-00763-1 · Cited by 8 · see citing works
ShockSurv: A machine learning model to accurately predict 28-day mortality for septic shock patients in the intensive care unit
Zheng, F., Wang, L., Pang, Y., Chen, Z., Lu, Y., Yang, Y., & Wu, J. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105146 · Cited by 8 · see citing works
Sepsis mortality prediction with Machine Learning Tecniques
Pérez-Tome, J. C., Carreño, T. P., Castaño-Fernández, A. B., Soriano, B. J. N., & Castro-Luna, G. M. (2024) · Medicina Intensiva · doi:10.1016/j.medin.2024.04.010 · Cited by 7 · see citing works
Surviving Sepsis Screening: The Unintended Consequences of Continuous Surveillance
Harrison, W. N., Workman, J. K., Bonafide, C. P., & Lockwood, J. M. (2020) · Hospital Pediatrics · doi:10.1542/hpeds.2020-002121 · Cited by 7 · see citing works
Perception Analysis and Early Warning of Home‐Based Care Health Information Based on the Internet of Things
Mao, Y., Zhang, L., & Wu, X. (2021) · Complexity · doi:10.1155/2021/6634575 · Cited by 7 · see citing works
BARRIERS to Early Detection of Deterioration in Hospitalized Infants Using Predictive Analytics
Sullivan, B. A., & Keim‐Malpass, J. (2021) · Hospital Pediatrics · doi:10.1542/hpeds.2020-004382 · Cited by 7 · see citing works
Qualitative Study of the Context of Health Information Technology in Sepsis Care Transitions: Facilitators, Barriers, and Strategies for Improvement
You, S. B., Hirschman, K. B., Stawnychy, M. A., Song, J., Sang, E., Pitcher, K. S., Oh, S., O’Connor, M. L., Garren, P. A., & Bowles, K. H. (2025) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2025.105606 · Cited by 6 · see citing works
Presenting Artificial Intelligence Predictions Based on Electronic Medical Records to Clinicians in Hospitals: A Systematic Review
Noselli, M., van der Vegt, A. H., Cordeil, M., Campbell, V. K., Wang, A. P., Shetty, A., & Scott, I. A. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3572494 · Cited by 6 · see citing works
AI-Guided Decision Support in Acute Cardiac Care: From Chest Pain to STEMI
Tran, H. H.-V., Thu, A., Twayana, A. R., Fuertes, A., Gonzalez, M., Basta, M., Mehta, K. A., James, M., Frishman, W. H., & Aronow, W. S. (2025) · Cardiology in Review · doi:10.1097/crd.0000000000001016 · Cited by 6 · see citing works
AI-Assisted Cardiovascular Risk Assessment by General Practitioners in Resource-Constrained Indonesian Settings Using a Conceptual Prototype: Randomized Controlled Study
Susanto, A. P., Lyell, D., Widyantoro, B., Juzar, D. A., Santoso, A., Berkovsky, S., & Magrabi, F. (2025) · Journal of Medical Internet Research · doi:10.2196/73131 · Cited by 6 · see citing works
Aid of a Machine Learning Algorithm Can Improve Clinician Predictions of Patient Quality of Life During Breast Cancer Treatments
Nuutinen, M., Hiltunen, A.-M., Korhonen, S., Haavisto, I., Poikonen‐Saksela, P., Mattson, J., Manikis, G. C., Kondylakis, H., Simos, P. G., Mazzocco, K., et al. (2022) · Health and Technology · doi:10.1007/s12553-023-00733-7 · Cited by 6 · see citing works

Clinician Perception of the Effectiveness of an Automated Early Warning and Response System for Sepsis in an Academic Medical Center
Guidi, J. L., Clark, K., Upton, M. T., Faust, H., Umscheid, C. A., Lane-Fall, M. B., Mikkelsen, M. E., Schweickert, W. D., Vanzandbergen, C. A., Betesh, J., et al. (2015) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201503-129oc · Cited by 43 · see citing works
Self-citation
Retrospective observational study of the clinical performance characteristics of a machine learning approach to early sepsis identification
Topiwala, R., Patel, K., Twigg, J., Rhule, J., & Meisenberg, B. (2019) · Critical Care Explorations · doi:10.1097/cce.0000000000000046 · Cited by 16 · see citing works
Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis
Adams, R., Henry, K. E., Sridharan, A., Soleimani, H., Zhan, A., Rawat, N., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01894-0 · Cited by 365 · see citing works
A Machine Learning Algorithm to Predict Severe Sepsis and Septic Shock: Development, Implementation, and Impact on Clinical Practice*
Giannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269 · see citing works
Self-citation
Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing
Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01895-z · Cited by 127 · see citing works

Authors: Ginestra, J. C., Giannini, H. M., Schweickert, W. D., Meadows, L., Lynch, M. J., Pavan, K., Chivers, C. J., Draugelis, M., Donnelly, P. J., Fuchs, B. D., & Umscheid, C. A.
Affiliations: Hospital of the University of Pennsylvania (US)
University of Pennsylvania Health System (US)
Penn Presbyterian Medical Center (US)
Pennsylvania Hospital (US)
ORCID: Ginestra, Jennifer C.
Giannini, Heather M.
Schweickert, William D.
Meadows, Laurie
Lynch, Michael J.
Chivers, Corey J.
Draugelis, Michael
Donnelly, Patrick J.
Umscheid, Craig A.
Venue: Critical Care Medicine
Publisher: Ovid Technologies (Wolters Kluwer Health)
Published: 2019-11
Volume: 47
Issue: 11
Pages: 1477-1484
ISSN: 0090-3493 (ISSN-L)
DOI: 10.1097/ccm.0000000000003803
PMC: PMC6791738
OpenAlex: W2945543078
Cited by: 159
References: 36
Citations by year: 2026: 11 · 2025: 28 · 2024: 23 · 2023: 30 · 2022: 29 · 2021: 19 · 2020: 15 · 2019: 4
FWCI: 12.75
Type: journal article
Language: en
Open access: green
www.ncbi.nlm.nih.gov
Locations: Critical Care Medicine (publishedVersion)
PubMed (publishedVersion)
PubMed Central (acceptedVersion)
Funders: National Institutes of Health (UL1TR000003)
National Center for Research Resources (UL1RR024134)
National Center for Advancing Translational Sciences
Topics: Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords: sepsis, septic shock, machine learning, clinical decision support, clinician perceptions, nurses, non-ICU patients, alert response
MeSH terms: Machine Learning, Algorithms, Attitude of Health Personnel, Diagnosis, Computer-Assisted, Hospitals, Teaching, Humans, Medical Staff, Hospital, Nursing Staff, Hospital, Practice Patterns, Physicians', Prospective Studies, Shock, Septic, Sepsis, Decision Support Systems, Clinical, Practice Patterns, Nurses', Electronic Health Records, Text Messaging

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Clinician perception of a machine learning–based early warning system designed to predict severe sepsis and septic shock

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks