Clinician Perception of the Effectiveness of an Automated Early Warning and Response System for Sepsis in an Academic Medical Center
Abstract
Rationale: We implemented an electronic early warning and response system (EWRS) to improve detection of and response to severe sepsis. Sustainability of such a system requires stakeholder acceptance. We hypothesized that clinicians receiving such alerts perceive them to be useful and effective. Objectives: To survey clinicians after EWRS notification about perceptions of the system. Methods: For a 6-week study period 1 month after EWRS implementation in a large tertiary referral medical center, bedside clinicians, including providers (physicians, advanced practice providers) and registered nurses (RNs), were surveyed confidentially within 2 hours of an alert. Measurements and Main Results: For the 247 alerts that triggered, 127 providers (51%) and 105 RNs (43%) completed the survey. Clinicians perceived most patients as stable before and after the alert. Approximately half (39% providers, 48% RNs) felt the alert provided new information, and about half (44% providers, 56% RNs) reported changes in management as a result of the alert, including closer monitoring and additional interventions. Over half (54% providers, 65% RNs) felt the alert was appropriately timed. Approximately one-third found the alert helpful (33% providers, 40% RNs) and fewer felt it improved patient care (24% providers, 35% RNs). Conclusions: A minority of responders perceived the EWRS to be useful, likely related to the perception that most patients identified were stable. However, management was altered half the time after an alert. These results suggest further improvements to the system are needed to enhance clinician perception of the system’s utility.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two previous studies examining predictive alerts for sepsis suggested that perceived utility decreased when the model frequently identified clinically stable patients.
“Similarly, 2 previous studies examining predictive alerts for sepsis suggested that the perceived utility decreased when the model frequently identified clinically stable patients [27,28].”
From Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study (2020) · cites this paper as [28]
The same previous studies found that false positives from other nonsepsis etiologies could increase alarm fatigue.
“They also found that false positives from other nonsepsis etiologies could increase alarm fatigue.”
From Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study (2020) · cites this paper as [28]
Many models have been developed to improve timely identification of sepsis, but their lack of adoption has led to an implementation gap in early warning systems for sepsis.
“Many models have been developed to improve timely identification of sepsis,4,5,6,7,8,9 but their lack of adoption has led to an implementation gap in early warning systems for sepsis.10,11”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [10]
Cited by
Sorted by citations
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
Giannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269 · see citing works
Self-citationIntegrating a Machine Learning System Into Clinical Workflows: Qualitative Study
Sandhu, S., Lin, A. L., Brajer, N., Sperling, J., Ratliff, W., Bedoya, A. D., Balu, S., O'Brien, C., & Sendak, M. P. (2020) · Journal of Medical Internet Research · doi:10.2196/22421 · Cited by 168 · see citing works
Ginestra, J. C., Giannini, H. M., Schweickert, W. D., Meadows, L., Lynch, M. J., Pavan, K., Chivers, C. J., Draugelis, M., Donnelly, P. J., Fuchs, B. D., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003803 · Cited by 159 · see citing works
Self-citationJoshi, M., Mecklai, K., Rozenblum, R., & Samal, L. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac022 · Cited by 46 · see citing works
Topiwala, R., Patel, K., Twigg, J., Rhule, J., & Meisenberg, B. (2019) · Critical Care Explorations · doi:10.1097/cce.0000000000000046 · Cited by 16 · see citing works
- Authors
- Guidi, J. L., Clark, K., Upton, M. T., Faust, H., Umscheid, C. A., Lane-Fall, M. B., Mikkelsen, M. E., Schweickert, W. D., Vanzandbergen, C. A., Betesh, J., Tait, G., Hanish, A., Smith, K., Feeley, D., & Fuchs, B. D.
- Affiliations
- Pediatrics and Genetics (US)
- University of Pennsylvania Health System (US)
- University of Pennsylvania (US)
- Information Services and
- Corresponding author
- Fuchs, Barry D.
- ORCID
- Venue
- Annals of the American Thoracic Society
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-10-01
- Volume
- 12
- Issue
- 10
- Pages
- 1514-1519
- ISSN
- 2325-6621 (ISSN-L)
- 2329-6933
- OpenAlex
- W1921056797
- Cited by
- 43
- References
- 0
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 3 · 2023: 4 · 2022: 7 · 2021: 4 · 2020: 4 · 2019: 8 · 2018: 2 · 2017: 1 · 2016: 2 · 2015: 2
- FWCI
- 1.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of the American Thoracic Society (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- sepsis, clinician perceptions, stakeholder acceptance, electronic health records, patient monitoring, academic medical centers
- MeSH terms
- Academic Medical Centers, Attitude of Health Personnel, Patient Care, Humans, Prospective Studies, Surveys and Questionnaires, Sepsis, Early Diagnosis, Medical Order Entry Systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review