Detecting Deteriorating Patients in the Hospital: Development and Validation of a Novel Scoring System
Open accessAbstract
Rationale: Late recognition of patient deterioration in hospital is associated with worse outcomes, including higher mortality. Despite the widespread introduction of early warning score (EWS) systems and electronic health records, deterioration still goes unrecognized. Objectives: To develop and externally validate a Hospital- wide Alerting via Electronic Noticeboard (HAVEN) system to identify hospitalized patients at risk of reversible deterioration. Methods: This was a retrospective cohort study of patients 16 years of age or above admitted to four UK hospitals. The primary outcome was cardiac arrest or unplanned admission to the ICU. We used patient data (vital signs, laboratory tests, comorbidities, and frailty) from one hospital to train a machine-learning model (gradient boosting trees). We internally and externally validated the model and compared its performance with existing scoring systems (including the National EWS, laboratory-based acute physiology score, and electronic cardiac arrest risk triage score). Measurements and Main Results: We developed the HAVEN model using 230,415 patient admissions to a single hospital. We validated HAVEN on 266,295 admissions to four hospitals. HAVEN showed substantially higher discrimination (c-statistic, 0.901 [95% confidence interval, 0.898-0.903]) for the primary outcome within 24 hours of each measurement than other published scoring systems (which range from 0.700 [0.696-0.704] to 0.863 [0.860-0.865]). With a precision of 10…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Models using readily available EHR data can accurately predict clinical deterioration and sepsis hours before they occur.
“Two prominent examples relevant to critical care are models using readily available EHR data that can accurately predict clinical deterioration and sepsis hours before they occur [28–33]; these models exemplify real-time CDSS that alert clinicians and prompt evaluation, testing, and interventions [33].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [28]
Cites
Comorbidity measures for use with administrative data.
Elixhauser, A., Steiner, C., Harris, D. R., & Coffey, R. M. (1998) · Medical Care · doi:10.1097/00005650-199801000-00004 · Cited by 9,921 · see citing works
Sterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., Wood, A. M., & Carpenter, J. R. (2009) · BMJ · doi:10.1136/bmj.b2393 · Cited by 7,299 · see citing works
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309 · see citing works
The practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246 · see citing works
A clinically applicable approach to continuous prediction of future acute kidney injury
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1390-1 · Cited by 1,086 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · European Urology · doi:10.1016/j.eururo.2014.11.025 · Cited by 439 · see citing works
Solomon, R. S., Corwin, G. S., Barclay, D. C., Quddusi, S. F., & Dannenberg, M. D. (2016) · Journal of Hospital Medicine · doi:10.1002/jhm.2554 · Cited by 225 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Use of artificial intelligence in critical care: opportunities and obstacles
Pinsky, M. R., Bedoya, A., Bihorac, A., Celi, L., Churpek, M., Economou-Zavlanos, N. J., Elbers, P., Saria, S., Liu, V., Lyons, P. G., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04860-z · Cited by 160 · see citing works
“…prominent examples relevant to critical care are models using readily available EHR data that can accurately predict clinical deterioration and sepsis hours before they occur [28–33]; these models exemplify real-time CDSS that alert clinicians and prompt evaluation, testing, and interventions [33].”
MethodEquitably Allocating Resources during Crises: Racial Differences in Mortality Prediction Models
Ashana, D. C., Anesi, G. L., Liu, V. X., Escobar, G. J., Chesley, C., Eneanya, N. D., Weissman, G. E., Miller, W. D., Harhay, M. O., & Halpern, S. D. (2021) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202012-4383oc · Cited by 98 · see citing works
- Authors
- Pimentel, M. A. F., Redfern, O. C., Malycha, J., Meredith, P., Prytherch, D., Briggs, J., Young, J. D., Clifton, D. A., Tarassenko, L., & Watkinson, P. J.
- Affiliations
- Institute of Biomedical Engineering (Japan) (JP)
- University of Oxford (GB)
- National Health Service (GB)
- University of Portsmouth (GB)
- Corresponding author
- Redfern, Oliver C.
- Venue
- American Journal of Respiratory and Critical Care Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-07-01
- Volume
- 204
- Issue
- 1
- Pages
- 44-52
- ISSN
- 1073-449X (ISSN-L)
- 1535-4970
- PMC
- PMC8437126
- OpenAlex
- W3128049448
- Cited by
- 84
- References
- 7
- Citations by year
- 2026: 11 · 2025: 17 · 2024: 11 · 2023: 16 · 2022: 17 · 2021: 12
- FWCI
- 4.95
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- American Journal of Respiratory and Critical Care Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Health Innovation Challenge Fund (HICF-R9-524, WT-103703/Z/14/Z)
- UK Department of Health and Social Care
- Wellcome Trust
- National Institute for Health Research Biomedical Research Centre, Oxford
- Drayson Research fellowships
- UK Department of Health and Social Care or the Wellcome Trust
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Medicine › Emergency Medicine › Cardiac Arrest and Resuscitation
- Keywords
- early warning scores, cardiac arrest prediction, unplanned ICU admission, machine learning, gradient boosting decision tree, National Early Warning Score, hospitalized patients
- MeSH terms
- Machine Learning, Clinical Deterioration, Early Warning Score, Adolescent, Adult, Aged, Aged, 80 and over, Algorithms, Female, United Kingdom, Humans, Male, Middle Aged, Retrospective Studies, Risk Factors, Reproducibility of Results, Cohort Studies, Guidelines as Topic, Risk Assessment, Young Adult, Vital Signs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
