The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Automated Identification of Adults at Risk for In-Hospital Clinical Deterioration
Open accessAbstract
BACKGROUND: Hospitalized adults whose condition deteriorates while they are in wards (outside the intensive care unit [ICU]) have considerable morbidity and mortality. Early identification of patients at risk for clinical deterioration has relied on manually calculated scores. Outcomes after an automated detection of impending clinical deterioration have not been widely reported. METHODS: On the basis of a validated model that uses information from electronic medical records to identify hospitalized patients at high risk for clinical deterioration (which permits automated, real-time risk-score calculation), we developed an intervention program involving remote monitoring by nurses who reviewed records of patients who had been identified as being at high risk; results of this monitoring were then communicated to rapid-response teams at hospitals. We compared outcomes (including the primary outcome, mortality within 30 days after an alert) among hospitalized patients (excluding those in the ICU) whose condition reached the alert threshold at hospitals where the system was operational (intervention sites, where alerts led to a clinical response) with outcomes among patients at hospitals where the system had not yet been deployed (comparison sites, where a patient's condition would have triggered a clinical response after an alert had the system been operational). Multivariate analyses adjusted for demographic characteristics, severity of illness, and burden of coexisting conditions. RESULTS: The program was deployed in a staggered fashion at 19 hospitals between August 1, 2016, and February 28, 2019. We identified 548,838 non-ICU hospitalizations involving 326,816 patients. A total of 43,949 hospitalizations (involving 35,669 patients) involved a patient whose condition reached the alert threshold; 15,487 hospitalizations were included in the intervention cohort, and 28,462 hospitalizations in the comparison cohort. Mortality within 30 days after an alert was lower in the intervention cohort than in the comparison cohort (adjusted relative risk, 0.84, 95% confidence interval, 0.78 to 0.90; P<0.001). CONCLUSIONS: The use of an automated predictive model to identify high-risk patients for whom interventions by rapid-response teams could be implemented was associated with decreased mortality. (Funded by the Gordon and Betty Moore Foundation and others.).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Translation of these approaches in clinical intervention studies has improved outcomes [16, 34, 35].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [34]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Pragmatic randomized trials and quasi-experimental designs offer advantages over pre-post designs or comparisons against historical controls [34, 52].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [34]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Other EWSs have influenced in-hospital mortality, sepsis or LOS, but not across the breadth of outcomes we observed4,5,26,27.”
From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, ease of use and integration into the workflow have been noted as important factors influencing adoption 26,28,47,48 .”
From Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing (2022) · cites this paper as [26]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While there are other existing multifactorial machine learning clinical decision support implementations that share similarities to ours27–29, the CONCERN intervention is distinct in its modeling approach by using metadata of nursing documentation as early predictors of patient deterioration in acute and intensive care settings, being fully integrated as a real-time EWS into the EHR and undergoing”
From Real-time surveillance system for patient deterioration: a pragmatic cluster-randomized controlled trial (2025) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
DEYO, R. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90133-8 · Cited by 10,452
Hosmer, D. W., Lemeshow, S. A., & May, S. J. (2008) · Wiley series in probability and statistics · doi:10.1002/9780470258019 · Cited by 3,851
Estimating the Relative Risk in Cohort Studies and Clinical Trials of Common Outcomes
McNutt, L. (2003) · American Journal of Epidemiology · doi:10.1093/aje/kwg074 · Cited by 2,098
Practical recommendations for reporting F ine‐ G ray model analyses for competing risk data
Austin, P. C., & Fine, J. P. (2017) · Statistics in Medicine · doi:10.1002/sim.7501 · Cited by 1,299
Design and analysis of stepped wedge cluster randomized trials.
Hussey, M. A., & Hughes, J. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.05.007 · Cited by 1,297
Smith, G. B., Prytherch, D. R., Meredith, P., Schmidt, P. E., & Featherstone, P. I. (2013) · Resuscitation · doi:10.1016/j.resuscitation.2012.12.016 · Cited by 1,077
Review of statistical methods for analysing healthcare resources and costs
Mihaylova, B., Briggs, A., O'Hagan, A., & Thompson, S. G. (2011) · Health Economics · doi:10.1002/hec.1653 · Cited by 767
Multilevel modelling of medical data
Goldstein, H., Browne, W. J., & Rasbash, J. (2002) · Statistics in Medicine · doi:10.1002/sim.1264 · Cited by 394
Priestley, G., Watson, W., Rashidian, A., Mozley, C. G., Russell, D., Wilson, J., Cope, J., Hart, D., Kay, D., Cowley, K., et al. (2004) · Intensive Care Medicine · doi:10.1007/s00134-004-2268-7 · Cited by 374
Flagged first, then by citations
Research works printed but not linked: 5
- Escobar GJ, Liu V, Kim YS, et al. Early detection of impending deterioration outside the ICU: a difference-in-differences (DiD) study. Am J Respir Crit Care Med 2016;93:A7614-A7614. abstract.
- Harrell F Jr. Regression modeling strategies with applications to linear models logistic regression and survival analysis. New York: Springer-Verlag 2010.
- Kalbfleisch JD Prentice RL. The statistical analysis of failure time data. 2nd ed. New York: John Wiley 2011.
- Harrell FE Jr. Semiparametric modeling of health care cost and resource utilization. Presented at the 24th Annual Midwest Biopharmaceutical Statistics Workshop Muncie IN May 21–23 2001.
- Collett D. Modelling survival data in medical research. 3rd ed. Boca Raton FL: Chapman and Hall/CRC Press 2014.
Cited by
Sorted by citations
Items 301–315 of 315
Chen, W., & Zhang, L. (2021) · Journal of Health Informatics and Digital Systems · doi:10.68159/q702272227
Blenis, R. C. (2026) · Journal of Advanced Nursing · doi:10.1111/jan.70773
A decision analytics framework for critical care escalation using electronic health records
Deif, M. A., Shwedeh, F., Hafez, M., & Khishe, M. (2026) · Decision Analytics Journal · doi:10.1016/j.dajour.2026.100751
Sheeba, R., Murugananandan, N., Tamilselvan, Mandal, T., Israel Jeba Prab, D., Jamunarani, P., Ilakkiya, N., Padmapriya, N., Nithin Manly, S., Golda Sahaya Rani, R., et al. (2026) · International Journal of Research in Medical Sciences · doi:10.18203/2320-6012.ijrms20263573
Enhancing Nurses’ Capability to Promptly Escalate Care Using the Early Warning Score
Er, J. B. T., Ang, E. N. K., Yue, L. T., & Thornlow, D. K. (2026) · Journal of Nursing Care Quality · doi:10.1097/ncq.0000000000000995
Equity-Aware Risk Prioritization for Mandatory Referral in Nursing-Home Telemedicine
Ávila-Jiménez, J. L., Reigal, N. L., Rich‐Ruiz, M., Cantón-Habas, V., & Sebastián, V. (2026) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms69103.2026.00162
do Nascimento, C. E., Reigal, N. L., Sánchez-Jiménez, F. J., Rich‐Ruiz, M., Ávila-Jiménez, J. L., & Cantón-Habas, V. (2026) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms69103.2026.00163
Yoo, M. S., Plimier, C., Kipnis, P., Escobar, G. J., Lydon, C. M., Schlessinger, D., Greene, J., Lawson, B. L., Brown, H. E., Hatton, J. R., et al. (2026) · Journal of Hospital Medicine · doi:10.1002/jhm.70500
Self-citationClinical heterogeneity in performance of a commercial prediction model for clinical deterioration
Taylor, S. L., Williams, C., Fan, S., Cortés‐Puch, I., Fazio, S., Liu, A., Callcut, R., Hall, B., & Adams, J. Y. (2026) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2026-102284
SOBRAL, V. B. L., Gonçalves, R. R., de Oliva, P. L. D., da Silveira, A. L. M., Sousa, J. D. M., da Silva, J. R., Pinheiro, P. D. V., dos Santos, E. B., Mendonça, M. D. B., Fonseca, I. D. S., et al. (2026) · Revista Ibero-Americana de Humanidades, Ciências e Educação · doi:10.51891/rease.v12i10.31167
Ko, D., Tachinardi, U., & D, B. M. K. M. (2026) · Productivity Press eBooks · doi:10.4324/9781003747628
Kim, M., Kim, K. H., KIM, M., Kwon, S. J., Park, M. S., Kim, A. J., Lee, M., Park, M. H., Cha, H. K., Lee, M., et al. (2026) · JMIR Research Protocols · doi:10.2196/104230
Clinical AI governance for scalable innovation in value based healthcare systems
Nguyen, D., Davidoff, R., Lee, S., Lalich, M., Tamaddon, K., Caivano, R., Loo, D., Anand, S., Tseng, W., Campos, M., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03409-4
Numeric versus embedding pipelines for optimized clinical risk prediction
Ge, A., Wu, J., Wu, X., Balasubramanian, J. B., Nautiyal, V., Jayakumar, R., Murali, C., Lee, A., Nguyen, A., Allen, M., et al. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag199
Choi, J., Marwaha, J. S., & Collins, P. G. S. (2026) · BMJ Digital Health & AI · doi:10.1136/bmjdh-2026-000219
Metadata
- Authors
- Escobar, G. J., Liu, V. X., Schuler, A., Lawson, B., Greene, J. D., & Kipnis, P.
- Affiliations
- Kaiser Permanente (US)
- ORCID
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2020-11-12
- Volume
- 383
- Issue
- 20
- Pages
- 1951-1960
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- PMC
- PMC7787261
- OpenAlex
- W3106297998
- Cited by
- 314
- References
- 39
- Citations by year
- 2026: 52 · 2025: 63 · 2024: 51 · 2023: 47 · 2022: 56 · 2021: 42 · 2020: 3
- FWCI
- 33.81
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Sidney Garfield Memorial Fund
- Gordon and Betty Moore Foundation (2663, 2663.01)
- The Permanente Medical Group, Inc.
- Kaiser Foundation Hospitals, Inc.
- National Institutes of Health (K23GM112018, R35GM128672)
- Agency for Healthcare Research and Quality (1R01HS018480-01)
- Topics
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Keywords
- clinical deterioration, early warning systems, rapid response teams, remote patient monitoring, 30-day mortality, electronic health records, hospitalized adults
- MeSH terms
- Alert Fatigue, Health Personnel, Laboratory Critical Values, Clinical Deterioration, Adult, Aged, Automation, Female, Hospitalization, Humans, Length of Stay, Male, Middle Aged, Models, Theoretical, Nursing Staff, Hospital, Patient Readmission, Telemetry, Multivariate Analysis, Hospital Mortality, Risk Assessment, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
