Effectiveness of an Artificial Intelligence–Enabled Intervention for Detecting Clinical Deterioration
Open accessAbstract
Importance: Inpatient clinical deterioration is associated with substantial morbidity and mortality but may be easily missed by clinicians. Early warning scores have been developed to alert clinicians to patients at high risk of clinical deterioration, but there is limited evidence for their effectiveness. Objective: To evaluate the effectiveness of an artificial intelligence deterioration model–enabled intervention to reduce the risk of escalations in care among hospitalized patients using a study design that facilitates stronger causal inference. Design, Setting, and Participants: This cohort study used a regression discontinuity design that controlled for confounding and was based on Epic Deterioration Index (EDI; Epic Systems Corporation) prediction model scores. Compared with other observational research, the regression discontinuity design facilitates causal analysis. Hospitalized adults were included from 4 general internal medicine units in 1 academic hospital from January 17, 2021, through November 16, 2022. Exposure: An artificial intelligence deterioration model–enabled intervention, consisting of alerts based on an EDI score threshold with an associated collaborative workflow among nurses and physicians. Main Outcomes and Measures: The primary outcome was escalations in care, including rapid response team activation, transfer to the intensive care unit, or cardiopulmonary arrest during hospitalization. Results: During the study, 9938 patients were admitted to 1 of the 4 units, with 963 patients (median [IQR] age, 76.1 [64.2-86.2] years; 498 males [52.3%]) included within the primary regression discontinuity analysis. The median (IQR) Elixhauser Comorbidity Index score in the primary analysis cohort was 10 (0-24). The intervention was associated with a −10.4–percentage point (95% CI, −20.1 to −0.8 percentage points; P = .03) absolute risk reduction in the primary outcome for patients at the EDI score threshold. There was no evidence of a discontinuity in measured confounders at the EDI score threshold. Conclusions and Relevance: Using a regression discontinuity design, this cohort study found that the implementation of an artificial intelligence deterioration model–enabled intervention was associated with a significantly decreased risk of escalations in care among inpatients. These results provide evidence for the effectiveness of this intervention and support its further expansion and testing in other care settings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI enhances decision-making capabilities by providing real-time data analysis and early warning scores.
“On the positive side, AI enhances decision-making capabilities by providing real-time data analysis and early warning scores.”
From Leading with AI in critical care nursing: challenges, opportunities, and the human factor (2024) · cites this paper as Gallo et al., (2024) [22]
AI has the potential to improve patient outcomes through early intervention and reduce the burden of routine tasks on healthcare professionals.
“This aligns with findings by Gallo et al., (2024) [22], which underscore AI’s potential to improve patient outcomes through early intervention and reduce the burden of routine tasks on healthcare professionals.”
From Leading with AI in critical care nursing: challenges, opportunities, and the human factor (2024) · cites this paper as Gallo et al., (2024) [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data
Quan, H., Sundararajan, V., Halfon, P., Fong, A., Burnand, B., Luthi, J.-C., Saunders, L. D., Beck, C. A., Feasby, T. E., & Ghali, W. A. (2005) · Medical Care · doi:10.1097/01.mlr.0000182534.19832.83 · Cited by 10,669 · see citing works
Robust nonparametric confidence intervals for regression-discontinuity designs
Calonico, S., Cattaneo, M. D., & Titiunik, R. (2014) · Econometrica · doi:10.3982/ecta11757 · Cited by 3,036 · see citing works
Automated Identification of Adults at Risk for In-Hospital Clinical Deterioration
Escobar, G. J., Liu, V. X., Schuler, A., Lawson, B., Greene, J. D., & Kipnis, P. (2020) · New England Journal of Medicine · doi:10.1056/nejmsa2001090 · Cited by 314 · see citing works
Moscoe, E., Bor, J. H., & Bärnighausen, T. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.06.021 · Cited by 233 · see citing works
Escobar, G. J., Greene, J., Gardner, M. N., Marelich, G. P., Quick, B., & Kipnis, P. (2010) · Journal of Hospital Medicine · doi:10.1002/jhm.817 · Cited by 113 · see citing works
Kollef, M. H., Chen, Y., Heard, K. M., LaRossa, G. N., Lu, C., Martin, N. R., Martin, N., Micek, S. T., & Bailey, T. C. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2193 · Cited by 91 · see citing works
Regression Discontinuity Design
Maciejewski, M. L., & Basu, A. (2020) · JAMA · doi:10.1001/jama.2020.3822 · Cited by 55 · see citing works
Association of Intravenous Radiocontrast With Kidney Function
Goulden, R., Rowe, B. H., Abrahamowicz, M., Strumpf, E., & Tamblyn, R. M. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.0916 · Cited by 49 · see citing works
Li, R., Smith, M. A., Lu, J., Avati, A., Wang, S. X. Y., Teuteberg, W., Shum, K., Hong, G., Seevaratnam, B., Westphal, J., et al. (2022) · NEJM Catalyst · doi:10.1056/cat.21.0457 · Cited by 37 · see citing works
Self-citationBosch, N. A., Law, A. C., Bor, J. H., Myers, L. C., Roubinian, N. H., Liu, V. X., & Walkey, A. J. (2022) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202109-1078oc · Cited by 22 · see citing works
Algorithms for Prediction of Clinical Deterioration on the General Wards: A Scoping Review
Peelen, R. V., Eddahchouri, Y., Koeneman, M., van de Belt, T. H., van Goor, H., & Bredie, S. J. H. (2021) · Journal of Hospital Medicine · doi:10.12788/jhm.3630 · Cited by 20 · see citing works
Advancing Quality Improvement with Regression Discontinuity Designs
Walkey, A. J., Drainoni, M., Cordella, N., & Bor, J. H. (2018) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201712-942ip · Cited by 19 · see citing works
Marafino, B. J., Escobar, G. J., Baiocchi, M., Liu, V. X., Plimier, C., & Schuler, A. (2021) · BMJ · doi:10.1136/bmj.n1747 · Cited by 15 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Early Warning Scores With and Without Artificial Intelligence
Edelson, D. P., Churpek, M. M., Carey, K. A., Lin, Z., Huang, C., Siner, J. M., Johnson, J. K., Krumholz, H. M., & Rhodes, D. J. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.38986 · Cited by 99 · see citing works
Arab, R. A. E., Moosa, O. A. A., Sagbakken, M., Ghannam, A., Abuadas, F. H. A., Somerville, J. G., & Mutair, A. S. A. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1619378 · Cited by 119 · see citing works
Leading with AI in critical care nursing: challenges, opportunities, and the human factor
Hassan, E. A., & El-Ashry, A. M. (2024) · BMC Nursing · doi:10.1186/s12912-024-02363-4 · Cited by 72 · see citing works
Morey, D. A., Rayo, M. F., & Woods, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01784-y · Cited by 17 · see citing works
Ruksakulpiwat, S., Phianhasin, L., Benjasirisan, C., Su, T., Riangkam, C., Thorngthip, S., Aldossary, H. M., Ahmed, B. H., Pongsuwun, K., & Angkaew, N. (2025) · International Nursing Review · doi:10.1111/inr.70080 · Cited by 30 · see citing works
Reuben, J. S., Meiri, H., & Arien‐Zakay, H. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1458811 · Cited by 23 · see citing works
Dai, Q., Li, M., Shi, S., Yang, M., Wang, Z., Liao, J., Li, Z., Liu, J., Deng, J. J., & Tao, L. (2025) · BMC Nursing · doi:10.1186/s12912-025-03748-9 · Cited by 25 · see citing works
Zhang, C., Yang, H., Liu, X., Wu, R., Zong, H., Wu, E., Zhou, Y. K., Li, J., & Shen, B. (2025) · Journal of Medical Internet Research · doi:10.2196/67201 · Cited by 15 · see citing works
Phillips, E., O’Donoghue, O., Zhang, Y., Tsimpos, P., Mallinger, L. A., Chatzidakis, S., Pohlmann, J. E., Du, Y., Kim, I. S. Y., Song, J. J., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01687-y · Cited by 8 · see citing works
Underrepresentation of children in public medical imaging datasets
Hua, S. B. Z., Heller, N., He, P., Towbin, A., Chen, I. Y., Lu, A. X., & Erdman, L. (2026) · Nature Health · doi:10.1038/s44360-026-00111-3 · Cited by 7 · see citing works
Application of Generative Artificial Intelligence in Dyslipidemia Care
Ahn, J., & Kim, B.-K. (2025) · Journal of Lipid and Atherosclerosis · doi:10.12997/jla.2025.14.1.77 · Cited by 6 · see citing works
Artificial intelligence-assisted nursing care: a concept analysis using Walker and Avant approach
Maleki, R. N., Shahbazi, S., Hoseinzadeh, M., Ghafourifard, M., Gougjehyaran, H. G., & Faravan, A. (2025) · BMC Nursing · doi:10.1186/s12912-025-03818-y · Cited by 12 · see citing works
Chen, L., Sun, Y., Kuek, S. Y., Chew, M. F., Tan, H. L., & Ho, S. (2025) · Resuscitation Plus · doi:10.1016/j.resplu.2025.101120 · Cited by 6 · see citing works
Shashikumar, S. P., Le, J. P., Yung, N., Ford, J., Singh, K., Malhotra, A., Nemati, S., & Wardi, G. (2024) · Critical Care Explorations · doi:10.1097/cce.0000000000001151 · Cited by 7 · see citing works
Integrating Artificial Intelligence Into Critical Care Nursing: Next Steps
Goforth, C. W., & Alderden, J. (2025) · Critical Care Nurse · doi:10.4037/ccn2025194 · Cited by 5 · see citing works
Integration of Artificial Intelligence in Radiation Oncology: A Narrative Review
Penberthy, D. R., & Mahase, S. S. (2025) · AI in Precision Oncology · doi:10.1089/aipo.2025.0005 · Cited by 3 · see citing works
Current Perspectives on the Artificial Intelligence in Critical Care Medicine
Lee, J., & Yoon, J. H. (2025) · Anesthesiology Clinics · doi:10.1016/j.anclin.2025.05.005 · Cited by 4 · see citing works
Top Ten Tips Palliative Care Clinicians Should Know About the Future of Artificial Intelligence
Patel, P., Patel, R., Kuperman, G. J., Consolandi, M., Collett, D., Nakagawa, S., Burry, N., Bezak, K. B., Strand, J., Kamdar, M. M., et al. (2025) · Journal of Palliative Medicine · doi:10.1177/10966218251366117 · Cited by 2 · see citing works
Slambrouck, L. V., Long, E., Kleunen, L. V., Vens, C., & Corte, W. D. (2026) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2026.104419 · Cited by 2 · see citing works
Adu-Gyamfi, E., Robins, R., RYAN, S., & VILLARENTE, F. (2024) · Nursing Research · doi:10.1097/nnr.0000000000000778 · Cited by 2 · see citing works
Chua, W. L., Azimirad, M., Chia, M., Jones, D. A., & Woon, P. (2025) · Critical Care Medicine · doi:10.1097/ccm.0000000000006960 · Cited by 1 · see citing works
Gebremeskel, T. G., & Romeo, F. (2025) · Health Science Reports · doi:10.1002/hsr2.71536 · Cited by 1 · see citing works
Readiness Assessment for AI in Nursing Care Projects: Multimethods Study
Seibert, K., Domhoff, D., Altona, J., Jäger, S., Bießmann, F., Nowak, A., Gubser, R., Schulte‐Althoff, M., Fürstenau, D., Pohle, J., et al. (2026) · JMIR Nursing · doi:10.2196/84148 · Cited by 1 · see citing works
Wang, J., Keyes, T., Liang, A. S., Ma, S. P., Shen, J., Liu, J., Ambers, N., Pandya, A., Pandya, R., Hom, J., et al. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2026.29579 · Cited by 1 · see citing works
Artificial Intelligence/Machine Learning in Palliative Care #492
Luonuansuu, T., Christensen, A. R., & Hutchinson, S. Z. (2024) · Journal of Palliative Medicine · doi:10.1089/jpm.2024.0401 · Cited by 1 · see citing works
Article Changed to Open Access
(2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.3693 · Cited by 1 · see citing works
Phillips, E., O’Donoghue, O., Zhang, Y., Tsimpos, P., Mallinger, L. A., Chatzidakis, S., Pohlmann, J. E., Du, Y., Kim, I. S. Y., Song, J. J., et al. (2024) · medRxiv · doi:10.1101/2024.11.13.24317229 · Cited by 1 · see citing works
Transforming rapid response team through artificial intelligence
Lee, K. (2025) · Acute and Critical Care · doi:10.4266/acc.000425 · Cited by 1 · see citing works
Clinical Evaluations of Early Warning Scores—Reply
Gallo, R. J., Geldsetzer, P., & Li, R. (2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.3053
Self-citationClinical Evaluations of Early Warning Scores
Goulden, R. (2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.3050
KI schützt vor klinischer Verschlechterung
Krome, S. (2024) · DMW - Deutsche Medizinische Wochenschrift · doi:10.1055/a-2329-7807
AI-Driven Data Engineering: Improving Patient Outcomes and Reducing Costs
Pathak, A. (2025) · European Journal of Computer Science and Information Technology · doi:10.37745/ejcsit.2013/vol13n182434
Use of Artificial Intelligence in Perioperative Care
Pal, A. D. (2025) · Journal of PeriAnesthesia Nursing · doi:10.1016/j.jopan.2025.06.002
Byrd, T. F., Mattson, M., Polt, M., Pint, K., Dimitrov, K., Willis, A., Lister, J., Beacom, E., & Tignanelli, C. (2025) · Applied Clinical Informatics · doi:10.1055/a-2648-4817 · Cited by 1 · see citing works
Readiness assessment for AI in nursing care projects (AINCRA): a multimethods study (Preprint)Preprint
Seibert, K., Domhoff, D., Altona, J., Jäger, S., Bießmann, F., Nowak, A., Gubser, R., Schulte‐Althoff, M., Fürstenau, D., Pohle, J., et al. (2025) · doi:10.2196/preprints.84148
The Future of Interprofessional Collaboration
Prentice, D. (2025) · IGI Global eBooks · doi:10.4018/979-8-3693-2230-7.ch013
Leung, C. K., Cheng, L., Walton, N. C., Nortjé, N., Adelson, K. B., & Hui, D. (2025) · Journal of Intensive Care Medicine · doi:10.1177/08850666251386399
Meka, P., & Stiller, E. (2025) · medRxiv · doi:10.1101/2025.10.23.25338637
Association of Adverse Events With Rapid Response Team Activations in Hospitalized Children
Rickey, L., Pimentel, M., Zakar, K., McBride, S. C., D'Ambrosi, G. R., & Landrigan, C. P. (2025) · Hospital Pediatrics · doi:10.1542/hpeds.2025-008488
Jose, A., Kaithathara, S. T., Mohan, R., Bhat, N., & Tapase, P. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-1454-9.ch005
Meka, P., & Stiller, E. (2026) · Next research. · doi:10.1016/j.nexres.2026.101654
Sneha, S., Awofisayo, R. M., Nino, V., Rischie, I., & Kumah, L. I. (2026) · JMIR AI · doi:10.2196/preprints.98274
Lin, C. Q., Ruiz, M., McLellan, M., & Gardner, M. J. (2026) · Journal of Orthopaedic Trauma · doi:10.1097/bot.0000000000003212
AI is making clinical reasoning optional—and that should worry us
Nghia, N. P. (2026) · BMJ · doi:10.1136/bmj.s871 · Cited by 1 · see citing works
Wang, Y., Biyun, X. I. A., Zhu, Q., Chai, F., Cao, M., & Ji, Y. (2026) · Nursing in Critical Care · doi:10.1111/nicc.70468
Daniel, C., Kalaydjian, A., Machnik, J. V., Ishaq, F., Malveaux, J. A., Homer, S., Griffin, L., Pletcher, S., & Nguyen, N.-A. (2026) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/j.jcjq.2026.07.006
Implementation of an AI-Triggered Rapid Response — Association with Mortality
NAHASS, T. A., Hanna, J. S., Liu, N., Martinez, N., Halm, E. A., Rockman, A., Roy, J., Gilligan, C. J., Murtha, A. P., Gracias, V. H., et al. (2026) · NEJM AI · doi:10.1056/aioa2500973
Almagharbeh, W. T., Alkubati, S. A., Alasmari, A. A., Alharbi, A. A., Alfanash, H. A., Abuadas, F. H. A., Arab, R. A. E., Alkhawaldeh, A., Alamri, A. A., Albalawi, H. M. H., et al. (2026) · Nursing in Critical Care · doi:10.1111/nicc.70606
Medical devices and diagnostics: enhancing patient care and outcomes
Chaudhri, B. B. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40572-3.00003-4
Wei, T., Guo, M., Yi, J., & Yang, Z. (2026) · Acta Biomaterialia · doi:10.1016/j.actbio.2026.09.021
Jang, S., & Chen, J. (2026) · Journal of the American Geriatrics Society · doi:10.1111/jgs.70704
Health Care in Crisis: Prospects and Directions
Amalberti, R., Wiig, S., & Vincent, C. A. (2026) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-032-29343-5_8
Makarewicz, A., Recław, R., Grzywacz, A., Chmielowiec, K., Jaworski, Ł. M., Kuczak-Wójtowicz, M., & Chmielowiec, J. (2026) · Brain Sciences · doi:10.3390/brainsci16101043
Vaishnav, V. (2026) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-026-03202-7
Kosmidis, D., Vlachopoulos, G., Nashwan, A. J., de Carvalho, R. L. R., de Araújo, B. S., Armeni, D., Sertaridou, E. N., & Koutsouki, S. (2026) · Nursing and Health Sciences · doi:10.1111/nhs.70431
Cui, X., Zhang, H., Wang, B., Luo, W., Guo, D., & Liu, X. (2026) · Anaesthesia and Intensive Care · doi:10.1177/0310057x261494610
Cleofas, J. V. (2026) · Journal of Advanced Nursing · doi:10.1111/jan.70780
- Authors
- Gallo, R. J., Shieh, L., Smith, M., Marafino, B. J., Geldsetzer, P., Asch, S. M., Shum, K., Lin, S., Westphal, J., Hong, G., & Li, R. C.
- Affiliations
- VA Palo Alto Health Care System (US)
- Stanford Medicine (US)
- Stanford Health Care (US)
- Kaiser Permanente (US)
- Chan Zuckerberg Biohub San Francisco (US)
- Corresponding author
- Gallo, Robert J.
- ORCID
- Venue
- JAMA Internal Medicine
- Publisher
- American Medical Association (AMA)
- Published
- 2024-05-01
- Volume
- 184
- Issue
- 5
- Pages
- 557
- ISSN
- 2168-6106 (ISSN-L)
- PMC
- PMC10964159
- OpenAlex
- W4393144098
- Cited by
- 58
- References
- 13
- Citations by year
- 2026: 23 · 2025: 24 · 2024: 11
- FWCI
- 24.31
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- cc-by
- Locations
- JAMA Internal Medicine (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Emergency Medicine › Hospital Admissions and Outcomes
- Keywords
- clinical deterioration, artificial intelligence, early warning scores, regression discontinuity design, treatment escalation, rapid response team activation, ICU transfer, cardiac arrest, hospitalized patients, general internal medicine, clinical decision support alerts, nurse-physician collaboration
- MeSH terms
- Clinical Deterioration, Early Warning Score, Aged, Artificial Intelligence, Female, Hospitalization, Humans, Intensive Care Units, Male, Middle Aged, Cohort Studies, Hospital Rapid Response Team
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
