Artificial Intelligence-powered tiered early warning framework addressing high false alarm rates for in-hospital mortality prediction
Open accessAbstract
Alert fatigue remains a major barrier to the effective deployment of predictive models in emergency care, particularly in the context of rare but critical outcomes such as in-hospital mortality (IHM), which often occurs in less than 5.0% of patients admitted from the emergency department (ED). Severe class imbalance leads to low positive predictive value (PPV), undermining the clinical utility of even high-performance predictive models. To address this issue, we propose AI-TEW (Artificial Intelligence-powered Tiered Early Warning), a novel two-stage early warning framework designed to reduce false alarms and improve clinical interpretability. In Stage 1, a robust machine learning model was developed and validated using data from 174,292 ED visits across three hospitals in China and the United States. The model demonstrated strong discriminative ability for IHM prediction, achieving AUROCs ranging from 0.84 (95% CI, 0.81-0.86) to 0.91 (95% CI, 0.90-0.91) in internal and external validation cohorts. In Stage 2, AI-TEW implements a tiered risk stratification strategy by optimizing decision thresholds to prioritize high-risk patients, thereby increasing PPV from baseline levels of 9.8-18.8% to 32.5-40.5% across sites, while maintaining a high negative predictive value (NPV) of over 98% for low-risk individuals. To further refine alert precision, a knowledge-based filtering layer is introduced, leveraging large language models (LLM) to interpret patient-specific risk factors derived from SHAP (Shapley Additive exPlanations) method. Integrating explainable AI with clinical reasoning enhances contextual understanding and reduces spurious alerts, leading to an 11.53% increase in PPV in external validation (p = 0.0092 for MedGemma). By integrating improved predictive efficiency with interpretable, knowledge-informed filtering, AI-TEW reduces alert burden while supporting timely clinical intervention, demonstrating a promising approach to mitigating the impact of class imbalance in emergency risk prediction.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
MIMIC-IV, a freely accessible electronic health record dataset
Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Hao, S., Moody, B., Gow, B., et al. (2023) · Scientific Data · doi:10.1038/s41597-022-01899-x · Cited by 3,401 · see citing works
Explainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967 · see citing works
Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators
CALSTER, B. V., Wynants, L., Verbeek, J. F. M., Verbakel, J. Y. J., Christodoulou, E., Vickers, A. J., Roobol, M. J., & Steyerberg, E. W. (2018) · European Urology · doi:10.1016/j.eururo.2018.08.038 · Cited by 1,289 · see citing works
Nohara, Y., Matsumoto, K., Soejima, H., & Nakashima, N. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106584 · Cited by 1,091 · see citing works
Bobadilla, A. V. P., Schmitt, V., Maier, C. S., Mensing, S., & Stodtmann, S. (2024) · Clinical and Translational Science · doi:10.1111/cts.70056 · Cited by 931 · see citing works
Decision curve analysis to evaluate the clinical benefit of prediction models
Vickers, A. J., & Holland, F. (2021) · The Spine Journal · doi:10.1016/j.spinee.2021.02.024 · Cited by 392 · see citing works
van den Goorbergh, R., van Smeden, M., Timmerman, D., & CALSTER, B. V. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac093 · Cited by 375 · see citing works
Song, X., Liu, X., Liu, F., & Wang, C. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104484 · Cited by 301 · see citing works
Handling imbalanced medical datasets: review of a decade of research
Salmi, M., Atif, D., Oliva, D. A., Abraham, A., & Ventura, S. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10884-2 · Cited by 287 · see citing works
Assessing GPT-4 for cell type annotation in single-cell RNA-seq analysis
Hou, W., & Ji, Z. (2024) · Nature Methods · doi:10.1038/s41592-024-02235-4 · Cited by 277 · see citing works
Benchmark evaluation of DeepSeek large language models in clinical decision-making
Sandmann, S., Hegselmann, S., Fujarski, M., Bickmann, L., Wild, B., Eils, R., & Varghese, J. (2025) · Nature Medicine · doi:10.1038/s41591-025-03727-2 · Cited by 233 · see citing works
What large language models know and what people think they know
Steyvers, M., Tejeda, H., Kumar, A., Belem, C., Karny, S., Hu, X., Mayer, L. W., & Smyth, P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00976-7 · Cited by 209 · see citing works
Dependency Analysis of Accuracy Estimates in k-Fold Cross Validation
Wong, T.-T., & Yang, N.-Y. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2740926 · Cited by 189 · see citing works
Minimal Impact of Implemented Early Warning Score and Best Practice Alert for Patient Deterioration*
Bedoya, A. D., Clement, M. E., Phelan, M., Steorts, R. C., O’Brien, C., & Goldstein, B. A. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003439 · Cited by 162 · see citing works
Wallace, M. L., Mentch, L., Wheeler, B. J., Tapia, A. L., Richards, M., Zhou, S., Yi, L., Redline, S. S., & Buysse, D. J. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01965-x · Cited by 126 · see citing works
Porto, B. M. (2024) · BMC Emergency Medicine · doi:10.1186/s12873-024-01135-2 · Cited by 124 · see citing works
Klug, M., Barash, Y., Bechler, S., Resheff, Y. S., Tron, T., Ironi, A., Soffer, S., Zimlichman, E., & Klang, E. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05512-7 · Cited by 120 · see citing works
Recommendation for the review of biological reference intervals in medical laboratories
Henny, J., Vassault, A., Boursier, G., Vukasovic, I., Mesko Brguljan, P., Lohmander, M., Ghita, I., Andreu, F. A. B., Kroupis, C., Sprongl, L., et al. (2016) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2016-0793 · Cited by 119 · see citing works
Williams, C. Y. K., Miao, B. Y., Kornblith, A. E., & Butte, A. J. (2024) · Nature Communications · doi:10.1038/s41467-024-52415-1 · Cited by 119 · see citing works
Brajer, N., Cozzi, B., Gao, M., Nichols, M., Revoir, M., Balu, S., Futoma, J., Bae, J., Setji, N. P., Hernandez, A. F., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.20733 · Cited by 107 · see citing works
Yuan, S., Yang, Z., Li, J., Wu, C., & Liu, S. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03048-x · Cited by 70 · see citing works
Self-citationMachine learning techniques for mortality prediction in emergency departments: a systematic review
Naemi, A., Schmidt, T., Mansourvar, M., Naghavi‐Behzad, M., Ebrahimi, A., & Wiil, U. K. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-052663 · Cited by 67 · see citing works
Ye, C., Wang, O., Liu, M., Zheng, L., Xia, M., Hao, S., Jin, B., Jin, H., Zhu, C., Huang, C.-J., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/13719 · Cited by 62 · see citing works
Honarmand, K., Wax, R. S., Penoyer, D., Lighthall, G., Danesh, V. C., Rochwerg, B. N., Cheatham, M. L., Davis, D. P., DeVita, M. A., Downar, J., et al. (2024) · Critical Care Medicine · doi:10.1097/ccm.0000000000006072 · Cited by 57 · see citing works
Au-Yeung, W.-T. M., Sahani, A. K., Isselbacher, E. M., & Armoundas, A. A. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0160-7 · Cited by 56 · see citing works
Sellergren, A., Kazemzadeh, S., Jaroensri, T., Kiraly, A., Traverse, M., Kohlberger, T., Xu, S., Jamil, F., Hughes, C., Lau, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.05201 · Cited by 49 · see citing works
Mahapatra, S., Gupta, V. R., Sahu, S. S., & Panda, G. (2021) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2021.3061300 · Cited by 42 · see citing works
Equivalent statistics for a one-sample t-test
Francis, G., & Jakicic, V. (2022) · Behavior Research Methods · doi:10.3758/s13428-021-01775-3 · Cited by 39 · see citing works
Logothetis, S. B., Green, D., Holland, M., & Moubayed, N. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-40661-0 · Cited by 37 · see citing works
Cartus, A. R., Samuels, E. A., Cerdá, M., & Marshall, B. D. L. (2023) · Addiction · doi:10.1111/add.16133 · Cited by 35 · see citing works
Xu, D., Liu, F., Ding, X., Ma, J., Suo, Y., Peng, Y.-Y., Li, J., & Fu, X. (2025) · BMC Nursing · doi:10.1186/s12912-025-03084-y · Cited by 35 · see citing works
Clark, J. S. C., Kulig, P., Podsiadło, K., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-29308-2 · Cited by 31 · see citing works
Using emergency department triage for machine learning-based admission and mortality prediction
Tschoellitsch, T., Seidl, P., Böck, C., Maletzky, A., Moser, P., Thumfart, S., Giretzlehner, M., Hochreiter, S., & Meier, J. M. (2023) · European Journal of Emergency Medicine · doi:10.1097/mej.0000000000001068 · Cited by 28 · see citing works
Alarm burden and the nursing care environment: a 213-hospital cross-sectional study
Ruppel, H., Dougherty, M., Bonafide, C. P., & Lasater, K. B. (2023) · BMJ Open Quality · doi:10.1136/bmjoq-2023-002342 · Cited by 27 · see citing works
Emergency Department Triage Accuracy and Delays in Care for High-Risk Conditions
Sax, D. R., Warton, E. M., Mark, D. G., & Reed, M. E. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.8498 · Cited by 27 · see citing works
Chi-square test under indeterminacy: an application using pulse count data
Aslam, M. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01400-z · Cited by 24 · see citing works
Explainable differential diagnosis with dual-inference large language models
Zhou, S., Lin, M., Ding, S., Wang, J., Chen, C., Melton, G. B., Zou, J., & Zhang, R. (2025) · npj Health Systems · doi:10.1038/s44401-025-00015-6 · Cited by 20 · see citing works
Son, B., Myung, J., Shin, Y., Kim, S., Kim, S. H., Chung, J., Noh, J., Cho, J., & Chung, H. S. (2023) · Scientific Reports · doi:10.1038/s41598-023-41544-0 · Cited by 19 · see citing works
Large Margin Weighted k-Nearest Neighbors Label Distribution Learning for Classification
Wang, J., & Geng, X. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3297261 · Cited by 18 · see citing works
Hsu, C.-C., Kao, Y., Hsu, C., Chen, C., Hsu, S., Liu, T.-L., Lin, H., Wang, J., Liu, C., & Huang, C. (2023) · BMC Endocrine Disorders · doi:10.1186/s12902-023-01437-9 · Cited by 18 · see citing works
Stats on the desats: alarm fatigue and the implications for patient safety
Anderson, H. R., Borgen, A., Christnacht, R., et al. (2023) · BMJ Open Quality · doi:10.1136/bmjoq-2023-002262 · Cited by 17 · see citing works
The use of decision tree analysis for improving age estimation standards from the acetabulum
Botha, D., & Steyn, M. (2022) · Forensic Science International · doi:10.1016/j.forsciint.2022.111514 · Cited by 15 · see citing works
Venkatesh, N. P., Kumar, R. P., Neelapu, B. C., Pal, K., & Sivaraman, J. (2023) · Physical and Engineering Sciences in Medicine · doi:10.1007/s13246-023-01274-z · Cited by 14 · see citing works
Kuo, K.-M., & Chang, C. S. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03010-x · Cited by 12 · see citing works
Evaluation of large language models for providing educational information in orthokeratology care
Huang, Y., Shi, R., Chen, C., Zhou, X., Zhou, X., Hong, J., & Chen, Z. (2025) · Contact Lens and Anterior Eye · doi:10.1016/j.clae.2025.102384 · Cited by 10 · see citing works
Potential Diagnostic Error for Emergency Conditions, Mortality, and Healthy Days at Home
Lin, M., Burke, R. C., Sabbatini, A. K., Latsko, E., Edlow, J. A., Orav, E. J., & Burke, L. G. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.16400 · Cited by 10 · see citing works
Ma, Y., Lv, H., Ma, Y., Wang, X., Lv, L., Liang, X., & Wang, L. (2025) · BioData Mining · doi:10.1186/s13040-025-00440-1 · Cited by 9 · see citing works
Wang, Y., Liu, J., Wang, J., Shang, J., & Gao, Y.-L. (2023) · Journal of Computational Biology · doi:10.1089/cmb.2023.0078 · Cited by 8 · see citing works
Hsieh, M., Hsu, N., Lin, Y., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-040837 · Cited by 7 · see citing works
Theodosakis, N., Yoon, J., Young, K., et al. (2022) · JAMA Dermatology · doi:10.1001/jamadermatol.2022.3872 · Cited by 7 · see citing works
Gosselin, A., Chabut, C., Duhamel, A., et al. (2020) · Journal of Population Therapeutics and Clinical Pharmacology · doi:10.15586/jptcp.v27i3.705 · Cited by 6 · see citing works
Guiding Clinical Decisions Through Predictive Risk Rules
Goldstein, B. A., & Bedoya, A. D. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.13101 · Cited by 6 · see citing works
Segal, G., Keidar, N., Lotan, R. M., Romano, Y., Herskovitz, M., & Yaniv, Y. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1184990 · Cited by 5 · see citing works
Lee, Y., Chang, C., & Yang, C. C. (2025) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-025-00194-9 · Cited by 5 · see citing works
Ray, P. P. (2025) · Aesthetic Plastic Surgery · doi:10.1007/s00266-025-05038-w · Cited by 5 · see citing works
Takefuji, Y. (2025) · Journal of Infection · doi:10.1016/j.jinf.2025.106575 · Cited by 4 · see citing works
Mortality after emergency department discharge: an analysis of 453599 cases
Maestre-Orozco, T., Ramos, J. M., Espinosa, B., et al. (2024) · Emergencias · doi:10.55633/s3me/018.2024 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Hundal, J., Veettil, A. A. V., Chung, C., & Hosseini, M. (2026) · American Society of Clinical Oncology Educational Book · doi:10.1200/edbk-26-517280 · Cited by 1 · see citing works
- Authors
- Wu, L., Mai, L., Wang, H., Huang, J., He, X., Zhan, X., Khalemsky, A., Kumar, V. A., Paxton, J. H., Tsilimingras, D., Hachimi-Idrissi, S., Liu, S. W., Savioli, G., Rathlev, N. K., Tazarourte, K., Slagman, A., Christ, M., Qureshi, M., Hariri, H., Grossman, S. A., Hu, B., Wang, H., He, B., Levy, P. D., O’Neil, B. J., Gemme, S., Kurland, L., Lang, E., Lin, J., Liang, H., Li, X., & Bellou, A.
- Affiliations
- Guangdong Academy of Medical Sciences (CN)
- North Sichuan Medical University (CN)
- Wayne State University (US)
- Vrije Universiteit Brussel (BE)
- Harvard University (US)
- University of Pavia (IT)
- University of Massachusetts Chan Medical School (US)
- Hospices Civils de Lyon (FR)
- Charité – Universitätsmedizin Berlin (DE)
- University of Lucerne (CH)
- Alfaisal University (SA)
- Beth Israel Deaconess Medical Center (US)
- Örebro University (SE)
- Alberta Health Services (CA)
- Shenzhen University (CN)
- Corresponding author
- Lin, Jinle
- Liang, Huiying
- Li, Xin
- Bellou, Abdelouahab
- ORCID
- Venue
- npj Digital Medicine
- Published
- 2026-03-14
- Volume
- 9
- Issue
- 1
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC13133136
- OpenAlex
- W7135397944
- Cited by
- 8
- References
- 57
- Citations by year
- 2026: 8
- FWCI
- 27.16
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- npj Digital Medicine (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- HAL (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Guangdong Natural Science Foundation General Project (2024A1515012112)
- Guangdong Medical Research Fund Project (A2024044)
- National Natural Science Fund of China (82302462)
- Guang Dong Basic and Applied Basic Research Foundation (2022A1515111206)
- National Key R&D Program of China-Intergovernmental Key Projects (2023YFE0114300)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Keywords
- in-hospital mortality prediction, early warning systems, alarm fatigue, false alarm reduction, class imbalance, positive predictive value, decision threshold optimization, large language models, SHAP, emergency department, clinical risk prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
