Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
This study aimed to develop a machine learning-based clinical decision support system for emergency departments based on the decision-making framework of physicians. We extracted 27 fixed and 93 observation features using data on vital signs, mental status, laboratory results, and electrocardiograms during emergency department stay. Outcomes included intubation, admission to the intensive care unit, inotrope or vasopressor administration, and in-hospital cardiac arrest. eXtreme gradient boosting algorithm was used to learn and predict each outcome. Specificity, sensitivity, precision, F1 score, area under the receiver operating characteristic curve (AUROC), and area under the precision-recall curve were assessed. We analyzed 303,345 patients with 4,787,121 input data, resampled into 24,148,958 1 h-units. The models displayed a discriminative ability to predict outcomes (AUROC > 0.9), and the model with lagging 6 and leading 0 displayed the highest value. The AUROC curve of in-hospital cardiac arrest had the smallest change, with increased lagging for all outcomes. With inotropic use, intubation, and intensive care unit admission, the range of AUROC curve change with the leading 6 was the highest according to different amounts of previous information (lagging). In this study, a human-centered approach to emulate the clinical decision-making process of emergency physicians has been adopted to enhance the use of the system. Machine learning-based clinical decision support systems customized according to clinical situations can help improve the quality of care.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
An overview of clinical decision support systems: Benefits, risks, and strategies for success
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0221-y · Cited by 3,019
Tabular data: Deep learning is not all you need
Shwartz-Ziv, R., & Armon, A. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.011 · Cited by 2,172
Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., Dugas, A., Linton, B., Kirsch, T., & Kelen, G. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.08.005 · Cited by 424
Strohm, L., Hehakaya, C., Ranschaert, E. R., Boon, W. P. C., & Moors, E. H. M. (2020) · European Radiology · doi:10.1007/s00330-020-06946-y · Cited by 329
ED overcrowding is associated with an increased frequency of medication errors
Kulstad, E. B., Sikka, R., Sweis, R. T., Kelley, K. M., & Rzechula, K. H. (2010) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2008.12.014 · Cited by 306
Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study
Sandhu, S., Lin, A. L., Brajer, N., Sperling, J., Ratliff, W., Bedoya, A. D., Balu, S., O'Brien, C., & Sendak, M. P. (2020) · Journal of Medical Internet Research · doi:10.2196/22421 · Cited by 168
Romero-Brufau, S., Wyatt, K. D., Boyum, P., Mickelson, M., Moore, M., & Cognetta-Rieke, C. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104072 · Cited by 143
Schwartz, J. M., George, M., Rossetti, S. C., Dykes, P. C., Minshall, S. R., Lucas, E., & Cato, K. D. (2022) · JMIR Human Factors · doi:10.2196/33960 · Cited by 88 · cited 1×
“The physicians did not trust the predictions when the logic behind those was unclear 17,39–42 .”
Background
Cited by
Gebrael, G., Sahu, K. K., Chigarira, B., Tripathi, N., Mathew Thomas, V., Sayegh, N., Maughan, B. L., Agarwal, N., Swami, U., & Li, H. (2023) · Cancers · doi:10.3390/cancers15143717 · Cited by 85
“These algorithms leverage machine learning techniques to analyze patient data and assist healthcare providers in rapidly categorizing patients based on the severity of their condition [15].”
MethodBayor, A. A., Li, J., Yang, I. A., & Varnfield, M. (2025) · Journal of Medical Internet Research · doi:10.2196/63733 · Cited by 46
“Machine learning–based systems develop knowledge from existing clinical data to apply it to new datasets [78, 79].”
The AI Future of Emergency Medicine
Petrella, R. J. (2024) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2024.01.031 · Cited by 43
Artificial Intelligence-Assisted Emergency Department Vertical Patient Flow Optimization
Hodgson, N. R., Saghafian, S., Martini, W. A., Feizi, A., & Orfanoudaki, A. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15060219 · Cited by 6
Metadata
- Authors
- Choi, A., Choi, S. Y., Chung, K., Chung, H. S., Song, T., Choi, B., & Kim, J. H.
- Venue
- Scientific Reports
- Published
- 2023-05-26
- Volume
- 13
- Issue
- 1
- Pages
- 8561-8561
- ISSN
- 2045-2322 (ISSN-L)
- OpenAlex
- W4378472098
- Cited by
- 47
- References
- 11
- FWCI
- 4.13
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department. https://app.cochat.ai/papers/10.1038/s41598-023-35617-3
- The paper
- Choi, A., Choi, S. Y., Chung, K., Chung, H. S., Song, T., Choi, B., & Kim, J. H. (2023). Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department. Scientific Reports. https://doi.org/10.1038/s41598-023-35617-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.