Machine Learning-Based Models for Prediction of Critical Illness at Community, Paramedic, and Hospital Stages
Open accessAbstract
Overcrowding of emergency department (ED) has put a strain on national healthcare systems and adversely affected the clinical outcomes of critically ill patients. Early identification of critically ill patients prior to ED visits can help induce optimal patient flow and allocate medical resources effectively. This study aims to develop ML-based models for predicting critical illness in the community, paramedic, and hospital stages using Korean National Emergency Department Information System (NEDIS) data. Random forest and light gradient boosting machine (LightGBM) were applied to develop predictive models. The predictive model performance based on AUROC in community stage, paramedic stage, and hospital stage was estimated to be 0.870 (95% CI: 0.869–0.871), 0.897 (95% CI: 0.896–0.898), and 0.950 (95% CI: 0.949–0.950) in random forest and 0.877 (95% CI: 0.876–0.878), 0.899 (95% CI: 0.898–0.900), and 0.950 (95% CI: 0.950–0.951) in LightGBM, respectively. The ML models showed high performance in predicting critical illness using variables available at each stage, which can be helpful in guiding patients to appropriate hospitals according to their severity of illness. Furthermore, a simulation model can be developed for proper allocation of limited medical resources.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI-driven predictive analytics could assist students in identifying patients needing urgent care.
“For example, AI-driven predictive analytics could assist students in identifying patients needing urgent care [24,25]”
From Reimagining Core Entrustable Professional Activities for Undergraduate Medical Education in the Era of Artificial Intelligence (2023) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Common pitfalls in statistical analysis: Logistic regression
Ranganathan, P., Pramesh, C. S., & Aggarwal, R. (2017) · Perspectives in Clinical Research · doi:10.4103/picr.picr_87_17 · Cited by 659 · see citing works
Tolles, J., & Meurer, W. J. (2016) · JAMA · doi:10.1001/jama.2016.7653 · Cited by 583 · see citing works
Emergency department triage prediction of clinical outcomes using machine learning models
Raita, Y., Goto, T., Faridi, M. K., Brown, D., Camargo, C. A., & Hasegawa, K. (2019) · Critical Care · doi:10.1186/s13054-019-2351-7 · Cited by 531 · see citing works
A Systematic Review on Imbalanced Data Challenges in Machine Learning
Kaur, H., Pannu, H. S., & Malhi, A. K. (2019) · ACM Computing Surveys · doi:10.1145/3343440 · Cited by 492 · see citing works
Overcrowding in emergency department: an international issue
Somma, S. D., Paladino, L., Vaughan, L., Lalle, I., Magrini, L., & Magnanti, M. (2014) · Internal and Emergency Medicine · doi:10.1007/s11739-014-1154-8 · Cited by 449 · see citing works
Emergency department crowding and thrombolysis delays in acute myocardial infarction
Schull, M. J., Vermeulen, M. J., Slaughter, G., Morrison, L. J., & Daly, P. (2004) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2004.05.004 · Cited by 402 · see citing works
Comparing Boosting and Bagging Techniques With Noisy and Imbalanced Data
Khoshgoftaar, T. M., Hulse, J. V., & Napolitano, A. (2010) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2010.2084081 · Cited by 310 · see citing works
McNarry, A. F., & Goldhill, D. R. (2003) · Anaesthesia · doi:10.1111/j.1365-2044.2004.03526.x · Cited by 191 · see citing works
Surge Capacity for Healthcare Systems: A Conceptual Framework
Kaji, A. H., Koenig, K. L., & Bey, T. A. (2006) · Academic Emergency Medicine · doi:10.1197/j.aem.2006.06.032 · Cited by 185 · see citing works
A Review on Imbalanced Data Handling Using Undersampling and Oversampling Technique
(2017) · International Journal of Recent Trends in Engineering and Research · doi:10.23883/ijrter.2017.3168.0uwxm · Cited by 167 · see citing works
Mistry, B., de Ramirez, S. S., Kelen, G. D., Schmitz, P. S. K., Balhara, K. S., Levin, S., Martínez, D. A., Psoter, K. J., Anton, X. P., & Hinson, J. S. (2017) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.09.036 · Cited by 160 · see citing works
Goh, K. J., Wong, J., Tien, C., Ng, S. Y., Wen, S. D., Phua, G. C., & Leong, C. K. (2020) · Critical Care · doi:10.1186/s13054-020-02916-4 · Cited by 157 · see citing works
Prediction of Critical Illness During Out-of-Hospital Emergency Care
Seymour, C., Kahn, J., Cooke, C., Watkins, T., Heckbert, S., & Rea, T. (2010) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.2010.1140 · Cited by 156 · see citing works
The Effect of Emergency Department Crowding on Patient Outcomes
Johnson, K. D., & Winkelman, C. (2011) · Advanced Emergency Nursing Journal · doi:10.1097/tme.0b013e318207e86a · Cited by 155 · see citing works
Sun, Y., Heng, B. H., Tay, S. Y., & Seow, E. (2011) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2011.01125.x · Cited by 154 · see citing works
Goldberg, S. B., Flemotomos, N., Martinez, V. R., Tanana, M. J., Kuo, P. B., Pace, B. T., Villatte, J. L., Georgiou, P. G., Van Epps, J., Imel, Z. E., et al. (2020) · Journal of Counseling Psychology · doi:10.1037/cou0000382 · Cited by 141 · see citing works
Litton, E., Bucci, T., Chavan, S., Ho, Y. Y., Holley, A., Howard, G., Huckson, S., Kwong, P., Millar, J., Nguyen, N., et al. (2020) · Medical Journal of Australia · doi:10.5694/mja2.50596 · Cited by 134 · see citing works
Kang, D.-Y., Cho, K.-J., Kwon, O., Kwon, J., Jeon, K., Park, H., Lee, Y., Park, J., & Oh, B. (2020) · Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine · doi:10.1186/s13049-020-0713-4 · Cited by 115 · see citing works
Self-citationSheikhbardsiri, H., Raeisi, A., Nekoei-Moghadam, M., & Rezaei, F. (2017) · Disaster Medicine and Public Health Preparedness · doi:10.1017/dmp.2016.178 · Cited by 103 · see citing works
Validation of deep-learning-based triage and acuity score using a large national dataset
Kwon, J., Lee, Y., Lee, Y., Lee, S., Park, H., & Park, J. (2018) · PLoS ONE · doi:10.1371/journal.pone.0205836 · Cited by 93 · see citing works
Self-citationA validation of machine learning-based risk scores in the prehospital setting
Spangler, D., Hermansson, T., Smékal, D., & Blomberg, H. (2019) · PLoS ONE · doi:10.1371/journal.pone.0226518 · Cited by 72 · see citing works
Off hour admission to an intensivist-led ICU is not associated with increased mortality
Meynaar, I. A., van der Spoel, J. I., Rommes, J. H., et al. (2009) · Critical Care · doi:10.1186/cc7904 · Cited by 69 · see citing works
Handberry, M., Bull–Otterson, L., Dai, M., Mann, N. C., Chaney, E., Ratto, J., Horiuchi, K., Siza, C. R., Kulkarni, A., Gundlapalli, A. V., et al. (2021) · Clinical Infectious Diseases · doi:10.1093/cid/ciab373 · Cited by 66 · see citing works
Predicting hospital admissions to reduce emergency department boarding
Golmohammadi, D. (2016) · International Journal of Production Economics · doi:10.1016/j.ijpe.2016.09.020 · Cited by 62 · see citing works
Lee, J.-T., Hsieh, C., Lin, C., Lin, Y.-J., & Kao, C. (2021) · Scientific Reports · doi:10.1038/s41598-021-98961-2 · Cited by 58 · see citing works
The epidemiology of critical illness in the elderly
Carson, S. S. (2003) · Critical Care Clinics · doi:10.1016/s0749-0704(03)00051-4 · Cited by 50 · see citing works
Surge Capacity for Healthcare Systems: A Conceptual Framework
Kaji, A. H., Koenig, K. L., & Bey, T. A. (2006) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2006.tb01641.x · Cited by 45 · see citing works
Mortality in out-of-hours emergency medical admissions – more than just a weekend effect
Maggs, F. G., & Mallet, M. (2010) · The Journal of the Royal College of Physicians of Edinburgh · doi:10.4997/jrcpe.2010.205 · Cited by 42 · see citing works
Leveraging LightGBM for Categorical Big Data
Hancock, J., & Khoshgoftaar, T. M. (2021) · IEEE International Conference on Big Data Computing Service and Applications (BigDataService) · doi:10.1109/bigdataservice52369.2021.00024 · Cited by 41 · see citing works
Multiple Options and Unique Pathways: A New Direction for EMS?
Neely, K. W., Drake, M. E. R., Moorhead, J. C., et al. (1997) · Annals of Emergency Medicine · doi:10.1016/s0196-0644(97)70051-9 · Cited by 36 · see citing works
The Dwindling Supply Of Empty Beds: Implications For Hospital Surge Capacity
DeLia, D., & Wood, E. (2008) · Health Affairs · doi:10.1377/hlthaff.27.6.1688 · Cited by 35 · see citing works
Wang, J., Zong, L., Zhang, J., Sun, H., Walline, J. H., Sun, P., Xu, S., Li, Y., Wang, C., Liu, J., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-039177 · Cited by 34 · see citing works
Lessons from a COVID-19 hospital, Republic of Korea
Kim, M., Lee, J. Y., Park, J. S., Kim, H. A., Hyun, M., Suh, Y. S., Nam, S., Chung, W. J., & Cho, C. (2020) · Bulletin of the World Health Organization · doi:10.2471/blt.20.261016 · Cited by 31 · see citing works
Bradt, D. A., Aitken, P., FitzGerald, G., et al. (2009) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2009.00501.x · Cited by 29 · see citing works
Trends in Boarding of Admitted Patients in US Emergency Departments 2003–2005
Carr, B. G., Hollander, J. E., Baxt, W. G., et al. (2009) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2008.04.035 · Cited by 26 · see citing works
Shahverdi, B., Miller-Hooks, E. D., Tariverdi, M., et al. (2022) · Disaster Medicine and Public Health Preparedness · doi:10.1017/dmp.2022.12 · Cited by 24 · see citing works
Rekha, G., Tyagi, A. K., & Reddy, V. K. (2019) · International Journal of Hybrid Intelligent Systems · doi:10.3233/his-190261 · Cited by 18 · see citing works
Yang, Y., Kim, H., & Hwang, J. (2020) · Journal of Korean Medical Science · doi:10.3346/jkms.2020.35.e429 · Cited by 18 · see citing works
Self-citationUnified Medical Language System Coverage of Emergency-medicine Chief Complaints
Travers, D. A., & Haas, S. W. (2006) · Academic Emergency Medicine · doi:10.1197/j.aem.2006.06.054 · Cited by 15 · see citing works
Shirakawa, T., Sonoo, T., Ogura, K., Fujimori, R., Hara, K., Goto, T., Hashimoto, H., Takahashi, Y., Naraba, H., & Nakamura, K. (2020) · JMIR Medical Informatics · doi:10.2196/20324 · Cited by 14 · see citing works
Ahn, J. Y., Ryoo, H. W., Park, J., et al. (2016) · Journal of Korean Medical Science · doi:10.3346/jkms.2016.31.5.806 · Cited by 13 · see citing works
Bautz, B., & Schneider, J. I. (2020) · Emergency Medicine Clinics of North America · doi:10.1016/j.emc.2020.01.009 · Cited by 9 · see citing works
Two-Phase Chief Complaint Mapping to the UMLS Metathesaurus in Korean Electronic Medical Records
Kang, B., Kim, D., & Kim, H. (2008) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2008.2007103 · Cited by 8 · see citing works
Organizational wellbeing: A model of a new Apulian COVID-19 designated hospital
Vimercati, L., Cannone, E. S. S., Sponselli, S., et al. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.963315 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Jacobs, S. M., Lundy, N. N., Issenberg, S. B., & Chandran, L. (2023) · JMIR Medical Education · doi:10.2196/50903 · Cited by 33 · see citing works
- Authors
- Lee, S., Park, H. J., Hwang, J., Lee, S. W., Han, K. S., Kim, W. Y., Jeong, J., Kang, H., Kim, A., Lee, C., & Kim, S. J.
- Affiliations
- Korea University (KR)
- Asan Medical Center (KR)
- Dong-A University (KR)
- Hanyang University (KR)
- Corresponding author
- Lee, Chulung
- Kim, Su Jin
- ORCID
- Venue
- Emergency Medicine International
- Publisher
- Wiley
- Published
- 2023-06-26
- Volume
- 2023
- Pages
- 1-11
- ISSN
- 2090-2840 (ISSN-L)
- 2090-2859
- PMC
- PMC10317605
- OpenAlex
- W4382045580
- Cited by
- 8
- References
- 44
- Citations by year
- 2025: 4 · 2024: 2 · 2023: 2
- FWCI
- 0.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Emergency Medicine International (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Korean Association of Regional Emergency Medical Centers (K1913901, K1915031, K2011351, K2012711)
- Korean Society of Emergency Medicine (K1913901, K1915031, K2011351, K2012711)
- Korea University (K1913901, K1915031, K2011351, K2012711)
- Brain Korea 21 FOUR (K1913901, K1915031, K2011351, K2012711)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- critical illness prediction, emergency department overcrowding, prehospital care, paramedics, patient flow, resource allocation, random forest, LightGBM, emergency department utilization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
