Early risk assessment for COVID-19 patients from emergency department data using machine learning
Open accessPublished version of a preprint
The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Since its emergence in late 2019, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused a pandemic with more than 55 million reported cases and 1.3 million estimated deaths worldwide. While epidemiological and clinical characteristics of COVID-19 have been reported, risk factors underlying the transition from mild to severe disease among patients remain poorly understood. In this retrospective study, we analysed data of 879 confirmed SARS-CoV-2 positive patients admitted to a two-site NHS Trust hospital in London, England, between January 1st and May 26th, 2020, with a majority of cases occurring in March and April. We extracted anonymised demographic data, physiological clinical variables and laboratory results from electronic healthcare records (EHR) and applied multivariate logistic regression, random forest and extreme gradient boosted trees. To evaluate the potential for early risk assessment, we used data available during patients’ initial presentation at the emergency department (ED) to predict deterioration to one of three clinical endpoints in the remainder of the hospital stay: admission to intensive care, need for invasive mechanical ventilation and in-hospital mortality. Based on the trained models, we extracted the most informative clinical features in determining these patient trajectories. Considering our inclusion criteria, we have identified 129 of 879 (15%) patients that required intensive care, 62 of 878 (7%) patients needing mechanical ventilation, and 193 of 619 (31%) cases of in-hospital mortality. Our models learned successfully from early clinical data and predicted clinical endpoints with high accuracy, the best model achieving area under the receiver operating characteristic (AUC-ROC) scores of 0.76 to 0.87 (F1 scores of 0.42–0.60). Younger patient age was associated with an increased risk of receiving intensive care and ventilation, but lower risk of mortality. Clinical indicators of a patient’s oxygen supply and selected laboratory results, such as blood lactate and creatinine levels, were most predictive of COVID-19 patient trajectories. Among COVID-19 patients machine learning can aid in the early identification of those with a poor prognosis, using EHR data collected during a patient’s first presentation at ED. Patient age and measures of oxygenation status during ED stay are primary indicators of poor patient outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020) · Nature · doi:10.1038/s41586-020-2649-2 · Cited by 23,826
Incidence of thrombotic complications in critically ill ICU patients with COVID-19
Klok, F. A., Kruip, M. J. H. A., van der Meer, N. J. M., Arbous, M. S., Gommers, D. A. M. P. J., Kant, K. M., Kaptein, F. H. J., van Paassen, J., Stals, M. A. M., Huisman, M. V., et al. (2020) · Thrombosis Research · doi:10.1016/j.thromres.2020.04.013 · Cited by 5,028
Clinical characteristics of 113 deceased patients with coronavirus disease 2019: retrospective study
Chen, T., Wu, D., Chen, H., Yan, W., Yang, D., Chen, G., Ma, K., Xu, D., Yu, H., Wang, H., et al. (2020) · BMJ · doi:10.1136/bmj.m1091 · Cited by 3,903
How will country-based mitigation measures influence the course of the COVID-19 epidemic?
Anderson, R. M., Heesterbeek, H., Klinkenberg, D., & Hollingsworth, T. D. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30567-5 · Cited by 3,648
Docherty, A. B., Harrison, E. M., Green, C. A., Hardwick, H. E., Pius, R., Norman, L., Holden, K. A., Read, J. M., Dondelinger, F., Carson, G., et al. (2020) · BMJ · doi:10.1136/bmj.m1985 · Cited by 3,481
Wynants, L., Van Calster, B., Bonten, M. M., Collins, G. S., Debray, T. P., De Vos, M., Haller, M. C., Heinze, G., Moons, K. G., Riley, R. D., et al. (2020) · doi:10.1101/2020.03.24.20041020 · Cited by 3,070 · cited 1×
“Combined with state-of-the-art machine learning algorithms, these data can help to predict patient outcomes with greater accuracy than traditional methods 8,9 .”
MethodCharacteristics and outcomes of 21 critically ill patients with COVID-19 in Washington State
Arentz, M., Yim, E., Klaff, L., Lokhandwala, S., Riedo, F. X., Chong, M., & Lee, M. (2020) · JAMA · doi:10.1001/jama.2020.4326 · Cited by 2,259
Visualizing the effects of predictor variables in black box supervised learning models
Apley, D. W., & Zhu, J. (2020) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12377 · Cited by 1,535
Fisher, A., Rudin, C., & Dominici, F. (2018) · arXiv · doi:10.48550/arxiv.1801.01489 · Cited by 1,165
Research works printed but not linked: 19
- Wu, Z. & McGoogan, J. M. Characteristics of and important lessons from the Coronavirus Disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention. JAMA 323, 1239–1242 (2020). · DOI · Paper page
- Yang, X. et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir. Med. 9, 475–481 (2020). · DOI · Paper page
- Vizcaychipi, M. P. et al. Early detection of severe COVID-19 disease patterns define near real-time personalised care, bioseverity in males, and decelerating mortality rates. medRxiv 22, 2413 (2020).
- Novel Coronavirus Pneumonia Emergency Response Epidemiology Team. The epidemiological characteristics of an outbreak of 2019 novel coronavirus diseases (COVID-19) in China. Chin. Cent. Dis. Control Prev. 41, 145–151 (2020).
- Wang, D. et al. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China. JAMA 323, 1061–1069 (2020). · DOI · Paper page
- Hu, L. et al. Risk factors associated with clinical outcomes in 323 COVID-19 patients in Wuhan, China. medRxiv 2020.03.25.20037721 (2020).
- Jiang, X. et al. Towards an artificial intelligence framework for data-driven prediction of coronavirus clinical severity. CMC-Comput. Mater. Contin. 63, 537–551 (2020). · DOI · Paper page
- Zhou, F. et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 395, 1054–1062 (2020). · DOI · Paper page
- Gong, J. et al. A tool for early prediction of severe Coronavirus Disease 2019 (COVID-19): a multicenter study using the risk Nomogram in Wuhan and Guangdong, China. Clin. Infect. Dis. 71, 833–840 (2020). · DOI · Paper page
- Xie, J. et al. Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19. medRxiv 2020.03.28.20045997 (2020). · DOI
- Pourhomayoun, M. & Shakibi, M. Predicting mortality risk in patients with COVID-19 using artificial intelligence to help medical decision-making. medRxiv 2020.03.30.20047308 (2020). · DOI
- Yan, L. et al. Prediction of criticality in patients with severe Covid-19 infection using three clinical features: a machine learning-based prognostic model with clinical data in Wuhan. medRxiv 2020.02.27.20028027 (2020).
- Lemaître, G., Nogueira, F. & Aridas, C. K. Imbalanced-learn: a Python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18, 1–5 (2017).
- Zou, H. & Hastie, T. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 67, 301–320 (2005). · DOI · Paper page
- Chen, T. & Guestrin, C. XGBoost: a scalable tree boosting system. in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 785–794 (2016). · DOI · Paper page
- McKinney, W. Data structures for statistical computing in Python. in Proc. 9th Python Sci. Conf. 56–61 (2010). · DOI · Paper page
- Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
- Hunter, J. D. Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9, 90–95 (2007). · DOI · Paper page
- Docherty, A. B. et al. Features of 16,749 hospitalised UK patients with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol. medRxiv 2020.04.23.20076042 (2020). · DOI
Cited by
Applications of artificial intelligence in battling against covid-19: A literature review
Tayarani N., M.-H. (2021) · Chaos, Solitons & Fractals · doi:10.1016/j.chaos.2020.110338 · Cited by 210
“[114] Heldt FS , Vizcaychipi MP , Peacock S , Cinelli M , McLachlan L , Andreotti F , et al.”
Syrowatka, A., Kuznetsova, M., Alsubai, A., Beckman, A. L., Bain, P. A., Craig, K. J. T., Hu, J., Jackson, G. P., Rhee, K., & Bates, D. W. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00459-8 · Cited by 130
A systematic review on AI/ML approaches against COVID-19 outbreak
Dogan, O., Tiwari, S., Jabbar, M. A., & Guggari, S. (2021) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00424-8 · Cited by 87
“[145] proposed a model that extracts the informative clinical features from the data.”
BackgroundDomain Shifts in Machine Learning Based Covid-19 Diagnosis From Blood Tests
Roland, T., Böck, C., Tschoellitsch, T., Maletzky, A., Hochreiter, S., Meier, J., & Klambauer, G. (2022) · Journal of Medical Systems · doi:10.1007/s10916-022-01807-1 · Cited by 42
(2023) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2024.04.001 · Cited by 4
Metadata
- Authors
- Heldt, F. S., Vizcaychipi, M. P., Peacock, S., Cinelli, M., McLachlan, L., Andreotti, F., Jovanović, S., Dürichen, R., Lipunova, N., Fletcher, R. A., Hancock, A., McCarthy, A., Pointon, R. A., Brown, A., Eaton, J., Liddi, R., Mackillop, L., Tarassenko, L., & Khan, R. T.
- Venue
- Scientific Reports
- Published
- 2021-02-18
- Volume
- 11
- Issue
- 1
- Pages
- 4200-4200
- ISSN
- 2045-2322 (ISSN-L)
- OpenAlex
- W3130958407
- Cited by
- 97
- References
- 11
- FWCI
- 7.76
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Funders
- Sensyne Health plc
- Chelsea and Westminster Hospital NHS Foundation Trust
- National Institute for Health Research (IS-BRC-1215-20008)
- Stroke Association (TSA BHF 2017/01)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Early risk assessment for COVID-19 patients from emergency department data using machine learning. https://app.cochat.ai/papers/10.1038/s41598-021-83784-y
- The paper
- Heldt, F. S., Vizcaychipi, M. P., Peacock, S., Cinelli, M., McLachlan, L., Andreotti, F., Jovanović, S., Dürichen, R., Lipunova, N., Fletcher, R. A., Hancock, A., McCarthy, A., Pointon, R. A., Brown, A., Eaton, J., Liddi, R., Mackillop, L., Tarassenko, L., & Khan, R. T. (2021). Early risk assessment for COVID-19 patients from emergency department data using machine learning. Scientific Reports. https://doi.org/10.1038/s41598-021-83784-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.