Progressive prediction of hospitalisation in the emergency department: uncovering hidden patterns to improve patient flow
Abstract
INTRODUCTION: One of the factors contributing to ED crowding is the lengthy delay in transferring an admitted patient from the ED to an inpatient department (ie, boarding time). An earlier start of the admission process using an automatic hospitalisation prediction model could potentially shorten these delays and reduce crowding. METHODS: Clinical, operational and demographic data were retrospectively collected on 80 880 visits to the ED of Rambam Health Care Campus in Haifa, Israel, from January 2011 to January 2012. Using these data, a logistic regression model was developed to predict patient disposition (hospitalisation vs discharge) at three progressive time points throughout the ED visit: within the first 10 min, within an hour and within 2 hours. The algorithm was trained on 50% of the data (n=40 440) and tested on the remaining 50%. RESULTS: During the study time period, 58 197 visits ended in discharge and 22 683 in hospitalisation. Within 1 hour of presentation, our model was able to predict hospitalisation with a specificity of 90%, sensitivity of 94% and an AUCof 0.97. Early clinical decisions such as testing for calcium levels were found to be highly predictive of hospitalisations. In the Rambam ED, the use of such a prediction system would have the potential to save more than 250 patient hours per day. CONCLUSIONS: Data collected by EDs in electronic medical records can be used within a progressive modelling framework to predict patient flow and improve clinical operations. This approach relies on commonly available data and can be applied across different healthcare settings.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Logistic regression and Naive Bayes are commonly used, with few studies using more complex algorithms like random-forests, artificial neural networks, and support vector machines.
“Logistic regression and Naive Bayes are commonly used [9,11,16,18–22], with few studies using more complex algorithms like random-forests, artificial neural networks, and support vector machines [12,15,17].”
From Predicting hospital admission at emergency department triage using machine learning (2018) · cites this paper as [20]
Some prior models predicting hospital admission at ED triage include additional features such as hospital usage statistics and past medical history.
“while some models include additional features such as hospital usage statistics and past medical history [9,12,20,21].”
From Predicting hospital admission at emergency department triage using machine learning (2018) · cites this paper as [20]
Many prior studies have been limited by technical factors, where continuous variables are often reduced to categorical variables through binning or to binary variables encoding presence or missing-ness of data due to the challenges of imputation.
“Furthermore, many prior studies have been limited by technical factors, where continuous variables are often reduced to categorical variables through binning or to binary variables encoding presence or missing-ness of data due to the challenges of imputation [9,15,16,19–21].”
From Predicting hospital admission at emergency department triage using machine learning (2018) · cites this paper as [20]
A progressive modeling approach that uses information available at later time-points has been able to achieve high predictive power and indicates the utility of these features.
“Notably, a progressive modeling approach that uses information available at later time-points, such as lab tests ordered, medications given, and diagnoses entered by the ED provider during the patient’s current visit, has been able to achieve high predictive power and indicates the utility of these features [20,21].”
From Predicting hospital admission at emergency department triage using machine learning (2018) · cites this paper as [20]
Barak-Corren et al. showed that the later datasets offered better predictions than at 10 minutes.
“they were able to show that the later datasets offered better predictions than at 10 minutes.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [18]
Barak-Corren et al.'s study demonstrates the potential that EHRs offer for improving on approaches that use triage data only.
“Their study demonstrates the potential that EHRs offer for improving on approaches that use triage data only.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [18]
Barak-Corren et al.'s study built progressive datasets from historical data, each intended to reflect the data usually available at 10, 60 and 120 minutes after presentation to the ED.
“They built progressive datasets from historical data, each intended to reflect the data usually available at 10, 60 and 120 minutes after presentation to the ED.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [18]
Using logistic regression, Barak-Corren et al. achieved an AUROC of 0.82 in their 10 minute model, compared with 0.85 at 15 minutes here without chief complaint; they achieved an AUROC of 0.83 after 120 minutes, compared with 0.90 here.
“Using logistic regression (LR), Barak-Corren et al.18 achieved an AUROC of 0.82 in their 10 minute model (augmented by free-text chief complaint data that was retrospectively encoded), compared with 0.85 at 15 minutes here without chief complaint; they achieved an AUROC of 0.83 after 120 minutes, compared with 0.90 here.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [18]
Some features used in other studies are not available here, particularly chief complaint and imaging.
“Some features used in other studies18,20 are not available here, particularly chief complaint and imaging.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [18]
Cites
Orange: data mining toolbox in python
Demšar, J., Curk, T., Erjavec, A., Gorup, Č., Hočevar, T., Milutinovič, M., Možina, M., Polajnar, M., Toplak, M., Starič, A., et al. (2013) · Journal of Machine Learning Research · Cited by 1,651 · see citing works
The Effect of Emergency Department Crowding on Clinically Oriented Outcomes
Bernstein, S. L., Aronsky, D., Duseja, R., Epstein, S., Handel, D., Hwang, U., McCarthy, M., John McConnell, K., Pines, J. M., Rathlev, N., et al. (2009) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2008.00295.x · Cited by 1,147 · see citing works
Trends and Characteristics of US Emergency Department Visits, 1997-2007
Tang, N., Stein, J. C., Hsia, R. Y.-J., Maselli, J. H., & Gonzales, R. (2010) · JAMA · doi:10.1001/jama.2010.1112 · Cited by 766 · see citing works
Guttmann, A., Schull, M. J., Vermeulen, M. J., & Stukel, T. A. (2011) · BMJ · doi:10.1136/bmj.d2983 · Cited by 726 · see citing works
Farrohknia, N., Castrén, M. K., Ehrenberg, A., Lind, L., Oredsson, S., Jonsson, H., Asplund, K., & Göransson, K. E. (2011) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/1757-7241-19-42 · Cited by 337 · see citing works
Demand at the emergency department front door: 10‐year trends in presentations
Lowthian, J. A., Curtis, A. J., Jolley, D. J., Stoelwinder, J. U., McNeil, J. J., & Cameron, P. A. (2012) · The Medical Journal of Australia · doi:10.5694/mja11.10955 · Cited by 280 · see citing works
Redesigning emergency department patient flows: application of Lean thinking to health care
King, D. L., Ben‐Tovim, D. I., & Bassham, J. (2006) · Emergency Medicine Australasia · doi:10.1111/j.1742-6723.2006.00872.x · Cited by 279 · see citing works
Myths versus facts in emergency department overcrowding and hospital access block
Richardson, D., & Mountain, D. (2009) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2009.tb02451.x · Cited by 263 · see citing works
Olshaker, J. S., & Rathlev, N. K. (2006) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2005.05.023 · Cited by 239 · see citing works
Systematic review of trends in emergency department attendances: an Australian perspective
Lowthian, J. A., Curtis, A. J., Cameron, P. A., Stoelwinder, J. U., Cooke, M. W., & McNeil, J. J. (2010) · Emergency Medicine Journal · doi:10.1136/emj.2010.099226 · Cited by 233 · see citing works
Emergency department overcrowding, mortality and the 4‐hour rule in Western Australia
Geelhoed, G. C., & de Klerk, N. H. (2012) · The Medical Journal of Australia · doi:10.5694/mja11.11159 · Cited by 226 · see citing works
Sun, Y., Heng, B. H., Tay, S. Y., & Seow, E. (2011) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2011.01125.x · Cited by 154 · see citing works
Predicting emergency department inpatient admissions to improve same-day patient flow
Peck, J. S., Benneyan, J. C., Nightingale, D. J., & Gaehde, S. A. (2012) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2012.01435.x · Cited by 138 · see citing works
Improving emergency department efficiency by patient streaming to outcomes-based teams
Kelly, A.-M., Bryant, M., Cox, L., & Jolley, D. (2007) · Australian Health Review · doi:10.1071/ah070016 · Cited by 124 · see citing works
Managing Emergency Department Overcrowding
Olshaker, J. S. (2009) · Emergency Medicine Clinics of North America · doi:10.1016/j.emc.2009.07.004 · Cited by 102 · see citing works
Emergency Department Overcrowding
McCabe, J. B. (2001) · Academic Medicine · doi:10.1097/00001888-200107000-00005 · Cited by 89 · see citing works
How well do paramedics predict admission to the hospital? A prospective study
Levine, S. D., Colwell, C. B., Pons, P. T., et al. (2006) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2005.08.007 · Cited by 87 · see citing works
Tracking Emergency Department Overcrowding in a Tertiary Care Academic Institution
Bullard, M. J., Villa‐Roel, C., Bond, K., et al. (2009) · Healthcare Quarterly · doi:10.12927/hcq.2013.20884 · Cited by 67 · see citing works
Forecasting Emergency Department Crowding: A Prospective, Real-time Evaluation
Hoot, N. R., LeBlanc, L. J., Jones, I., Levin, S. R., Zhou, C., Gadd, C. S., & Aronsky, D. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2772 · Cited by 64 · see citing works
A Bayesian model for triage decision support
Sadeghi, S., Barzi, A., Sadeghi, N., & King, B. R. (2005) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2005.07.028 · Cited by 61 · see citing works
Established and Novel Initiatives to Reduce Crowding in Emergency Departments
Liu, S., Hamedani, A. G., Brown, D., Asplin, B. R., & Camargo, C. A. (2013) · Western Journal of Emergency Medicine · doi:10.5811/westjem.2012.11.12171 · Cited by 47 · see citing works
Can emergency medical service staff predict the disposition of patients they are transporting?
Clesham, K., Mason, S. M., Gray, J. T., et al. (2008) · Emergency Medicine Journal · doi:10.1136/emj.2007.054924 · Cited by 47 · see citing works
Predicting hospital admission for Emergency Department patients using a Bayesian network.
Leegon, J., Jones, I. R., Lanaghan, K., & Aronsky, D. (2005) · PubMed Central · Cited by 42 · see citing works
Richards, J. R., Ozery, G., Notash, M., et al. (2011) · Emergency Medicine International · doi:10.1155/2011/840459 · Cited by 37 · see citing works
Leegon, J., Jones, I. R., Lanaghan, K., & Aronsky, D. (2006) · Europe PMC (PubMed Central) · Cited by 33 · see citing works
Simon, H. K., Hirsh, D. A., Rogers, A. J., et al. (2008) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2007.10.090 · Cited by 32 · see citing works
Arslanian-Engoren, C. (2004) · Journal of Cardiovascular Nursing · doi:10.1097/00005082-200407000-00008 · Cited by 31 · see citing works
The emergency department prediction of disposition (EPOD) study
Vaghasiya, M. R., Murphy, M., O’Flynn, D., & Shetty, A. (2014) · Australasian Emergency Nursing Journal · doi:10.1016/j.aenj.2014.07.003 · Cited by 25 · see citing works
Derivation andInternal Validation of a Rule to Predict Hospital Admission in Prehospital Patients
Meisel, Z. F., Pollack, C. V., Mechem, C. C., & Pines, J. M. (2008) · Prehospital Emergency Care · doi:10.1080/10903120802096647 · Cited by 23 · see citing works
Inside the health blogosphere: quality, governance and the new innovation leaders.
Seeman, N. (2009) · PubMed · Cited by 14 · see citing works
Using Prediction to Improve Patient Flow in a Health Care Delivery Chain
Peck, J. S. (2011) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2 · see citing works
Cohen, R., & Elhadad, M. (2013) · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Predicting hospital admission at emergency department triage using machine learning
Hong, W. S., Haimovich, A. D., & Taylor, R. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201016 · Cited by 326 · see citing works
Machine learning for real-time aggregated prediction of hospital admission for emergency patients
King, Z., Farrington, J., Utley, M., Kung, E., Elkhodair, S., Harris, S., Sekula, R., Gillham, J., Li, K., & Crowe, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00649-y · Cited by 87 · see citing works
De Hond, A., Raven, W., Schinkelshoek, L., Gaakeer, M., Ter Avest, E., Sir, O., Lameijer, H., Hessels, R. A., Reijnen, R., De Jonge, E., et al. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104496 · Cited by 66 · see citing works
- Authors
- Barak-Corren, Y., Israelit, S. H., & Reis, B. Y.
- Affiliations
- Rambam Health Care Campus (IL)
- Boston Children's Hospital (US)
- ORCID
- Reis, Ben Y
- Venue
- Emergency Medicine Journal
- Publisher
- BMJ
- Published
- 2017-02-10
- Volume
- 34
- Issue
- 5
- Pages
- 308-314
- ISSN
- 1351-0622 (ISSN-L)
- 1472-0205
- 1472-0213
- OpenAlex
- W2587730730
- Cited by
- 52
- References
- 32
- Citations by year
- 2026: 4 · 2025: 9 · 2024: 3 · 2023: 4 · 2022: 6 · 2021: 8 · 2020: 7 · 2019: 5 · 2018: 6
- FWCI
- 6.26
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Emergency Medicine Journal (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- U.S. National Library of Medicine
- Topics
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Health Sciences › Health Professions › Emergency Medical Services › Healthcare Operations and Scheduling Optimization
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- emergency department overcrowding, patient boarding, patient flow, patient disposition, logistic regression, early admission, electronic health records, delayed hospital admission
- MeSH terms
- Adolescent, Adult, Aged, Aged, 80 and over, Child, Child, Preschool, Crowding, Emergency Service, Hospital, Female, Hematologic Tests, Hospitalization, Humans, Infant, Israel, Length of Stay, Male, Middle Aged, Predictive Value of Tests, Retrospective Studies, Sensitivity and Specificity, Time Factors, Multivariate Analysis, Logistic Models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review