Prospective External Validation of a Commercial Model Predicting the Likelihood of Inpatient Admission From the Emergency Department
Abstract
Study objective Early notification of admissions from the emergency department (ED) may allow hospitals to plan for inpatient bed demand. This study aimed to assess Epic's ED Likelihood to Occupy an Inpatient Bed predictive model and its application in improving hospital bed planning workflows. Methods All ED adult (18 years and older) visits from September 2021 to August 2022 at a large regional health care system were included. The primary outcome was inpatient admission. The predictive model is a random forest algorithm that uses demographic and clinical features. The model was implemented prospectively, with scores generated every 15 minutes. The area under the receiver operator curves (AUROC) and precision-recall curves (AUPRC) were calculated using the maximum score prior to the outcome and for each prediction independently. Test characteristics and lead time were calculated over a range of model score thresholds. Results Over 11 months, 329,194 encounters were evaluated, with an incidence of inpatient admission of 25.4%. The encounter-level AUROC was 0.849 (95% confidence interval [CI], 0.848 to 0.851), and the AUPRC was 0.643 (95% CI, 0.640 to 0.647). With a prediction horizon of 6 hours, the AUROC was 0.758 (95% CI, 0.758 to 0.759,) and the AUPRC was 0.470 (95% CI, 0.469 to 0.471). At a predictive model threshold of 40, the sensitivity was 0.49, the positive predictive value was 0.65, and the median lead-time warning was 127 minutes before the inpatient bed request.…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Epic ED to Inpatient model had an AUROC of 0.84 (95% CI 0.84-0.85).
“Dadabhoy 2023 46 Brigham and Women's Hospital Prospective Inpatient Admissions 329,194 25.4% 0.84 (0.84, 0.85)”
From A Systematic Review and Meta-analysis of Externally Validated Epic Clinical Decision Support Tools (2026) · cites this paper as [46]
Cites
The Effect of Emergency Department Crowding on Clinically Oriented Outcomes
Bernstein, S. L., Aronsky, D., Duseja, R., Epstein, S., Handel, D., Hwang, U., McCarthy, M., John McConnell, K., Pines, J. M., Rathlev, N., et al. (2009) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2008.00295.x · Cited by 1,147 · see citing works
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
The Clinician and Dataset Shift in Artificial Intelligence
Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., & Saria, S. (2021) · New England Journal of Medicine · doi:10.1056/nejmc2104626 · Cited by 855 · see citing works
Federated learning for predicting clinical outcomes in patients with COVID-19
Dayan, I., Roth, H. R., Zhong, A., Harouni, A., Gentili, A., Abidin, A. Z., Liu, A., Costa, A. B., Wood, B. J., Tsai, C.-S., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01506-3 · Cited by 842 · see citing works
Predicting hospital admission at emergency department triage using machine learning
Hong, W. S., Haimovich, A. D., & Taylor, R. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201016 · Cited by 326 · see citing works
Cabitza, F., Campagner, A., Soares, F., García de Guadiana-Romualdo, L., Challa, F., Sulejmani, A., Seghezzi, M., & Carobene, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106288 · Cited by 284 · see citing works
Singh, K., Valley, T. S., Tang, S., Li, B. Y., Kamran, F., Sjoding, M. W., Wiens, J., Otles, E., Donnelly, J. P., Wei, M. Y., et al. (2021) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202006-698oc · Cited by 111 · see citing works
Predicting hospital admission at the emergency department triage: A novel prediction model
Parker, C. A., Liu, N., Wu, S. X., Shen, Y., Lam, S. S. W., & Ong, M. E. H. (2019) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2018.10.060 · Cited by 100 · see citing works
Machine learning for real-time aggregated prediction of hospital admission for emergency patients
King, Z., Farrington, J., Utley, M., Kung, E., Elkhodair, S., Harris, S., Sekula, R., Gillham, J., Li, K., & Crowe, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00649-y · Cited by 87 · see citing works
Kamran, F., Tang, S., Otles, E., McEvoy, D. S., Saleh, S. N., Gong, J., Li, B. Y., Dutta, S., Liu, X., Medford, R. J., et al. (2022) · BMJ · doi:10.1136/bmj-2021-068576 · Cited by 60 · see citing works
Self-citationQuantification of Sepsis Model Alerts in 24 US Hospitals Before and During the COVID-19 Pandemic
Wong, A., Cao, J., Lyons, P. G., Dutta, S., Major, V. J., Ötleş, E., & Singh, K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.35286 · Cited by 57 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
You, J. G., Hernandez-Boussard, T., Pfeffer, M. A., Landman, A., & Mishuris, R. G. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01506-4 · Cited by 59 · see citing works
A Systematic Review and Meta-analysis of Externally Validated Epic Clinical Decision Support Tools
Patel, H., Crusco, S., Hansen, D., Morales, J., Patale, T., Chiuzan, C., Sands, M., Naidich, J., Myers, K., Davidson, K. W., et al. (2026) · Journal of General Internal Medicine · doi:10.1007/s11606-026-10381-y · Cited by 4 · see citing works
- Authors
- Dadabhoy, F. Z., Driver, L., McEvoy, D. S., Stevens, R., Rubins, D., & Dutta, S.
- Affiliations
- Brigham and Women's Hospital (US)
- Mass General Brigham (US)
- Harvard University (US)
- Corresponding author
- Dutta, Sayon
- Venue
- Annals of Emergency Medicine
- Publisher
- Elsevier BV
- Published
- 2023-06
- Volume
- 81
- Issue
- 6
- Pages
- 738-748
- ISSN
- 0196-0644 (ISSN-L)
- OpenAlex
- W4317780979
- Cited by
- 10
- References
- 11
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 3 · 2023: 1
- FWCI
- 3.11
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of Emergency Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (R01 HS027170)
- Topics
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Emergency Medical Services › Healthcare Operations and Scheduling Optimization
- Keywords
- emergency department, hospital bed planning, prediction model validation, random forest, lead time
- MeSH terms
- Adult, Emergency Service, Hospital, Hospitalization, Humans, Inpatients, Patient Admission, Prospective Studies, Retrospective Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
