Prediction across healthcare settings: a case study in predicting emergency department disposition
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Several approaches exist today for developing predictive models across multiple clinical sites, yet there is a lack of comparative data on their performance, especially within the context of EHR-based prediction models. We set out to provide a framework for prediction across healthcare settings. As a case study, we examined an ED disposition prediction model across three geographically and demographically diverse sites. We conducted a 1-year retrospective study, including all visits in which the outcome was either discharge-to-home or hospitalization. Four modeling approaches were compared: a ready-made model trained at one site and validated at other sites, a centralized uniform model incorporating data from all sites, multiple site-specific models, and a hybrid approach of a ready-made model re-calibrated using site-specific data. Predictions were performed using XGBoost. The study included 288,962 visits with an overall admission rate of 16.8% (7.9–26.9%). Some risk factors for admission were prominent across all sites (e.g., high-acuity triage emergency severity index score, high prior admissions rate), while others were prominent at only some sites (multiple lab tests ordered at the pediatric sites, early use of ECG at the adult site). The XGBoost model achieved its best performance using the uniform and site-specific approaches (AUC = 0.9–0.93), followed by the calibrated-model approach (AUC = 0.87–0.92), and the ready-made approach (AUC = 0.62–0.85). Our results show that site-specific customization is a key driver of predictive model performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579
The future of digital health with federated learning
Rieke, N., Hancox, J., Li, W., Milletarì, F., Roth, H. R., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B. A., Maier-Hein, K., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00323-1 · Cited by 4,015
The Future of Digital Health with Federated Learning
Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B., Maier-Hein, K., et al. (2020) · arXiv · doi:10.48550/arxiv.2003.08119 · Cited by 3,495
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506 · cited 2×
“Four main approaches are commonly used today to address the challenges of prediction across multiple sites: (1) Training and testing a model at each site separately in order to build site-specific models; (2) creating a centralized database of data from all sites in order to build a single uniform model(25,26); (3) ap…”
MethodAssessing the Generalizability of Prognostic Information
Justice, A. C., Covinsky, K. E., & Berlin, J. A. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-6-199903160-00016 · Cited by 1,160
Overview of artificial intelligence in medicine
Amisha, Malik, P., Pathania, M., & Rathaur, V. K. (2019) · Journal of Family Medicine and Primary Care · doi:10.4103/jfmpc.jfmpc_440_19 · Cited by 1,106 · cited 2×
“3): (1) In the sitespecific approach, separate prediction models were developed and validated at each of the three participating sites.”
MethodSMART on FHIR: a standards-based, interoperable apps platform for electronic health records
Mandel, J. C., Kreda, D. A., Mandl, K. D., Kohane, I. S., & Ramoni, R. B. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv189 · Cited by 829
Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., Dugas, A., Linton, B., Kirsch, T., & Kelen, G. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.08.005 · Cited by 424
Agniel, D., Kohane, I. S., & Weber, G. M. (2018) · BMJ · doi:10.1136/bmj.k1479 · Cited by 391
Cited by
Yang, J., Soltan, A. A. S., & Clifton, D. A. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00614-9 · Cited by 181
Machine learning for real-time aggregated prediction of hospital admission for emergency patients
King, Z., Farrington, J., Utley, M., Kung, E., Elkhodair, S., Harris, S., Sekula, R., Gillham, J., Li, K., & Crowe, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00649-y · Cited by 87
“Using XGBoost, Barak-Corren et al.(31) achieved AUROC of 0.”
Cunningham, J. W., Singh, P., Reeder, C., Claggett, B., Marti-Castellote, P. M., Lau, E. S., Khurshid, S., Batra, P., Lubitz, S. A., Maddah, M., et al. (2024) · JAMA Cardiology · doi:10.1001/jamacardio.2023.4859 · Cited by 38
Metadata
- Authors
- Barak-Corren, Y., Chaudhari, P., Perniciaro, J., Waltzman, M., Fine, A. M., & Reis, B. Y.
- Venue
- npj Digital Medicine
- Published
- 2021-12-15
- Volume
- 4
- Issue
- 1
- Pages
- 169-169
- ISSN
- 2398-6352 (ISSN-L)
- OpenAlex
- W4200271217
- Cited by
- 37
- References
- 15
- FWCI
- 1.76
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Prediction across healthcare settings: a case study in predicting emergency department disposition. https://app.cochat.ai/papers/10.1038/s41746-021-00537-x
- The paper
- Barak-Corren, Y., Chaudhari, P., Perniciaro, J., Waltzman, M., Fine, A. M., & Reis, B. Y. (2021). Prediction across healthcare settings: a case study in predicting emergency department disposition. npj Digital Medicine. https://doi.org/10.1038/s41746-021-00537-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.