The DOI matches its Crossref record; 57 of the 58 references it cites were found in a registry or an index; none of the 46 sources its claims cite is checked for support yet.
Predicting hospital admission at emergency department triage using machine learning
Open accessAbstract
OBJECTIVE: To predict hospital admission at the time of ED triage using patient history in addition to information collected at triage. METHODS: This retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model. RESULTS: A total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91). CONCLUSION: Machine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Machine learning models using electronic health record data predicted hospital admission at ED triage, with full-variable models reaching test AUCs of 0.91–0.92. Adding patient history significantly improved performance over triage information alone.
Key findings
- Among 560,486 adult ED visits with an overall admission risk of 29.7%, full-variable models achieved test AUC 0.91 for logistic regression, 0.92 for XGBoost, and 0.92 for deep neural networks; triage-only and history-only models achieved AUCs around 0.86–0.87.
- A low-dimensional XGBoost model using ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91–0.91).
- All algorithms reached maximum performance at 50% of the training set or less; logistic regression reached it at 10%, while XGBoost and DNN reached it at 50%.
- XGBoost and DNN outperformed logistic regression on the full dataset, with no significant difference between XGBoost and DNN across the three dataset types.
Methods
- Retrospective study of all adult ED visits between March 2014 and July 2017 from one academic and two community EDs that ended in admission or discharge; 972 variables were extracted per visit, and 560,486 visits were randomly partitioned into training (80%), validation (10%), and test (10%) sets.
- Nine binary classifiers were trained using logistic regression, XGBoost, and deep neural networks on three dataset types: triage information only, patient history only, and the full variable set; hyperparameters were optimized by average validation AUC across five validation sets, and test AUCs were calculated with DeLong 95% CIs.
- Additional analyses trained full-variable models on increasing fractions of the training set (1%, 10%, 30%, 50%, 80%, 100%) and used information gain from 100 XGBoost iterations to identify important variables and build a low-dimensional model.
Limitations
- The paper states that patient history was restricted to previous ED visits, and expanding to outpatient or inpatient data may improve performance but could present EHR integration and differing-standards difficulties.
- The paper states that data came from one hospital system with multiple EDs and a large catchment area, and extending the approach to dense urban areas with multiple independent EDs may be difficult because patients may not consistently present to the same system.
- The paper states that the outcome label was the ED provider’s prior disposition decision, so appropriateness of individual clinical decisions could not be assessed, and it does not address implementation or efficacy barriers in clinical practice; evident from the text, the study is retrospective, adult-only, excludes non-admission/discharge dispositions, and uses a held-out test set from the same health system rather than external validation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 46 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86–0.87), 0.87 for XGBoost (95% CI 0.87–0.88) and 0.87 for DNN (95% CI 0.87–0.88).”
§ Predicting hospital admission at emergency department triage using machine learning > Abstract > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the addition of patient history improves predictive performance significantly compared to using triage information alone”
§ Predicting hospital admission at emergency department triage using machine learning > Discussion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“as did prior hospital usage statistics such as the number of admissions within the past year and the disposition of the previous ED visit.”
§ Predicting hospital admission at emergency department triage using machine learning > Discussion
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For LR, the 95% CI of the AUC of the model trained on 10% of the training set contained the AUC of the model trained on the entire training set.”
§ Predicting hospital admission at emergency department triage using machine learning > Results > Testing the benefit of additional training samples
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86–0.87), 0.87 for XGBoost (95% CI 0.87–0.87) and 0.87 for DNN (95% CI 0.87–0.88).”
§ Predicting hospital admission at emergency department triage using machine learning > Abstract > Results
Strength:plain· not yet calibratedn = 560,486Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91–0.91), 0.92 for XGBoost (95% CI 0.92–0.93) and 0.92 for DNN (95% CI 0.92–0.92).”
§ Predicting hospital admission at emergency department triage using machine learning > Abstract > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variables correlated with age and markers of socioeconomic status such as insurance type were some of the other features identified by our model”
§ Predicting hospital admission at emergency department triage using machine learning > Discussion
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All algorithms reached maximum performance at 50% of the training set or less.”
§ Predicting hospital admission at emergency department triage using machine learning > Abstract > Results
Strength:plain· not yet calibratedn = 560,486Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“XGBoost and DNN outperformed LR on the full dataset, while there was no significant difference in performance between XGBoost and DNN across all three dataset types.”
§ Predicting hospital admission at emergency department triage using machine learning > Results > Model performance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91–0.91), 0.92 for XGBoost (95% CI 0.92–0.93) and 0.92 for DNN (95% CI 0.92–0.92).”
§ Predicting hospital admission at emergency department triage using machine learning > Abstract > Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 5
- The quote only partly states this claim. 1
- Probably not a substantial claim. 10
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All three algorithms reach maximum performance at 50% of the training set or less.”
- Also stated in: § Discussion · boosted
“we train our models on increasing fractions of the training set and show that the AUC plateaus well below the full training set.”
§ Predicting hospital admission at emergency department triage using machine learning > Results > Testing the benefit of additional training samples
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
- Also stated in: § Discussion · boosted
In the paper
“While most emergency department (ED) visits end in discharge, EDs represent the largest source of hospital admissions [1].”
§ Predicting hospital admission at emergency department triage using machine learning > Introduction
What the sources say
- Not checked yetcites [1] Moore B, Stocks C, Owens P. Trends in Emergency Department Visits, 2006–2014. HCUP Statistical Brief #227 [Internet]. Agency for Healthcare Research and Quality; 2017 Sep. Available: https://www.hcup-us.ahrq.gov/reports/statbriefs/sb227-Emergency-Department-Visit-Trends.pdf
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“It is increasingly understood that ED crowding is correlated with poorer patient outcomes [10].”
§ Predicting hospital admission at emergency department triage using machine learning > Introduction
What the sources say
- Not checked yetcites [10] SL Bernstein, 2009, The Effect of Emergency Department Crowding on Clinically Oriented Outcomes, Acad Emerg Med source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“the importance of medication counts in the model may either reflect a proxy feature for medical complexity or indicate that polypharmacy itself is a risk factor [42,43].”
§ Predicting hospital admission at emergency department triage using machine learning > Discussion
What the sources say
- Not checked yetcites [42] CM Hohl, 2001, Polypharmacy, adverse drug-related events, and potential adverse drug interactions in elderly patients presenting to an emergency department, Ann Emerg Med source
- Not checked yetcites [43] A Banerjee, 2011, The prevalence of polypharmacy in elderly attenders to an emergency department—a problem with a need for an effective solution, Int J Emerg Med source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Notably, a progressive modeling approach that uses information available at later time-points, such as lab tests ordered, medications given, and diagnoses entered by the ED provider during the patient’s current visit, has been able to achieve high predictive power and indicates the utility of these features [20,21].”
§ Predicting hospital admission at emergency department triage using machine learning > Introduction
What the sources say
- Not checked yetcites [20] Y Barak-Corren, 2017, Progressive prediction of hospitalisation in the emergency department: uncovering hidden patterns to improve patient flow, Emerg Med J source
- Not checked yetcites [21] Y Barak-Corren, 2017, Early Prediction Model of Patient Hospitalization From the Pediatric Emergency Department, Pediatrics source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets.”
- Also stated in: § Model fitting and evaluation · plain
“A randomly chosen test set of 56,000 (10%) samples was held out, then the remaining 504,486 (90%) samples split randomly five times to create five independent validation sets of 56,000 (10%) and training sets of 448,486 (80%).”
§ Predicting hospital admission at emergency department triage using machine learning > Abstract > Methods
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Also stated in: § Model fitting and evaluation · plain
None of the sources cited for this claim have been checked yet.
In the paper
“that have been previously linked to hospital admission [46–48].”
§ Predicting hospital admission at emergency department triage using machine learning > Discussion
What the sources say
- Not checked yetcites [46] AW Selassie, 2003, The Influence of Insurance, Race, and Gender on Emergency Department Disposition, Acad Emerg Med source
- Not checked yetcites [47] Jiang HJ, Russo CA, Barrett ML. Nationwide Frequency and Costs of Potentially Preventable Hospitalizations, 2006. HCUP Statistical Brief #72 [Internet]. Agency for Healthcare Research and Quality; 2009 Apr. Available: http://www.hcup-us.ahrq.gov/reports/statbriefs/sb72.pdf
- Not checked yetcites [48] S Kangovi, 2013, Understanding Why Patients Of Low Socioeconomic Status Prefer Hospitals Over Ambulatory Care, Health Aff (Millwood) source
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“As a recent work showed that using all elements of the electronic health record can robustly predict in-patient outcomes [26]”
§ Predicting hospital admission at emergency department triage using machine learning > Introduction
What the sources say
- Not checked yetcites [26] A Rajkomar, 2018, Scalable and accurate deep learning for electronic health records, Npj Digit Med source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Logistic regression and Naive Bayes are commonly used [9,11,16,18–22], with few studies using more complex algorithms like random-forests, artificial neural networks, and support vector machines [12,15,17].”
§ Predicting hospital admission at emergency department triage using machine learning > Introduction
What the sources say
- Not checked yetcites [9] Y Sun, 2011, Predicting hospital admissions at emergency department triage using routine administrative data, Acad Emerg Med source
- Not checked yetcites [11] JS Peck, 2012, Predicting emergency department inpatient admissions to improve same-day patient flow, Acad Emerg Med source
- Not checked yetcites [16] J Leegon, 2005, Predicting Hospital Admission for Emergency Department Patients using a Bayesian Network, AMIA Annu Symp Proc
- Not checked yetcites [18] DW Savage, 2017, P112: Predicting patient admission from the emergency department using triage administrative data, Can J Emerg Med source
- Not checked yetcites [19] JA Lucke, 2017, Early prediction of hospital admission for emergency department patients: a comparison between patients younger or older than 70 years, Emerg Med J
- Not checked yetcites [20] Y Barak-Corren, 2017, Progressive prediction of hospitalisation in the emergency department: uncovering hidden patterns to improve patient flow, Emerg Med J source
- Not checked yetcites [21] Y Barak-Corren, 2017, Early Prediction Model of Patient Hospitalization From the Pediatric Emergency Department, Pediatrics source
- Not checked yetcites [22] MM Dinh, 2016, The Sydney Triage to Admission Risk Tool (START) to predict Emergency Department Disposition: A derivation and internal validation study using retrospective state-wide data from New South Wales, Australia, BMC Emerg Med source
- Not checked yetcites [12] S Levin, 2017, Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index, Ann Emerg Med
- Not checked yetcites [15] J Leegon, 2006, Predicting Hospital Admission in a Pediatric Emergency Department using an Artificial Neural Network, AMIA Annu Symp Proc
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Prior models that incorporate past medical history utilize simplified chronic disease categories such as heart disease or diabetes [9,12] while leaving out rich historical information accessible from the electronic health record (EHR) such as outpatient medications and historical labs and vitals”
§ Predicting hospital admission at emergency department triage using machine learning > Introduction
What the sources say
- Not checked yetcites [9] Y Sun, 2011, Predicting hospital admissions at emergency department triage using routine administrative data, Acad Emerg Med source
- Not checked yetcites [12] S Levin, 2017, Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index, Ann Emerg Med
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Notification of administrators and inpatient teams regarding potential admissions may help alleviate this problem [11].”
§ Predicting hospital admission at emergency department triage using machine learning > Introduction
What the sources say
- Not checked yetcites [11] JS Peck, 2012, Predicting emergency department inpatient admissions to improve same-day patient flow, Acad Emerg Med source
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
None of the sources cited for this claim have been checked yet.
In the paper
“while some models include additional features such as hospital usage statistics and past medical history [9,12,20,21].”
§ Predicting hospital admission at emergency department triage using machine learning > Introduction
What the sources say
- Not checked yetcites [9] Y Sun, 2011, Predicting hospital admissions at emergency department triage using routine administrative data, Acad Emerg Med source
- Not checked yetcites [12] S Levin, 2017, Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index, Ann Emerg Med
- Not checked yetcites [20] Y Barak-Corren, 2017, Progressive prediction of hospitalisation in the emergency department: uncovering hidden patterns to improve patient flow, Emerg Med J source
- Not checked yetcites [21] Y Barak-Corren, 2017, Early Prediction Model of Patient Hospitalization From the Pediatric Emergency Department, Pediatrics source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“Lastly, this study does not address the implementation and efficacy barriers present in clinical practice [53].”
§ Predicting hospital admission at emergency department triage using machine learning > Discussion
What the sources say
- Not checked yetcites [53] SM Schenkel, 2018, Triage, Machine Learning, Algorithms, and Becoming the Borg, Ann Emerg Med
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
Gradient boosting and deep neural networks have not been applied to the task of predicting admission at ED triage to date.
“While gradient boosting and deep neural networks have been shown to be powerful tools for predictive modeling, neither has been applied to the task of predicting admission at ED triage to date.”
The paper relies on this without support.
The three EDs studied are representative of EDs generally, so the models' performance generalizes to other settings.
“Machine learning can robustly predict hospital admission using triage information and patient history.”
The paper relies on this without support.
The historical information available in the EHR at the time of triage is sufficiently complete and accurate to predict admission.
“For each patient visit, we collected a total of 972 variables, divided into major categories shown in Table 1.”
The paper relies on this without support.
The ED provider's prior decision is a valid ground truth for hospital admission.
“Throughout this study, we predict patient disposition by using the ED provider's prior decision as our true label.”
The paper relies on this without support.
The variables used in the models are available at the time of emergency department triage.
“Our study shows that machine learning can robustly predict hospital admission at emergency department (ED) triage”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This represents state-of-the-art performance comparable with recent admission models (e.g. AUROC 0.825–0.87722–25).”
From Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19 (2021) · cites this paper as [24]
In the citing paper
“Hospital services can be managed more efficiently if accurate short-term forecasts for emergency demand are available8,9.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [9]
In the citing paper
“Hong et al.9 showed that ML algorithms like gradient-boosted trees and deep neural networks, applied to a large EHR dataset of 972 variables, improved predictive performance.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [9]
In the citing paper
“Using XGBoost, Barak-Corren et al.31 achieved AUROC of 0.87–0.93 with a 60 minute model, compared with 0.89 here, and Hong et al9 achieved 0.87 with triage data.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Conflicting descriptions of how model cutoffs were chosen
MinorThe methods say Youden's index determined the cutoff for each model. Table 3 instead says every model's cutoff was set to a fixed specificity of 0.85, chosen using Youden's index only on the full XGBoost model.
“Youden's index was used to find the optimal cutoff point on the ROC curve to calculate the sensitivity, specificity, positive predictive value, and negative predictive value for each model [32,33].”
“The cutoff threshold for each model was set to match a fixed specificity of 0.85 to facilitate comparison. The value of 0.85 was chosen by using Youden’s Index on the full XGBoost model.”
Citations
Cites
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697
Building Predictive Models in R Using the caret Package
Kuhn, M. (2008) · Journal of Statistical Software · doi:10.18637/jss.v028.i05 · Cited by 8,863
An introduction to variable and feature selection
GuyonIsabelle, & ElisseeffAndré (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944968 · Cited by 8,354
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648
Scalable and accurate deep learning with electronic health records.
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · eScholarship (California Digital Library) · Cited by 2,242
Can machine-learning improve cardiovascular risk prediction using routine clinical data?
Weng, S. F., Reps, J., Kai, J., Garibaldi, J. M., & Qureshi, N. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174944 · Cited by 1,346
Ruopp, M. D., Perkins, N. J., Whitcomb, B. W., & Schisterman, E. F. (2008) · Biometrical Journal · doi:10.1002/bimj.200710415 · Cited by 1,320
The Effect of Emergency Department Crowding on Clinically Oriented Outcomes
Bernstein, S. L., Aronsky, D., Duseja, R., Epstein, S., Handel, D., Hwang, U., McCarthy, M., John McConnell, K., Pines, J. M., Rathlev, N., et al. (2009) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2008.00295.x · Cited by 1,147
Moons, K. G. M., Kengne, A. P., Woodward, M., Royston, P., Vergouwe, Y., Altman, D. G., & Grobbee, D. E. (2012) · Heart · doi:10.1136/heartjnl-2011-301246 · Cited by 889
The Treatment of Missing Values and its Effect on Classifier Accuracy
Acuña, E., & Rodriguez, C. (2004) · Springer eBooks · doi:10.1007/978-3-642-17103-1_60 · Cited by 576
Taylor, R. A., Pare, J. R., Venkatesh, A. K., Mowafi, H. O., Melnick, E. R., Fleischman, W. M., & Hall, M. K. (2015) · Academic Emergency Medicine · doi:10.1111/acem.12876 · Cited by 513
Self-citationDesautels, T., Calvert, J., Hoffman, J., Jay, M., Kerem, Y., Shieh, L., Shimabukuro, D., Chettipally, U., Feldman, M. D., Barton, C., et al. (2016) · JMIR Medical Informatics · doi:10.2196/medinform.5909 · Cited by 509
Hohl, C. M., Dankoff, J., Colacone, A., & Afilalo, M. (2001) · Annals of Emergency Medicine · doi:10.1067/mem.2001.119456 · Cited by 477
Understanding Why Patients Of Low Socioeconomic Status Prefer Hospitals Over Ambulatory Care
Kangovi, S., Barg, F. K., Carter, T., Long, J. A., Shannon, R., & Grande, D. (2013) · Health Affairs · doi:10.1377/hlthaff.2012.0825 · Cited by 445
Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., Dugas, A., Linton, B., Kirsch, T., & Kelen, G. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.08.005 · Cited by 424
Ford, E. M., Carroll, J. A., Smith, H. E., Scott, D., & Cassell, J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv180 · Cited by 411
Reliability and validity of scores on The Emergency Severity Index version 3
Tanabe, P., Gimbel, R., Yarnold, P. R., Kyriacou, D. N., & Adams, J. G. (2004) · Academic Emergency Medicine · doi:10.1197/j.aem.2003.06.013 · Cited by 326
Horng, S., Sontag, D. A., Halpern, Y., Jernite, Y., Shapiro, N. I., & Nathanson, L. A. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174708 · Cited by 310
Classification, Clustering, and Data Mining Applications
Banks, D., McMorris, F. R., Arabie, P., & Gaul, W. (2004) · Springer eBooks · doi:10.1007/978-3-642-17103-1 · Cited by 216
Predictive Modeling With Big Data: Is Bigger Really Better ?
de Fortuny, E. J., Martens, D., & Provost, F. J. (2013) · Big Data · doi:10.1089/big.2013.0037 · Cited by 181
Predicting the Risk of Suicide by Analyzing the Text of Clinical Notes
Poulin, C., Shiner, B. R., Thompson, P., Vepštas, L., Young‐Xu, Y., Goertzel, B., Watts, B. V., Flashman, L. A., & McAllister, T. W. (2014) · PLoS ONE · doi:10.1371/journal.pone.0085733 · Cited by 168
Sun, Y., Heng, B. H., Tay, S. Y., & Seow, E. (2011) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2011.01125.x · Cited by 154
Lucini, F. R., Fogliatto, F. S., da Silveira, G. J. C., Neyeloff, J. L., Anzanello, M. J., Kuchenbecker, R. S., & Schaan, B. D. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.01.001 · Cited by 139
Exploiting missing clinical data in Bayesian network modeling for predicting medical problems
Lin, J.-H., & Haug, P. J. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.06.001 · Cited by 138
Predicting emergency department inpatient admissions to improve same-day patient flow
Peck, J. S., Benneyan, J. C., Nightingale, D. J., & Gaehde, S. A. (2012) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2012.01435.x · Cited by 138
Sabbatini, A. K., Kocher, K. E., Basu, A., & Hsia, R. Y.-J. (2016) · JAMA · doi:10.1001/jama.2016.0649 · Cited by 136
An Electronic Emergency Triage System to Improve Patient Distribution by Critical Outcomes
Dugas, A. F., Kirsch, T. D., Toerper, M., Korley, F., Yenokyan, G., France, D., Hager, D., & Levin, S. (2016) · The Journal of Emergency Medicine · doi:10.1016/j.jemermed.2016.02.026 · Cited by 132
missForest: Nonparametric missing value imputation using random forest
Stekhoven, D. J. (2015) · Astrophysics Source Code Library · Cited by 125
Baumann, M. R., & Strout, T. D. (2007) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2006.04.011 · Cited by 121
kerasR: R Interface to the Keras Deep Learning Library
Arnold, T. (2017) · The Journal of Open Source Software · doi:10.21105/joss.00296 · Cited by 105
Approaches to Patient Health Information Exchange and Their Impact on Emergency Medicine
Shapiro, J. S., Kannry, J. L., Lipton, M., Goldberg, E., Conocenti, P., Stuard, S. S., Wyatt, B. M., & Kuperman, G. J. (2006) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2006.03.032 · Cited by 101
Arya, R., Wei, G., McCoy, J. V., Crane, J., Ohman-Strickland, P., & Eisenstein, R. M. (2013) · Academic Emergency Medicine · doi:10.1111/acem.12249 · Cited by 78
Banerjee, A., Mbamalu, D., Ebrahimi, S., Khan, A. A., & Chan, T. F. (2011) · International Journal of Emergency Medicine · doi:10.1186/1865-1380-4-22 · Cited by 75
Nationwide Frequency and Costs of Potentially Preventable Hospitalizations, 2006
Jiang, H., Russo, C. A., & Barrett, M. L. (2009) · Cited by 71
The Influence of Insurance, Race, and Gender on Emergency Department Disposition
Selassie, A. W., McCarthy, M. L., & Pickelsimer, E. E. (2003) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2003.tb00611.x · Cited by 63
The Influence of Insurance, Race, and Gender on Emergency Department Disposition
Selassie, A. W., McCarthy, M. L., & Pickelsimer, E. E. (2003) · Academic Emergency Medicine · doi:10.1197/s1069-6563(03)00497-4 · Cited by 63
Lucke, J. A., de Gelder, J., Clarijs, F., Heringhaus, C., de Craen, A. J. M., Fogteloo, A. J., Blauw, G. J., de Groot, B., & Mooijaart, S. P. (2017) · Emergency Medicine Journal · doi:10.1136/emermed-2016-205846 · Cited by 58
Early Prediction Model of Patient Hospitalization From the Pediatric Emergency Department
Barak-Corren, Y., Fine, A. M., & Reis, B. Y. (2017) · PEDIATRICS · doi:10.1542/peds.2016-2785 · Cited by 56
Barak-Corren, Y., Israelit, S. H., & Reis, B. Y. (2017) · Emergency Medicine Journal · doi:10.1136/emermed-2014-203819 · Cited by 52
Dinh, M. M., Russell, S. B., Bein, K. J., Rogers, K., Muscatello, D., Paoloni, R., Hayman, J., Chalkley, D. R., & Ivers, R. (2016) · BMC Emergency Medicine · doi:10.1186/s12873-016-0111-4 · Cited by 47
Predicting hospital admission for Emergency Department patients using a Bayesian network.
Leegon, J., Jones, I. R., Lanaghan, K., & Aronsky, D. (2005) · PubMed Central · Cited by 42
Health Information Exchange Improves Identification Of Frequent Emergency Department Users
Shapiro, J. S., Johnson, S. A., Angiolillo, J., Fleischman, W. M., Onyile, A., & Kuperman, G. J. (2013) · Health Affairs · doi:10.1377/hlthaff.2013.0167 · Cited by 42
Leegon, J., Jones, I. R., Lanaghan, K., & Aronsky, D. (2006) · Europe PMC (PubMed Central) · Cited by 33
Predicting admission at triage: are nurses better than a simple objective score?
Cameron, A., Ireland, A., McKay, G. A., Stark, A., & Lowe, D. J. (2016) · Emergency Medicine Journal · doi:10.1136/emermed-2014-204455 · Cited by 31
Ensuring the Quality of Quality Metrics for Emergency Care
Adams, J. G. (2016) · JAMA · doi:10.1001/jama.2015.19484 · Cited by 17
Comparison of Glasgow Admission Prediction Score and Amb Score in predicting need for inpatient care
Cameron, A., Jones, D., Logan, E., O'Keeffe, C. A., Mason, S. M., & Lowe, D. J. (2018) · Emergency Medicine Journal · doi:10.1136/emermed-2017-207246 · Cited by 16
Haimovich, J. S., Venkatesh, A. K., Shojaee, A., Coppi, A., Warner, F., Li, S.-X., & Krumholz, H. M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0172049 · Cited by 15
The Sydney Triage to Admission Risk Tool (START): A prospective validation study
Ebker‐White, A. A., Bein, K. J., & Dinh, M. M. Q. (2018) · Emergency Medicine Australasia · doi:10.1111/1742-6723.12940 · Cited by 14
flusight: interactive visualizations for infectious disease forecasts
Tushar, A., & Reich, N. G. (2017) · The Journal of Open Source Software · doi:10.21105/joss.00231 · Cited by 8
Triage, Machine Learning, Algorithms, and Becoming the Borg
Schenkel, S. M., & Wears, R. L. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2018.02.010 · Cited by 4
P112: Predicting patient admission from the emergency department using triage administrative data
Savage, D. W., Weaver, B., & Wood, D. (2017) · CJEM · doi:10.1017/cem.2017.314 · Cited by 2
Flagged first, then by citations
1 of the 58 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- M Kuhn, 2013, Chapter 3. Data Pre-processing. Applied Predictive Modeling, Springer Science & Business Media, 27
Cited by
Sorted by citations
Emergency department triage prediction of clinical outcomes using machine learning models
Raita, Y., Goto, T., Faridi, M. K., Brown, D., Camargo, C. A., & Hasegawa, K. (2019) · Critical Care · doi:10.1186/s13054-019-2351-7 · Cited by 556
Fernandes, M. P., Vieira, S. M., Leite, F., Palos, C., Finkelstein, S. N., & Sousa, J. M. C. (2019) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101762 · Cited by 341
Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review
Verma, S., Boonsanong, V., Hoang, M., Hines, K., Dickerson, J. P., & Shah, C. S. (2024) · ACM Computing Surveys · doi:10.1145/3677119 · Cited by 723
Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review
Verma, S., Boonsanong, V., Hoang, M., Hines, K. E., Dickerson, J. P., & Shah, C. (2020) · arXiv · doi:10.48550/arxiv.2010.10596 · Cited by 267
Application of the XGBoost Machine Learning Method in PM2.5 Prediction: A Case Study of Shanghai
Ma, J., Yu, Z., Qu, Y., Xu, J., & Cao, Y. (2019) · Aerosol and Air Quality Research · doi:10.4209/aaqr.2019.08.0408 · Cited by 242
Metadata
- Authors
- Hong, W. S., Haimovich, A. D., & Taylor, R. A.
- Affiliations
- Yale University (US)
- Corresponding author
- Taylor, R. Andrew
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-07-20
- Volume
- 13
- Issue
- 7
- Pages
- e0201016
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6054406
- OpenAlex
- W2884597820
- Semantic Scholar
- d012519a924e41aa7ff49d9b6be58033bd60fd9c
- Cited by
- 326
- References
- 58
- Citations by year
- 2026: 51 · 2025: 55 · 2024: 47 · 2023: 39 · 2022: 46 · 2021: 39 · 2020: 30 · 2019: 18 · 2018: 1
- FWCI
- 12.22
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- National Institutes of Health (1F30CA196191, T32GM007205)
- Yale School of Medicine
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- emergency department triage, hospital admission prediction, medical history, Emergency Severity Index, hospital utilization, XGBoost, deep neural network, logistic regression, feature importance, information gain, low-dimensional models, training data size, model generalization, clinical decision support
- MeSH terms
- Machine Learning, Adult, Emergency Service, Hospital, Female, Humans, Male, Middle Aged, Patient Admission, Patient Discharge, Triage, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
