The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index
Abstract
Study objective Standards for emergency department (ED) triage in the United States rely heavily on subjective assessment and are limited in their ability to risk-stratify patients. This study seeks to evaluate an electronic triage system (e-triage) based on machine learning that predicts likelihood of acute outcomes enabling improved patient differentiation.Methods A multisite, retrospective, cross-sectional study of 172,726 ED visits from urban and community EDs was conducted. E-triage is composed of a random forest model applied to triage data (vital signs, chief complaint, and active medical history) that predicts the need for critical care, an emergency procedure, and inpatient hospitalization in parallel and translates risk to triage level designations. Predicted outcomes and secondary outcomes of elevated troponin and lactate levels were evaluated and compared with the Emergency Severity Index (ESI).Results E-triage predictions had an area under the curve ranging from 0.73 to 0.92 and demonstrated equivalent or improved identification of clinical patient outcomes compared with ESI at both EDs. E-triage provided rationale for risk-based differentiation of the more than 65% of ED visits triaged to ESI level 3. Matching the ESI patient distribution for comparisons, e-triage identified more than 10% (14,326 patients) of ESI level 3 patients requiring up triage who had substantially increased risk of critical care or emergency procedure (1.7% ESI level 3 versus 6.2% up triaged) and hospitalization (18.9% versus 45.4%) across EDs.Conclusion E-triage more accurately classifies ESI level 3 patients and highlights opportunities to use predictive analytics to support triage decisionmaking. Further prospective validation is needed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In a multisite retrospective study of 172,726 ED visits, e-triage predictions had areas under the receiver operating characteristic curve ranging from 0.73 to 0.92.”
From Wearables and the Medical Revolution (2018) · cites this paper as [53]
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Chobanian, A. V. (2003) · JAMA · doi:10.1001/jama.289.19.2560 · Cited by 25,386
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106
Surviving Sepsis Campaign: International Guidelines for Management of Severe Sepsis and Septic Shock
Dellinger, R. P., Levy, M. M., Rhodes, A., Annane, D., Gerlach, H., Opal, S. M., Sevransky, J. E., Sprung, C. L., Douglas, I. S., Jaeschke, R., et al. (2013) · Critical Care Medicine · doi:10.1097/ccm.0b013e31827e83af · Cited by 9,694
Dellinger, R. P., Levy, M. M., Rhodes, A., Annane, D., Gerlach, H., Opal, S. M., Sevransky, J., Sprung, C. L., Douglas, I. S., Jaeschke, R., et al. (2013) · Intensive Care Medicine · doi:10.1007/s00134-012-2769-8 · Cited by 7,371
Third universal definition of myocardial infarction
the Writing Group on behalf of the Joint ESC/ACCF/AHA/WHF Task Force for the Universal Definition of Myocardial Infarction, Thygesen, K., Alpert, J. S., Jaffe, A. S., Simoons, M. L., CHAITMAN, B. R., & White, H. D. (2012) · Nature Reviews Cardiology · doi:10.1038/nrcardio.2012.122 · Cited by 6,585
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951
Research works printed but not linked: 1
- Gilboy N, Tanabe T, Travers D, et al. Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care, Version 4. Implementation Handbook 2012 Edition. Rockville, MD: Agency for Healthcare Research and Quality; 2012.
Cited by
Sorted by citations
Artificial Intelligence in Pharmaceutical Technology and Drug Delivery Design
Vora, L. K., Gholap, A. D., Jetha, K. L., Singh, T. R. R., Solanki, H. K., & Chavda, V. P. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15071916 · Cited by 1,193
Wearables and the Medical Revolution
Dunn, J., Runge, R., & Snyder, M. (2018) · Personalized Medicine · doi:10.2217/pme-2018-0044 · Cited by 654
Emergency department triage prediction of clinical outcomes using machine learning models
Raita, Y., Goto, T., Faridi, M. K., Brown, D., Camargo, C. A., & Hasegawa, K. (2019) · Critical Care · doi:10.1186/s13054-019-2351-7 · Cited by 556
What is Machine Learning? A Primer for the Epidemiologist
Bi, Q., Goodman, K. E., Kaminsky, J., & Lessler, J. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz189 · Cited by 579
Predicting hospital admission at emergency department triage using machine learning
Hong, W. S., Haimovich, A. D., & Taylor, R. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201016 · Cited by 326
Metadata
- Authors
- Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., Dugas, A., Linton, B., Kirsch, T., & Kelen, G.
- Affiliations
- Johns Hopkins University (US)
- University of Maryland, College Park (US)
- Uniformed Services University of the Health Sciences (US)
- Corresponding author
- Levin, Scott
- ORCID
- Venue
- Annals of Emergency Medicine
- Publisher
- Elsevier BV
- Published
- 2018-05
- Volume
- 71
- Issue
- 5
- Pages
- 565-574.e2
- ISSN
- 0196-0644 (ISSN-L)
- OpenAlex
- W2752349109
- Cited by
- 424
- References
- 52
- Citations by year
- 2026: 71 · 2025: 60 · 2024: 64 · 2023: 53 · 2022: 54 · 2021: 37 · 2020: 39 · 2019: 29 · 2018: 16
- FWCI
- 29.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of Emergency Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (R21HS023641)
- Topics
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- emergency department triage, Emergency Severity Index, risk stratification, random forest, machine learning prediction, hospitalization prediction, troponin, lactate, triage decision support
- MeSH terms
- Machine Learning, Adult, Algorithms, Cross-Sectional Studies, Emergency Service, Hospital, Female, Humans, Male, Retrospective Studies, Triage, United States, Area Under Curve, Vital Signs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
