Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index
Abstract
Study objective Standards for emergency department (ED) triage in the United States rely heavily on subjective assessment and are limited in their ability to risk-stratify patients. This study seeks to evaluate an electronic triage system (e-triage) based on machine learning that predicts likelihood of acute outcomes enabling improved patient differentiation.Methods A multisite, retrospective, cross-sectional study of 172,726 ED visits from urban and community EDs was conducted. E-triage is composed of a random forest model applied to triage data (vital signs, chief complaint, and active medical history) that predicts the need for critical care, an emergency procedure, and inpatient hospitalization in parallel and translates risk to triage level designations. Predicted outcomes and secondary outcomes of elevated troponin and lactate levels were evaluated and compared with the Emergency Severity Index (ESI).Results E-triage predictions had an area under the curve ranging from 0.73 to 0.92 and demonstrated equivalent or improved identification of clinical patient outcomes compared with ESI at both EDs. E-triage provided rationale for risk-based differentiation of the more than 65% of ED visits triaged to ESI level 3. Matching the ESI patient distribution for comparisons, e-triage identified more than 10% (14,326 patients) of ESI level 3 patients requiring up triage who had substantially increased risk of critical care or emergency procedure (1.7% ESI level 3 versus 6.2% up triaged) and hospitalization (18.9% versus 45.4%) across EDs.Conclusion E-triage more accurately classifies ESI level 3 patients and highlights opportunities to use predictive analytics to support triage decisionmaking. Further prospective validation is needed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In a multisite retrospective study of 172,726 ED visits, e-triage predictions had areas under the receiver operating characteristic curve ranging from 0.73 to 0.92.
“In a multisite retrospective study of 172,726 ED visits, e-triage predictions had areas under the receiver operating characteristic curve ranging from 0.73 to 0.92.”
From Wearables and the Medical Revolution (2018) · cites this paper as [53]
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106 · see citing works
Surviving Sepsis Campaign: International Guidelines for Management of Severe Sepsis and Septic Shock
Dellinger, R. P., Levy, M. M., Rhodes, A., Annane, D., Gerlach, H., Opal, S. M., Sevransky, J. E., Sprung, C. L., Douglas, I. S., Jaeschke, R., et al. (2013) · Critical Care Medicine · doi:10.1097/ccm.0b013e31827e83af · Cited by 9,694 · see citing works
Dellinger, R. P., Levy, M. M., Rhodes, A., Annane, D., Gerlach, H., Opal, S. M., Sevransky, J., Sprung, C. L., Douglas, I. S., Jaeschke, R., et al. (2013) · Intensive Care Medicine · doi:10.1007/s00134-012-2769-8 · Cited by 7,282 · see citing works
Third universal definition of myocardial infarction
the Writing Group on behalf of the Joint ESC/ACCF/AHA/WHF Task Force for the Universal Definition of Myocardial Infarction, Thygesen, K., Alpert, J. S., Jaffe, A. S., Simoons, M. L., CHAITMAN, B. R., & White, H. D. (2012) · Nature Reviews Cardiology · doi:10.1038/nrcardio.2012.122 · Cited by 6,583 · see citing works
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · see citing works
Some Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443 · see citing works
Rokach, L. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-009-9124-7 · Cited by 2,657 · see citing works
Third Universal Definition of Myocardial Infarction
Thygesen, K. A., Alpert, J. S., Jaffe, A. S., Simoons, M. L., CHAITMAN, B. R., White, H. D., Thygesen, K. A., Alpert, J. S., White, H. D., Jaffe, A. S., et al. (2012) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.08.001 · Cited by 2,619 · see citing works
The Effect of Emergency Department Crowding on Clinically Oriented Outcomes
Bernstein, S. L., Aronsky, D., Duseja, R., Epstein, S., Handel, D., Hwang, U., McCarthy, M., John McConnell, K., Pines, J. M., Rathlev, N., et al. (2009) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2008.00295.x · Cited by 1,147 · see citing works
Chalfin, D. B., Trzeciak, S., Likourezos, A., Baumann, B. M., & Dellinger, R. P. (2007) · Critical Care Medicine · doi:10.1097/01.ccm.0000266585.74905.5a · Cited by 1,022 · see citing works
Early stroke treatment associated with better outcome
Marler, J. R., Tilley, B. C., Lü, M., Brott, T. G., Lyden, P., Grotta, J. C., Broderick, J. P., Levine, S. R., Frankel, M., Horowitz, S. H., et al. (2000) · Neurology · doi:10.1212/wnl.55.11.1649 · Cited by 856 · see citing works
National Hospital Ambulatory Medical Care Survey: 2019 Emergency Department Summary Tables
Cairns, C., & Kang, K. (2022) · doi:10.15620/cdc:115748 · Cited by 725 · see citing works
A roadmap for national action on clinical decision support
Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2334 · Cited by 707 · see citing works
(1998) · Machine Learning · doi:10.1023/a:1017181826899 · Cited by 638 · see citing works
Reliability and Validity of a New Five‐level Triage Instrument
Wuerz, R. C., Milne, L. W., Eitel, D. R., Travers, D., & Gilboy, N. (2000) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2000.tb01066.x · Cited by 554 · see citing works
Emergency Department Crowding Is Associated With Poor Care for Patients With Severe Pain
Pines, J. M., & Hollander, J. E. (2007) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2007.07.008 · Cited by 515 · see citing works
Modern Triage in the Emergency Department
Christ, M., Grossmann, F. F., Winter, D., Bingisser, R. M., & Platz, E. (2010) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2010.0892 · Cited by 511 · see citing works
Emergency department crowding and thrombolysis delays in acute myocardial infarction
Schull, M. J., Vermeulen, M. J., Slaughter, G., Morrison, L. J., & Daly, P. (2004) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2004.05.004 · Cited by 402 · see citing works
Austin, P. C., Tu, J. V., Ho, J. E., Levy, D. A., & Lee, D. S. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.11.008 · Cited by 340 · see citing works
Optimizing Emergency Department Front-End Operations
Wiler, J. L., Gentle, C., Halfpenny, J. M., Heins, A., Mehrotra, A., Mikhail, M. G., & Fite, D. (2010) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2009.05.021 · Cited by 339 · see citing works
The Emergency Severity Index Triage Algorithm Version 2 Is Reliable and Valid
Eitel, D. R., Travers, D. A., Rosenau, A. M., Gilboy, N., & Wuerz, R. C. (2003) · Academic Emergency Medicine · doi:10.1197/s1069-6563(03)00350-6 · Cited by 291 · see citing works
Pines, J. M., Localio, A. R., Hollander, J. E., Baxt, W. G., Lee, H., Phillips, C., & Metlay, J. P. (2007) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2007.07.021 · Cited by 287 · see citing works
Fee, C. R., Weber, E. J., Maak, C. A., & Bacchetti, P. (2007) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2007.08.003 · Cited by 257 · see citing works
Shapiro, N. I., Trzeciak, S., Hollander, J. E., Birkhahn, R., Otero, R., Osborn, T. M., Moretti, E., Nguyen, H. B., Gunnerson, K. J., MILZMAN, D. P., et al. (2008) · Critical Care Medicine · doi:10.1097/ccm.0b013e318192fd9d · Cited by 240 · see citing works
Pines, J. M., Hollander, J. E., Localio, A. R., & Metlay, J. P. (2006) · Academic Emergency Medicine · doi:10.1197/j.aem.2006.03.568 · Cited by 231 · see citing works
Barfod, C., Lauritzen, M. M. P., Danker, J. K., Sölétormos, G., Forberg, J. L., Berlac, P. A., Lippert, F. K., Lundstrøm, L. H., Antonsen, K., & Lange, K. H. W. (2012) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/1757-7241-20-28 · Cited by 194 · see citing works
Khot, U. N., Johnson, M. L., Ramsey, C., Khot, M. B., Todd, R., Shaikh, S. R., & Berg, W. J. (2007) · Circulation · doi:10.1161/circulationaha.106.677401 · Cited by 167 · see citing works
McHugh, M., Tanabe, P. J., McClelland, M. S., & Khare, R. K. (2011) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2011.01240.x · Cited by 159 · see citing works
Rincón, F., Mayer, S. A., Rivolta, J., Stillman, J. I., Boden-Albala, B. M., Elkind, M. S. V., Marshall, R. S., & Chong, J. Y. (2010) · Neurocritical Care · doi:10.1007/s12028-010-9347-0 · Cited by 139 · see citing works
An Electronic Emergency Triage System to Improve Patient Distribution by Critical Outcomes
Dugas, A. F., Kirsch, T. D., Toerper, M., Korley, F., Yenokyan, G., France, D., Hager, D., & Levin, S. (2016) · The Journal of Emergency Medicine · doi:10.1016/j.jemermed.2016.02.026 · Cited by 132 · see citing works
Self-citationThe impact of a delay in intensive care unit admission for community-acquired pneumonia
Phua, J., Ngerng, W. J., & Lim, T. K. (2010) · European Respiratory Journal · doi:10.1183/09031936.00154209 · Cited by 104 · see citing works
Carpenter, C. R., Raja, A. S., & Brown, M. D. (2015) · Academic Emergency Medicine · doi:10.1111/acem.12820 · Cited by 99 · see citing works
Reliability of Computerized Emergency Triage
Dong, S. L., Bullard, M. J., Meurer, D. P., Blitz, S., Ohinmaa, A. E., Holroyd, B. R., & Rowe, B. H. (2006) · Academic Emergency Medicine · doi:10.1197/j.aem.2005.10.014 · Cited by 95 · see citing works
Mapping Out the Emergency Department Disposition Decision for High-Acuity Patients
Calder, L., Forster, A. J., Stiell, I. G., Carr, L. K., Perry, J. J., Vaillancourt, C., & Brehaut, J. (2012) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2012.04.013 · Cited by 86 · see citing works
Kulstad, E. B., & Kelley, K. M. (2009) · International Journal of Emergency Medicine · doi:10.1007/s12245-009-0107-x · Cited by 81 · see citing works
Triage: an investigation of the process and potential vulnerabilities
Hitchcock, M., Gillespie, B. M., Crilly, J. L., & Chaboyer, W. P. (2013) · Journal of Advanced Nursing · doi:10.1111/jan.12304 · Cited by 79 · see citing works
Arya, R., Wei, G., McCoy, J. V., Crane, J., Ohman-Strickland, P., & Eisenstein, R. M. (2013) · Academic Emergency Medicine · doi:10.1111/acem.12249 · Cited by 78 · see citing works
Yurkova, I., & Wolf, L. A. (2011) · Journal of Emergency Nursing · doi:10.1016/j.jen.2011.01.016 · Cited by 72 · see citing works
Undertriage in the Manchester triage system: an assessment of severity and options for improvement
Seiger, N., van Veen, M., Steyerberg, E. W., Ruige, M., van Meurs, A. H. J., & Moll, H. A. (2011) · Archives of Disease in Childhood · doi:10.1136/adc.2010.206797 · Cited by 60 · see citing works
Placing Physician Orders at Triage: The Effect on Length of Stay
Russ, S. E., Jones, I. R., Aronsky, D., Dittus, R. S., & Slovis, C. M. (2010) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2010.02.006 · Cited by 58 · see citing works
Intas, G., Stergiannis, P. I., Chalari, E., Tsoumakas, K., & Fildissis, G. (2012) · Advanced Emergency Nursing Journal · doi:10.1097/tme.0b013e318251515f · Cited by 47 · see citing works
Principles of Anatomy and Physiology, 13th Edition
(2014) · Medicine & Science in Sports & Exercise · doi:10.1249/01.mss.0000446521.17381.6e · Cited by 39 · see citing works
Emergency Department Abnormal Vital Sign “Triggers” Program Improves Time to Therapy
McGillicuddy, D. C., O’Connell, F., Shapiro, N. I., Calder, S. A., Mottley, L., Roberts, J. C., & Sánchez, L. D. (2011) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2011.01056.x · Cited by 37 · see citing works
An integrated computerized triage system in the emergency department.
Aronsky, D., Jones, I. R., Raines, B., Hemphill, R. R., Mayberry, S. R., Luther, M. A., & Slusser, T. (2008) · PubMed Central · Cited by 31 · see citing works
Elliott, D. J., Williams, K. D., Wu, P., Kher, H. V., Michalec, B., Reinbold, N., Coletti, C., Patel, B., & Dressler, R. M. (2015) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(15)41071-2 · Cited by 24 · see citing works
The effects of computerized triage on nurse work behavior.
Levin, S., France, D. J., Mayberry, R. S., Stonemetz, S., Jones, I. R., & Aronsky, D. (2006) · Europe PMC (PubMed Central) · Cited by 14 · see citing works
Self-citationFrance, D. J., Levin, S., Ding, R., Hemphill, R. R., Han, J. H., Russ, S. E., Aronsky, D., & Weinger, M. (2016) · Journal of Patient Safety · doi:10.1097/pts.0000000000000242 · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Artificial Intelligence in Pharmaceutical Technology and Drug Delivery Design
Vora, L. K., Gholap, A. D., Jetha, K. L., Singh, T. R. R., Solanki, H. K., & Chavda, V. P. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15071916 · Cited by 1,152 · see citing works
Wearables and the Medical Revolution
Dunn, J., Runge, R., & Snyder, M. (2018) · Personalized Medicine · doi:10.2217/pme-2018-0044 · Cited by 654 · see citing works
Emergency department triage prediction of clinical outcomes using machine learning models
Raita, Y., Goto, T., Faridi, M. K., Brown, D., Camargo, C. A., & Hasegawa, K. (2019) · Critical Care · doi:10.1186/s13054-019-2351-7 · Cited by 531 · see citing works
What is Machine Learning? A Primer for the Epidemiologist
Bi, Q., Goodman, K. E., Kaminsky, J., & Lessler, J. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz189 · Cited by 522 · see citing works
Predicting hospital admission at emergency department triage using machine learning
Hong, W. S., Haimovich, A. D., & Taylor, R. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201016 · Cited by 326 · see citing works
Fernandes, M. P., Vieira, S. M., Leite, F., Palos, C., Finkelstein, S. N., & Sousa, J. M. C. (2019) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101762 · Cited by 314 · see citing works
Triage Performance in Emergency Medicine: A Systematic Review
Hinson, J. S., Martínez, D. A., Cabral, S., George, K., Whalen, M. M., Hansoti, B., & Levin, S. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2018.09.022 · Cited by 298 · see citing works
Self-citationGoto, T., Camargo, C. A., Faridi, M. K., Freishtat, R. J., & Hasegawa, K. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.6937 · Cited by 285 · see citing works
Natural language processing in medicine: A review
Locke, S., Bashall, A., Al-Adely, S., Moore, J., Wilson, A. J., & Kitchen, G. B. (2021) · Trends in Anaesthesia and Critical Care · doi:10.1016/j.tacc.2021.02.007 · Cited by 252 · see citing works
Ellahham, S., Ellahham, N., & Simsekler, M. C. E. (2020) · American Journal of Medical Quality · doi:10.1177/1062860619878515 · Cited by 177 · see citing works
Bridging the implementation gap of machine learning in healthcare
Seneviratne, M. G., Shah, N. H., & Chu, L. (2020) · BMJ Innovations · doi:10.1136/bmjinnov-2019-000359 · Cited by 173 · see citing works
A Path for Translation of Machine Learning Products into Healthcare Delivery
(2020) · EMJ Innovations · doi:10.33590/emjinnov/19-00172 · Cited by 172 · see citing works
Artificial intelligence and machine learning in emergency medicine
Stewart, J. E., Sprivulis, P., & Dwivedi, G. (2018) · Emergency Medicine Australasia · doi:10.1111/1742-6723.13145 · Cited by 168 · see citing works
Machine learning approaches for predicting disposition of asthma and COPD exacerbations in the ED
Goto, T., Camargo, C. A., Faridi, M. K., Yun, B. J., & Hasegawa, K. (2018) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2018.06.062 · Cited by 164 · see citing works
Mistry, B., de Ramirez, S. S., Kelen, G. D., Schmitz, P. S. K., Balhara, K. S., Levin, S., Martínez, D. A., Psoter, K. J., Anton, X. P., & Hinson, J. S. (2017) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.09.036 · Cited by 160 · see citing works
Self-citationAn explainable machine learning framework for lung cancer hospital length of stay prediction
Alsinglawi, B. S., Alshari, O., Alorjani, M. S., Mubin, O., Alnajjar, F. S. K., Novoa, M., & Darwish, O. (2022) · Scientific Reports · doi:10.1038/s41598-021-04608-7 · Cited by 155 · see citing works
Soltan, A. A. S., Kouchaki, S., Zhu, T., Kiyasseh, D., Taylor, T., Hussain, Z. B., Peto, T., Brent, A. J., Eyre, D. W., & Clifton, D. A. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30274-0 · Cited by 150 · see citing works
Chenais, G., Lagarde, E., & Gil-Jardiné, C. (2023) · Journal of Medical Internet Research · doi:10.2196/40031 · Cited by 142 · see citing works
A review of triage accuracy and future direction
Tam, H. L., Chung, S. F., & Lou, C. K. (2018) · BMC Emergency Medicine · doi:10.1186/s12873-018-0215-0 · Cited by 141 · see citing works
Parakh, A., Lee, H., Lee, J. H., Eisner, B. H., Sahani, D. V., & Do, S. (2019) · Radiology Artificial Intelligence · doi:10.1148/ryai.2019180066 · Cited by 135 · see citing works
Klug, M., Barash, Y., Bechler, S., Resheff, Y. S., Tron, T., Ironi, A., Soffer, S., Zimlichman, E., & Klang, E. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05512-7 · Cited by 120 · see citing works
Machine learning methods applied to triage in emergency services: A systematic review
Sánchez-Salmerón, R., Gómez‐Urquiza, J. L., Albendín‐García, L., Correa‐Rodríguez, M., Martos-Cabrera, M. B., Velando‐Soriano, A., & Suleiman‐Martos, N. (2021) · International Emergency Nursing · doi:10.1016/j.ienj.2021.101109 · Cited by 117 · see citing works
Sterling, N. W., Patzer, R. E., Di, M., & Schrager, J. D. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.06.008 · Cited by 112 · see citing works
Sax, D. R., Warton, E. M., Mark, D. G., Vinson, D. R., Kene, M. V., Ballard, D. W., Vitale-McDowell, T. J., McGaughey, K. R., Beardsley, A., Pines, J. M., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.3404 · Cited by 113 · see citing works
How artificial intelligence could transform emergency department operations
Berlyand, Y., Raja, A. S., Dorner, S. C., Prabhakar, A. M., Sonis, J. D., Gottumukkala, R. V., Succi, M. D., & Yun, B. J. (2018) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2018.01.017 · Cited by 115 · see citing works
Porto, B. M. (2024) · BMC Emergency Medicine · doi:10.1186/s12873-024-01135-2 · Cited by 124 · see citing works
Machine learning to assist clinical decision-making during the COVID-19 pandemic
Debnath, S., Barnaby, D. P., Coppa, K., Makhnevich, A., Kim, E. J., Chatterjee, S., Tóth, V., Levy, T. J., Paradis, M. D., Cohen, S. L., et al. (2020) · Bioelectronic Medicine · doi:10.1186/s42234-020-00050-8 · Cited by 104 · see citing works
Ivanov, O., Wolf, L. A., Brecher, D., Lewis, E., Masek, K., Montgomery, K., Andrieiev, Y., McLaughlin, M., Liu, S., Dunne, R., et al. (2020) · Journal of Emergency Nursing · doi:10.1016/j.jen.2020.11.001 · Cited by 110 · see citing works
Applications of Machine Learning Approaches in Emergency Medicine; a Review Article.
Shafaf, N., & Malek, H. (2019) · Archives of Academic Emergency Medicine · doi:10.22037/aaem.v7i1.410 · Cited by 97 · see citing works
Artificial intelligence in emergency radiology: A review of applications and possibilities
Katzman, B. D., van der Pol, C. B., Soyer, P., & Patlas, M. N. (2023) · Diagnostic and Interventional Imaging · doi:10.1016/j.diii.2022.07.005 · Cited by 97 · see citing works
Validation of deep-learning-based triage and acuity score using a large national dataset
Kwon, J., Lee, Y., Lee, Y., Lee, S., Park, H., & Park, J. (2018) · PLoS ONE · doi:10.1371/journal.pone.0205836 · Cited by 93 · see citing works
Mothilal, R. K., Mahajan, D., Tan, C., & Sharma, A. (2020) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462597 · Cited by 157 · see citing works
Boonstra, A., & Laven, M. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08070-7 · Cited by 87 · see citing works
Human intelligence versus Chat-GPT: who performs better in correctly classifying patients in triage?
Zaboli, A., Brigo, F., Sibilio, S., Mian, M., & Turcato, G. (2024) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2024.02.008 · Cited by 85 · see citing works
Fernandes, M. P., Mendes, R., Vieira, S. M., Leite, F., Palos, C., Johnson, A. E. W., Finkelstein, S. N., Horng, S., & Celi, L. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229331 · Cited by 84 · see citing works
Jiang, H., Mao, H., Lu, H., Lin, P., Garry, W., Lu, H., Yang, G., Rainer, T. H., & Chen, X. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104326 · Cited by 83 · see citing works
Schwartz, J., Moy, A. J., Rossetti, S. C., Elhadad, N., & Cato, K. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa296 · Cited by 79 · see citing works
Artificial Intelligence and Machine Learning in Emergency Medicine
Tang, K. J. W., Ang, C. K. E., Constantinides, T., Rajinikanth, V., Acharya, U. R., & Cheong, K. H. (2020) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2020.12.002 · Cited by 83 · see citing works
Li, D., Xiao, C., Liu, Y., Chen, Z., Hassan, H., Su, L., Liu, J., Li, H., Xie, W., Zhong, W., et al. (2022) · Diagnostics · doi:10.3390/diagnostics12081788 · Cited by 71 · see citing works
Salman, O. H., Taha, Z. K., Alsabah, M. Q., Hussein, Y. S., Mohammed, A. S., & Aal-Nouman, M. I. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106357 · Cited by 81 · see citing works
Choi, S. W., Ko, T., Hong, K. J., & Kim, K. H. (2019) · Healthcare Informatics Research · doi:10.4258/hir.2019.25.4.305 · Cited by 80 · see citing works
Exploring the Role of Artificial Intelligence in an Emergency and Trauma Radiology Department
Jalal, S., Parker, W., Ferguson, D., & Nicolaou, S. (2020) · Canadian Association of Radiologists Journal · doi:10.1177/0846537120918338 · Cited by 80 · see citing works
Xie, F., Zhou, J. R., Lee, J. W., Tan, M., Li, S., Rajnthern, L. S. O., Chee, M. L., Chakraborty, B., Wong, A.-K. I., Dagan, A. S., et al. (2022) · Scientific Data · doi:10.1038/s41597-022-01782-9 · Cited by 84 · see citing works
Miles, J., Turner, J., Jacques, R., Williams, J., & Mason, S. M. (2020) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00084-1 · Cited by 77 · see citing works
A validation of machine learning-based risk scores in the prehospital setting
Spangler, D., Hermansson, T., Smékal, D., & Blomberg, H. (2019) · PLoS ONE · doi:10.1371/journal.pone.0226518 · Cited by 72 · see citing works
Errors in nurse-led triage: An observational study
Ausserhofer, D., Zaboli, A., Pfeifer, N., Solazzo, P., Magnarelli, G., Marsoner, T., Siller, M., & Turcato, G. (2020) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2020.103788 · Cited by 71 · see citing works
Feng, Y., Huang, X., & Sester, M. (2022) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2022.2048835 · Cited by 73 · see citing works
Roquette, B. P., Nagano, H., Marujo, E. C., & Maiorano, A. C. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.03.012 · Cited by 70 · see citing works
“Just” accuracy? Procedural fairness demands explainability in AI-based medical resource allocations
Rueda, J., Delgado, J., Jounou, I. P., Hortal-Carmona, J., Ausín, T., & Rodríguez‐Arias, D. (2022) · AI & Society · doi:10.1007/s00146-022-01614-9 · Cited by 73 · see citing works
Machine learning techniques for mortality prediction in emergency departments: a systematic review
Naemi, A., Schmidt, T., Mansourvar, M., Naghavi‐Behzad, M., Ebrahimi, A., & Wiil, U. K. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-052663 · Cited by 67 · see citing works
The future of artificial intelligence in medicine: Medical-legal considerations for health leaders
Jassar, S., Adams, S., Zarzeczny, A., & Burbridge, B. (2022) · Healthcare Management Forum · doi:10.1177/08404704221082069 · Cited by 65 · see citing works
Kareemi, H., Vaillancourt, C., Rosenberg, H., Fournier, K., & Yadav, K. (2020) · Academic Emergency Medicine · doi:10.1111/acem.14190 · Cited by 61 · see citing works
How artificial intelligence could transform emergency care
Kachman, M. M., Brennan, I., Oskvarek, J. J., Waseem, T. C., & Pines, J. M. (2024) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2024.04.024 · Cited by 62 · see citing works
Machine learning and artificial intelligence: applications in healthcare epidemiology
Hamilton, A. J., Strauss, A. T., Martinez, D. A., Hinson, J. S., Levin, S., Lin, G., & Klein, E. Y. (2021) · Antimicrobial Stewardship & Healthcare Epidemiology · doi:10.1017/ash.2021.192 · Cited by 58 · see citing works
Self-citationMartínez, D. A., Levin, S., Klein, E. Y., Parikh, C. R., Menez, S. P., Taylor, R. A., & Hinson, J. S. (2020) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2020.05.026 · Cited by 61 · see citing works
Self-citationImpact of Artificial Intelligence–Based Triage Decision Support on Emergency Department Care
Taylor, R. A., Chmura, C., Hinson, J. S., Steinhart, B., Sangal, R. B., Venkatesh, A. K., Xu, H., Cohen, I., Faustino, I. V., & Levin, S. (2025) · NEJM AI · doi:10.1056/aioa2400296 · Cited by 63 · see citing works
Self-citationJoseph, J. W., Leventhal, E. L., Grossestreuer, A. V., Wong, M. L., Joseph, L. J., Nathanson, L. A., Donnino, M. W., Elhadad, N., & Sanchez, L. D. (2020) · doi:10.1101/2020.05.02.20089052 · Cited by 57 · see citing works
Sarno, L. D., Caroselli, A., Tonin, G., Graglia, B., Pansini, V., Causio, F. A., Gatto, A., & Chiaretti, A. (2024) · Biomedicines · doi:10.3390/biomedicines12061220 · Cited by 58 · see citing works
Machine Learning and Initial Nursing Assessment-Based Triage System for Emergency Department
Yu, J. Y., Jeong, G., Jeong, O. S., Chang, D. K., & Cha, W. C. (2020) · Healthcare Informatics Research · doi:10.4258/hir.2020.26.1.13 · Cited by 52 · see citing works
Choi, A., Choi, S. Y., Chung, K. S., Chung, H. S., Song, T., Choi, B., & Kim, J. H. (2023) · Scientific Reports · doi:10.1038/s41598-023-35617-3 · Cited by 51 · see citing works
Machine Learning in Relation to Emergency Medicine Clinical and Operational Scenarios: An Overview
Lee, S., Mohr, N. M., Street, W., & Nadkarni, P. M. (2019) · Western Journal of Emergency Medicine · doi:10.5811/westjem.2019.1.41244 · Cited by 50 · see citing works
Applications of natural language processing at emergency department triage: A narrative review
Stewart, J. E., Lü, J., Goudie, A., Arendts, G., Meka, S. A., Freeman, S., Walker, K., Sprivulis, P., Sanfilippo, F. M., Bennamoun, M., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0279953 · Cited by 52 · see citing works
Artificial Intelligence in Emergency Medicine: Benefits, Risks, and Recommendations
Vearrier, L., Derse, A. R., Basford, J. B., Larkin, G. L., & Moskop, J. C. (2022) · The Journal of Emergency Medicine · doi:10.1016/j.jemermed.2022.01.001 · Cited by 49 · see citing works
Elhaj, H., Achour, N., Tania, M. H., & Açıksarı, K. (2023) · Array · doi:10.1016/j.array.2023.100281 · Cited by 48 · see citing works
The aspects of running artificial intelligence in emergency care; a scoping review
(2023) · Archives of Academic Emergency Medicine · doi:10.22037/aaem.v11i1.1974 · Cited by 45 · see citing works
Benda, N. C., Das, L. T., Abramson, E. L., Blackburn, K., Thoman, A., Kaushal, R., Zhang, Y., & Ancker, J. S. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa021 · Cited by 45 · see citing works
Fernandes, M. P., Mendes, R., Vieira, S. M., Leite, F., Palos, C., Johnson, A. E. W., Finkelstein, S. N., Horng, S., & Celi, L. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0230876 · Cited by 45 · see citing works
Kaieski, N., da Costa, C. A., Righi, R. D. R., Lora, P. S., & Eskofier, B. M. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106612 · Cited by 45 · see citing works
Artificial Intelligence in Critical Care
Mathur, P., & Burns, M. L. (2019) · International Anesthesiology Clinics · doi:10.1097/aia.0000000000000221 · Cited by 45 · see citing works
Wang, B., Li, W., Bradlow, A., Bazuaye, E., & Chan, A. T. Y. (2022) · Decision Support Systems · doi:10.1016/j.dss.2022.113899 · Cited by 42 · see citing works
Chan, S. L., Lee, J. W., Ong, M. E. H., Siddiqui, F. J., Graves, N., Ho, A. F. W., & Liu, N. (2023) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2023.02.001 · Cited by 42 · see citing works
Yun, H., Choi, J., & Park, J. H. (2021) · JMIR Medical Informatics · doi:10.2196/30770 · Cited by 45 · see citing works
Sills, M. R., Ozkaynak, M., & Jang, H. A. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104468 · Cited by 41 · see citing works
Using Machine Learning Techniques to Predict Hospital Admission at the Emergency Department
Feretzakis, G., Karlis, G., Loupelis, E., Kalles, D., Chatzikyriakou, R., Τράκας, Ν., Karakou, E., Sakagianni, A., Tzelves, L. I., Petropoulou, S., et al. (2022) · The Journal of Critical Care Medicine · doi:10.2478/jccm-2022-0003 · Cited by 40 · see citing works
Machine learning-based prediction of critical illness in children visiting the emergency department
Hwang, S., & Lee, B. (2022) · PLoS ONE · doi:10.1371/journal.pone.0264184 · Cited by 40 · see citing works
Race and Ethnicity and Primary Language in Emergency Department Triage
Joseph, J. W., Kennedy, M., Landry, A., Marsh, R. H., Baymon, D. E., Im, D. D., Chen, P. C., Samuels‐Kalow, M. E., Nentwich, L. M., Elhadad, N., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.37557 · Cited by 40 · see citing works
Hunter-Zinck, H., Peck, J. S., Strout, T. D., & Gaehde, S. A. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz171 · Cited by 39 · see citing works
Carlile, M., Hurt, B., Hsiao, A., Hogarth, M. A., Longhurst, C., & Dameff, C. J. (2020) · Journal of the American College of Emergency Physicians Open · doi:10.1002/emp2.12297 · Cited by 37 · see citing works
Cheng, C., Chen, C.-C., Cheng, F., Chen, H., Su, Y.-S., Yeh, C., Chung, I., & Liao, C. (2020) · JMIR Medical Informatics · doi:10.2196/19416 · Cited by 38 · see citing works
Prediction across healthcare settings: a case study in predicting emergency department disposition
Barak‐Corren, Y., Chaudhari, P. P., Perniciaro, J. L., Waltzman, M. L., Fine, A. M., & Reis, B. Y. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00537-x · Cited by 37 · see citing works
Liu, Y., Gao, J., Liu, J., Walline, J. H., Liu, X., Zhang, T., Wu, Y., Wu, J. Y., Zhu, H., & Zhu, W. (2021) · Scientific Reports · doi:10.1038/s41598-021-03104-2 · Cited by 36 · see citing works
Logothetis, S. B., Green, D., Holland, M., & Moubayed, N. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-40661-0 · Cited by 37 · see citing works
Liu, N., Xie, F., Siddiqui, F. J., Ho, A. F. W., Chakraborty, B., Nadarajan, G. D., Tan, K. B. K., & Ong, M. E. H. (2021) · JMIR Research Protocols · doi:10.2196/34201 · Cited by 35 · see citing works
Chang, Y.-H., Lin, Y.-C., Huang, F.-W., Chen, D.-M., Chung, Y.-T., Chen, W.-K., & Wang, C. C. N. (2024) · BMC Emergency Medicine · doi:10.1186/s12873-024-01152-1 · Cited by 35 · see citing works
Bulstra, A. E. J., Buijze, G. A., Bulstra, A. E. J., Cohen, A., Colaris, J. W., Court-Brown, C. M., Doornberg, J. N., Duckworth, A. D., Goslings, J. C., Gray, A. J., et al. (2022) · The Journal Of Hand Surgery · doi:10.1016/j.jhsa.2022.02.023 · Cited by 34 · see citing works
Machine learning based early mortality prediction in the emergency department
Li, C., Zhang, Z., Ren, Y., Nie, H., Lei, Y., Qiu, H., Xu, Z., & Pu, X. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104570 · Cited by 33 · see citing works
Challenges in evaluating the accuracy of AI-containing digital triage systems: A systematic review
Ilicki, J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0279636 · Cited by 33 · see citing works
Emergency Severity Index Version 4 and Triage of Pediatric Emergency Department Patients
Sax, D. R., Warton, E. M., Kene, M. V., Ballard, D. W., Vitale-McDowell, T. J., Timm, J. A., Adams, E. S., McGauhey, K. R., Pines, J. M., Reed, M., et al. (2024) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2024.2671 · Cited by 33 · see citing works
Shin, Y., Lee, M., Lee, Y. K., Kim, K., & Kim, T. J. (2025) · Life · doi:10.3390/life15040654 · Cited by 32 · see citing works
Hinson, J. S., Klein, E., Smith, A., Toerper, M., Dungarani, T., Hager, D., Hill, P., Kelen, G., Niforatos, J. D., Stephens, R. S., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00646-1 · Cited by 30 · see citing works
Self-citationHinson, J. S., Levin, S., Steinhart, B. D., Chmura, C., Sangal, R. B., Venkatesh, A. K., & Taylor, R. A. (2024) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2024.10.014 · Cited by 30 · see citing works
Self-citationJiang, W., Siddiqui, S., Barnes, S., Barouch, L. A., Korley, F. K., Martínez, D. A., Toerper, M. F., Cabral, S., Hamrock, E., & Levin, S. (2019) · JMIR Medical Informatics · doi:10.2196/14756 · Cited by 30 · see citing works
Self-citationMachine learning for healthcare that matters: Reorienting from technical novelty to equitable impact
Balagopalan, A., Baldini, I., Celi, L. A., Gichoya, J. W., McCoy, L. G., Naumann, T., Shalit, U., van der Schaar, M., & Wagstaff, K. L. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000474 · Cited by 31 · see citing works
Yazaki, M., Maki, S., Furuya, T., Inoue, K., Nagai, K., Nagashima, Y., Maruyama, J., Toki, Y., Kitagawa, K., Iwata, S., et al. (2025) · Prehospital Emergency Care · doi:10.1080/10903127.2024.2374400 · Cited by 29 · see citing works
Chang, Y.-H., Shih, H.-M., Wu, J.-E., Huang, F.-W., Chen, W.-K., Chen, D.-M., Chung, Y.-T., & Wang, C. C. N. (2022) · BMC Emergency Medicine · doi:10.1186/s12873-022-00632-6 · Cited by 29 · see citing works
Chang, H., Yu, J. Y., Yoon, S., Kim, T., & Cha, W. C. (2022) · Scientific Reports · doi:10.1038/s41598-022-14422-4 · Cited by 29 · see citing works
Sociodemographic Disparities in Queue Jumping for Emergency Department Care
Sangal, R. B., Su, H., Khidir, H., Parwani, V. L., Liebhardt, B., Pinker, E. J., Meng, L., Venkatesh, A. K., & Ulrich, A. S. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.26338 · Cited by 29 · see citing works
Ahmed, A., Ashour, O. M., Ali, H., & Firouz, M. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117314 · Cited by 31 · see citing works
Zhang, X., Bellolio, F., Medrano−Gracia, P., Werys, K., Yang, S., & Mahajan, P. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-1006-6 · Cited by 28 · see citing works
Nguyen, M., Corbin, C. K., Eulalio, T. Y., Ostberg, N. P., Machiraju, G., Marafino, B. J., Baiocchi, M., Rose, C. C., & Chen, J. H. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab118 · Cited by 29 · see citing works
- Authors
- Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., Dugas, A., Linton, B., Kirsch, T., & Kelen, G.
- Affiliations
- Johns Hopkins University (US)
- University of Maryland, College Park (US)
- Uniformed Services University of the Health Sciences (US)
- Corresponding author
- Levin, Scott
- ORCID
- Venue
- Annals of Emergency Medicine
- Publisher
- Elsevier BV
- Published
- 2018-05
- Volume
- 71
- Issue
- 5
- Pages
- 565-574.e2
- ISSN
- 0196-0644 (ISSN-L)
- OpenAlex
- W2752349109
- Cited by
- 424
- References
- 52
- Citations by year
- 2026: 71 · 2025: 60 · 2024: 64 · 2023: 53 · 2022: 54 · 2021: 37 · 2020: 39 · 2019: 29 · 2018: 16 · 2017: 1
- FWCI
- 29.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of Emergency Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (R21HS023641)
- Topics
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- emergency department triage, Emergency Severity Index, risk stratification, random forest, machine learning prediction, hospitalization prediction, troponin, lactate, triage decision support
- MeSH terms
- Machine Learning, Adult, Algorithms, Cross-Sectional Studies, Emergency Service, Hospital, Female, Humans, Male, Retrospective Studies, Triage, United States, Area Under Curve, Vital Signs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
