Validation and comparison of triage-based screening strategies for sepsis
Open accessAbstract
Objective This study sought to externally validate and compare proposed methods for stratifying sepsis risk at emergency department (ED) triage.Methods This nested case/control study enrolled ED patients from four hospitals in Utah and evaluated the performance of previously-published sepsis risk scores amenable to use at ED triage based on their area under the precision-recall curve (AUPRC, which balances positive predictive value and sensitivity) and area under the receiver operator characteristic curve (AUROC, which balances sensitivity and specificity). Score performance for predicting whether patients met Sepsis-3 criteria in the ED was compared to patients' assigned ED triage score (Canadian Triage Acuity Score [CTAS]) with adjustment for multiple comparisons.Results Among 2000 case/control patients, 981 met Sepsis-3 criteria on final adjudication. The best performing sepsis risk scores were the Predict Sepsis version #3 (AUPRC 0.183, 95 % CI 0.148-0.256; AUROC 0.859, 95 % CI 0.843-0.875) and Borelli scores (AUPRC 0.127, 95 % CI 0.107-0.160, AUROC 0.845, 95 % CI 0.829-0.862), which significantly outperformed CTAS (AUPRC 0.038, 95 % CI 0.035-0.042, AUROC 0.650, 95 % CI 0.628-0.671, p < 0.001 for all AUPRC and AUROC comparisons). The Predict Sepsis and Borelli scores exhibited sensitivity of 0.670 and 0.678 and specificity of 0.902 and 0.834, respectively, at their recommended cutoff values and outperformed Systemic Inflammatory Response Syndrome (SIRS) criteria (AUPRC 0.083, 95 % CI 0.070-0.102, p = 0.052 and p = 0.078, respectively; AUROC 0.775, 95 % CI 0.756-0.795, p < 0.001 for both scores).Conclusions The Predict Sepsis and Borelli scores exhibited improved performance including increased specificity and positive predictive values for sepsis identification at ED triage compared to CTAS and SIRS criteria.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Compared with similar previous studies, our AUPRC results demonstrate more advantages.
“Additionally, compared with similar previous studies16,24 that was reported AUPRC 0.15 to 0.183, our AUPRC results demonstrate more advantages.”
From Development and validation of a screening tool for sepsis without laboratory results in the emergency department: a machine learning study (2025) · cites this paper as [24]
The Results report an AUPRC of 0.183 for Predict Sepsis, supporting the cited upper value. They do not report 0.15 as a point estimate; 0.148 is the lower bound of the 95% confidence interval for the 0.183 estimate. The citing authors’ conclusion that their own results have an advantage is not a finding attributed to this source. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250 · see citing works
Bone, R. C., BALK, R. A., Cerra, F. B., Dellinger, R. P., Fein, A. M., Knaus, W. A., Schein, R. M. H., & Sibbald, W. J. (1992) · CHEST Journal · doi:10.1378/chest.101.6.1644 · Cited by 13,192 · see citing works
Seymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., Rubenfeld, G., Kahn, J. M., Shankar-Hari, M., Singer, M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0288 · Cited by 3,939 · see citing works
Time to treatment and mortality during mandated emergency care for sepsis.
Seymour, C. W., Gesten, F., Prescott, H. C., Friedrich, M. E., Iwashyna, T. J., Phillips, G. S., Lemeshow, S., Osborn, T., Terry, K. M., & Levy, M. M. (2017) · New England Journal of Medicine · doi:10.1056/nejmoa1703058 · Cited by 2,225 · see citing works
Validation of a modified Early Warning Score in medical admissions
Subbe, C. P. (2001) · QJM · doi:10.1093/qjmed/94.10.521 · Cited by 1,767 · see citing works
Adjusted P-Values for Simultaneous Inference
Wright, S. P. (1992) · Biometrics · doi:10.2307/2532694 · Cited by 1,026 · see citing works
Paoli, C. J., Reynolds, M. A., Sinha, M., Gitlin, M. D., & Crouser, E. D. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003342 · Cited by 746 · see citing works
Collins, G. S., Ogundimu, E. O., & Altman, D. G. (2016) · Statistics in Medicine · doi:10.1002/sim.6787 · Cited by 665 · see citing works
Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) Guidelines 2016
Bullard, M. J., Musgrave, E., Warren, D., Unger, B., Skeldon, T., Grierson, R., van der Linde, E., & Swain, J. (2017) · Canadian Journal of Emergency Medicine · doi:10.1017/cem.2017.365 · Cited by 497 · see citing works
Minimum sample size for external validation of a clinical prediction model with a binary outcome
Riley, R. D., Debray, T. P. A., Collins, G. S., Archer, L., Ensor, J., van Smeden, M., & Snell, K. I. E. (2021) · Statistics in Medicine · doi:10.1002/sim.9025 · Cited by 400 · see citing works
Ozenne, B., Subtil, F., & Maucort-Boulch, D. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.02.010 · Cited by 379 · see citing works
Olsson, T., Terént, A., & Lind, L. (2004) · Journal of Internal Medicine · doi:10.1111/j.1365-2796.2004.01321.x · Cited by 320 · see citing works
Performance of triage systems in emergency care: a systematic review and meta-analysis
Zachariasse, J. M., van der Hagen, V., Seiger, N., Mackway-Jones, K. C., van Veen, M., & Moll, H. A. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026471 · Cited by 320 · see citing works
The Importance of External Validity
Steckler, A., & McLeroy, K. R. (2007) · American Journal of Public Health · doi:10.2105/ajph.2007.126847 · Cited by 295 · see citing works
Usman, O. A., Usman, A. A., & Ward, M. A. (2018) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2018.10.058 · Cited by 276 · see citing works
Leisman, D. E., Harhay, M. O., Lederer, D. J., Abramson, M., Adjei, A. A., Bakker, J., Ballas, Z. K., Barreiro, E., Bell, S. C., Bellomo, R., et al. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004246 · Cited by 271 · see citing works
ED door-to-antibiotic time and long-term mortality in sepsis.
Peltan, I. D., Brown, S. M., Bledsoe, J. R., Sorensen, J., Samore, M. H., Allen, T. L., & Hough, C. L. (2019) · Chest · doi:10.1016/j.chest.2019.02.008 · Cited by 243 · see citing works
Self-citationRevisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) adult guidelines
Bullard, M. J., Unger, B., Spence, J. M., Grafstein, E., & the CTAS National Working Group (2008) · Canadian Journal of Emergency Medicine · doi:10.1017/s1481803500009854 · Cited by 234 · see citing works
Delahanty, R. J., Alvarez, J., Flynn, L. M., Sherwin, R. L., & Jones, S. S. (2019) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2018.11.036 · Cited by 206 · see citing works
Minimum sample size for external validation of a clinical prediction model with a continuous outcome
Archer, L., Snell, K. I. E., Ensor, J., Hudda, M. T., Collins, G. S., & Riley, R. D. (2021) · Statistics in Medicine · doi:10.1002/sim.8766 · Cited by 198 · see citing works
McNarry, A. F., & Goldhill, D. R. (2003) · Anaesthesia · doi:10.1111/j.1365-2044.2004.03526.x · Cited by 191 · see citing works
Validation of the National Early Warning Score in the prehospital setting
Silcock, D. J., Corfield, A. R., Gowens, P., & Rooney, K. (2015) · Resuscitation · doi:10.1016/j.resuscitation.2014.12.029 · Cited by 187 · see citing works
Alam, N., Vegting, I. L., Houben, E. N. G., van Berkel, B., Vaughan, L., Kramer, M. H. H., & Nanayakkara, P. W. B. (2015) · Resuscitation · doi:10.1016/j.resuscitation.2015.02.011 · Cited by 179 · see citing works
IDSA Sepsis Task Force, Kalil, A. C., Gilbert, D. N., Winslow, D. L., Masur, H., & Klompas, M. (2017) · Clinical Infectious Diseases · doi:10.1093/cid/cix997 · Cited by 178 · see citing works
Development, implementation, and impact of an automated early warning and response system for sepsis
Umscheid, C. A., Betesh, J., VanZandbergen, C., Hanish, A., Tait, G., Mikkelsen, M. E., French, B. C., & Fuchs, B. D. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2259 · Cited by 167 · see citing works
Revised National Estimates of Emergency Department Visits for Sepsis in the United States*
Wang, H. E., Jones, A. R., & Donnelly, J. P. (2017) · Critical Care Medicine · doi:10.1097/ccm.0000000000002538 · Cited by 145 · see citing works
A quality improvement project to improve early sepsis care in the emergency department
Gatewood, M., Wemple, M. L., Greco, S. A., Kritek, P., & Durvasula, R. V. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003552 · Cited by 128 · see citing works
Talmor, D. S., Jones, A. E., Rubinson, L., Howell, M. D., & Shapiro, N. I. (2007) · Critical Care Medicine · doi:10.1097/01.ccm.0000262385.95721.cc · Cited by 117 · see citing works
Building a Comprehensive Clinical Information System from Components
Narus, S. P., Huff, S. M., Pryor, T. A., Haug, P. J., Larkin, T., Matney, S. A., Evans, R. S., Rocha, B. H., Bowes, W., Holston, F. T., et al. (2003) · Methods of Information in Medicine · doi:10.1055/s-0038-1634203 · Cited by 109 · see citing works
Bayer, O., Schwarzkopf, D., Stumme, C., Stacke, A., Hartog, C. S., Hohenstein, C., Kabisch, B., Reichel, J., Reinhart, K., & Winning, J. (2015) · Academic Emergency Medicine · doi:10.1111/acem.12707 · Cited by 105 · see citing works
An approach to antibiotic treatment in patients with sepsis
Martínez, M. L., Plata-Menchaca, E. P., Ruiz‐Rodríguez, J. C., & Ferrer, R. (2020) · Journal of Thoracic Disease · doi:10.21037/jtd.2020.01.47 · Cited by 105 · see citing works
Emergency Department Crowding Is Associated With Delayed Antibiotics for Sepsis
Peltan, I. D., Bledsoe, J. R., Oniki, T. A., Sorensen, J., Jephson, A. R., Allen, T. L., Samore, M. H., Hough, C. L., & Brown, S. M. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2018.10.007 · Cited by 87 · see citing works
Self-citationExternal validity, generalisability, applicability and directness: a brief primer
Murad, M. H., Katabi, A., Benkhadra, R., & Montori, V. M. (2018) · BMJ Evidence-Based Medicine · doi:10.1136/ebmed-2017-110800 · Cited by 82 · see citing works
Identification of adult septic patients in the prehospital setting
Wallgren, U. M., Castrén, M. K., Svensson, A., & Kurland, L. (2013) · European Journal of Emergency Medicine · doi:10.1097/mej.0000000000000084 · Cited by 80 · see citing works
A prehospital screening tool utilizing end-tidal carbon dioxide predicts sepsis and severe sepsis
Hunter, C. L., Silvestri, S., Ralls, G. A., Stone, A. M., Walker, A., & Papa, L. (2016) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2016.01.017 · Cited by 63 · see citing works
Ruangsomboon, O., Boonmee, P., Limsuwat, C., Chakorn, T., & Monsomboon, A. (2021) · BMC Emergency Medicine · doi:10.1186/s12873-020-00396-x · Cited by 49 · see citing works
Predictive validity comparison of two five-level triage acuity scales
Worster, A., Fernandes, C. M. B., Eva, K. W., & Upadhye, S. (2007) · European Journal of Emergency Medicine · doi:10.1097/mej.0b013e3280adc956 · Cited by 48 · see citing works
Screening strategies to identify sepsis in the prehospital setting: a validation study
Lane, D. J., Wunsch, H., Saskin, R., Cheskes, S., Lin, S., Morrison, L. J., & Scales, D. C. (2020) · Canadian Medical Association Journal · doi:10.1503/cmaj.190966 · Cited by 40 · see citing works
Wallgren, U. M., Sjölin, J., Järnbert‐Pettersson, H., & Kurland, L. (2020) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-020-00745-6 · Cited by 40 · see citing works
Nieves-Ortega, R., Rosin, C., Bingisser, R. M., & Nickel, C. H. (2019) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2019.06.036 · Cited by 39 · see citing works
Hooper, G. A., Klippel, C., McLean, S. R., Stenehjem, E. A., Webb, B. J., Murnin, E., Hough, C. L., Bledsoe, J. R., Brown, S. M., & Peltan, I. D. (2023) · Clinical Infectious Diseases · doi:10.1093/cid/ciad101 · Cited by 38 · see citing works
Self-citationChallenges and Opportunities for Emergency Department Sepsis Screening at Triage
Filbin, M. R., Thorsen, J. E., Lynch, J., Gillingham, T. D., Pasakarnis, C. L., Capp, R., Shapiro, N. I., Mooncai, T., Hou, P. C., Heldt, T., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-29427-1 · Cited by 35 · see citing works
Rare Events in the ICU: An Emerging Challenge in Classification and Prediction
Leisman, D. E. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000002943 · Cited by 30 · see citing works
Durr, D., Niemi, T., Despraz, J., Tusgul, S., Dami, F., Akrour, R., Carron, P., Pogam, M. L., Calandra, T., & Meylan, S. (2022) · Antibiotics · doi:10.3390/antibiotics11111518 · Cited by 30 · see citing works
Mellhammar, L., Linder, A., Tverring, J., Christensson, B., Boyd, J. H., Åkesson, P., & Kahn, F. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229210 · Cited by 29 · see citing works
Skitch, S., Tam, B., Xu, M. K., McInnis, L., Vu, A., & Fox‐Robichaud, A. (2017) · Canadian Journal of Emergency Medicine · doi:10.1017/cem.2017.21 · Cited by 27 · see citing works
Early recognition of sepsis through emergency medical services pre-hospital screening
Borrelli, G., Koch, E., Sterk, E. J., Lovett, S., & Rech, M. A. (2018) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2018.10.036 · Cited by 27 · see citing works
Trends in False-Positive Code Sepsis Activations in the Emergency Department
Taylor, S. P., Rozario, N. L., Kowalkowski, M. A., Gunn, E., Ellerman, J., Heffner, A. C., Gibbs, M. A., & Taylor, B. (2019) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201910-757rl · Cited by 22 · see citing works
Zaboli, A., Ausserhofer, D., Pfeifer, N., Solazzo, P., Magnarelli, G., Siller, M., & Turcato, G. (2020) · Journal of Critical Care · doi:10.1016/j.jcrc.2020.05.019 · Cited by 21 · see citing works
Brann, F., Sterling, N. W., Frisch, S. O., & Schrager, J. D. (2023) · JMIR AI · doi:10.2196/49784 · Cited by 21 · see citing works
Smyth, M. A., Gallacher, D., Kimani, P., Ragoo, M., Ward, M., & Perkins, G. D. (2019) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-019-0642-2 · Cited by 12 · see citing works
Comparison of an ED triage sepsis screening tool and qSOFA in identifying CMS SEP-1 patients
Sterk, E. J., Hyun, B. H., & Rech, M. A. (2020) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2020.06.030 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Jiang, S., Dai, S., Li, Y., Zhou, X., Jiang, C., Tian, C., Yuan, Y., Li, C., & Zhao, Y. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.103048 · Cited by 18 · see citing works
“Additionally, compared with similar previous studies 16,24 that was reported AUPRC 0.15 to 0.183, our AUPRC results demonstrate more advantages.”
InfluentialPhysician Variation in Early Sepsis Management
Peltan, I. D., Groat, D., Butler, J., Bledsoe, J. R., Ofori-Atta, B. S., Wu, C., Presson, A. P., Greene, T. H., Christensen, M. A., Schroeder, F. M., et al. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.56945 · Cited by 4 · see citing works
Self-citationPerformance Evaluation of Prehospital Sepsis Prediction Models
Peltan, I. D., Rahmati, K., Bledsoe, J. R., Yoneoka, Y., Alvarez, F., Plendl, M., Taillac, P. P., Youngquist, S. T., Samore, M. M., Hough, C. L., et al. (2025) · Critical Care Medicine · doi:10.1097/ccm.0000000000006586 · Cited by 3 · see citing works
Self-citationAlsabri, M., Elkarargy, M. A., Ata, I. M., Khalifa, M., Hamid, A. K., Elsnhory, A. B., Abdelwahab, O. A., & Urbon, S. C. (2025) · BMC Emergency Medicine · doi:10.1186/s12873-025-01347-0 · Cited by 2 · see citing works
- Authors
- Rahmati, K., Brown, S. M., Bledsoe, J. R., Passey, P., Taillac, P. P., Youngquist, S. T., Samore, M. M., Hough, C. L., & Peltan, I. D.
- Affiliations
- University of California, Los Angeles (US)
- University of Utah (US)
- Intermountain Medical Center (US)
- University of Washington (US)
- Corresponding author
- Peltan, Ithan D.
- ORCID
- Venue
- The American Journal of Emergency Medicine
- Publisher
- Elsevier BV
- Published
- 2024-11
- Volume
- 85
- Pages
- 140-147
- ISSN
- 0735-6757 (ISSN-L)
- PMC
- PMC11525104
- OpenAlex
- W4402177388
- Cited by
- 4
- References
- 52
- Citations by year
- 2026: 1 · 2025: 3
- FWCI
- 1.22
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (am, 314 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Locations
- The American Journal of Emergency Medicine (acceptedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- Funders
- Intermountain Research and Medical Foundation
- National Institutes of Health
- National Institute of General Medical Sciences (K23GM129661, R35GM151147)
- Topics
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Health Sciences › Medicine › Emergency Medicine › Emergency and Acute Care Studies
- Life Sciences › Immunology and Microbiology › Immunology › Inflammation biomarkers and pathways
- Keywords
- sepsis, emergency department triage, Sepsis-3 criteria, systemic inflammatory response syndrome, risk stratification, external validation, sepsis screening, sensitivity and specificity, precision-recall curve, receiver operating characteristic
- MeSH terms
- Adult, Aged, Emergency Service, Hospital, Female, Humans, Male, Mass Screening, Middle Aged, ROC Curve, Sensitivity and Specificity, Triage, Utah, Case-Control Studies, Risk Assessment, Sepsis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
