Factors Associated With Variability in the Performance of a Proprietary Sepsis Prediction Model Across 9 Networked Hospitals in the US
Open accessAbstract
This cohort study uses data from electronic health records to assess variability in a sepsis prediction model across 9 hospitals.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Vendors increasingly provide proprietary "turnkey" CDSS solutions for identifying clinical deterioration and sepsis.
“Vendors increasingly provide proprietary “turnkey” CDSS solutions for identifying clinical deterioration and sepsis [42–44].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [42]
The Methods and Abstract (article pp. 611–612) describe the Epic Sepsis Model as a proprietary sepsis prediction model evaluated across nine hospitals, supporting it as an example of the cited type of tool. The paper does not establish that vendors increasingly provide such tools, describe them as “turnkey,” or address clinical deterioration beyond sepsis. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe Epic Sepsis Model version 1 was found to have poor discrimination and calibration in several external validations across different health systems.
“Despite its widespread deployment, this model was found to have poor discrimination and calibration in several external validations across different health systems.12,13,14,15”
From Multicenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model (2026) · cites this paper as [13]
The Results report that ESM discrimination varied across nine hospitals in one network, with a C-statistic of 0.55 at the largest hospital. The Methods describe measuring discrimination, not calibration; the Discussion only suggests that increased COVID-19-related alerting may reflect miscalibration. This paper therefore supports poor discrimination at some hospitals, but not poor calibration established by several validations across different health systems. (LLM, conf=0.87)
Experimental — accuracy not yet measuredPatient complexity and baseline sepsis rates affect the performance of sepsis prediction models.
“These factors support prior evidence that patient complexity and baseline sepsis rates affect the performance of sepsis prediction models.13”
From Multicenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model (2026) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
Self-citationThe Clinician and Dataset Shift in Artificial Intelligence
Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., & Saria, S. (2021) · New England Journal of Medicine · doi:10.1056/nejmc2104626 · Cited by 855 · see citing works
Self-citationAdams, R., Henry, K. E., Sridharan, A., Soleimani, H., Zhan, A., Rawat, N., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01894-0 · Cited by 365 · see citing works
Quantification of Sepsis Model Alerts in 24 US Hospitals Before and During the COVID-19 Pandemic
Wong, A., Cao, J., Lyons, P. G., Dutta, S., Major, V. J., Ötleş, E., & Singh, K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.35286 · Cited by 57 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [3]Tarabichi Y, Cheng A, Bar-Shain D, et al. Improving timeliness of antibiotic administration using a provider and pharmacist facing sepsis early warning system in the emergency department setting: a randomized controlled quality improvement initiative. Crit Care Med. 2022;50(3):418-427. [DOI] [PubMed] [Google Scholar]
Cited by
Sorted by citations
Impact of a deep learning sepsis prediction model on quality of care and survival
Boussina, A., Shashikumar, S. P., Malhotra, A., Owens, R. L., El-Kareh, R., Longhurst, C. A., Quintero, K., Donahue, A., Chan, T. C., Nemati, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-023-00986-6 · Cited by 170 · see citing works
Use of artificial intelligence in critical care: opportunities and obstacles
Pinsky, M. R., Bedoya, A., Bihorac, A., Celi, L., Churpek, M., Economou-Zavlanos, N. J., Elbers, P., Saria, S., Liu, V., Lyons, P. G., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04860-z · Cited by 160 · see citing works
“Also, performance variation across settings, case mix and time must be measured and addressed [42, 43].”
BackgroundSelf-citationZhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99 · see citing works
Evaluation of sepsis prediction models before onset of treatment
Kamran, F., Tjandra, D., Heiler, A., Virzi, J., Singh, K., King, J. E., Valley, T. S., & Wiens, J. (2024) · NEJM AI · doi:10.1056/aioa2300032 · Cited by 74 · see citing works
Self-citationMulticenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model
Wong, A., Currey, D., Schwinne, M., Park-Egan, B., Meyer, S., Gutting, A., Cao, J., Khan, S., Dantes, R., Pan, T., et al. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2026.0181 · Cited by 21 · see citing works
Self-citationA Systematic Review and Meta-analysis of Externally Validated Epic Clinical Decision Support Tools
Patel, H., Crusco, S., Hansen, D., Morales, J., Patale, T., Chiuzan, C., Sands, M., Naidich, J., Myers, K., Davidson, K. W., et al. (2026) · Journal of General Internal Medicine · doi:10.1007/s11606-026-10381-y · Cited by 4 · see citing works
- Authors
- Lyons, P. G., Hofford, M. R., Yu, S. C., Michelson, A. P., Payne, P. R. O., Hough, C. L., & Singh, K.
- Affiliations
- Oregon Health & Science University (US)
- Washington University in St. Louis (US)
- University of Michigan (US)
- Corresponding author
- Lyons, Patrick G.
- ORCID
- Venue
- JAMA Internal Medicine
- Publisher
- American Medical Association (AMA)
- Published
- 2023-06-01
- Volume
- 183
- Issue
- 6
- Pages
- 611
- ISSN
- 2168-6106 (ISSN-L)
- PMC
- PMC10071393
- OpenAlex
- W4362506020
- Cited by
- 62
- References
- 4
- Citations by year
- 2026: 14 · 2025: 20 · 2024: 19 · 2023: 9
- FWCI
- 12.38
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- JAMA Internal Medicine (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (UL1TR002345, KL2TR002346)
- National Center for Advancing Translational Sciences
- Topics
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- sepsis, sepsis prediction model, hospital networks, electronic health records, clinical decision support
- MeSH terms
- Patient Care, Hospitals, Humans, Prognosis, Models, Statistical, Sepsis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
