The Nature and Variability of Automated Practice Alerts Derived from Electronic Health Records in a U.S. Nationwide Critical Care Research Network
Open accessAbstract
Rationale: The nature, variability, and extent of early warning clinical practice alerts derived from automated query of electronic health records (e-alerts) currently used in acute care settings for clinical care or research is unknown. Objectives: To describe e-alerts in current use in acute care settings at medical centers participating in a nationwide critical care research network. Methods: We surveyed investigators at 38 institutions involved in the National Institutes of Health–funded Clinical Trials Network for the Prevention and Early Treatment of Acute Lung Injury (PETAL) for quantitative and qualitative analysis. Measurements and Main Results: Thirty sites completed the survey (79% response rate). All sites used electronic health record systems. Epic Systems was used at 56% of sites; the others used alternate commercially available vendors or homegrown systems. Respondents at 57% of sites represented in this survey used e-alerts. All but 1 of these 17 sites used an e-alert for early detection of sepsis-related syndromes, and 35% used an e-alert for pneumonia. E-alerts were triggered by abnormal laboratory values (37%), vital signs (37%), or radiology reports (15%) and were used about equally for clinical decision support and research. Only 59% of sites with e-alerts have evaluated them either for accuracy or for validity. Conclusions: A majority of the research network sites participating in this survey use e-alerts for early notification of potential threats to hospitalized patients; however, there was significant variability in the nature of e-alerts between institutions. Use of one common electronic health record vendor at more than half of the participating sites suggests that it may be possible to standardize e-alerts across multiple sites in research networks, particularly among sites using the same medical record platform.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
More than half of surveyed US health systems report using electronic alerts, with nearly all using an alert system for sepsis.
“More than half of surveyed US health systems report using electronic alerts, with nearly all using an alert system for sepsis.14”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Health systems and services: the role of acute care
Hirshon, J. M., Risko, N., Calvello, E. J., Stewart de Ramirez, S., Narayan, M., Theodosis, C., O’Neill, J., & for the Acute Care Research Collaborative at the University of Maryland Global Health Initiative (2013) · Bulletin of the World Health Organization · doi:10.2471/blt.12.112664 · Cited by 270 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- 1 Finlay HE, Cassorla L, Feiner J, Toy P. Designing and testing a computer-based screening system for transfusion-related acute lung injury. Am J Clin Pathol 2005;124:601 -609.
- 2 Herasevich V, Yilmaz M, Khan H, Hubmayr RD, Gajic O. Validation of an electronic surveillance system for acute lung injury. Intensive Care Med 2009;35:1018 -1023.
- 3 Kukhareva PV, Kawamoto K, Shields DE, Barfuss DT, Halley AM, Tippetts TJ, Warner PB, Bray BE, Staes CJ. Clinical decision support-based quality measurement (CDS-QM) framework: prototype implementation, evaluation, and future directions. AMIA Annu Symp Proc 2014;2014:825 -834.
- 4 Blumenthal D, Tavenner M. The ' meaningful use ' regulation for electronic health records. N Engl J Med 2010;363:501 -504.
- 6 Schmickl CN, Li M, Li G, Wetzstein MM, Herasevich V, Gajic O, Benzo RP. The accuracy and ef fi ciency of electronic screening for recruitment into a clinical trial on COPD. Respir Med 2011;105: 1501 -1506.
- 7 Thadani SR, Weng C, Bigger JT, Ennever JF, Wajngurt D. Electronic screening improves ef fi ciency in clinical trial recruitment. J Am Med Inform Assoc 2009;16:869 -873.
- 8 Embi PJ, Jain A, Clark J, Bizjack S, Hornung R, Harris CM. Effect of a clinical trial alert system on physician participation in trial recruitment. Arch Intern Med 2005;165:2272 -2277.
- 10 Herasevich V, Tsapenko M, Kojicic M, Ahmed A, Kashyap R, Venkata C, Shahjehan K, Thakur SJ, Pickering BW, Zhang J, et al . Limiting ventilator-induced lung injury through individual electronic medical record surveillance. Crit Care Med 2011;39: 34 -39. in a large U.S.-based clinical trials network. Therefore, bias in the favor of larger, academic institutions is possible. The use of e-alerts may differ in other settings. Another limitation is the use of a mixed-methods survey. Because the data were qualitative, it is possible that information on individualized e-alerts designed for speci fi c locations or patient/disease populations within an institution may not have been identi fi ed. Finally, this was an investigation of the use of e-alerts and does not have the capability to determine the impact of e-alerts on the clinical care or research.
Cited by
Sorted by citations
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
Implementing Artificial Intelligence in Critical Care Medicine: a consensus of 22.
Cecconi, M., Greco, M., Shickel, B., Angus, D. C., Bailey, H., Bignami, E., Calandra, T., Celi, L. A., Einav, S., Elbers, P., et al. (2025) · Critical Care · doi:10.1186/s13054-025-05532-2 · Cited by 80 · see citing works
“Networking refers to collaborative research consortia [76] that align protocols and pool clinical research data across institutions.”
Self-citationMulticenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model
Wong, A., Currey, D., Schwinne, M., Park-Egan, B., Meyer, S., Gutting, A., Cao, J., Khan, S., Dantes, R., Pan, T., et al. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2026.0181 · Cited by 21 · see citing works
- Authors
- Benthin, C., Pannu, S., Khan, A., & Gong, M.
- Affiliations
- Oregon Health & Science University (US)
- The Ohio State University Wexner Medical Center (US)
- Albert Einstein College of Medicine (US)
- Corresponding author
- Khan, Akram
- Venue
- Annals of the American Thoracic Society
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-10-01
- Volume
- 13
- Issue
- 10
- Pages
- 1784-1788
- ISSN
- 2325-6621 (ISSN-L)
- 2329-6933
- PMC
- PMC5122489
- OpenAlex
- W2461415823
- Cited by
- 18
- References
- 1
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 1 · 2023: 1 · 2022: 2 · 2021: 2 · 2020: 2 · 2019: 2 · 2018: 2 · 2017: 1
- FWCI
- 2.57
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Annals of the American Thoracic Society (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (RR024140)
- Oregon Clinical and Translational Research Institute (UL1RR024140, RR024140)
- National Heart, Lung, and Blood Institute (HL123020, HL123018, U01HL122989, HL123023, U01HL123009, U01HL123008, U01HL123018, U01HL123027, U01HL123031, HL123004, U01HL123023, U01HL123010, HL123009, HL123027, HL122998, HL123010, HL123033, U01HL122998, U01HL123022, HL123008, U01HL123020, U01HL123004, HL122989, HL123022, HL123031, U01HL123033)
- National Center for Research Resources (UL1RR024140)
- National Center for Advancing Translational Sciences
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Keywords
- electronic health record alerts, automated alerts, early warning systems, sepsis detection, pneumonia detection, clinical decision support, acute care, petals, Epic Systems
- MeSH terms
- Clinical Decision-Making, Critical Care, Humans, Pneumonia, Surveys and Questionnaires, Respiratory Distress Syndrome, United States, Randomized Controlled Trials as Topic, Sepsis, Decision Support Systems, Clinical, Biomedical Research, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review