Cited by 33Electronic Health Records SystemsHealthcare Technology and Patient MonitoringPersonal Information Management and User Behavior

Evaluating the impact of interruptive alerts within a health system: use, response time, and cumulative time burden

Open access
Elias, Pierre, Peterson, Eric, Wachter, Bob, Ward, Cary, Poon, Eric, Navar, Ann Marie·2019·Applied Clinical Informatics

Abstract

Background Health systems often employ interruptive alerts through the electronic health record to improve patient care. However, concerns of “alert fatigue” have been raised, highlighting the importance of understanding the time burden and impact of these alerts on providers. Objectives Our main objective was to determine the total time providers spent on interruptive alerts in both inpatient and outpatient settings. Our secondary objectives were to analyze dwell time for individual alerts and examine both provider and alert-related factors associated with dwell time variance. Methods We retrospectively evaluated use and response to the 75 most common interruptive (“popup”) alerts between June 1st, 2015 and November 1st, 2016 in a large academic health care system. Alert “dwell time” was calculated as the time between the alert appearing on a provider's screen until it was closed. The total number of alerts and dwell times per provider per month was calculated for inpatient and outpatient alerts and compared across alert type. Results The median number of alerts seen by a provider was 12 per month (IQR 4–34). Overall, 67% of inpatient and 39% of outpatient alerts were closed in under 3 seconds. Alerts related to patient safety and those requiring more than a single click to proceed had significantly longer median dwell times of 5.2 and 6.7 seconds, respectively. The median total monthly time spent by providers viewing alerts was 49 seconds on inpatient alerts and 28 seconds on outpatient alerts. Conclusion Most alerts were closed in under 3 seconds and a provider's total time spent on alerts was less than 1 min/mo. Alert fatigue may lie in their interruptive and noncritical nature rather than time burden. Monitoring alert interaction time can function as a valuable metric to assess the impact of alerts on workflow and potentially identify routinely ignored alerts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1055/s-0039-1700869

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • At a large academic healthcare system, interruptive alerts fired 53,106 times per month in the inpatient setting, representing 73.6% of all alerts fired in the system and equivalent to approximately 71 hours of total provider effort per month.

    “Another study showed that at a large academic healthcare system, interruptive alerts fired 53,106 times per month in the inpatient setting, representing 73.6% of all alerts fired in the system and was equivalent to approximately 71 hours of total provider effort per month.”

    From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [14]

    The Results report that 73.6% of alerts occurred during inpatient encounters. Table 1 gives 53,103 inpatient alerts per month, close to the claimed 53,106, but reports 3,524 minutes of inpatient provider time per month—about 59 hours, not 71. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 16

  1. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998;280(15):1311-1316
  2. Baysari MT, Tariq A, Day RO, Westbrook JI. Alert override as a habitual behavior-a new perspective on a persistent problem. J Am Med Inform Assoc 2017;24(02):409-412
  3. Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from "inaccurate warning" overrides in the intensive care unit. Appl Clin Inform 2018;9(02):268-274
  4. Meslin SMM, Zheng WY, Day RO, Tay EMY, Baysari MT. Evaluation of clinical relevance of drug-drug interaction alerts prior to implementation. Appl Clin Inform 2018;9(04):849-855
  5. Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017;8(03):910-923
  6. Payne TH, Hines LE, Chan RC, et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc 2015;22(06):1243-1250
  7. Ojeleye O, Avery A, Gupta V, Boyd M. The evidence for the effectiveness of safety alerts in electronic patient medication record systems at the point of pharmacy order entry: a systematic review. BMC Med Inform Decis Mak 2013;13(01):69
  8. Weingart SN, Simchowitz B, Shiman L, et al. Clinicians' assess- ments of electronic medication safety alerts in ambulatory care. Arch Intern Med 2009;169(17):1627-1632
  9. Phansalkar S, van der Sijs H, Tucker AD, et al. Drug-drug inter- actions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20(03):489-493
  10. Dexheimer JW, Kirkendall ES, Kouril M, et al. The effects of medication alerts on prescriber response in a pediatric hospital. Appl Clin Inform 2017;8(02):491-501
  11. Makam AN, Lanham HJ, Batchelor K, et al. Use and satisfaction with key functions of a common commercial electronic health record: a survey of primary care providers. BMC Med Inform Decis Mak 2013;13(01):86
  12. Varghese P, Wright A, Andersen JM, Yoshida EI, Bates DW. Clinical decision support: the experience at Brigham and women's hospi- tal/partners healthcare. In: Clinical Decision Support Systems. New York, NY: Springer Nature; 2016:227-244
  13. Kuperman GJ, Bobb A, Payne TH, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14(01):29-40 19
  14. Gregory ME, Russo E, Singh H. Electronic health record alert- related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017;8(03):686-697
  15. Murphy DR, Meyer AN, Russo E, Sittig DF, Wei L, Singh H. Burden of inbox notifications in commercial electronic health records. JAMA Intern Med 2016;176(04):559-560
  16. Dombrowski W, Clay B, O'Brien JD. Electronic health records quantify previously existing phenomenon. JAMA Intern Med 2016;176(08):1234-1235

Cited by

Sorted by citations

Authors
Elias, P., Peterson, E., Wachter, B., Ward, C., Poon, E., & Navar, A. M.
Affiliations
  • Columbia University (US)
  • Duke University (US)
  • University of California, San Francisco (US)
  • Duke Clinical Research Institute, Duke University, Durham, North Carolina, United States
Venue
Applied Clinical Informatics
Published
2019-10
Volume
10
Issue
05
Pages
909-917
ISSN
1869-0327 (ISSN-L)
OpenAlex
W2991215871
Cited by
33
References
2
Citations by year
2026: 6 · 2025: 5 · 2024: 4 · 2023: 5 · 2022: 11 · 2020: 2
FWCI
3.54
Type
journal article
Language
en
Open access
Locations
  • Applied Clinical Informatics (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
  • Social Sciences › Decision Sciences › Information Systems and Management › Personal Information Management and User Behavior
Keywords
interruptive alerts, alarm fatigue, electronic health records, dwell time, alert response time, patient safety alerts, clinical workflow
MeSH terms
Health Personnel, Humans, Time Factors, Medical Order Entry Systems, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review