Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index; 18 of the 56 sources its claims cite were checked for support.

Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system

Open access
with the HITEC Investigators, Ancker, Jessica S., Edwards, Alison, Nosal, Sarah, Hauser, Diane, Mauer, Elizabeth, Kaushal, Rainu · 2017 · BMC Medical Informatics and Decision Making

Abstract

Background: Although alert fatigue is blamed for high override rates in contemporary clinical decision support systems, the concept of alert fatigue is poorly defined. We tested hypotheses arising from two possible alert fatigue mechanisms: (A) cognitive overload associated with amount of work, complexity of work, and effort distinguishing informative from uninformative alerts, and (B) desensitization from repeated exposure to the same alert over time. Methods: Retrospective cohort study using electronic health record data (both drug alerts and clinical practice reminders) from January 2010 through June 2013 from 112 ambulatory primary care clinicians. Results: On average, one-quarter of drug alerts received by a primary care clinician, and one-third of clinical reminders, were repeats for the same patient within the same year. Alert acceptance was associated with work complexity and repeated alerts, but not with the amount of work. Likelihood of reminder acceptance dropped by 30% for each additional reminder received per encounter, and by 10% for each five reminders received per day.

Overview

CoChat Summary

TL;DR

In a retrospective cohort of 112 primary care clinicians, alert acceptance fell with greater work complexity and more repeated alerts, but not with amount of workload or desensitization over time. Reducing within-patient repeat alerts may be a promising target for reducing alert fatigue.

Key findings

  • Alert acceptance decreased with repeated alerts and complexity: for best-practice advisories (BPAs), each additional BPA per encounter reduced acceptance by 30% (IRR 0.70, p < .001), and each 5-percentage-point increase in repeated BPAs reduced acceptance by 10% (IRR 0.90, p < .001).
  • Workload amount was not associated with acceptance: patients per year and encounters per year were not significant predictors. Patient comorbidity was associated with lower BPA acceptance in bivariate analysis (IRR 0.27, p = .008) but narrowly lost significance in the multivariable model (IRR 0.49, p = .06).
  • No desensitization pattern was found: among 6 newly deployed BPAs, only 1 showed an early peak followed by decline (slopes +0.16 to −0.34; p = .03 to .24), likely due to flu seasonality; the others had no clear temporal pattern.
  • Nurse practitioners were more than 4 times as likely as physicians to accept drug alerts (multivariable IRR 4.56, p = .002; bivariate IRR 3.97, p = .002). Repeated alerts were highly sticky: if the first BPA was overridden, 87.9% of subsequent instances were overridden; if the first was accepted, 51.9% of subsequent instances were overridden. For drug alerts, the corresponding override rates were 99.9% and 58.4%.

Methods

  • Retrospective cohort using EHR data from January 2010 through June 2013 at the Institute for Family Health, including 112 ambulatory primary care clinicians (93 physicians, 19 nurse practitioners), 430,803 encounters, 99,649 patients, 1,266,325 BPAs, and 326,203 drug-drug interaction/drug-allergy interaction alerts.
  • Clinician-level negative binomial models with an offset of log(number of alerts fired) and cluster-robust standard errors; predictors included clinician type, meaningful use attestation, sex, patients per year, encounters per year, alerts per encounter, patient comorbidity, percent within-patient repeats, order sets used, and labs ordered.
  • Desensitization was tested on 6 newly deployed BPAs available continuously for ≥6 months and presented to ≥10 clinicians ≥5 times each; monthly acceptance rates for 46 clinicians were graphed with lowess smoothing and modeled with Poisson, zero-inflated Poisson, negative binomial, or zero-inflated negative binomial models selected by likelihood ratio and Vuong tests.

Limitations

  • The paper states it was observational, used novel metrics because no validated metrics existed, and relied on a single commercial EHR with interruptive alerts, limiting generalizability. It had no DDI/DAI severity information and no manual review to distinguish appropriate from inappropriate overrides; repeats were assumed to be potentially uninformative, though relevance depends on the clinical situation.
  • The paper also states that secondary EHR data lacked ambient noise, interruptions, time pressure, quality of patient-clinician interaction, and multitasking, likely biasing findings toward the null; high alert volume may have produced ceiling effects; and desensitization models used robust rather than cluster-robust errors.
  • Evident from the text: drug alert acceptance was <1%, creating floor effects that weakened DDI/DAI analyses, and the clinician-level observational design cannot establish causality.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12911-017-0430-8

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 The average acceptance rate across clinicians over time for a specified alert show no clear pattern that would suggest desensitization over time, with the potential exception of the flu immunization alert (see text for interpretation). The curves represent the average acceptance rates (lowess smoothed). Each dot represents one clinician’s acceptance rate for the alert during one month; these are displayed to demonstrate that the alerts included in this analysis had different numbers of clinicians and different numbers of months. The zeros on the x-axis indicate the initial month that the alert was deployed
Fig. 1 The average acceptance rate across clinicians over time for a specified alert show no clear pattern that would suggest desensitization over time, with the potential exception of the flu immunization alert (see text for interpretation). The curves represent the average acceptance rates (lowess smoothed). Each dot represents one clinician’s acceptance rate for the alert during one month; these are displayed to demonstrate that the alerts included in this analysis had different numbers of clinicians and different numbers of months. The zeros on the x-axis indicate the initial month that the alert was deployedCC BY · PubMed Central

Verification

Claims the paper makes

11 Valid · 0 Not valid · 17 Not checked · 46 Own results and interpretations

29 more claims

Create a free account to see all 74 claims and to verify the 3 references still open.

See all claims — free account
  • Probably not a substantial claim. 27
  • It may state two separate claims as one. 14
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 43 references

Cited by

See all 709

Metadata

Authors
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R.
Affiliations
  • Weill Cornell Medicine (US)
  • Institute for Family Health (US)
Corresponding author
Ancker, Jessica S.
Venue
BMC Medical Informatics and Decision Making
Published
2017-04-10
Volume
17
Issue
1
Pages
36-36
ISSN
1472-6947 (ISSN-L)
OpenAlex
W2605753964
Cited by
709
References
43
Citations by year
2026: 160 · 2025: 87 · 2024: 76 · 2023: 82 · 2022: 90 · 2021: 74 · 2020: 73 · 2019: 38 · 2018: 26 · 2017: 2
FWCI
52.24
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • New York State Department of Health (C025877)
  • Agency for Healthcare Research and Quality (K01HS021531)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords
alarm fatigue, clinical decision support, drug alerts, alert override, cognitive overload, workload, job complexity, desensitization, nurse practitioners, primary care providers, electronic health records
MeSH terms
Alert Fatigue, Health Personnel, Adult, Humans, Nurse Practitioners, Retrospective Studies, Workload, Decision Support Systems, Clinical, Electronic Health Records, Physicians, Primary Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. https://app.cochat.ai/papers/10.1186/s12911-017-0430-8
The paper
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017). Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-017-0430-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.