Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study
Open accessAbstract
OBJECTIVE: Inadequate participant recruitment is a major problem facing clinical research. Recent studies have demonstrated that electronic health record (EHR)-based, point-of-care, clinical trial alerts (CTA) can improve participant recruitment to certain clinical research studies. Despite their promise, much remains to be learned about the use of CTAs. Our objective was to study whether repeated exposure to such alerts leads to declining user responsiveness and to characterize its extent if present to better inform future CTA deployments. METHODS: During a 36-week study period, we systematically documented the response patterns of 178 physician users randomized to receive CTAs for an ongoing clinical trial. Data were collected on: (1) response rates to the CTA; and (2) referral rates per physician, per time unit. Variables of interest were offset by the log of the total number of alerts received by that physician during that time period, in a Poisson regression. RESULTS: Response rates demonstrated a significant downward trend across time, with response rates decreasing by 2.7% for each advancing time period, significantly different from zero (flat) (p<0.0001). Even after 36 weeks, response rates remained in the 30%-40% range. Subgroup analyses revealed differences between community-based versus university-based physicians (p=0.0489). DISCUSSION: CTA responsiveness declined gradually over prolonged exposure, although it remained reasonably high even after 36 weeks of exposure. There were also notable differences between community-based versus university-based users. CONCLUSIONS: These findings add to the limited literature on this form of EHR-based alert fatigue and should help inform future tailoring, deployment, and further study of CTAs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
One study showed that response to clinical trial recruiting alerts in the EHR declined by approximately 2.7% per week, falling from about 50% to about 35% over 36 weeks.
“One study supporting the desensitization explanation showed that response to clinical trial recruiting alerts in the EHR declined by approximately 2.7% per week, falling from about 50% to about 35% over 36 weeks [19].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [19]
The Results (page not provided) report that responses to EHR-based clinical trial alerts fell from about 50% initially to 35% at week 36. The reported decline was 2.7% per two-week period, not per week as the citing sentence states. (LLM, conf=0.92)
Experimental — accuracy not yet measuredThe desensitization model holds that repeated exposure to alerts leads to declining responsiveness.
“A second conceptual model of alert fatigue is that repeated exposure to alerts leads to declining responsiveness [9, 19, 20], a phenomenon that can be called desensitization .”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The American Journal of Managed Care
(2020) · The American Journal of Managed Care · doi:10.37765/ajmc · Cited by 3,238 · see citing works
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117 · see citing works
Physicians' Decisions to Override Computerized Drug Alerts in Primary Care
Weingart, S. N., Toth, M., Sands, D. Z., Aronson, M. D., Davis, R. B., & Phillips, R. S. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.21.2625 · Cited by 487 · see citing works
Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · see citing works
Some unintended consequences of clinical decision support systems.
Ash, J. S., Sittig, D. F., Campbell, E. M., Guappone, K. P., & Dykstra, R. H. (2007) · PubMed Central · Cited by 298 · see citing works
Effect of a Clinical Trial Alert System on Physician Participation in Trial Recruitment
Embí, P. J., Jain, A., Clark, J. R., Bizjack, S., Hornung, R., & Harris, C. M. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.19.2272 · Cited by 183 · see citing works
Siminoff, L. A., Zhang, A., Colabianchi, N., Sturm, C. M. S., & Shen, Q. (2000) · Journal of Clinical Oncology · doi:10.1200/jco.2000.18.6.1203 · Cited by 149 · see citing works
To err is not entirely human: Complex technology and user cognition
Horsky, J., Zhang, J., & Patel, V. L. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.05.002 · Cited by 138 · see citing works
Cash, J. J. (2009) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp090181 · Cited by 126 · see citing works
Barriers to clinical trials: Part III: Knowledge and attitudes of health care providers
Mansour, E. G. (1994) · Cancer · doi:10.1002/1097-0142(19941101)74:9+ · Cited by 103 · see citing works
Organizational barriers to physician participation in cancer clinical trials.
Somkin, C. P., Altschuler, A., Ackerson, L., Geiger, A. M., Greene, S. M., Mouchawar, J., Holup, J. L., Fehrenbacher, L., Nelson, A., Glass, A. G., et al. (2005) · PubMed · Cited by 102 · see citing works
Status of Clinical Research in Academic Health Centers
Campbell, E. G., Weissman, J. S., Moy, E. M., et al. (2001) · JAMA · doi:10.1001/jama.286.7.800 · Cited by 99 · see citing works
Clinical Research and the NIH — A Report Card
Nathan, D. G., & Wilson, J. D. (2003) · New England Journal of Medicine · doi:10.1056/nejmsb035066 · Cited by 92 · see citing works
Embí, P. J., Jain, A., Clark, J. A., & Harris, C. M. (2005) · Europe PMC (PubMed Central) · Cited by 90 · see citing works
Obstacles to the accrual of patients to clinical trials in the community setting.
Winn, R. J. (1994) · PubMed · Cited by 81 · see citing works
Embí, P. J., Jain, A. K., & Harris, C. M. (2008) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-8-13 · Cited by 66 · see citing works
Comparison of Electronic Physician Prompts versus Waitroom Case-Finding on Clinical Trial Enrollment
Rollman, B. L., Fischer, G. S., Zhu, F., & Belnap, B. H. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0449-0 · Cited by 56 · see citing works
Research subject enrollment by primary care pediatricians using an electronic health record.
Grundmeier, R. W., Swietlik, M., & Bell, L. M. (2007) · PubMed · Cited by 33 · see citing works
Khan, Y. M., Roth, C., Payne, P., et al. (2013) · PubMed · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0430-8 · Cited by 709 · see citing works
“7% per week, falling from about 50% to about 35% over 36 weeks [19].”
BackgroundStrategies to improve recruitment to randomised trials
Treweek, S., Pitkethly, M., Cook, J., Fraser, C., Mitchell, E., Sullivan, F., Jackson, C., Taskila, T. K., & Gardner, H. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000013.pub6 · Cited by 521 · see citing works
A Clinical Case of Electronic Health Record Drug Alert Fatigue: Consequences for Patient Outcome
Carspecken, C. W., Sharek, P. J., Longhurst, C., & Pageler, N. M. (2013) · Pediatrics · doi:10.1542/peds.2012-3252 · Cited by 142 · see citing works
Rayo, M. F., & Moffatt-Bruce, S. D. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003373 · Cited by 115 · see citing works
Olakotan, O. O., & Mohd Yusof, M. (2021) · Health Informatics Journal · doi:10.1177/14604582211007536 · Cited by 101 · see citing works
Epic Sepsis Model Inpatient Predictive Analytic Tool: A Validation Study
Cull, J., Brevetta, R., Gerac, J., Kothari, S., & Blackhurst, D. (2023) · Critical Care Explorations · doi:10.1097/cce.0000000000000941 · Cited by 37 · see citing works
Horn, J., & Ueng, S. (2019) · Annals of Pharmacotherapy · doi:10.1177/1060028019863419 · Cited by 26 · see citing works
Assessing the use of a clinical decision support tool for pain management in primary care
Apathy, N. C., Sanner, L., Adams, M. C. B., Mamlin, B. W., Grout, R. W., Fortin, S., Hillstrom, J., Saha, A., Teal, E., Vest, J. R., et al. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac074 · Cited by 9 · see citing works
- Authors
- Embi, P. J., & Leonard, A. C.
- Affiliations
- University of Cincinnati (US)
- ORCID
- Leonard, A. C.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-06-01
- Volume
- 19
- Issue
- e1
- Pages
- e145-e148
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3392862
- OpenAlex
- W2148794355
- Cited by
- 199
- References
- 19
- Citations by year
- 2026: 12 · 2025: 10 · 2024: 13 · 2023: 21 · 2022: 18 · 2021: 16 · 2020: 16 · 2019: 17 · 2018: 18 · 2017: 10 · 2016: 7 · 2015: 16 · 2014: 18 · 2013: 5 · 2012: 2
- FWCI
- 21.30
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- U.S. National Library of Medicine (R01LM009533)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Ethics in Clinical Research
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Keywords
- alarm fatigue, electronic health records, participant recruitment, clinical research, community-based physicians
- MeSH terms
- Fatigue, Humans, Physicians, User-Computer Interface, Randomized Controlled Trials as Topic, Patient Selection, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
