CoChat
Sign Up
Cited by 37Electronic Health Records SystemsHealthcare Technology and Patient MonitoringMedical Coding and Health Information

Optimizing best practice advisory alerts in electronic medical records with a multi-pronged strategy at a tertiary care hospital in Singapore

Open access
Ng, Hannah Jia Hui, Kansal, Amit, Abdul Naseer, Jishana Farhad, Hing, Wee Chuan, Goh, Carmen Jia Man, Poh, Hermione, D’souza, Jared Louis Andre, Lim, Er Luen +1 more·2023·JAMIA Open
DOI 10.1093/jamiaopen/ooad056PMC PMC10393867
Open this paper in CoChat — free accountOpen PDF

Abstract

Objective: Clinical decision support (CDS) alerts can aid in improving patient care. One CDS functionality is the Best Practice Advisory (BPA) alert notification system, wherein BPA alerts are automated alerts embedded in the hospital’s electronic medical records (EMR). However, excessive alerts can change clinician behavior; redundant and repetitive alerts can contribute to alert fatigue. Alerts can be optimized through a multipronged strategy. Our study aims to describe these strategies adopted and evaluate the resultant BPA alert optimization outcomes. Materials and Methods: This retrospective single-center study was done at Jurong Health Campus. Aggregated, anonymized data on patient demographics and alert statistics were collected from January 1, 2018 to December 31, 2021. “Preintervention” period was January 1–December 31, 2018, and “postintervention” period was January 1–December 31, 2021. The intervention period was the intervening period. Categorical variables were reported as frequencies and proportions and compared using the chi-square test. Continuous data were reported as median (interquartile range, IQR) and compared using the Wilcoxon rank-sum test. Statistical significance was defined at P < .05. Results: There was a significant reduction of 59.6% in the total number of interruptive BPA alerts, despite an increase in the number of unique BPAs from 54 to 360 from pre- to postintervention. There was a 74% reduction in the number of alerts from the 7 BPAs that were optimized from the pre- to postintervention period. There was a significant increase in percentage of overall interruptive BPA alerts with action taken (8 [IQR 7.7–8.4] to 54.7 [IQR 52.5–58.9], P-value < .05) and optimized BPAs with action taken (32.6 [IQR 32.3–32.9] to 72.6 [IQR 64.3–73.4], P-value < .05). We estimate that the reduction in alerts saved 3600 h of providers’ time per year. Conclusions: A significant reduction in interruptive alert volume, and a significant increase in action taken rates despite manifold increase in the number of unique BPAs could be achieved through concentrated efforts focusing on governance, data review, and visualization using a system-embedded tool, combined with the CDS Five Rights framework, to optimize alerts. Improved alert compliance was likely multifactorial—due to decreased repeated alert firing for the same patient; better awareness due to stakeholders’ involvement; and less fatigue since unnecessary alerts were removed. Future studies should prospectively focus on patients’ clinical chart reviews to assess downstream effects of various actions taken, identify any possibility of harm, and collect end-user feedback regarding the utility of alerts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/jamiaopen/ooad056

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Previous research has shown interruptive alert optimization by using strategies of eliciting end-user feedback, reducing alert volume, and reducing inappropriate firing of alerts, similar to the themes identified by participants in our study.

    “Previous research has shown interruptive alert optimization by using strategies of eliciting end-user feedback, reducing alert volume, and reducing inappropriate firing of alerts, similar to the themes identified by participants in our study. 34 35 36 37”

    From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [34]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 29

  1. [2]Kawamanto K, Flynn MC, Kukhareva P, et al A pragmatic guide to establishing clinical decision support governance and addressing decision support fatigue: a case study. AMIA Annu Symp Proc 2018; 2018: 624– 33. 30815104 PMC6371304
  2. [3]Bejjanki H, Mramba L, Beal S, et al The role of a best practice alert in the electronic medical record in reducing repetitive lab tests. Clinicoecon Outcomes Res 2018; 10: 611– 8. 30323637 PMC6181108
  3. [4]Barnes GD, Spranger E, Sippola E, et al Assessment of a Best Practice Alert and referral process for preprocedure antithrombotic medication management for patients undergoing gastrointestinal endoscopic procedures. JAMA Netw Open 2020; 3 (2): e1920548. 32022882 PMC7416616
  4. [5]Singh H, Spitzmueller C, Petersen NJ, et al Information overload and missed test results in electronic health record–based settings. JAMA Intern Med 2013; 173 (8): 702– 4. 23460235 PMC3822526
  5. [6]Van Dort BA, Zheng WY, Baysari MT. Prescriber perceptions of medication-related computerized decision support systems in hospitals: a synthesis of qualitative research. Int J Med Inform 2019; 129: 285– 95. 31445268
  6. [7]van der Sijs H, Aarts J, Vulto A, et al Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (2): 138– 47. 16357358 PMC1447540
  7. [8]Weingart SN, Toth M, Sands DZ, et al Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163 (21): 2625– 31. 14638563
  8. [9]Phansalkar S, van der Sijs H, Tucker AD, et al Drug–drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (3): 489– 93. 23011124 PMC3628052
  9. [11]McDaniel RB, Burlison JD, Baker DK, et al Alert dwell time: introduction of a measure to evaluate interruptive clinical decision support alerts. J Am Med Inform Assoc 2016; 23 (e1): e138–41– e141. 26499101 PMC4954613
  10. [12]Schreiber R, Gregoire JA, Shaha JE, et al Think time: a novel approach to analysis of clinicians’ behavior after reduction of drug-drug interaction alerts. Int J Med Inform 2017; 97: 59– 67. 27919396
  11. [13]Baysari MT, Tariq A, Day RO, et al Alert override as a habitual behavior – a new perspective on a persistent problem. J Am Med Inform Assoc 2017; 24 (2): 409– 12. 27274015 PMC7651895
  12. [14]Simpao AF, Ahumada LM, Desai BR, et al Optimization of drug–drug interaction alert rules in a pediatric hospital’s electronic health record system using a visual analytics dashboard. J Am Med Inform Assoc 2015; 22 (2): 361– 9. 25318641 PMC11749158
  13. [15]Van Dort BA, Zheng WY, Sundar V, et al Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals. J Am Med Inform Assoc 2021; 28 (1): 177– 83. 33186438 PMC7810441
  14. [16]Chaparro JD, Hussain C, Lee JA, et al Reducing interruptive alert burden using quality improvement methodology. Appl Clin Inform 2020; 11 (1): 46– 58. 31940671 PMC6962089
  15. [17]Bhakta SB, Colavecchia AC, Haines L, et al A systematic approach to optimize electronic health record medication alerts in a health system. Am J Health Syst Pharm 2019; 76 (8): 530– 6. 31361861
  16. [18]Helmons PJ, Suijkerbuijk BO, Nannan Panday PV, et al Drug-drug interaction checking assisted by clinical decision support: a return on investment analysis. J Am Med Inform Assoc 2015; 22 (4): 764– 72. 25670751 PMC11737639
  17. [19]Parke C, Santiago E, Zussy B, et al Reduction of clinical support warnings through recategorization of severity levels. Am J Health Syst Pharm 2015; 72 (2): 144– 8. 25550138
  18. [20]Baysari MT, Zheng WY, Van Dort B, et al A late attempt to involve end users in the design of medication-related alerts: survey study. J Med Internet Res 2020; 22 (3): e14855. 32167479 PMC7101499
  19. [21]Liberati EG, Ruggiero F, Galuppo L, et al What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci 2017; 12 (1): 113. 28915822 PMC5602839
  20. [22]Varonen H, Kortteisto T, Kaila M; EBMeDS Study Group. What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians. Family Practice 2008; 25 (3): 162– 7. 18504253
  21. [23]Osheroff Jerome A, Teich J, Levick D, et al Improving Outcomes with Clinical Decision Support: An Implementer’s Guide. 2nd ed. New York: HIMSS Publishing; 2021. doi: 10.4324/9780367806125. · DOI
  22. [25]Smithburger PL, Buckley MS, Bejian S, et al A critical evaluation of clinical decision support for the detection of drug–drug interactions. Expert Opin Drug Saf 2011; 10 (6): 871– 82. 21542665
  23. [26]Wang L, Goh KH, Yeow A, et al Habit and automaticity in medical alert override: cohort study. J Med Internet Res 2022; 24 (2): e23355. 35171102 PMC8892274
  24. [27]Westbrook JI, Woods A, Rob MI, Dunsmuir WTM, Day RO. Association of interruptions with an increased risk and severity of medication administration errors. Arch Intern Med 2010; 170 (8): 683– 90. 20421552
  25. [28]Westbrook JI, Coiera E, Dunsmuir WTM, et al The impact of interruptions on clinical task completion. Qual Saf Health Care 2010; 19 (4): 284– 9. 20463369
  26. [29]Grundgeiger T, Sanderson P. Interruptions in healthcare: theoretical views. Int J Med Inform 2009; 78 (5): 293– 307. 19081295
  27. [30]Cuéllar Monreal MJ. Reducción de la fatiga por alertas en un sistema de prescripción electrónica asistida, mediante la. Farm Hosp 2017; 41 (1): 14– 30. doi: 10.7399/fh.2017.41.1.10434. 28045650 · DOI
  28. [31]Chien S-C, Chin Y-P, Yoon C-H, et al A tool to retrieve alert dwell time from a homegrown computerized physician order entry (CPOE) system of an academic medical center: an exploratory analysis. Appl Sci 2021; 11 (24): 12004. PMC7872273 33561178
  29. [32]Baysari MT, Westbrook JI, Richardson K, et al Optimising computerised alerts within electronic medication management systems: a synthesis of four years of research. Stud Health Technol Inform 2014; 204: 1– 6. 25087519

Cited by

Sorted by citations

Authors
Ng, H. J. H., Kansal, A., Abdul Naseer, J. F., Hing, W. C., Goh, C. J. M., Poh, H., D’souza, J. L. A., Lim, E. L., & Tan, G.
Affiliations
  • Ng Teng Fong General Hospital (SG)
  • Department of Medical Informatics, Ng Teng Fong General Hospital , Singapore, Singapore
Corresponding author
Ng, Hannah Jia Hui
ORCID
Venue
JAMIA Open
Publisher
Oxford University Press (OUP)
Published
2023-07-04
Volume
6
Issue
3
Pages
ooad056-ooad056
ISSN
2574-2531 (ISSN-L)
DOI
10.1093/jamiaopen/ooad056
PMC
PMC10393867
OpenAlex
W4385443247
Cited by
37
References
3
Citations by year
2026: 14 · 2025: 15 · 2024: 5 · 2023: 3
FWCI
7.13
Type
journal article
Language
en
Open access
Licence
Locations
  • JAMIA Open (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • National University of Singapore (submittedVersion)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
  • Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
Keywords
clinical decision support alerts, alarm fatigue, electronic health records, interruptive alerts, tertiary care hospital, Southeast Asia

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review