CoChat
Sign Up
Cited by 75Electronic Health Records SystemsArtificial Intelligence in Healthcare and EducationHealthcare Technology and Patient Monitoring

Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals

Open access
Van Dort, Bethany A, Zheng, Wu Yi, Sundar, Vivek, Baysari, Melissa T·2021·Journal of the American Medical Informatics Association
DOI 10.1093/jamia/ocaa279PMC PMC7810441
Open this paper in CoChat — free account

Abstract

Objective: To identify and summarize the current internal governance processes adopted by hospitals, as reported in the literature, for selecting, optimizing, and evaluating clinical decision support (CDS) alerts in order to identify effective approaches. Materials and methods: Databases (Medline, Embase, CINAHL, Scopus, Web of Science, IEEE Xplore Digital Library, CADTH, and WorldCat) were searched to identify relevant papers published from January 2010 to April 2020. All paper types published in English that reported governance processes for selecting and/or optimizing CDS alerts in hospitals were included. Results: Eight papers were included in the review. Seven papers focused specifically on medication-related CDS alerts. All papers described the use of a multidisciplinary committee to optimize alerts. Other strategies included the use of clinician feedback, alert data, literature and drug references, and a visual dashboard. Six of the 8 papers reported evaluations of their CDS alert modifications following the adoption of optimization strategies, and of these, 5 reported a reduction in alert rate. Conclusions: A multidisciplinary committee, often in combination with other approaches, was the most frequent strategy reported by hospitals to optimize their CDS alerts. Due to the limited number of published processes, variation in system changes, and evaluation results, we were unable to compare the effectiveness of different strategies, although employing multiple strategies appears to be an effective approach for reducing CDS alert numbers. We recommend hospitals report on descriptions and evaluations of governance processes to enable identification of effective strategies for optimization of CDS alerts in hospitals.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/jamia/ocaa279

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Effective governance of CDS has been demonstrated in other institutions, resulting in the transition of alert and reminder content to other more appropriate areas in the EHR, downgrading the severity of alerts, and assessing the effect of CDS on clinician behavior and patient care.

    “Effective governance of CDS has been demonstrated in other institutions, resulting in the transition of alert and reminder content to other more appropriate areas in the EHR, downgrading the severity of alerts (often leading to them no longer firing), 38 39 and assessing the effect of CDS on clinician behavior and patient care. 40 41”

    From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [38]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 41

  1. [1]1. Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: prescribing and transcribing2016. AmJHealth Syst Pharm 2017; 74 (17): 1336-52.
  2. [3]3. Marcilly R, Ammenwerth E, Vasseur F, et al. Usability flaws of medication-related alerting functions: a systematic qualitative review. J Biomed Inform 2015; 55: 260-71.
  3. [4]4. Makam AN, Nguyen OK, Auerbach AD. Diagnostic accuracy and effectiveness of automated electronic sepsis alert systems: a systematic review. J Hosp Med 2015; 10 (6): 396-402.
  4. [5]5. Page N, Baysari MT, Westbrook JI. A systematic review of the effectiveness of interruptive medication prescribing alerts in hospital CPOE systems to change prescriber behavior and improve patient safety. Int J Med Inform 2017; 105: 22-30.
  5. [6]6. Bright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (1): 29-43.
  6. [7]7. Jaspers MWM, Smeulers M, Vermeulen H, et al. Effects of clinical decision-support systems on practitioner performance and patient out- - comes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011; 18 (3): 327-34.
  7. [8]8. Phansalkar S, Edworthy J, Hellier E, et al. Areview of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc 2010; 17 (5): 493-501.
  8. [9]9. van der Sijs H, Aarts J, Vulto A, et al. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (2): 138-47.
  9. [10]10. Van Dort BA, Zheng WY, Baysari MT. Prescriber perceptions of medication-related computerized decision support systems in hospitals: a synthesis of qualitative research. Int J Med Inform 2019; 129: 285-95.
  10. [12]12. Wright A, Ai A, Ash J, et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 25 (5): 496-506.
  11. [13]13. Boxwala AA, Tu S, Peleg M, et al. Toward a representation format for sharable clinical guidelines. J Biomed Inform 2001; 34 (3): 157-69.
  12. [14]14. Sittig DF, Wright A, Osheroff JA, et al. Grand challenges in clinical decision support. J Biomed Inform 2008; 41 (2): 387-92.
  13. [15]15. Yoshida E, Fei S, Bavuso K, et al. The value of monitoring clinical decision support interventions. Appl Clin Inform 2018; 09 (01): 163-73.
  14. [16]16. Sligo J, Roberts V, Gauld R, et al. Achecklist for healthcare organisations undergoing transformational change associated with large-scale health information systems implementation. Health Policy Technol 2019; 8 (3): 237-47.
  15. [17]17. Ash JS, Sittig DF, Guappone KP, et al. Recommended practices for computerized clinical decision support and knowledge management in community settings: a qualitative study. BMC Med Inform Decis Mak 2012; 12 (1): 6.
  16. [18]18. Khalifa M, Alswailem O. Clinical decision support knowledge management: strategies for success. ICIMTH 2015; 213: 67-70.
  17. [19]19. Tolley CL, Slight SP, Husband AK, et al. Improving medication-related clinical decision support. Bull Am Soc Hosp Pharm 2018; 75 (4): 239-46.
  18. [20]20. Moher D, Liberati A, Tetzlaff J, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6 (7): e1000097.
  19. [21]21. Bhakta SB, Colavecchia AC, Haines L, et al. A systematic approach to optimize electronic health record medication alerts in a health system. Am J Health Syst Pharm 2019; 76 (8): 530-36.
  20. [22]22. Chaparro JD, Hussain C, Lee JA, et al. Reducing interruptive alert burden using quality improvement methodology. Appl Clin Inform 2020; 11 (01): 46-58.
  21. [23]23. Liberati EG, Ruggiero F, Galuppo L, et al. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci 2017; 12 (1): 113.
  22. [24]24. Hatton RC, Rosenberg AF, Morris CT, et al. Evaluation of contraindicated drug-drug interaction alerts in a hospital setting. Ann Pharmacother 2011; 45 (3): 297-308.
  23. [25]25. Helmons PJ, Suijkerbuijk BO, Nannan Panday PV, et al. Drug-drug interaction checking assisted by clinical decision support: a return on investment analysis. J Am Med Inform Assoc 2015; 22 (4): 764-72.
  24. [26]26. Parke C, Santiago E, Zussy B, et al. Reduction of clinical support warnings through recategorization of severity levels. AmJ Health Syst Pharm 2015; 72 (2): 144-8.
  25. [27]27. Simpao AF, Ahumada LM, Desai BR, et al. Optimization of drug-drug interaction alert rules in a pediatric hospital's electronic health record system using a visual analytics dashboard. J Am Med Inform Assoc 2015; 22 (2): 361-9.
  26. [28]28. Zenziper Y, Kurnik D, Markovits N, et al. Implementation of a clinical decision support system for computerized drug prescription entries in a large tertiary care hospital. Isr Med Assoc J 2014; 16 (5): 289-94.
  27. [29]29. Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in US hospitals. NEngl J Med 2009; 360 (16): 1628-38.
  28. [30]30. Phansalkar S, van der Sijs H, Tucker AD, et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (3): 489-93.
  29. [31]31. Cash JJ. Alert fatigue. AmJHealth Syst Pharm 2009; 66 (23): 2098-101.
  30. [33]33. Smithburger PL, Buckley MS, Bejian S, et al. A critical evaluation of clinical decision support for the detection of drug-drug interactions. Expert Opin Drug Saf 2011; 10 (6): 871-82.
  31. [34]34. Baysari MT, Zheng WY, Van Dort B, et al. A late attempt to involve end users in the design of medication-related alerts: survey study. J Med Internet Res 2020; 22 (3): e14855.
  32. [35]35. Liberati EG, Ruggiero F, Galuppo L, et al. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implementation Sci 2017; 12 (1): 113.
  33. [36]36. Varonen H, Kortteisto T, Kaila M. What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians. Fam Pract 2008; 25 (3): 162-7.
  34. [37]37. Baysari MT, Reckmann MH, Li L, et al. Failure to utilize functions of an electronic prescribing system and the subsequent generation of 'technically preventable' computerized alerts. J Am Med Inform Assoc 2012; 19 (6): 1003-10.
  35. [38]38. Baysari MT, Westbrook JI, Richardson KL, et al. Optimising computerised alerts within electronic medication management systems: a synthesis of four years of research. Stud Health Technol Inform 2014; 204: 1-6.
  36. [39]39. Cullinan S, Hansen CR, Byrne S, et al. Challenges of deprescribing in the multimorbid patient. Eur J Hosp Pharm 2017; 24 (1): 43-6.
  37. [41]41. Davies A, Mueller J, Moulton G. Core competencies for clinical informaticians: a systematic review. Int J Med Inform 2020; 141: 104237.
  38. [42]42. Gardner RM, Overhage JM, Steen EB, et al. ; for the AMIA Board of Directors. Core content for the subspecialty of clinical informatics. J Am Med Inform Assoc 2009; 16 (2): 153-7.
  39. [43]43. Caban JJ, Gotz D. Visual analytics in healthcare-opportunities and research challenges. J Am Med Inform Assoc 2015; 22 (2): 260-2.
  40. [44]44. Stadler JG, Donlon K, Siewert JD, et al. Improving the efficiency and ease of healthcare analysis through use of data visualization dashboards. Big Data 2016; 4 (2): 129-35.
  41. [45]45. McCoy AB, Thomas EJ, Krousel-Wood M, et al. Clinical decision support alert appropriateness: a review and proposal for improvement. Ochsner J 2014; 14 (2): 195-202.

Cited by

Sorted by citations

Authors
Van Dort, B. A., Zheng, W. Y., Sundar, V., & Baysari, M. T.
Affiliations
  • Discipline of Biomedical Informatics and Digital Health, School of Medical Sciences, Charles Perkins Centre, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
  • The University of Sydney (AU)
  • Macquarie University (AU)
ORCID
Venue
Journal of the American Medical Informatics Association
Publisher
Oxford University Press (OUP)
Published
2020-11-13
Volume
28
Issue
1
Pages
177-183
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
DOI
10.1093/jamia/ocaa279
PMC
PMC7810441
OpenAlex
W3103574025
Cited by
75
References
4
Citations by year
2026: 16 · 2025: 23 · 2024: 11 · 2023: 16 · 2022: 5 · 2021: 4
FWCI
0.95
Type
journal article
Language
en
Open access
Licence
academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
Locations
  • Journal of the American Medical Informatics Association (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
The University of Sydney
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
Keywords
clinical decision support alerts, electronic health records, multidisciplinary committee, clinician feedback, clinical decision support
MeSH terms
Alert Fatigue, Health Personnel, Hospital Information Systems, Humans, Decision Support Systems, Clinical, Medical Order Entry Systems, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review