Cited by 43Electronic Health Records SystemsPharmaceutical Practices and Patient OutcomesPatient Safety and Medication Errors

Renal medication-related clinical decision support (CDS) alerts and overrides in the inpatient setting following implementation of a commercial electronic health record: implications for designing more effective alerts

Open access
Shah, Sonam N, Amato, Mary G, Garlo, Katherine G, Seger, Diane L, Bates, David W·2021·Journal of the American Medical Informatics Association

Abstract

Objective: To assess the appropriateness of medication-related clinical decision support (CDS) alerts associated with renal insufficiency and the potential/actual harm from overriding the alerts. Materials and Methods: Override rate frequency was recorded for all inpatients who had a renal CDS alert trigger between 05/2017 and 04/2018. Two random samples of 300 for each of 2 types of medication-related CDS alerts associated with renal insufficiency—“dose change” and “avoid medication”—were evaluated by 2 independent reviewers using predetermined criteria for appropriateness of alert trigger, appropriateness of override, and patient harm. Results: We identified 37 100 “dose change” and 5095 “avoid medication” alerts in the population evaluated, and 100% of each were overridden. Dose change triggers were classified as 12.5% appropriate and overrides of these alerts classified as 90.5% appropriate. Avoid medication triggers were classified as 29.6% appropriate and overrides 76.5% appropriate. We identified 5 adverse drug events, and, of these, 4 of the 5 were due to inappropriately overridden alerts. Conclusion: Alerts were nearly always presented inappropriately and were all overridden during the 1-year period studied. Alert fatigue resulting from receiving too many poor-quality alerts may result in failure to recognize errors that could lead to patient harm. Although medication-related CDS alerts associated with renal insufficiency had previously been found to be the most clinically beneficial alerts in a legacy system, in this system they were ineffective. These findings underscore the need for improvements in alert design, implementation, and monitoring of alert performance to make alerts more patient-specific and clinically appropriate.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/jamia/ocaa222

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Interruptive alerts are one of the dominant forms of CDS and have the potential to distract from patient care and increase clinician's cognitive load.

    “Interruptive alerts, or pop ups that require end-user action or acknowledgment, are one of the dominant forms of CDS and have the potential to distract from patient care and increase clinician’s cognitive load.”

    From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [6]

    The Introduction and Discussion (page unavailable) describe alert fatigue from excessive or inappropriate CDS alerts and say clinicians may become desensitized to important alerts. They do not establish that interruptive pop-ups are a dominant form of CDS or specifically say they distract from patient care and increase cognitive load. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. [1]IOM (Institute of Medicine). Evidence-Based Medicine and the Changing Nature of Health Care: 2007 IOM Annual Meeting Summary. Washington, DC: The National Academies Press; 2008. 21391346
  2. [2]Beijer HJ, de Blaey CJ. Hospitalisations caused by adverse drug reactions (ADR): a meta-analysis of observational studies. Pharm World Sci 2002; 24 (2): 46– 54. 12061133
  3. [3]Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA 1998; 279 (15): 1200– 5. 9555760
  4. [4]Budnitz DS, Pollock DA, Weidenbach KN, et al National surveillance of emergency department visits for outpatient adverse drug events. JAMA 2006; 296 (15): 1858– 66. 17047216
  5. [5]Makary MA, Daniel M. Medical error- the third leading cause of death in the US. BMJ 2016; 353: i2139. 27143499
  6. [6]Davies EC, Green CF, Taylor S, et al Adverse drug reactions in hospital in-patients: a prospective analysis of 3695 patient-episodes. PLoS One 2009; 4 (2): e4439. 19209224 PMC2635959
  7. [7]Onder G, Petrovic M, Tangiisuran B, et al Development and validation of a score to assess risk of adverse drug reactions among in-hospital patients 65 years or older: the GerontoNet ADR risk score. Arch Intern Med 2010; 170 (13): 1142– 8. 20625022
  8. [8]Corsonello A, Pedone C, Corica F, et al Concealed renal insufficiency and adverse drug reactions in elderly hospitalized patients. Arch Intern Med 2005; 165 (7): 790– 5. 15824299
  9. [9]HelldéN A, Bergman U, von Euler M, et al Adverse drug reactions and impaired renal function in elderly patients admitted to the emergency department. Drugs Aging 2009; 26 (7): 595– 606. 19655826
  10. [10]Manley HJ, McClaran ML, Overbay DK, et al Factors associated with medication-related problems in ambulatory hemodialysis patients. Am J Kidney Dis 2003; 41 (2): 386– 93. 12552501
  11. [11]Verbeeck RK, Musuamba FT. Pharmacokinetics and dosage adjustment in patients with renal dysfunction. Eur J Clin Pharmacol 2009; 65 (8): 757– 73. 19543887
  12. [12]Leape LL, Brennan TA, Laird N, et al The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991; 324 (6): 377– 84. 1824793
  13. [13]Classen DC, Pestotnik SL, Evans RS, et al Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA 1997; 277 (4): 301– 6. 9002492
  14. [14]Bates DW, Spell N, Cullen DJ, et al The costs of adverse drug events in hospitalized patients. JAMA 1997; 277 (4): 307– 11. 9002493
  15. [15]Chertow GM, Lee J, Kuperman GJ, et al Guided medication dosing for inpatients with renal insufficiency. JAMA 2001; 286 (22): 2839– 44. 11735759
  16. [16]Bates DW, Leape LL, Cullen DJ, et al Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998; 280 (15): 1311– 6. 9794308
  17. [17]Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163 (12): 1409– 16. 12824090
  18. [18]Nuckols TK, Smith-Spangler C, Morton SC, et al The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3 (1): 56. 24894078 PMC4096499
  19. [19]Shamliyan TA, Duval S, Du J, Kane RL. Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res 2007; 43 (1p1): 32– 5. PMC2323150 18211517
  20. [20]Wolfstadt JI, Gurwitz JH, Field TS, et al The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008; 23 (4): 451– 8. 18373144 PMC2359507
  21. [21]Bates DW, Teich JM, Lee J, et al The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999; 6 (4): 313– 21. 10428004 PMC61372
  22. [22]Lin CP, Payne TH, Nichol P, Hoey PJ, Anderson CL, Gennari JH. Evaluating clinical decision support systems: monitoring CPOE order check override rates in the Department of Veterans Affairs’ Computerized Patient Record System. J Am Med Inform Assoc 2008; 15 (5): 620– 6. 18579840 PMC2528033
  23. [23]Nanji KC, Slight SP, Seger DL, et al Overrides of medication-related clinical decision support alerts in inpatients. J Am Med Inform Assoc 2018; 25 (5): 476– 81. 29092059 PMC7646870
  24. [24]Kuperman GJ, Bobb A, Payne TH, et al M edication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14 (1): 29– 40. 17068355 PMC2215064
  25. [25]Payne TH, Nichol WP, Hoey P, Savarino J. Characteristics and override rates of order checks in a practitioner order entry system. Proc AMIA Symp 2002; 2002: 602– 6. PMC2244252 12463894
  26. [27]Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163 (21): 2625– 31. 14638563
  27. [28]Hsieh TC, Kuperman GJ, Jaggi T, et al Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004; 11 (6): 482– 91. 15298998 PMC524628
  28. [29]Wong A, Amato MG, Seger DL, et al Evaluation of medication-related clinical decision support alert overrides in the intensive care unit. J Crit Care 2017; 39: 156– 61. 28259059
  29. [30]Wong A, Amato MG, Seger DL, et al Prospective evaluation of medication related clinical decision support over-rides in the intensive care unit. BMJ Qual Saf 2018; 27 (9): 718– 24. 29440481
  30. [31]Morimoto T, Gandhi TK, Seger AC, et al Adverse drug events and medication errors; detection and classification methods. Qual Saf Health Care 2004; 13 (4): 306– 14. 15289635 PMC1743868
  31. [32]Phansalkar S, Edworthy J, Hellier E, et al A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc 2010; 17 (5): 493– 501. 20819851 PMC2995688
  32. [33]Cho I, Slight SP, Nanji KC, et al Understanding physicians’ behavior toward alerts about nephrotoxic medications in outpatients: a cross-sectional analysis. BMC Nephrol 2014; 15 (1): 200. 25511564 PMC4279964
  33. [34]Leung AA, Schiff G, Keohane C, et al Impact of vendor computerized physician order entry on patients with renal impairment in community hospitals. J Hosp Med 2013; 8 (10): 545– 52. 24101539

Cited by

Sorted by citations

Authors
Shah, S. N., Amato, M. G., Garlo, K. G., Seger, D. L., & Bates, D. W.
Affiliations
  • Brigham and Women's Hospital (US)
  • Clinical and Quality Analysis, Partners HealthCare, Somerville, Massachusetts, USA
Corresponding author
Shah, Sonam N
Venue
Journal of the American Medical Informatics Association
Published
2021-01-31
Volume
28
Issue
6
Pages
1081-1087
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
OpenAlex
W3126229035
Cited by
43
References
1
Citations by year
2026: 8 · 2025: 5 · 2024: 4 · 2023: 9 · 2022: 11 · 2021: 6
FWCI
5.52
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
  • Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
Keywords
clinical decision support alerts, alert override, renal failure, electronic health records, alarm fatigue, inappropriate medication use, adverse drug reactions, inpatient medication safety
MeSH terms
Academic Medical Centers, Alert Fatigue, Health Personnel, Boston, Humans, Inpatients, Medication Errors, Quality of Health Care, Decision Support Systems, Clinical, Medical Order Entry Systems, Renal Insufficiency, Electronic Health Records, Drug-Related Side Effects and Adverse Reactions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review