Drug interaction alert override rates in the Meaningful Use era: No evidence of progress
Open accessAbstract
Summary: Background: Interruptive drug interaction alerts may reduce adverse drug events and are required for Stage I Meaningful Use attestation. For the last decade override rates have been very high. Despite their widespread use in commercial EHR systems, previously described interventions to improve alert frequency and acceptance have not been well studied. Objectives: (1) To measure override rates of inpatient medication alerts within a commercial clinical decision support system, and assess the impact of local customization efforts. (2) To compare override rates between drug-drug interaction and drug-allergy interaction alerts, between attending and resident physicians, and between public and academic hospitals. (3) To measure the correlation between physicians’ individual alert quantities and override rates as an indicator of potential alert fatigue. Methods: We retrospectively analyzed physician responses to drug-drug and drug-allergy interaction alerts, as generated by a common decision support product in a large teaching hospital system. Results: (1) Over four days, 461 different physicians entered 18,354 medication orders, resulting in 2,455 visible alerts; 2,280 alerts (93%) were overridden. (2) The drug-drug alert override rate was 95.1%, statistically higher than the rate for drug-allergy alerts (90.9%) (p < 0.001). There was no significant difference in override rates between attendings and residents, or between hospitals. (3) Physicians saw a mean of 1.3 alerts per day, and the number of alerts per physician was not significantly correlated with override rate (R2 = 0.03, p = 0.41). Conclusions: Despite intensive efforts to improve a commercial drug interaction alert system and to reduce alerting, override rates remain as high as reported over a decade ago. Alert fatigue does not seem to contribute. The results suggest the need to fundamentally question the premises of drug interaction alert systems. Citation: Bryant AD, Fletcher GS, Payne TH. Drug interaction alert override rates in the Meaningful Use era: No evidence of progress. Appl Clin Inf 2014; 5: 802–813 http://dx.doi.org/10.4338/ACI-2013-12-RA-0103
AI snapshot
Written from the paper’s full text.
TL;DR
In a 96-hour retrospective audit at two UW Medicine teaching hospitals using a commercial Cerner/Multum CDS, 93% of 2,455 physician-visible drug interaction alerts were overridden, with drug-drug alerts overridden more often than drug-allergy alerts (95.1% vs 90.9%). Despite local reclassification efforts, override rates remained as high as historical reports, and alert quantity was not correlated with override rate.
Key findings
- Over four days, 461 physicians entered 18,354 medication orders, generating 2,455 visible alerts; 2,280 alerts (93%) were overridden.
- Drug-drug alert override rate was 95.1%, significantly higher than drug-allergy alerts at 90.9% (p < 0.001); there was no significant difference between attendings and residents (p = 0.11) or between hospitals (p = 0.25).
- Physicians saw a mean of 1.3 alerts per day; individual alert quantity was not significantly correlated with override rate (R² = 0.03, p = 0.41).
- Exact allergy matches were 123/1,302 (9%) of allergy alerts, and 94/123 (76%) of these exact-match alerts were overridden; antibiotic allergy alerts had the highest compliance among drug-allergy classes (121/172, 70% overridden).
Methods
- Retrospective review of major medication alerts at UW Medical Center and Harborview Medical Center from Monday, June 10, 2013 00:00 to Thursday, June 13, 2013 23:59 (96 hours), using Cerner Millennium with the Multum knowledge base; only major interactions generated interruptive physician alerts.
- Alerts were included if they occurred in the study period, were handled by a physician, and occurred on an inpatient; there were no exclusion criteria. Records were filtered to MD, DO, DDS, or MBBS credentials, and coincident alerts in the same window were counted as a single override in the main analysis.
- Override rates across categories were compared with Pearson’s chi-squared tests; alert quantity and override rate were correlated with simple linear regression and a 2-tailed F test; significance was α = 0.05; analysis used Microsoft Excel.
Limitations
- Stated: the study examined only medication alert override rates and associated metadata, over a short time span, within a single vendor’s CDS; it did not assess alert or override appropriateness, review adverse events, or examine behavior outside the alert window, and could not determine whether alert fatigue or other factors increased inappropriate overrides. The authors also note that some physicians may override alerts but later cancel orders, leading to falsely high estimates of noncompliance, and that high baseline override rates may have masked differences due to alert fatigue or other variability.
- Evident from the text: the findings come from one health system’s two hospitals over four days, are limited to physicians, and do not include patient outcomes or independent appropriateness review; override reasons were coded and most physicians selected the first option, which may limit interpretation of override intent.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 12 cited sources checked · 0 supported · 0 flagged
- Findingown data
Opioids accounted for the majority of allergy alert triggers.
“Opioids accounted for the majority of allergy alert triggers; drug-drug alert triggers were more diverse.”
Quote in the paperNot a substantial claim - Findingreasoning
Physicians override cross-reaction alerts more often than exact allergy matches.
“Physicians override cross-reaction alerts more often than exact allergy matches, and likely view the former with a healthy skepticism that is only enforced by daily false alarms.”
Quote in the paperSubstantial claim - Findingreasoning
Antibiotic allergy alerts had the highest compliance rate.
“Antibiotic allergy alerts had the highest compliance rate.”
Quote in the paperNot a substantial claim - Findingown data
Most (76% or 94/123) of exact allergy alerts were overridden.
“Most (76% or 94/123) of these exact allergy alerts were overridden.”
Quote in the paperSubstantial claim - Findingown data
Individual override rates varied widely, even among physicians with identical alert quantities.
“Individual override rates varied widely, even among physicians with identical alert quantities.”
Quote in the paperNot a substantial claim
- Findingown data
Over a four-day audit period, 461 distinct physicians entered 18,354 medication orders directly.
“Over four days, 461 different physicians entered 18,354 medication orders, resulting in 2,455 visible alerts; 2,280 alerts (93%) were overridden.”
- Also stated in: § 4. Results · plain
“During the four day audit period providers entered 43,287 medication orders, but only 18,354 were entered directly by physicians.”
Quote in the paperNot a substantial claim - Also stated in: § 4. Results · plain
- Findingown data
Physicians entered an average of 10 medication orders per day.
“Physicians triggered a median of 3 alerts over the 4 day period, with a mean of 1.3 alerts per day, while entering an average of 10 medication orders per day.”
Quote in the paperNot a substantial claim - Findingown data
The four-day audit period captured 2,455 distinct alerts.
“The four day audit period captured 2,455 distinct alerts, involving 461 distinct physicians, for an overall alerting rate of 13.4% (2,455 alerts per 18,354 orders)”
Quote in the paperNot a substantial claim - Findingown data
There was no significant difference in override rates between attending and resident physicians (p = 0.11).
“There was no significant difference in override rates between the two hospital sites (p = 0.25), or between attending and resident physicians (p = 0.11).”
Quote in the paperNot a substantial claim - Findingown data
Fewer than 5% of individuals viewed more than 4 alerts per day.
“Fewer than 5% of individuals viewed more than 4 alerts per day.”
Quote in the paperNot a substantial claim - Findingown data
Physicians triggered a median of 3 alerts over the 4-day period.
“Physicians triggered a median of 3 alerts over the 4 day period, with a mean of 1.3 alerts per day, while entering an average of 10 medication orders per day.”
Quote in the paperNot a substantial claim - Findingown data
There was no significant difference in override rates between the two hospital sites (p = 0.25).
“There was no significant difference in override rates between the two hospital sites (p = 0.25), or between attending and resident physicians (p = 0.11).”
Quote in the paperNot a substantial claim - Findingown data
Most triggering medication classes had drug-drug override rates greater than 90% when grouped by class.
“Most triggering medication classes had drug-drug override rates greater than 90% when grouped by class (▶ Table 2).”
Quote in the paperSubstantial claim - Findingown data
Physicians saw a mean of 1.3 alerts per day.
“Physicians saw a mean of 1.3 alerts per day, and the number of alerts per physician was not significantly correlated with override rate (R2 = 0.03, p = 0.41).”
Quote in the paperNot a substantial claim - Findingown data
The remainders of allergy alerts were triggered by cross-reactivity with other drugs.
“The remainders of allergy alerts were triggered by cross-reactivity with other drugs.”
Quote in the paperNot a substantial claim - Findingown data
The number of alerts per physician was not significantly correlated with override rate (R2 = 0.03, p = 0.41).
“the number of alerts per physician was not significantly correlated with override rate (R2 = 0.03, p = 0.41).”
- Also stated in: § 4. Results · plain
“There was no significant association between physicians' alert quantity and their override rate (R 2 = 0.03, p = 0.41, ▶ Figure 3).”
Quote in the paperSubstantial claim - Also stated in: § 4. Results · plain
- Findingown data
Multiple alerts had a similar override rate (92%) as single alerts.
“Multiple alerts had a similar override rate (92%) as single alerts.”
Quote in the paperNot a substantial claim - Findingown data
Drug-drug alert triggers were more diverse than allergy alert triggers.
“Opioids accounted for the majority of allergy alert triggers; drug-drug alert triggers were more diverse.”
Quote in the paperNot a substantial claim - Findingown data
Most overriding physicians entered the reason 'Provider Approved' for both alert types.
“Most overriding physicians entered the reason”
Quote in the paperNot a substantial claim - Findingown data
18% of drug-drug alert override reasons inappropriately used an override indication for allergies.
“Most overriding physicians entered the reason”
Quote in the paperNot a substantial claim - Findingown data
2,280 alerts, or 93%, were overridden.
“2,280 alerts (93%) were overridden.”
Quote in the paperSubstantial claim - Findingown data
The overall alerting rate was 13.4%, calculated as 2,455 alerts per 18,354 orders.
“The four day audit period captured 2,455 distinct alerts, involving 461 distinct physicians, for an overall alerting rate of 13.4% (2,455 alerts per 18,354 orders)”
Quote in the paperNot a substantial claim - Findingown data
Only 9% of allergy alerts (123/1302) were triggered by exact matches between the ordered medication and the allergy.
“Only 9% of allergy alerts (123/1302) were triggered by exact matches between the ordered medication and the allergy.”
Quote in the paperSubstantial claim - Interpretationreasoning
High drug-allergy override rates suggest that these alerts may not be preventing harm from true drug allergies.
“High drug-allergy override rates suggest that these alerts may not be preventing harm from true drug allergies, where a single dose of medication may cause serious reactions.”
Quote in the paperSubstantial claim - Interpretationown data
Despite intensive efforts to improve a commercial drug interaction alert system and reduce alerting, override rates remain as high as reported over a decade ago.
“Despite intensive efforts to improve a commercial drug interaction alert system and to reduce alerting, override rates remain as high as reported over a decade ago.”
- Also stated in: § 6. Conclusions · plain
“Our analysis of medication order alerts shows that override rates remain as high as more than a decade ago, and before the Meaningful Use program.”
Quote in the paperSubstantial claim - Also stated in: § 6. Conclusions · plain
- Interpretationown data
The similar override rate between two hospitals with a shared staff and EHR implies that unique patient demographics or medication use patterns are not influencing overrides.
“The similar rate between two hospitals with a shared staff and EHR also implies that unique patient demographics or medication use patterns are not influencing overrides.”
Quote in the paperSubstantial claim - Interpretationown data
Alert fatigue does not seem to contribute to override behavior.
“Alert fatigue does not seem to contribute.”
- Also stated in: § 5. Discussion · hedged
“The low number of alerts per physician makes it less likely that quantity-based alert fatigue is a major contributor to dismissal of alerts.”
Quote in the paperSubstantial claim- The authors state this as an assumption: If alert fatigue contributes to overrides, physicians with higher alert quantities will have higher override rates.
- Also stated in: § 5. Discussion · hedged
- Premiseown data
There has been no apparent decline in override rates of CDS alerts in the 15 years of reported literature.
“There has been no apparent decline in override rates of CDS alerts in the 15 years of reported literature.”
Quote in the paperSubstantial claim - Premisecitation
Drug-drug interaction alerting with commercial EHR software is used in over 85% of U.S. hospitals.
“It is now used in over 85% of U.S. hospitals [16].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [16] Charles D, King J, Furukawa MF, Patel V. Hospital Adoption of Electronic Health Record Technology to Meet Meaningful Use Objectives: 2008-2012. ONC Data Brief, no. 10. Washington, DC: Office of the National Coordinator for Health Information Technology; 2013
Not checked in the free run.
- Interpretationown data
The study results question the value of the Meaningful Use requirement for drug-drug interaction alerts in certified EHRs.
“With Meaningful Use mandating the use of drug-drug interaction alerts, our study results question the value of this requirement for certified EHRs.”
Quote in the paperSubstantial claim - Interpretationown data
The currently recommended methods to reduce alert quantity and improve relevance will not be enough to improve physician acceptance of interruptive alerts.
“Taken together, these findings suggest that the currently recommended methods to reduce alert quantity and improve relevance will not be enough to improve physician acceptance of interruptive alerts.”
Quote in the paperSubstantial claim - Interpretationown data
Commercial EHRs as federally required to meet core Meaningful Use objectives have not managed to improve the high override rate issue.
“the importance of this study lies in the finding that commercial EHRs as federally required to meet core Meaningful Use objectives have not managed to improve this issue.”
Quote in the paperSubstantial claim - Interpretationcitation
The current study's override rates are at or above the rates in previous studies, including those at the regional VA hospital.
“Our current rates are at or above the rates in previous studies, including those at our regional VA hospital [7,8].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [7] Payne TH, Nichol WP, Hoey P, Savarino J. Characteristics and override rates of order checks in a practitioner order entry system. Proc AMIA Symp 2002: 602-606
Not checked in the free run.
- cites [8] source
Not checked in the free run.
- Rests on an assumption: Historical override rates reported in prior studies are comparable to the current study's rates.
- May combine two claims. 11
- Classified as probably not a substantial claim. 10
May combine two claims.
Findingown dataThe drug-drug alert override rate was 95.1%, significantly higher than the drug-allergy alert override rate of 90.9% (p < 0.001).
“the drug-drug alert override rate was significantly higher than that for drug-allergy alerts (95.1% vs. 90.9%,p < 0.001)”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataAbout 20% of all alerts occurred as multiples within the same alert window.
“About 20% of all alerts occurred as multiples within the same alert window.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenA similar review of allergy cross-reactivity rules before implementation found that most were appropriate.
“A similar review of the allergy cross-reactivity rules before implementation found that most were appropriate.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOverride rates for drug-allergy alerts were similarly high among all classes, with the notable exception of antibiotics.
“Override rates for drug-allergy alerts were similarly high among all classes, with the notable exception of antibiotics.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingcitationAll interactions were reviewed before CDS implementation, causing 65% of all major interactions to be reclassified to lower levels.
“All interactions were reviewed before the implementation of CDS, causing 65% of all major interactions to be reclassified to lower levels [15].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [15] source
Not checked in the free run.
May combine two claims.
FindingcitationMost allergy alerts were triggered by narcotic allergies or by cross-reactions of unknown significance.
“Most of our allergy alerts were triggered by narcotic allergies, or by cross-reactions of unknown significance [29,30].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [29] Abookire SA, Teich JM, Sandige H, Paterno MD, Martin MT, Kuperman GJ, Bates DW. Improving allergy alerting in a computerized physician order entry system. Proc AMIA Symp 2000: 2-6
Not checked in the free run.
- cites [30] source
Not checked in the free run.
May combine two claims.
Findingown dataThe olanzapine-lorazepam drug interaction alert occurred 92 times and was overridden every time.
“For example, the olanzapine-lorazepam drug interaction alert occurred 92 times and was overridden every time.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSimilar override rates of over 90% have been found in drug dose alerts.
“Similar override rates of over 90% have been found in drug dose alerts [21].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [21] source
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenThe study period was a continuous 96-hour order entry period that provided adequate data and excluded dates of resident and faculty turnover.
“This continuous 96 hour order entry period provided adequate data and excluded dates of resident and faculty turnover.”
Quote in the paperSubstantial claim- Rests on an assumption: The four-day audit period is representative of typical inpatient medication ordering and alerting at the study hospitals.
May combine two claims.
Study design and conductcitationThe panel used the Hansten and Horn operational classification scheme to reassign major drug-drug interactions to lower severity levels, decreasing interruptive alerting.
“The panel uses the operational classification scheme described by Hansten and Horn to reassign major drug-drug interactions to lower severity levels, and thus decrease interruptive alerting [14].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [14] Hansten, 2001, J Am Pharm Assoc source
Not checked in the free run.
Classified as probably not a substantial claim.
Study design and conductreasoningCoincident alerts that could not be overridden independently were counted as a single override in the main analysis.
“Because the override of multiple alerts represented a single decision, these coincident alerts were included as a single override in the main analysis.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe frequent overrides of most alert types including exact allergy matches, and the often inappropriate override reasons, suggest that physicians dismiss alerts without assessing the specific risk of each alert.
“The frequent overrides of most alert types including exact allergy matches, and the often inappropriate (unmatched to the alert) override reasons suggest that physicians dismiss alerts without assessing the specific risk of each alert.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationcitationThe overall risk of ADEs is low, but very high override rates could increase the absolute number of inappropriate overrides, raising the ADE risk.
“While the overall risk of ADEs is low, our very high override rates could increase the absolute number of inappropriate overrides, raising the ADE risk [5,28].”
Quote in the paperSubstantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe study was approved by the University of Washington Human Subjects Division.
“This study was approved by the University of Washington Human Subjects Division.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductreasoningBecause of confounding factors, the analysis focused on physicians alone to assess the impact of alert reclassification.
“Because of these confounding factors, we focused on physicians alone to assess the impact of alert reclassification.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe lack of correlation between alert quantity and override rate supports prior evidence that reducing the number of alerts will not appreciably increase physician acceptance or reduce overrides.
“The lack of correlation between alert quantity and override rate also supports prior evidence that reducing the number of alerts will not appreciably increase physician acceptance or reduce overrides [23].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [23] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenA panel of physicians, pharmacists, and IT staff met monthly since 2008 to review all major drug-drug interaction alerts.
“A panel of physicians, pharmacists, and information technology staff has met monthly since 2008 to review all major drug-drug interaction alerts.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningThe study period fell at the end of the academic year to ensure that all providers had several months experience with the order entry system.
“The study period fell at the end of the academic year to ensure that all providers had several months experience with the order entry system.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationnone givenThe authors' experience illustrates substantial obstacles when trying to customize a commercial product to local needs.
“Further, our experience illustrates the substantial obstacles when trying to customize a commercial product to local needs.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationAlerts appear 'too late', when the prescribing decision has been made and negotiated with the patient, and consequently do not fit correctly into the workflow and are likely to be ignored by providers.
“As Hayward et al. describe, alerts appear”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [24] source
Not checked in the free run.
The paper relies on this without support.
Override rates are a valid measure of clinical decision support acceptance and utility.
“We are the first to report a lack of improvement in decision support acceptance (as measured by overrides) after Meaningful Use requirements took effect.”
The paper relies on this without support.
The four-day audit period is representative of typical inpatient medication ordering and alerting at the study hospitals.
“This continuous 96 hour order entry period provided adequate data and excluded dates of resident and faculty turnover.”
The paper relies on this without support.
The study's commercial CDS system is similar to those used in many U.S. hospitals.
“We demonstrated this lack of improvement within a popular commercial decision support system, similar to those used in many hospitals nationwide.”
The paper relies on this without support.
The local customization efforts represent a strong effort to improve CDS acceptance.
“University of Washington made a strong effort to improve CDS and its acceptance based on suggestions in the literature using local expertise to modify the default alerts within the constraints of our commercial system.”
The paper relies on this without support.
Historical override rates reported in prior studies are comparable to the current study's rates.
“Our current rates are at or above the rates in previous studies, including those at our regional VA hospital [7,8].”
The authors state this as an assumption.
If alert fatigue contributes to overrides, physicians with higher alert quantities will have higher override rates.
“Further, we hypothesized that physicians who saw higher quantities of alerts would have higher override rates, as predicted by the theory of alert fatigue.”
The paper relies on this without support.
The coded override reason selected by a physician accurately reflects the physician's reason for overriding the alert.
“Most overriding physicians entered the reason”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
High override rates of 56.3%-95.7% have already been described for DDI alerts.
“High override rates of 56.3%-95.7% have already been described for DDI alerts [13, 21, 27–32].”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [21]
The Results report a high drug–drug interaction alert override rate of 95.1%, which falls within the cited range. Our copy does not report 56.3% or 95.7% as the bounds of a range, so this source supports the high-rate claim but not the precise range. (LLM, conf=0.86)
Experimental — accuracy not yet measuredAlerts fire so frequently that most are ignored, with prior research demonstrating rates of ignoring as high as 96%.
“Optimal alert volume is unknown, but alerts fire so frequently that most are ignored, with prior research demonstrating rates of ignoring as high as 96%, almost entirely defeating their purpose.”
From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [15]
The Results report that most alerts were overridden: 93% overall and 95.1% of drug-drug alerts. But the paper does not report a prior rate of 96%, and its Discussion says the low number of alerts per physician and lack of correlation between alert quantity and override rate argue against frequent firing as the reason alerts were dismissed. (LLM, conf=0.87)
Experimental — accuracy not yet measuredThe inherent design of our site's EHR makes interruptive alerts one of the easiest and least expensive forms of CDS to implement, even though they are known to be disruptive and often ineffective.
“The inherent design of our site’s EHR makes interruptive alerts one of the easiest and least expensive forms of CDS to implement, even though they are known to be disruptive and often ineffective. 5 15 16”
From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [15]
The Introduction (page unavailable) says order-checking alerts interrupt workflow and are frequently overridden; the Discussion (page unavailable) questions whether drug-drug alerts provide value. The paper does not say that the citing site's EHR design makes interruptive alerts particularly easy or inexpensive to implement. (LLM, conf=0.86)
Experimental — accuracy not yet measuredBryant and colleagues found that alert volume was not correlated with override rate.
“Bryant and colleagues also recently found that alert volume was not correlated with override rate [24].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [24]
Our study results are in agreement with previous reports.
“Our study results are in agreement with previous reports.”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [21]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Exact-match alerts are not 9% of all alerts
MinorThe table labels its percentage column "% of all alerts," but 123 exact-match alerts are 5.0% of the 2,455 total alerts, not 9%. The 9% figure instead uses drug-allergy alerts as its denominator.
Reported 9 · recomputed 5.010183
“Total 2455 1153 1302 123 1179 480 1975 1200 1255 % of all alerts 100 47 53 9 48 20 80 49 51”
Cites
Bates, D. W. (1998) · JAMA · doi:10.1001/jama.280.15.1311 · Cited by 2,107 · see citing works
The Impact of Computerized Physician Order Entry on Medication Error Prevention
Bates, D. W., Teich, J. M., Lee, J., Seger, D., Kuperman, G. J., Ma'Luf, N., Boyle, D., & Leape, L. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.00660313 · Cited by 1,260 · see citing works
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117 · see citing works
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763 · see citing works
Self-citationDrug-Drug Interactions Among Elderly Patients Hospitalized for Drug Toxicity
Juurlink, D. N., Mamdani, M., Kopp, A., Laupacis, A., & Redelmeier, D. A. (2003) · JAMA · doi:10.1001/jama.289.13.1652 · Cited by 759 · see citing works
Physicians' Decisions to Override Computerized Drug Alerts in Primary Care
Weingart, S. N., Toth, M., Sands, D. Z., Aronson, M. D., Davis, R. B., & Phillips, R. S. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.21.2625 · Cited by 487 · see citing works
Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · see citing works
Absence of Cross-Reactivity between Sulfonamide Antibiotics and Sulfonamide Nonantibiotics
Strom, B. L., Schinnar, R., Apter, A. J., Margolis, D. J., Lautenbach, E., Hennessy, S., Bilker, W. B., & Pettitt, D. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa022963 · Cited by 349 · see citing works
Tiering Drug-Drug Interaction Alerts by Severity Increases Compliance Rates
Paterno, M. D., Maviglia, S. M., Gorman, P. N., Seger, D. L., Yoshida, E., Seger, A. C., Bates, D. W., & Gandhi, T. K. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2808 · Cited by 263 · see citing works
Phansalkar, S., van der Sijs, H., Tucker, A. D., Desai, A. A., Bell, D. S., Teich, J. M., Middleton, B., & Bates, D. W. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001089 · Cited by 239 · see citing works
Hsieh, T. C., Kuperman, G. J., Jaggi, T., Hojnowski-Diaz, P., Fiskio, J., Williams, D. H., Bates, D. W., & Gandhi, T. K. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1556 · Cited by 232 · see citing works
Overrides of medication-related clinical decision support alerts in outpatients
Nanji, K. C., Slight, S. P., Seger, D. L., Cho, I., Fiskio, J. M., Redden, L. M., Volk, L. A., & Bates, D. W. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001813 · Cited by 219 · see citing works
Turning Off Frequently Overridden Drug Alerts: Limited Opportunities for Doing It Safely
van der Sijs, H., Aarts, J., van Gelder, T., Berg, M., & Vulto, A. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2311 · Cited by 175 · see citing works
The costs of adverse drug events in community hospitals
Hug, B. L., Keohane, C., Seger, D. L., Yoon, C., & Bates, D. W. (2012) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(12)38016-1 · Cited by 169 · see citing works
A Clinical Case of Electronic Health Record Drug Alert Fatigue: Consequences for Patient Outcome
Carspecken, C. W., Sharek, P. J., Longhurst, C., & Pageler, N. M. (2013) · Pediatrics · doi:10.1542/peds.2012-3252 · Cited by 142 · see citing works
Characteristics and override rates of order checks in a practitioner order entry system.
Payne, T. H., Nichol, W. P., Hoey, P., & Savarino, J. E. (2002) · PubMed Central · Cited by 129 · see citing works
Self-citationImproving allergy alerting in a computerized physician order entry system.
Abookire, S. A., Teich, J. M., Sandige, H. L., Paterno, M. D., Martin, M. T., Kuperman, G. J., & Bates, D. W. (2000) · PubMed Central · Cited by 120 · see citing works
Lin, C.-P., Payne, T. H., Nichol, W. P., Hoey, P. J., Anderson, C. L., & Gennari, J. H. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2453 · Cited by 110 · see citing works
Self-citationORCA: OpeRational ClassificAtion of Drug Interactions
Hansten, P. D., Horn, J. R., & Hazlet, T. K. (2001) · Journal of the American Pharmaceutical Association (1996) · doi:10.1016/s1086-5802(16)31244-x · Cited by 98 · see citing works
Slight, S. P., Seger, D. L., Nanji, K. C., Cho, I., Maniam, N., Dykes, P. C., & Bates, D. W. (2013) · PLoS ONE · doi:10.1371/journal.pone.0085071 · Cited by 86 · see citing works
Hayward, J., Thomson, F., Milne, H., Buckingham, S., Sheikh, A., Fernando, B., Cresswell, K., Williams, R., & Pinnock, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001484 · Cited by 48 · see citing works
Wright, A. T., Kuperman, G. J., Vaida, A. J., Bobb, A., Jenders, R. A., Payne, T. H., Halamka, J. D., Bloomrosen, M., Bates, D. W., & Phansalkar, S. (2011) · Applied Clinical Informatics · doi:10.4338/aci-2010-04-ra-0026 · Cited by 30 · see citing works
Self-citationKaushal, R., Shojania, K. G., & Bates, D. W. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.12.1409 · Cited by 30 · see citing works
Customizing clinical decision support to prevent excessive drug–drug interaction alerts
Horn, J. R., Hansten, P. D., Osborn, J. D., Wareham, P., & Somani, S. (2011) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp100465 · Cited by 26 · see citing works
Evaluation of medication dose alerts in pediatric inpatients
Scharnweber, C., Lau, B. D., Mollenkopf, N., Thiemann, D. R., Veltri, M. A., & Lehmann, C. U. (2013) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2013.04.002 · Cited by 26 · see citing works
Clinical decision support for drug–drug interactions: Improvement needed
Horn, J. R., Gumpper, K. F., Hardy, J., Mcdonnell, P. J., Phansalkar, S., & Reilly, C. (2013) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp120405 · Cited by 21 · see citing works
Coded entry versus free-text and alert overrides: What you get depends on how you ask
Seidling, H. M., Paterno, M. D., Haefeli, W. E., et al. (2010) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2010.08.003 · Cited by 17 · see citing works
Impact of alert specifications on clinicians' adherence.
Langemeijer, M., Peute, L., & Jaspers, M. (2011) · PubMed · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Langemeijer, 2011, Stud Health Technol Inform, 169, 930
Cited by
Sorted by citations
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0430-8 · Cited by 709 · see citing works
Schwartz, P. J., & Woosley, R. L. (2016) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2015.12.063 · Cited by 329 · see citing works
“Unfortunately, the beneficial impact of these CDSS has been limited because in 75% to 95% of cases, physicians and pharmacists override or ignore the alerts (50,52,69), and no CDSS has yet been found to have sufficient impact on prescribing to reduce the”
BackgroundClinical Decision Support: a 25 Year Retrospective and a 25 Year Vision
Sittig, D. F., Wright, A. T., & Middleton, B. (2016) · Yearbook of Medical Informatics · doi:10.15265/iys-2016-s034 · Cited by 276 · see citing works
“Given this variation, it is no surprise that many clinicians view CDS as arising at an inopportune time in their workflow [219, 220], and many CDS alerts are simply ignored or overridden [221-224].”
BackgroundElectronic health record alerts for acute kidney injury: multicenter, randomized clinical trial
Wilson, F. P., Martin, M., Yamamoto, Y., Partridge, C., Moreira, E., Arora, T., Biswas, A., Feldman, H., Garg, A. X., Greenberg, J. H., et al. (2021) · BMJ · doi:10.1136/bmj.m4786 · Cited by 200 · see citing works
Medication-related clinical decision support alert overrides in inpatients
Nanji, K. C., Seger, D. L., Slight, S. P., Amato, M. G., Beeler, P. E., Her, Q. L., Dalleur, O., Eguale, T., Wong, A., Silvers, E. R., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx115 · Cited by 179 · see citing works
Missing clinical and behavioral health data in a large electronic health record (EHR) system
Madden, J. M., Lakoma, M. D., Rusinak, D., Lu, C. Y., & Soumerai, S. B. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw021 · Cited by 173 · see citing works
Jacobs, M., He, J., F. Pradier, M., Lam, B., Ahn, A. C., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445385 · Cited by 177 · see citing works
“For example, studies in medical informatics found that when DST alerts arise at inopportune times (from clinicians’ vantage), the alerts become ignored or overridden [1, 35, 43].”
BackgroundZheng, W. Y., Richardson, L. C., Li, L., Day, R. O., Westbrook, J. I., & Baysari, M. T. (2017) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-017-2357-5 · Cited by 170 · see citing works
Colicchio, T. K., Cimino, J., & Fiol, G. D. (2019) · Journal of Medical Internet Research · doi:10.2196/13313 · Cited by 139 · see citing works
“Given the complexity and high cost involved in implementing a commercial EHR, health care organizations are unlikely to change an EHR vendor anytime soon, causing a saturation of the US EHR market.”
Hussain, M., Reynolds, T. L., & Zheng, K. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz095 · Cited by 137 · see citing works
“12,13 The results have been mixed.”
Appropriateness of Overridden Alerts in Computerized Physician Order Entry: Systematic Review
Poly, T. N., Islam, M. M., Yang, H.-C., & Li, Y.-C. (. (2020) · JMIR Medical Informatics · doi:10.2196/15653 · Cited by 118 · see citing works
“1 DDI June 10-13, 2013 PTHcc ROS Bryant et al [28]”
BackgroundKane‐Gill, S. L., O’Connor, M. F., Rothschild, J. M., Selby, N. M., McLean, B. A., Bonafide, C. P., Cvach, M. M., Hu, X., Konkani, A., Pelter, M. M., et al. (2017) · Critical Care Medicine · doi:10.1097/ccm.0000000000002580 · Cited by 115 · see citing works
Segal, G., Segev, A., Brom, A., Lifshitz, Y., Wasserstrum, Y., & Zimlichman, E. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz135 · Cited by 108 · see citing works
“12,13 Moreover, most systems focus on identifying the potential errors at the prescribing event but fail to identify ADEs that occur in ongoing active prescriptions due to the frequent changes, which are characteristic of patients during their in-hospital stay.”
BackgroundOlakotan, O. O., & Mohd Yusof, M. (2021) · Health Informatics Journal · doi:10.1177/14604582211007536 · Cited by 101 · see citing works
“3,38,42,45,86,99 Therefore, CDSS alerts must be designed to provide clinicians with patient-specific assessment and recommendation (i.e. appropriate alert) during task execution to improve the quality of care and clinical outcomes.”
BackgroundCredibleMeds.org: What does it offer?
Woosley, R. L., Black, K., Heise, C. W., & Romero, K. (2017) · Trends in Cardiovascular Medicine · doi:10.1016/j.tcm.2017.07.010 · Cited by 99 · see citing works
Clinical Decision Support Systems for Drug Allergy Checking: Systematic Review
Légat, L., Laere, S. V., Nyssen, M., Steurbaut, S., Dupont, A. G., & Cornu, P. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8206 · Cited by 91 · see citing works
Alert override as a habitual behavior – a new perspective on a persistent problem
Baysari, M. T., Tariq, A., Day, R. O., & Westbrook, J. I. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw072 · Cited by 81 · see citing works
“For example, a detailed retrospective review of alerts triggered by physicians over a 4-day period at two US primary teaching hospitals showed that even highly relevant alerts (eg, exact allergy matches) were frequently overridden.(8) In a study where alert content was modified to include more relevant patient informa…”
BackgroundRecommendations for selecting drug–drug interactions for clinical decision support
Tilson, H. H., Hines, L. E., McEvoy, G. K., Weinstein, D. M., Hansten, P. D., Matuszewski, K. A., le Comte, M., Higby-Baker, S., Hanlon, J. T., Pezzullo, L., et al. (2016) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp150565 · Cited by 78 · see citing works
McEvoy, D. S., Sittig, D. F., Hickman, T.-T., Aaron, S., Ai, A., Amato, M., Bauer, D. W., Fraser, G. M., Harper, J., Kennemer, A., et al. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw114 · Cited by 75 · see citing works
Kawamoto, K., & McDonald, C. J. (2020) · Annals of Internal Medicine · doi:10.7326/m19-0875 · Cited by 72 · see citing works
“However, clinicians have ignored up to 93% of alerts (15), and they complain about excessive rates of false-positive reminders (16–18).”
BackgroundOlakotan, O. O., & Yusof, M. M. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103453 · Cited by 75 · see citing works
Trinkley, K. E., Kahn, M. G., Bennett, T. D., Glasgow, R. E., Haugen, H. A., Kao, D. P., Kroehl, M., Lin, C., Malone, D. C., & Matlock, D. D. (2020) · Journal of Medical Internet Research · doi:10.2196/19676 · Cited by 56 · see citing works
“Up to 95% of CDS tools are dismissed [5], 52%-66% improve process outcomes such as appropriate drug selection, and only 21%-43% lead to improvements in clinical outcomes [6-9].”
BackgroundArtificial intelligence and machine learning for clinical pharmacology
Ryan, D. K., Maclean, R., Balston, A. J., Scourfield, A., Shah, A. D., & Ross, J. (2023) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15930 · Cited by 63 · see citing works
Kim, H. S., McCarthy, D. M., Hoppe, J. A., Courtney, D. M., & Lambert, B. L. (2017) · Academic Emergency Medicine · doi:10.1111/acem.13273 · Cited by 55 · see citing works
Precision Population Medicine in Primary Care: The Sanford Chip Experience
Christensen, K. D., Bell, M., Zawatsky, C. L. B., Galbraith, L. N., Green, R. C., Hutchinson, A. M., Jamal, L., LeBlanc, J. L., Leonhard, J. R., Moore, M. M., et al. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2021.626845 · Cited by 54 · see citing works
Van De Sijpe, G., Quintens, C., Walgraeve, K., Van Laer, E., Penny, J., De Vlieger, G., Schrijvers, R., De Munter, P., Foulon, V., Casteels, M., et al. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01783-z · Cited by 54 · see citing works
A comparison of potential psychiatric drug interactions from six drug interaction database programs
Monteith, S., & Glenn, T. (2019) · Psychiatry Research · doi:10.1016/j.psychres.2019.03.041 · Cited by 47 · see citing works
Automation to optimise physician treatment of individual patients: examples in psychiatry
Bauer, M., Monteith, S., Geddes, J., Gitlin, M. J., Grof, P., Whybrow, P. C., & Glenn, T. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30041-0 · Cited by 44 · see citing works
McDaniel, R. B., Burlison, J. D., Baker, D. K., Hasan, M., Robertson, J., Hartford, C. M., Howard, S. C., Sablauer, A., & Hoffman, J. M. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv144 · Cited by 45 · see citing works
Muylle, K. M., Gentens, K., Dupont, A. G., & Cornu, P. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104393 · Cited by 45 · see citing works
Wan, P. K., Satybaldy, A., Huang, L., Holtskog, H., & Nowostawski, M. (2020) · Journal of Medical Internet Research · doi:10.2196/22013 · Cited by 42 · see citing works
“Bryant et al [38] pointed out that despite intensive efforts to reduce irrelevant alerts of commercial systems, overriding rates remain as high as reported over a decade ago.”
BackgroundScott, I. A., Sullivan, C. M., & Staib, A. N. (2018) · Australian Health Review · doi:10.1071/ah17153 · Cited by 39 · see citing works
Cornu, P., Steurbaut, S., Gentens, K., de Velde, R. V., & Dupont, A. G. (2015) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2015.05.005 · Cited by 35 · see citing works
“Because accreditation and legislation requirements in different countries boost the use of EHRs and CDSS, further attention should go to the development of CDSS with high enough specificity so that they are useful and support health care professionals [43,51,52].”
BackgroundKovačević, M., Kovačević, S. V., Radovanović, S., et al. (2019) · Current Medical Research and Opinion · doi:10.1080/03007995.2019.1647021 · Cited by 34 · see citing works
Heringa, M., Floor‐Schreudering, A., Tromp, P. C., et al. (2015) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3915 · Cited by 31 · see citing works
Bykov, K., Schneeweiß, S., Glynn, R. J., Mittleman, M. A., & Gagne, J. J. (2019) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1376 · Cited by 30 · see citing works
“Electronic prescribing and pharmacy software generate an excessive number of DDI alerts, most of which are routinely overridden, although not always appropriately.(5,6) There are growing concerns that the current DDI decision support systems are failing to achieve their goal of reducing DDI- associated patient harm(4,…”
BackgroundA systematic review of drug allergy alert systems
Luri, M., Leache, L., Gastaminza, G., Idoate, A., & Ortega, A. (2021) · Int. J. Medical Informatics · doi:10.1016/j.ijmedinf.2021.104673 · Cited by 24 · see citing works
Mulder-Wildemors, L. G. M., Heringa, M., Floor‐Schreudering, A., Jansen, P. A. F., & Bouvy, M. L. (2019) · Drugs & Aging · doi:10.1007/s40266-019-00728-y · Cited by 29 · see citing works
Orenstein, E. W., Kandaswamy, S., Muthu, N., Chaparro, J. D., Hagedorn, P. A., Dziorny, A. C., Moses, A., Hernandez, S., Khan, A. N., Huth, H. B., et al. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab179 · Cited by 26 · see citing works
Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis
Chien, S.-C., Chen, Y.-L., Chien, C., Chin, Y.-P., Yoon, C. H., Chen, C.-Y., Yang, H., & Li, Y. (2022) · Healthcare · doi:10.3390/healthcare10040601 · Cited by 30 · see citing works
Overriding Drug-Drug Interaction Alerts in Clinical Decision Support Systems: A Scoping Review
Zapata, L. V., Subbian, V., Boyce, R. D., Hansten, P. D., Horn, J. R., Gephart, S. M., Romero, A. V., & Malone, D. C. (2022) · Studies in health technology and informatics · doi:10.3233/shti220101 · Cited by 28 · see citing works
Murad, D. A., Tsugawa, Y., Elashoff, D. A., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac191 · Cited by 27 · see citing works
“Visit is with patient’s PCP, n (%) No 14 706 (26) Yes 40 943 (74) Appointment length, n (%) 15 min or less 21 809 (39) 20–30 min 27 943 (50) Greater than 30 min 5897 (11) Encounter year, n (%) 2017 2205 (4) 2018 14 127 (25) 2019 21 969 (39) 2020 14 759 (27) 2021 2589 (5) Competing BPA alert count, n (%) 0 21 203 (38)…”
Schneider, K. L., Kastenmüller, K., Weckbecker, K., Bleckwenn, M., Böhme, M., & Stingl, J. (2018) · Drugs & Aging · doi:10.1007/s40266-018-0550-6 · Cited by 26 · see citing works
Felisberto, M., Lima, G. D. S., Celuppi, I. C., Fantonelli, M. D. S., Zanotto, W. L., Dias de Oliveira, J. M., Mohr, E. T. B., dos Santos, R. A., Scandolara, D. H., Cunha, C. L., et al. (2024) · Health Informatics Journal · doi:10.1177/14604582241263242 · Cited by 28 · see citing works
“Most studies demonstrated high methodological quality, 21,23 – 27, 32,33,35,”
InfluentialGrizzle, A. J., Horn, J. R., Collins, C. J., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/11182 · Cited by 25 · see citing works
“These factors may contribute to the >90% override rate consistently reported for clinicians [4].”
Wang, L., Blackley, S. V., Blumenthal, K. G., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa042 · Cited by 24 · see citing works
“The study method was comprised of 5 steps: (1) allergen consolidation, (2) refinement of reaction value set, (3) reaction reconciliation from free-text entries, (4) development of dynamic reaction picklist,”
MethodPrevalence of Clinically Significant Drug-Drug Interactions Across US Children’s Hospitals
Antoon, J. W., Hall, M., Herndon, A. C., et al. (2020) · PEDIATRICS · doi:10.1542/peds.2020-0858 · Cited by 24 · see citing works
Drug–Drug–Gene Interactions in Cardiovascular Medicine
Asiimwe, I. G., & Pirmohamed, M. (2022) · Pharmacogenomics and Personalized Medicine · doi:10.2147/pgpm.s338601 · Cited by 23 · see citing works
User-centered design of multi-gene sequencing panel reports for clinicians
Cutting, E. M., Banchero, M., Beitelshees, A. L., Cimino, J. J., Fiol, G. D., Gürses, A. P., Hoffman, M. A., Jeng, L. J. B., Kawamoto, K., Kelemen, M. D., et al. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.07.014 · Cited by 22 · see citing works
“Participants agreed that the way the GMA Genetic Test Report tab was structured to provide optional access to information resources along with concise information summaries could help to overcome issues with alert fatigue (or desensitization to frequent alerts) [38].”
BackgroundA Late Attempt to Involve End Users in the Design of Medication-Related Alerts: Survey Study
Baysari, M. T., Zheng, W. Y., Dort, B. A. V., Reid-Anderson, H., Gronski, M., & Kenny, E. (2020) · Journal of Medical Internet Research · doi:10.2196/14855 · Cited by 23 · see citing works
“Despite international efforts to improve medication-related alerts, alert override rates remain as high as they were over a decade ago [6].”
BackgroundKnight, A. M., Maygers, J., Foltz, K. A., John, I. S., Yeh, H. C., & Brotman, D. J. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-3400447 · Cited by 22 · see citing works
Hammar, T., Lidström, B., Petersson, G., Gustafson, Y., & Eiermann, B. (2015) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-015-0146-8 · Cited by 21 · see citing works
“high despite substantial alert optimization [44, 45].”
Ethics, Medicine, and Information Technology
Goodman, K. W. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781139600330 · Cited by 40 · see citing works
Lower alert rates by clustering of related drug interaction alerts
Heringa, M., Siderius, H., Floor‐Schreudering, A., Smet, P. A. G. M. D., & Bouvy, M. L. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw049 · Cited by 19 · see citing works
“23 Therefore, there is a need for exploration of additional strategies.”
BackgroundBain, K. T., Knowlton, C. H., & Turgeon, J. (2017) · Clinics in Geriatric Medicine · doi:10.1016/j.cger.2017.01.009 · Cited by 20 · see citing works
Evaluation of Clinical Relevance of Drug–Drug Interaction Alerts Prior to Implementation
Meslin, S. M. M., Zheng, W. Y., Day, R. O., Tay, E., & Baysari, M. T. (2018) · Applied Clinical Informatics · doi:10.1055/s-0038-1676039 · Cited by 20 · see citing works
Heringa, M., Floor‐Schreudering, A., Smet, P. A. G. M. D., et al. (2017) · Drugs & Aging · doi:10.1007/s40266-017-0497-z · Cited by 19 · see citing works
“%) are not uncommon in routinely used CDSSs [27–30].”
Generating Evidence of Clinical Outcomes of Drug–Drug Interactions
Bykov, K., & Gagne, J. J. (2017) · Drug Safety · doi:10.1007/s40264-016-0496-8 · Cited by 18 · see citing works
An Investigation of Drug–Drug Interaction Alert Overrides at a Pediatric Hospital
Humphrey, K. E., Jorina, M., Harper, M. B., et al. (2018) · Hospital Pediatrics · doi:10.1542/hpeds.2017-0124 · Cited by 18 · see citing works
A systematic approach to optimize electronic health record medication alerts in a health system
Bhakta, S. B., Colavecchia, A. C., Haines, L., et al. (2019) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxz012 · Cited by 18 · see citing works
Survey of psychiatrist use of digital technology in clinical practice
Bauer, R., Glenn, T., Monteith, S., Whybrow, P. C., & Bauer, M. (2020) · International Journal of Bipolar Disorders · doi:10.1186/s40345-020-00194-1 · Cited by 18 · see citing works
Monteith, S., Glenn, T., Gitlin, M. J., & Bauer, M. (2020) · Pharmacopsychiatry · doi:10.1055/a-1156-4193 · Cited by 17 · see citing works
“Alert fatigue remains a major issue as the majority of DDI alerts are overridden [86, 87].”
BackgroundClinician Perceptions of Timing and Presentation of Drug-Drug Interaction Alerts
Humphrey, K. E., Mirica, M., Phansalkar, S., Ozonoff, A., & Harper, M. B. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1714276 · Cited by 17 · see citing works
Monteith, S., & Glenn, T. (2021) · Human Psychopharmacology Clinical and Experimental · doi:10.1002/hup.2802 · Cited by 17 · see citing works
“Physicians override most DDI alerts, with override rates of about 55%–95% (Bryant et al., 2014; Isaac et al., 2009; Poly et al., 2020), including nearly 90% of alerts for the highest severity DDI (Edrees et al., 2020).”
BackgroundBetter specification of triggers to reduce the number of drug interaction alerts in primary care
Heringa, M., van der Heide, A., Floor‐Schreudering, A., et al. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.11.005 · Cited by 17 · see citing works
“generation to the most severe drug interactions, and context-specific alert generation) [27,28], to reach an approach customized for the specific setting.”
Cornu, P., Phansalkar, S., Seger, D. L., Cho, I., Pontefract, S. K., Robertson, A., Bates, D. W., & Slight, S. P. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.12.027 · Cited by 16 · see citing works
Muylle, K. M., Gentens, K., Dupont, A. G., & Cornu, P. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104013 · Cited by 16 · see citing works
“The main problems are the low specificity of the alerts and their perceived lack of clinical importance [4,12,16,17].”
BackgroundWide variation and patterns of physicians’ responses to drug–drug interaction alerts
Cho, I., Lee, Y., Lee, J., & Bates, D. W. (2018) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzy102 · Cited by 16 · see citing works
Evaluation of Drug–Drug Interactions in Pharmacoepidemiologic Research
Chen, C., Nguyen, T. P. P., Hughes, J. E., Hennessy, S., Leonard, C. E., Miano, T. A., Douros, A., Gagne, J. J., & Bykov, K. (2025) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.70088 · Cited by 16 · see citing works
“This uncertainty makes DDI compendia unreliable [12–16] and leads to excessive DDI alerts in computerized provider order entry systems [17], the vast majority of which are routinely overridden [18–20].”
Bykov, K., Schneeweiß, S., Glynn, R. J., Mittleman, M. A., Bates, D. W., & Gagne, J. J. (2017) · Drug Safety · doi:10.1007/s40264-017-0556-8 · Cited by 14 · see citing works
Improving the specificity of drug-drug interaction alerts: Can it be done?
Reese, T., Wright, A. T., Liu, S., Boyce, R. D., Romero, A. V., Fiol, G. D., Kawamoto, K., & Malone, D. C. (2022) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxac045 · Cited by 14 · see citing works
Jiang, S., Mathias, P. C., Hendrix, N., Shirts, B. H., Tarczy-Hornoch, P. Z., Veenstra, D. L., Malone, D. C., & Devine, B. (2022) · The Pharmacogenomics Journal · doi:10.1038/s41397-022-00275-7 · Cited by 17 · see citing works
Clinician Response to Pharmacogenetic Clinical Decision Support Alerts
Lemke, L. K., Cicali, E. J., Williams, R., Nguyen, K. A., Cavallari, L. H., & Wiisanen, K. (2023) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3051 · Cited by 15 · see citing works
Electronic health records and patient safety: should we be discouraged?
Payne, T. H. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004039 · Cited by 13 · see citing works
Self-citationOptimizing vitamin D naming conventions in computerized order entry to support high-value care
White, A. A., McKinney, C. M., Hoffman, N. G., et al. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw038 · Cited by 13 · see citing works
Tukukino, C., & Wallerstedt, S. M. (2019) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/bcpt.13294 · Cited by 13 · see citing works
Role of Opioid-Involved Drug Interactions in Chronic Pain Management
Bain, K. T., & Knowlton, C. H. (2019) · Journal of Osteopathic Medicine · doi:10.7556/jaoa.2019.136 · Cited by 13 · see citing works
Nudging hospitals towards evidence‐based decision support for medication management
Westbrook, J. I., & Baysari, M. T. (2019) · The Medical Journal of Australia · doi:10.5694/mja2.50028 · Cited by 12 · see citing works
Gunn, L., Tunney, R. K., & Kelly, K. (2018) · Applied Clinical Informatics · doi:10.1055/s-0038-1626726 · Cited by 12 · see citing works
Baysari, M. T., Zheng, W. Y., Li, L., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026034 · Cited by 12 · see citing works
“17 There is now little doubt that improving alert specificity is critical for reducing frequent interruptions to prescriber workflow (ie, too many alerts) and improving the effectiveness of computerised alerts to prevent errors.”
BackgroundMeaningful time-related aspects of alerts in Clinical Decision Support Systems. A unified framework
Cánovas-Segura, B., Morales, A., Juárez, J. M., & Campos, M. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104397 · Cited by 11 · see citing works
Wan, Y.-K. J., Wright, M. C., McFarland, M. M., Dishman, D., Nies, M. A., Rush, A., Madaras-Kelly, K., Jeppesen, A., & Del Fiol, G. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad203 · Cited by 11 · see citing works
Schjøtt, J. D., & Aßmus, J. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0821-0 · Cited by 11 · see citing works
“A hospital study found that 93% of alerts from an electronic medical record and clinical decision system were overridden with no difference between experienced or unexperienced physicians [31].”
BackgroundModulators Influencing Medication Alert Acceptance: An Explorative Review
Bittmann, J. A., Haefeli, W. E., & Seidling, H. M. (2022) · Applied Clinical Informatics · doi:10.1055/s-0042-1748146 · Cited by 12 · see citing works
Colicchio, T. K., & Cimino, J. J. (2020) · Applied Clinical Informatics · doi:10.1055/s-0040-1710310 · Cited by 11 · see citing works
Bassir, F., Varghese, S., Wang, L., Chin, Y. P., & Zhou, L. (2022) · Immunology and Allergy Clinics of North America · doi:10.1016/j.iac.2022.01.004 · Cited by 10 · see citing works
Bole, C. B., Nagode, K., Pišlar, M., Mrhar, A., Grabnar, I., & Vovk, T. (2023) · Medicina · doi:10.3390/medicina59020284 · Cited by 12 · see citing works
Karajizadeh, M., Zand, F., Vazin, A., Saeidnia, H. R., Lund, B. D., Tummuru, S. P., & Sharifian, R. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105135 · Cited by 10 · see citing works
Lofthus, D. M., Gadgil, J. Y., & de Lemos, J. A. (2015) · Clinical Chemistry · doi:10.1373/clinchem.2014.236406 · Cited by 10 · see citing works
Drug Interactions in Space: a Cause for Concern?
Berman, E., & Eyal, S. (2019) · Pharmaceutical Research · doi:10.1007/s11095-019-2649-9 · Cited by 10 · see citing works
Mo, Y., Karakas-Torgut, A., & Pham, A. Q. (2018) · Journal of Pharmacy Practice · doi:10.1177/0897190018788264 · Cited by 10 · see citing works
“Finally, previous studies suggest that override rates of DDI alerts are as high as 95%.(16) At least 223 DOAC-drug interacting alerts were overridden during the study period, but override rates were not fully investigated in this study.”
BackgroundHave Electronic Health Records Improved the Quality of Patient Care?
Krenn, L., & Schlossman, D. (2017) · PM&R · doi:10.1016/j.pmrj.2017.04.001 · Cited by 8 · see citing works
Colicchio, T. K., & Cimino, J. J. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104508 · Cited by 8 · see citing works
Habit and Automaticity in Medical Alert Override: Cohort Study
Wang, L., Goh, K. H., Yeow, A. Y. K., Poh, H. M. N., Li, K., Yeow, J. J. L., Tan, G. Y. H., & Soh, C. (2022) · Journal of Medical Internet Research · doi:10.2196/23355 · Cited by 9 · see citing works
“However, the ubiquitous use of such alerts has led to a significant number of alert dismissals [2-8].”
BackgroundWright, A. P., Embí, P. J., Nelson, S. D., Smith, J., Turchin, A., & Mize, D. L. E. (2022) · Journal of Diabetes Science and Technology · doi:10.1177/19322968221119788 · Cited by 9 · see citing works
Health Information Technology in Long-Term Care: Potential for the Future
Little, M. O., Rantz, M., & Lynn, G. A. (2016) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2016.02.029 · Cited by 8 · see citing works
An Exploration of the Terminology of Clinical Cognition and Reasoning.
Cimino, J. J., Li, Z., & Weng, C. (2018) · PubMed · Cited by 8 · see citing works
Reese, T. J., Fiol, G. D., Morgan, K. L., Hurwitz, J. T., Kawamoto, K., Gómez-Lumbreras, A., Brown, M., Thiess, H., Vazquez, S. R., Nelson, S. D., et al. (2021) · JMIR Human Factors · doi:10.2196/28618 · Cited by 8 · see citing works
“Most electronic health records (EHRs) implement DDI clinical decision support functionality with underlying logic provided by drug knowledge base vendors, but DDI alerts continue to be overridden at rates as high as 90% [9-11].”
BackgroundDo user preferences align with human factors assessment scores of drug–drug interaction alerts?
Lowenstein, D., Zheng, W. Y., Burke, R. A., Kenny, E., Sandhu, A., Makeham, M., Westbrook, J. I., Day, R. O., & Baysari, M. T. (2019) · Health Informatics Journal · doi:10.1177/1460458219840210 · Cited by 7 · see citing works
Colicchio, T. K., Dissanayake, P. I., & Cimino, J. J. (2020) · PubMed Central · Cited by 7 · see citing works
- Authors
- Bryant, A. D., Fletcher, G. S., & Payne, T. H.
- Affiliations
- University of Washington (US)
- Corresponding author
- Payne, T.H.
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2014
- Volume
- 05
- Issue
- 03
- Pages
- 802-813
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC4187095
- OpenAlex
- W1155385988
- Semantic Scholar
- 030961ec3d58fe1888e09ba078dd667a6f9136e9
- Cited by
- 172
- References
- 28
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 5 · 2023: 14 · 2022: 21 · 2021: 13 · 2020: 20 · 2019: 25 · 2018: 14 · 2017: 18 · 2016: 15 · 2015: 12
- FWCI
- 27.37
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Applied Clinical Informatics (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Keywords
- drug interaction alerts, clinical decision support, alarm fatigue, meaningful use, drug-drug interactions, drug allergy alerts, hospital comparison
- MeSH terms
- Humans, Medication Errors, Washington, Incidence, Prevalence, Clinical Pharmacy Information Systems, Adverse Drug Reaction Reporting Systems, Evidence-Based Medicine, Electronic Prescribing, Pharmacovigilance, Meaningful Use, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
