The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index; none of the 12 sources its claims cite is checked for support yet.
Drug interaction alert override rates in the Meaningful Use era: No evidence of progress
Open accessAbstract
Summary: Background: Interruptive drug interaction alerts may reduce adverse drug events and are required for Stage I Meaningful Use attestation. For the last decade override rates have been very high. Despite their widespread use in commercial EHR systems, previously described interventions to improve alert frequency and acceptance have not been well studied. Objectives: (1) To measure override rates of inpatient medication alerts within a commercial clinical decision support system, and assess the impact of local customization efforts. (2) To compare override rates between drug-drug interaction and drug-allergy interaction alerts, between attending and resident physicians, and between public and academic hospitals. (3) To measure the correlation between physicians’ individual alert quantities and override rates as an indicator of potential alert fatigue. Methods: We retrospectively analyzed physician responses to drug-drug and drug-allergy interaction alerts, as generated by a common decision support product in a large teaching hospital system. Results: (1) Over four days, 461 different physicians entered 18,354 medication orders, resulting in 2,455 visible alerts; 2,280 alerts (93%) were overridden. (2) The drug-drug alert override rate was 95.1%, statistically higher than the rate for drug-allergy alerts (90.9%) (p < 0.001). There was no significant difference in override rates between attendings and residents, or between hospitals. (3) Physicians saw a mean of 1.3 alerts per day, and the number of alerts per physician was not significantly correlated with override rate (R2 = 0.03, p = 0.41). Conclusions: Despite intensive efforts to improve a commercial drug interaction alert system and to reduce alerting, override rates remain as high as reported over a decade ago. Alert fatigue does not seem to contribute. The results suggest the need to fundamentally question the premises of drug interaction alert systems. Citation: Bryant AD, Fletcher GS, Payne TH. Drug interaction alert override rates in the Meaningful Use era: No evidence of progress. Appl Clin Inf 2014; 5: 802–813 http://dx.doi.org/10.4338/ACI-2013-12-RA-0103
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
In a 96-hour retrospective audit at two UW Medicine teaching hospitals using a commercial Cerner/Multum CDS, 93% of 2,455 physician-visible drug interaction alerts were overridden, with drug-drug alerts overridden more often than drug-allergy alerts (95.1% vs 90.9%). Despite local reclassification efforts, override rates remained as high as historical reports, and alert quantity was not correlated with override rate.
Key findings
- Over four days, 461 physicians entered 18,354 medication orders, generating 2,455 visible alerts; 2,280 alerts (93%) were overridden.
- Drug-drug alert override rate was 95.1%, significantly higher than drug-allergy alerts at 90.9% (p < 0.001); there was no significant difference between attendings and residents (p = 0.11) or between hospitals (p = 0.25).
- Physicians saw a mean of 1.3 alerts per day; individual alert quantity was not significantly correlated with override rate (R² = 0.03, p = 0.41).
- Exact allergy matches were 123/1,302 (9%) of allergy alerts, and 94/123 (76%) of these exact-match alerts were overridden; antibiotic allergy alerts had the highest compliance among drug-allergy classes (121/172, 70% overridden).
Methods
- Retrospective review of major medication alerts at UW Medical Center and Harborview Medical Center from Monday, June 10, 2013 00:00 to Thursday, June 13, 2013 23:59 (96 hours), using Cerner Millennium with the Multum knowledge base; only major interactions generated interruptive physician alerts.
- Alerts were included if they occurred in the study period, were handled by a physician, and occurred on an inpatient; there were no exclusion criteria. Records were filtered to MD, DO, DDS, or MBBS credentials, and coincident alerts in the same window were counted as a single override in the main analysis.
- Override rates across categories were compared with Pearson’s chi-squared tests; alert quantity and override rate were correlated with simple linear regression and a 2-tailed F test; significance was α = 0.05; analysis used Microsoft Excel.
Limitations
- Stated: the study examined only medication alert override rates and associated metadata, over a short time span, within a single vendor’s CDS; it did not assess alert or override appropriateness, review adverse events, or examine behavior outside the alert window, and could not determine whether alert fatigue or other factors increased inappropriate overrides. The authors also note that some physicians may override alerts but later cancel orders, leading to falsely high estimates of noncompliance, and that high baseline override rates may have masked differences due to alert fatigue or other variability.
- Evident from the text: the findings come from one health system’s two hospitals over four days, are limited to physicians, and do not include patient outcomes or independent appropriateness review; override reasons were coded and most physicians selected the first option, which may limit interpretation of override intent.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
26 Own result · 9 Not checked yet · 9 Own interpretation · 6 Own method · 3 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Opioids accounted for the majority of allergy alert triggers; drug-drug alert triggers were more diverse.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Physicians override cross-reaction alerts more often than exact allergy matches, and likely view the former with a healthy skepticism that is only enforced by daily false alarms.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 5. Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Antibiotic allergy alerts had the highest compliance rate.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 5. Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most (76% or 94/123) of these exact allergy alerts were overridden.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Individual override rates varied widely, even among physicians with identical alert quantities.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Over four days, 461 different physicians entered 18,354 medication orders, resulting in 2,455 visible alerts; 2,280 alerts (93%) were overridden.”
- Also stated in: § 4. Results · plain
“During the four day audit period providers entered 43,287 medication orders, but only 18,354 were entered directly by physicians.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4. Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Physicians triggered a median of 3 alerts over the 4 day period, with a mean of 1.3 alerts per day, while entering an average of 10 medication orders per day.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The four day audit period captured 2,455 distinct alerts, involving 461 distinct physicians, for an overall alerting rate of 13.4% (2,455 alerts per 18,354 orders)”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There was no significant difference in override rates between the two hospital sites (p = 0.25), or between attending and resident physicians (p = 0.11).”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Fewer than 5% of individuals viewed more than 4 alerts per day.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 11
- Probably not a substantial claim. 10
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the drug-drug alert override rate was significantly higher than that for drug-allergy alerts (95.1% vs. 90.9%,p < 0.001)”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“About 20% of all alerts occurred as multiples within the same alert window.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A similar review of the allergy cross-reactivity rules before implementation found that most were appropriate.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.2 Knowledge base customization
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Override rates for drug-allergy alerts were similarly high among all classes, with the notable exception of antibiotics.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“All interactions were reviewed before the implementation of CDS, causing 65% of all major interactions to be reclassified to lower levels [15].”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.2 Knowledge base customization
What the sources say
- Not checked yetcites [15] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Most of our allergy alerts were triggered by narcotic allergies, or by cross-reactions of unknown significance [29,30].”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 5. Discussion
What the sources say
- Not checked yetcites [29] Abookire SA, Teich JM, Sandige H, Paterno MD, Martin MT, Kuperman GJ, Bates DW. Improving allergy alerting in a computerized physician order entry system. Proc AMIA Symp 2000: 2-6
- Not checked yetcites [30] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, the olanzapine-lorazepam drug interaction alert occurred 92 times and was overridden every time.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 4. Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Similar override rates of over 90% have been found in drug dose alerts [21].”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 5. Discussion
What the sources say
- Not checked yetcites [21] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This continuous 96 hour order entry period provided adequate data and excluded dates of resident and faculty turnover.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.3 Design and scope
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Rests on an assumption: The four-day audit period is representative of typical inpatient medication ordering and alerting at the study hospitals.
In the paper
“The panel uses the operational classification scheme described by Hansten and Horn to reassign major drug-drug interactions to lower severity levels, and thus decrease interruptive alerting [14].”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.2 Knowledge base customization
What the sources say
- Not checked yetcites [14] Hansten, 2001, J Am Pharm Assoc source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Because the override of multiple alerts represented a single decision, these coincident alerts were included as a single override in the main analysis.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.4 Analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The frequent overrides of most alert types including exact allergy matches, and the often inappropriate (unmatched to the alert) override reasons suggest that physicians dismiss alerts without assessing the specific risk of each alert.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 5. Discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“While the overall risk of ADEs is low, our very high override rates could increase the absolute number of inappropriate overrides, raising the ADE risk [5,28].”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 5. Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This study was approved by the University of Washington Human Subjects Division.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.4 Analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Because of these confounding factors, we focused on physicians alone to assess the impact of alert reclassification.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.4 Analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“The lack of correlation between alert quantity and override rate also supports prior evidence that reducing the number of alerts will not appreciably increase physician acceptance or reduce overrides [23].”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 5. Discussion
What the sources say
- Not checked yetcites [23] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“A panel of physicians, pharmacists, and information technology staff has met monthly since 2008 to review all major drug-drug interaction alerts.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.2 Knowledge base customization
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The study period fell at the end of the academic year to ensure that all providers had several months experience with the order entry system.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 3.3 Design and scope
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, our experience illustrates the substantial obstacles when trying to customize a commercial product to local needs.”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 6. Conclusions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“As Hayward et al. describe, alerts appear”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > 5. Discussion
What the sources say
- Not checked yetcites [24] source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
Override rates are a valid measure of clinical decision support acceptance and utility.
“We are the first to report a lack of improvement in decision support acceptance (as measured by overrides) after Meaningful Use requirements took effect.”
The paper relies on this without support.
The four-day audit period is representative of typical inpatient medication ordering and alerting at the study hospitals.
“This continuous 96 hour order entry period provided adequate data and excluded dates of resident and faculty turnover.”
The paper relies on this without support.
The study's commercial CDS system is similar to those used in many U.S. hospitals.
“We demonstrated this lack of improvement within a popular commercial decision support system, similar to those used in many hospitals nationwide.”
The paper relies on this without support.
The local customization efforts represent a strong effort to improve CDS acceptance.
“University of Washington made a strong effort to improve CDS and its acceptance based on suggestions in the literature using local expertise to modify the default alerts within the constraints of our commercial system.”
The paper relies on this without support.
Historical override rates reported in prior studies are comparable to the current study's rates.
“Our current rates are at or above the rates in previous studies, including those at our regional VA hospital [7,8].”
The authors state this as an assumption.
If alert fatigue contributes to overrides, physicians with higher alert quantities will have higher override rates.
“Further, we hypothesized that physicians who saw higher quantities of alerts would have higher override rates, as predicted by the theory of alert fatigue.”
The paper relies on this without support.
The coded override reason selected by a physician accurately reflects the physician's reason for overriding the alert.
“Most overriding physicians entered the reason”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“High override rates of 56.3%-95.7% have already been described for DDI alerts [13, 21, 27–32].”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Optimal alert volume is unknown, but alerts fire so frequently that most are ignored, with prior research demonstrating rates of ignoring as high as 96%, almost entirely defeating their purpose.”
From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The inherent design of our site’s EHR makes interruptive alerts one of the easiest and least expensive forms of CDS to implement, even though they are known to be disruptive and often ineffective. 5 15 16”
From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Bryant and colleagues also recently found that alert volume was not correlated with override rate [24].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [24]
The cited source says what the claim attributes to it.
In the citing paper
“Our study results are in agreement with previous reports.”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [21]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Total 2455 1153 1302 123 1179 480 1975 1200 1255 % of all alerts 100 47 53 9 48 20 80 49 51”
§ Drug interaction alert override rates in the Meaningful Use era No evidence of progress > Interaction type
Reported 9 · recomputed 5.010183
Citations
Cites
Bates, D. W. (1998) · JAMA · doi:10.1001/jama.280.15.1311 · Cited by 2,107
The Impact of Computerized Physician Order Entry on Medication Error Prevention
Bates, D. W., Teich, J. M., Lee, J., Seger, D., Kuperman, G. J., Ma'Luf, N., Boyle, D., & Leape, L. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.00660313 · Cited by 1,260
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117
Drug-Drug Interactions Among Elderly Patients Hospitalized for Drug Toxicity
Juurlink, D. N., Mamdani, M., Kopp, A., Laupacis, A., & Redelmeier, D. A. (2003) · JAMA · doi:10.1001/jama.289.13.1652 · Cited by 766
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763
Self-citationPhysicians' Decisions to Override Computerized Drug Alerts in Primary Care
Weingart, S. N., Toth, M., Sands, D. Z., Aronson, M. D., Davis, R. B., & Phillips, R. S. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.21.2625 · Cited by 487
Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353
Absence of Cross-Reactivity between Sulfonamide Antibiotics and Sulfonamide Nonantibiotics
Strom, B. L., Schinnar, R., Apter, A. J., Margolis, D. J., Lautenbach, E., Hennessy, S., Bilker, W. B., & Pettitt, D. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa022963 · Cited by 349
Tiering Drug-Drug Interaction Alerts by Severity Increases Compliance Rates
Paterno, M. D., Maviglia, S. M., Gorman, P. N., Seger, D. L., Yoshida, E., Seger, A. C., Bates, D. W., & Gandhi, T. K. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2808 · Cited by 263
Phansalkar, S., van der Sijs, H., Tucker, A. D., Desai, A. A., Bell, D. S., Teich, J. M., Middleton, B., & Bates, D. W. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001089 · Cited by 239
Research works printed but not linked: 1
- Langemeijer, 2011, Stud Health Technol Inform, 169, 930
Cited by
Sorted by citations
Items 101–164 of 164
Naeem, A., Alwadie, A. F., Alshehri, A., Alharbi, L. M., Nawaz, M. U., AlHadidi, R. A., Alshammari, R. S., Alsufyani, M. A., Babsail, L., Alshamrani, S. A., et al. (2022) · Cureus · doi:10.7759/cureus.31542 · Cited by 6
“These findings are in agreement with previous studies that found DDIs had the highest override rate (87%) and generate a large number of useless alerts due to a lack of relevance and specificity [18-21].”
ResultPhysician response to a medication alert system in inpatients with levodopa-treated diseases
Morris, M. T., Willis, A. W., Nielsen, S. S., McCann, F., Birke, A., & Racette, B. A. (2015) · Neurology · doi:10.1212/wnl.0000000000001745 · Cited by 8
“When dozens of alerts are triggered from a medication list, physicians can become desensitized to them and override alerts,(8) although at least one study found no correlation between drug-drug alert override rates and the number of alerts.(9) Other studies have suggested that alert fatigue can result in adverse outco…”
BackgroundNature and management of duplicate medication alerts
Heringa, M., Floor‐Schreudering, A., Meijer, W. M., Smet, P. A. G. M. D., & Bouvy, M. L. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv021 · Cited by 7
Heringa, M., Floor‐Schreudering, A., Smet, P. A. G. M. D., & Bouvy, M. L. (2017) · Patient Education and Counseling · doi:10.1016/j.pec.2017.11.010 · Cited by 7
Clinical Utility of Definitive Drug–Drug Interaction Testing in Primary Care
Peabody, J. W., Tran, M. T., Paculdo, D. R., Schrecker, J. P., Valdenor, C., & Jeter, E. K. (2018) · Journal of Clinical Medicine · doi:10.3390/jcm7110384 · Cited by 7
“These disparities and the rising tendency for physicians to override system alerts [18] suggests these tools are not working.”
BackgroundShakowski, C., Page, R. L., II, Wright, G. C., Lunowa, C., Marquez, C., Suresh, K., Allen, L. A., Glasgow, R. E., Lin, C., Wick, A., et al. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad109 · Cited by 5
Baysari, M. T., Hilmer, S. N., Day, R. O., Dort, B. A. V., Zheng, W. Y., Quirk, R., Deidun, D., Moran, M., Stanceski, K., Aryal, N. R., et al. (2025) · BMJ Quality & Safety · doi:10.1136/bmjqs-2024-018243 · Cited by 10
“Implementing these tailored DDI alerts, by either”
Bryan, A. L., & Lammers, J. C. (2020) · Journal of Professions and Organization · doi:10.1093/jpo/joaa023 · Cited by 6
Prescribers’ reported acceptance and use of drug-drug interaction alerts: An Australian survey
Baysari, M. T., Dort, B. A. V., Zheng, W. Y., Li, L., Hilmer, S. N., Westbrook, J. I., & Day, R. O. (2022) · Health Informatics Journal · doi:10.1177/14604582221100678 · Cited by 7
Clinical Decision Support: History and Basic Concepts
McCallie, D. P. (2015) · Health informatics · doi:10.1007/978-3-319-20765-0_1 · Cited by 6
“Some studies suggest that even with these additional factors, it is diffi cult to avoid generating too many alerts [ 8 ].”
BackgroundExamination of medication clinical decision support using Bayes’ theorem
Payne, T. H., & Desai, B. R. (2016) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp150964 · Cited by 6
Self-citationZhang, Y., Long, X., Chen, W., Li, H., Duan, H., & Shang, Q. (2016) · SpringerPlus · doi:10.1186/s40064-016-3701-4 · Cited by 6
“Recent related work shows override rates can be as high as 53.6% (Nanji et al. 2014), 87.6% (Topaz et al. 2015), and 93% (Bryant et al. 2013) respectively.”
BackgroundInfluentialPilot evaluation of a method to assess prescribers’ information processing of medication alerts
Russ, A. L., Melton, B. L., Daggy, J., & Saleem, J. J. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.11.011 · Cited by 7
Adverse events targeted by drug-drug interaction alerts in hospitalized patients
Gatenby, J., Blomqvist, M., Burke, R. A., et al. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104266 · Cited by 6
Effect of Replacing Vendor QTc Alerts with a Custom QTc Risk Alert in Inpatients
Stettner, S., Adie, S. K., Hanigan, S., Thomas, M. P., Pogue, K. T., & Zimmerman, C. (2022) · Applied Clinical Informatics · doi:10.1055/s-0041-1740483 · Cited by 7
Allergic Adverse Drug Events After Alert Overrides in Hospitalized Patients
de Manzanos, M. L. F., Gastaminza, G., Idoate, A., & Ortega, A. (2022) · Journal of Patient Safety · doi:10.1097/pts.0000000000001034 · Cited by 7
Chazard, E., Boudry, A., Beeler, P. E., Dalleur, O., Hubert, H., Tréhou, E., Beuscart, J., & Bates, D. W. (2020) · JMIR Medical Informatics · doi:10.2196/20862 · Cited by 4
“If the number of DDIs is large, however, the resulting over-alerting [11-15] may produce “alert fatigue” [11], a mental state close to overwork caused by the clinician's exposure to a continuous flow of alerts, regardless of whether or not they are relevant [11-16].”
Sundermann, M., Clendon, O., McNeill, R., Doogue, M. P., & Chin, P. K. L. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105418 · Cited by 6
Improving Bridging from Informatics Practice to Theory
Lehmann, C. U., & Gundlapalli, A. (2015) · Methods of Information in Medicine · doi:10.3414/me15-01-0138 · Cited by 4
“…settings including immunosuppressive therapy in transplant centers [18], medication administration and laboratory alerts in retirement/nursing homes [19, 20], as well as the continued high override rate of drug-drug alerts [21] requiring a tailoring of drug-drug alerts to specific user groups [22].”
BackgroundLee, H., Choi, H. S., & Ji, E. (2019) · Korean Journal of Clinical Pharmacy · doi:10.24304/kjcp.2019.29.1.9 · Cited by 5
Drug-Induced Long QT Syndrome and Torsades de Pointes
Woosley, R. L., & Schwartz, P. J. (2019) · Springer eBooks · doi:10.1007/978-3-030-22672-5_10 · Cited by 5
Design and Evaluation of Trust–Eliciting Cues in Drug–Drug Interaction Alerts
Chattopadhyay, D., Verma, N., Duke, J. D., & Bolchini, D. P. (2017) · Interacting with Computers · doi:10.1093/iwc/iwx020 · Cited by 5
Navigating the Minefield of QTc Interval‐Prolonging Therapy in Nursing Facility Residents
Zarowitz, B. J. M., & Tisdale, J. E. (2019) · Journal of the American Geriatrics Society · doi:10.1111/jgs.15810 · Cited by 4
“As many as 95% of EMR warnings are overridden, some perhaps appropriately, but many due to alert fatigue.(47,48) In follow-up to the publication of high-priority drug-drug interaction pairs, an investigation found inconsistent implementation and display of alerts (27%-93% implementation).”
BackgroundSomogyi‐Végh, A., Ludányi, Z., Erdős, Á., & Botz, L. (2019) · BMC Pharmacology and Toxicology · doi:10.1186/s40360-019-0311-0 · Cited by 5
Raven, K., Zhang, J., Vagliano, I., Jaspers, M., Peute, L., & Medlock, S. K. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106011 · Cited by 3
Kijtawornrat, A., Boonyarattanasoonthorn, T., & Khemawoot, P. (2021) · Veterinary Sciences · doi:10.3390/vetsci8040060 · Cited by 3
“Many healthcare providers dislike using drug interaction databases [27] for several reasons, including alert fatigue [28,29], workflow disturbance [30], and belief that there is no clinical significance related to most DDI alerts [31].”
BackgroundSafely prescribing opioids with Nirmatrelvir/Ritonavir - Case report and management recommendations.
Admane, S., Clark, M. D., Reddy, A., Narayanan, S., & Bruera, E. (2023) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2023.09.027 · Cited by 2
Barriers to Adoption of Tailored Drug–Drug Interaction Clinical Decision Support
Zhang, T., Gephart, S. M., Subbian, V., Boyce, R. D., Zapata, L. V., Tan, M. S., Horn, J. R., Gómez-Lumbreras, A., Romero, A. V., & Malone, D. C. (2023) · Applied Clinical Informatics · doi:10.1055/s-0043-1772686 · Cited by 3
Redinger, K. E., Rozin, E., Schiller, T., et al. (2021) · Journal of Emergency Medicine · doi:10.1016/j.jemermed.2021.09.014 · Cited by 3
Colicchio, T. K., ElHalta, D., Fiol, G. D., Kawamoto, K., Strasberg, H. R., & Cimino, J. J. (2025) · Applied Clinical Informatics · doi:10.1055/a-2546-5954 · Cited by 2
Gheorghita, F.-I., Bocanet, V.-I., & Iantovics, L. B. (2025) · Applied Sciences · doi:10.3390/app15158653 · Cited by 2
Zhong, H., Jack, M., O’Leary, K. J., et al. (2025) · Clinical and Translational Science · doi:10.1111/cts.70193 · Cited by 2
Finkel, N., & Galvin, H. K. (2017) · Hospital Medicine Clinics · doi:10.1016/j.ehmc.2016.11.007 · Cited by 2
INTO THE BLACK BOX: DESIGNING FOR TRANSPARENCY IN ARTIFICIAL INTELLIGENCE
Vorm, E. S. (2019) · IUScholarWorks (Indiana University) · doi:10.13140/rg.2.2.12257.33120 · Cited by 2
Sheehan, K. N., Cioci, A., Lucioni, T., & Hernandez, S. (2024) · Applied Clinical Informatics · doi:10.1055/s-0044-1786682 · Cited by 1
John, A., Alhajj, R., & Rokne, J. G. (2026) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-026-00763-x · Cited by 1
Endorsement, Prior Action, and Language
Chattopadhyay, D., Duke, J. D., & Bolchini, D. (2016) · doi:10.1145/2851581.2892315 · Cited by 1
“Although intended for safe prescribing of drugs, studies found that up to 95% of DDI alerts are ignored by physicians [2, 13].”
BackgroundWechselwirkungen mit zielgerichtet wirksamen oralen Tumortherapeutika
Lipp, H. (2016) · best practice onkologie · doi:10.1007/s11654-016-5285-4 · Cited by 1
“eine ernüchternde Antwort: Aufgrund einer teilweise zu hohen Rate an Warnmeldungen („over-alerting“) wurden in der täglichen Praxis über 90% der „Alarme“ von Ärzten außer Kra gesetzt [15].”
Electronic medication management: enthusiasm justified but caution required
Shakib, S. (2017) · Journal of Pharmacy Practice and Research · doi:10.1002/jppr.1343 · Cited by 1
Snapshot at Mid-stride: Current State of EHRs and Their Use by Clinicians from a CMIO’s Perspective
Payne, T. H. (2015) · Health informatics · doi:10.1007/978-3-319-20765-0_4 · Cited by 1
Self-citationWechselwirkungsrisiken mit zielgerichtet wirksamen, oralen Tumortherapeutika – ein Überblick
Lipp, H. (2017) · Deutsche Zeitschrift für Onkologie · doi:10.1055/s-0043-101110 · Cited by 1
“Da sie oft viele In-vitro-Befunde berücksichtigen, besteht das Risiko einer zu hohen Warnmeldungsrate, sodass in der täglichen Praxis sehr häufig entsprechende „Alarme“ ignoriert werden [1].”
BackgroundPanich, J., Gooden, A., Shirazi, F. “., et al. (2018) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2018.09.008 · Cited by 1
Martell, J. (2019) · DalSpace (Dalhousie University) · Cited by 1
Harris, S. T., Bell, P. D., & Baker, E. A. (2017) · The Health Care Manager · doi:10.1097/hcm.0000000000000184
“Physicians can be overwhelmed with the number of alerts received in a day,(11) which may cause them to override alerts, which should not be overridden.(12) Cell phones can carry several persistent viruses on their surfaces and so act as reservoirs of infection.”
BackgroundClinical Decision Support Improves Adherence to Pediatric Antimicrobial Renal Dosing Guidelines
Quach, A., Ateya, M., & Zimmerman, C. (2018) · ACI Open · doi:10.1055/s-0038-1667296
Sistem Entri Resep Dokter Terkomputerisasi dalam Sistem Informasi Rumah Sakit: Literature Review
Ilhamsyah, R., & Setyonugroho, W. (2020) · JATISI (Jurnal Teknik Informatika dan Sistem Informasi) · doi:10.35957/jatisi.v7i2.300
da Silva, P. I., de Souza, R. R., Cascão, P. C., et al. (2021) · Research Society and Development · doi:10.33448/rsd-v10i7.15759
Srinivasan, M., White, A., Lott, J. P., Williamson, T. E., Kong, S. X., & Plouffe, L. (2021) · Gates Open Research · doi:10.12688/gatesopenres.13430.1
Yusof, M. M., Olakotan, O. O., & Puteh, S. E. W. (2019) · doi:10.2196/preprints.16809
Abdulkadir, S. A., Wettermark, B., & Hammar, T. (2023) · Pharmacy · doi:10.3390/pharmacy11010035
“Alert overrides means that the health care provider may not follow information presented due to alert fatigue, caused by alerts that are not clinically relevant or an unreasonable number of alerts [13].”
BackgroundAlert Fatigue and Alert Override Significance in Relation to CDSS Success
Eagen, A. (2023) · Acta Scientific Medical Sciences · doi:10.31080/asms.2023.07.1559
Lenert, L. A. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-91200-6.00015-2
Felisberto, M., Lima, G. D. S., Celuppi, I. C., Fantonelli, M. D. S., Zanotto, W. L., de Oliveira, J. M. D., Mohr, E. T. B., Santos, R. A., Scandolara, D. H., Cunha, C. L., et al. (2023) · Research Square · doi:10.21203/rs.3.rs-2977577/v1
Evaluation of an electronic health record Drug Interaction Customization Editor (DICE)
Romero, A. V., Gómez-Lumbreras, A., Zapata, L. V., Tan, M. S., Horn, J. R., & Malone, D. C. (2024) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxae169
Smith, S. N., Lanham, M. S. M., Seagull, F. J., et al. (2024) · JMIR Formative Research · doi:10.2196/64674
“Clinical alerts are often prone to high false alarm rates and are frequently ignored or overridden [16].”
BackgroundPeaper, D. R., Rathod, S. N., Sussman, L. S., et al. (2025) · Infection Control and Hospital Epidemiology · doi:10.1017/ice.2025.30
“40 While overall rates of testing did not differ between the interactions that received a phone call and those that did not, there were clinicians who may not have pursued C. difficile testing based on the phone conversation.”
Zimmerman, C. R., Adie, S. K., & Bhave, N. M. (2025) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxaf283
O’Sullivan, C., Starkey, T., & Pirmohamed, M. (2025) · doi:10.2196/preprints.88578 · Cited by 1
Crupi, L., Létinier, L., Jouhet, V., et al. (2026) · Drug Safety · doi:10.1007/s40264-025-01637-3
Elektronische Verordnung im stationären Sektor
Grabe, S. (2026) · Springer eBooks · doi:10.1007/978-3-662-71044-9_20
Masud, J. H. B., Cimino, J. J., & Colicchio, T. K. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag010
“17,18 Although many overrides are clinically justified, inappropriate overrides pose significant risks, including a 6-fold increase in the risk of adverse drug events (ADEs).”
Beyond alert fatigue: Automation bias, deskilling, and clinical decision-making in the age of AI
Nelson, S. D., & Wright, A. T. (2026) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxag227 · Cited by 1
Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support
Khan, A., & Schnipper, J. (2026) · Applied Clinical Informatics · doi:10.1055/a-2955-9555
Metadata
- Authors
- Bryant, A. D., Fletcher, G. S., & Payne, T. H.
- Affiliations
- University of Washington (US)
- Corresponding author
- Payne, T.H.
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2014
- Volume
- 05
- Issue
- 03
- Pages
- 802-813
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC4187095
- OpenAlex
- W1155385988
- Semantic Scholar
- 030961ec3d58fe1888e09ba078dd667a6f9136e9
- Cited by
- 172
- References
- 28
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 5 · 2023: 14 · 2022: 21 · 2021: 13 · 2020: 20 · 2019: 25 · 2018: 14 · 2017: 18 · 2016: 15 · 2015: 12
- FWCI
- 27.37
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Applied Clinical Informatics (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Patient Safety and Medication Errors
- Keywords
- drug interaction alerts, clinical decision support, alarm fatigue, meaningful use, drug-drug interactions, drug allergy alerts, hospital comparison
- MeSH terms
- Humans, Medication Errors, Washington, Incidence, Prevalence, Clinical Pharmacy Information Systems, Adverse Drug Reaction Reporting Systems, Evidence-Based Medicine, Electronic Prescribing, Pharmacovigilance, Meaningful Use, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
