The DOI matches its Crossref record; 34 of the 36 references it cites were found in a registry or an index; 33 of the 45 sources its claims cite were checked for support.
Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey
Open accessAbstract
Background: Clinical decision support systems are implemented in many hospitals to prevent medication errors and associated harm. They are however associated with a high burden of false positive alerts and alert fatigue. The aim of this study was to evaluate a drug–drug interaction (DDI) clinical decision support system in terms of its performance, uptake and user satisfaction and to identify barriers and opportunities for improvement. Methods: A quantitative evaluation and end-user survey were performed in a large teaching hospital. First, very severe DDI alerts generated between 2019 and 2021 were evaluated retrospectively. Data collection comprised alert burden, override rates, the number of alert overrides reviewed by pharmacists and the resulting pharmacist recommendations as well as their acceptance rate. Second, an e-survey was carried out among prescribers to assess satisfaction, usefulness and relevance of DDI alerts as well as reasons for overriding. Results: A total of 38,409 very severe DDI alerts were generated, of which 88.2% were overridden by the prescriber. In 3.2% of reviewed overrides, a recommendation by the pharmacist was provided, of which 79.2% was accepted. False positive alerts were caused by a too broad screening interval and lack of incorporation of patient-specific characteristics, such as QTc values. Co-prescribing of a non-vitamin K oral anticoagulant and a low molecular weight heparin accounted for 49.8% of alerts, of which 92.2% were overridden. In 88 (1.1%) of these overridden alerts, concurrent therapy was still present. Despite the high override rate, the e-survey revealed that the DDI clinical decision support system was found useful by prescribers. Conclusions: Identified barriers were the lack of DDI-specific screening intervals and inclusion of patient-specific characteristics, both leading to a high number of false positive alerts and risk for alert fatigue. Despite these barriers, the added value of the DDI clinical decision support system was recognized by prescribers. Hence, integration of DDI-specific screening intervals and patient-specific characteristics is warranted to improve the performance of the DDI software.
Overview
CoChat Summary
TL;DR
This study evaluated a drug–drug interaction (DDI) clinical decision support system (CDSS) in a large Belgian teaching hospital using retrospective alert data and a prescriber survey. It found high override rates and false-positive alerts driven by a broad screening interval and lack of patient-specific data, although prescribers still rated the system useful.
Key findings
- Of 38,409 very severe DDI alerts across 64 DDI pairs, 33,888 (88.2%) were overridden by prescribers, giving an initial acceptance rate of 11.8%.
- CMA pharmacists reviewed 14,320 overrides; 453 (3.2%) led to a pharmacist recommendation, of which 79.2% were accepted. NOAC plus another anticoagulant accounted for 19,143 (49.8%) alerts and was overridden in 92.2% of cases; in 88 (1.1%) overridden alerts, concurrent therapy was still present, and all recommendations for these were accepted (100%).
- QTc-prolonging drug pairs accounted for 12,227 (31.8%) alerts and were overridden in 83.6% of cases; 146 (3.7%) reviewed QTc overrides led to a recommendation, with 76.2% acceptance.
- The survey was completed by 175 physicians (10.6% response rate); 153 of 173 analyzed respondents (88.4%) rated the DDI module moderately to very useful. Main override reasons were that the DDI was not clinically relevant (n = 65, 37.6%) and that the DDI was no longer present (n = 61, 35.3%).
Methods
- Retrospective quantitative evaluation of all CPOE medication orders for non-critically ill hospitalized patients and day care patients at UZ Leuven from January 2019 to January 2021, using very severe DDI alerts generated by a DelphiCare-based CDSS with a fixed 7-day before/after screening interval and secondary pharmacist review through the Check of Medication Appropriateness service.
- Cross-sectional anonymous e-survey in February 2021 sent to all 1,631 physicians, with nine questions on satisfaction, usefulness, relevance, and reasons for overriding; quantitative data were analyzed descriptively and open-ended answers by thematic analysis.
Limitations
- Stated by the authors: only very severe DDIs were analyzed; prescriber override motivations were not thoroughly evaluated; the survey had a limited 10% response rate, self-selection bias cannot be excluded, and the open-ended question was answered by only 34 prescribers, limiting generalisability.
- Evident from the text but not listed as a limitation by the authors: the study was conducted at a single tertiary teaching hospital, so findings may not generalise to other settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
17 Valid · 0 Not valid · 4 Not checked · 88 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Alert fatigue might cause prescribers to override both irrelevant and clinically relevant alerts due to an overload of irrelevant alerts, which compromises the desired safety effect of the CDSSs [13].”
§ Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey > Background
What the sources say
- Supportedcites [13] van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2):138–47. source
The Abstract says alert fatigue may cause clinicians to override important and unimportant alerts, compromising the safety benefit of decision support. The Results link alert fatigue to too many irrelevant or low-credibility alerts and say important alerts may be overridden along with unimportant ones. (LLM, conf=0.94)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Different studies have shown a substantial reduction in DDI alert burden by applying patient and context-specific factors [14–18].”
§ Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey > Background
What the sources say
- Not checked — no full text of the sourcecites [14] Tolley CL, Slight SP, Husband AK, Watson N, Bates DW. Improving medication-related clinical decision support. Am J Health Syst Pharm. 2018;75(4):239–46. source
- Partly supportedcites [15] Muylle KM, Gentens K, Dupont AG, Cornu P. Evaluation of an optimized context-aware clinical decision support system for drug–drug interaction screening. Int J Med Inform. 2021;148:104393. sourceExperimental — accuracy not yet measured
Section 3.1 (page not available in our copy) reports a substantial 61% reduction in alerts for the 19 DDIs screened in both periods. But the system introduced several optimization strategies at once, so the reduction cannot be attributed specifically to patient- and context-specific factors; after 67 additional DDIs were added, the overall alert burden increased by 142%. (LLM, conf=0.87)
- Partly supportedcites [16] Shah SN, Seger DL, Fiskio JM, Horn JR, Bates DW. Comparison of medication alerts from two commercial applications in the USA. Drug Saf. 2021;44(6):661–8. sourceExperimental — accuracy not yet measured
The Conclusion (page unavailable) attributes a projected nearly twentyfold reduction in overall medication-alert load to real-time use of patient- and medication-specific factors. The Methods, Technology Assessed section includes drug–drug interactions among the alert types, but the reported reduction is not specific to DDI alerts; the paper also says real-time alert-load reduction still needs testing. (LLM, conf=0.86)
- Supportedcites [17] Chou E, Boyce RD, Balkan B, Subbian V, Romero A, Hansten PD, et al. Designing and evaluating contextualized drug–drug interaction algorithms. JAMIA Open. 2021;4(1):ooab023. source
The Discussion reports that contextualized algorithms had the potential to reduce interruptive DDI alerts by more than 50% in retrospective data. It also describes other studies that substantially reduced alert volume using contextual factors, including a study that restricted alerts by patients’ potassium values. The paper distinguishes its projected reduction from a reduction demonstrated after clinical implementation. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [18] Horn J, Ueng S. The effect of patient-specific drug–drug interaction alerting on the frequency of alerts: a pilot study. Ann Pharmacother. 2019;53(11):1087–92. source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“Edrees et al. conducted a retrospective study of overridden high-priority DDI alerts and found an override rate of 87.3% for the highest severity DDI alerts [27].”
§ Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey > Discussion
What the sources say
- Supportedcites [27] Edrees H, Amato MG, Wong A, Seger DL, Bates DW. High-priority drug–drug interaction clinical decision support overrides in a newly implemented commercial computerized provider order-entry system: Override appropriateness and adverse drug events. J Am Med Inform Assoc. 2020;27(6):893–900. source
Materials and Methods describes a retrospective study of overridden high-priority DDI alerts. Results reports that 193 of 221 highest-severity alerts presented to providers were overridden, an override rate of 87.3%. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Moreover, it underlines the added value of a secondary review of overridden DDI alerts by a specified trained person (e.g. a pharmacist) [25].”
§ Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey > Discussion
What the sources say
- Partly supportedcites [25] Quintens C, Van de Sijpe G, Van der Linden L, Spriet I. Computerised prescribing support still needs a human touch. Age Ageing. 2020;50(6):e3–4. sourceExperimental — accuracy not yet measured
In the letter’s “Sir” section (p. 1), the authors recommend that a specified trained person, such as a pharmacist, assess software-generated recommendations for clinical relevance. They also report low acceptance of severe drug–drug interaction alerts, but do not specifically describe a secondary review of alerts that physicians have overridden. (LLM, conf=0.87)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A total of 38,409 very severe DDI alerts, divided over 64 DDI pairs, were generated.”
§ Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey > Results > Quantitative evaluation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
50 more claims
Create a free account to see all 109 claims and to verify the 4 references still open.
- Probably not a substantial claim. 27
- It may state two separate claims as one. 32
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey > Abstract > Results
“In 88 (1.1%) of these overridden alerts, concurrent therapy was still present.”
§ Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey > Results > Quantitative evaluation
“Among all override motivations, 8159 (57.0%) concerned the interaction between a NOAC and another anticoagulant. A recommendation was given to the treating physician for 88 (1.1%) alerts because co-prescription was indeed present.”
Citations
Cites
To Err is Human: Building a Safer Health System
Hendee, W. R. (2001) · Journal of Vascular and Interventional Radiology · doi:10.1016/s1051-0443(01)70072-3 · Cited by 10,103
Incidence of Adverse Drug Events and Potential Adverse Drug Events: Implications for Prevention
Bates, D. W. (1995) · JAMA · doi:10.1001/jama.1995.03530010043033 · Cited by 2,127
Steffel, J., Collins, R., Antz, M., Cornu, P., Desteghe, L., Haeusler, K. G., Oldgren, J., Reinecke, H., Roldan-Schilling, V., Rowell, N., et al. (2021) · EP Europace · doi:10.1093/europace/euab065 · Cited by 1,132
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117 · cited 10×
“Shah et al. compared two commercial CDSSs and showed a 94% reduction in alert burden, with much higher sensitivity and specificity [16].”
BackgroundInfluentialDrug-Related Problems in Hospitals: A Review of the Recent Literature
Kr??henb??hl-Melcher, A., Schlienger, R., Lampert, M., Haschke, M., Drewe, J., & Kr??henb??hl, S. (2007) · Drug Safety · doi:10.2165/00002018-200730050-00003 · Cited by 540
Using information technology to reduce rates of medication errors in hospitals
Bates, D. W. (2000) · BMJ · doi:10.1136/bmj.320.7237.788 · Cited by 483 · cited 1×
“Computerized physician order entry (CPOE) linked with clinical decision support systems (CDSSs) has seen a broad uptake in many hospitals to prevent medication errors and associated harm [9].”
BackgroundEpidemiology of Adverse Drug Reactions in Europe: A Review of Recent Observational Studies
Bouvy, J. C., De Bruin, M. L., & Koopmanschap, M. A. (2015) · Drug Safety · doi:10.1007/s40264-015-0281-0 · Cited by 466
Prevalence, Incidence and Nature of Prescribing Errors in Hospital Inpatients: A Systematic Review
Lewis, P. J., Dornan, T., Taylor, D., Tully, M. P., Wass, V., & Ashcroft, D. M. (2009) · Drug Safety · doi:10.2165/00002018-200932050-00002 · Cited by 383
Nuckols, T. K., Smith-Spangler, C., Morton, S. C., Asch, S. M., Patel, V. M., Anderson, L. J., Deichsel, E. L., & Shekelle, P. G. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-56 · Cited by 340
Epidemiology and characteristics of adverse drug reactions caused by drug–drug interactions
Magro, L., Moretti, U., & Leone, R. (2012) · Expert Opinion on Drug Safety · doi:10.1517/14740338.2012.631910 · Cited by 287 · cited 1×
“More specifically, 5–17% of adverse drug events among inpatients are caused by drug–drug interactions (DDIs) [1, 6].”
Background
Cited by
Learning motif-based graphs for drug–drug interaction prediction via local–global self-attention
Zhong, Y., Li, G., Ji, Y., Zheng, H., Yu, Y., Zhang, J., Luo, H., Wang, B., & Weng, Z. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00888-6 · Cited by 69
“However, the general practice to ensure the safety of polypharmacy is to search pairwise DDIs between all the possible pairs of drugs 46,47 ; MeTDDI can be directly deployed to predict DDIs between multiple drugs by enumerating all the drug pairs.”
Hansten, P. D., Tan, M. S., Horn, J. R., Gómez-Lumbreras, A., Zapata, L. V., Boyce, R. D., Subbian, V., Romero, A. V., Gephart, S. M., & Malone, D. C. (2022) · Drug Safety · doi:10.1007/s40264-022-01265-1 · Cited by 57
A systematic review of clinicians’ acceptance and use of clinical decision support systems over time
Newton, N., Bamgboje‐Ayodele, A., Forsyth, R., Tariq, A., & Baysari, M. T. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01662-7 · Cited by 60
Hughes, J. E., Moriarty, F., Bennett, K., & Cahir, C. (2023) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15970 · Cited by 42
Li, L., Baker, J., Quirk, R., et al. (2024) · Drug Safety · doi:10.1007/s40264-024-01412-w · Cited by 29
“DDI alerts have been the focus of much research [14, 15], however, there have been limited controlled studies examining the effectiveness of DDI alerts to reduce cDDIs and patient harm [13].”
Metadata
- Authors
- Van De Sijpe, G., Quintens, C., Walgraeve, K., Van Laer, E., Penny, J., De Vlieger, G., Schrijvers, R., De Munter, P., Foulon, V., Casteels, M., Van der Linden, L., & Spriet, I.
- Affiliations
- Universitair Ziekenhuis Leuven (BE)
- KU Leuven (BE)
- ORCID
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2022-02-22
- Volume
- 22
- Issue
- 1
- Pages
- 48-48
- ISSN
- 1472-6947 (ISSN-L)
- PMC
- PMC8864797
- OpenAlex
- W4213099779
- Cited by
- 54
- References
- 36
- Citations by year
- 2026: 12 · 2025: 15 · 2024: 16 · 2023: 7 · 2022: 4
- FWCI
- 7.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Lirias (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Patient Safety and Medication Errors
- Keywords
- drug-drug interaction alerts, clinical decision support systems, alarm fatigue, false positive alerts, alert override, pharmacist recommendations, direct oral anticoagulants, low molecular weight heparin, screening frequency, medication errors, teaching hospitals
- MeSH terms
- Drug Interactions, Humans, Medication Errors, Retrospective Studies, Decision Support Systems, Clinical, Medical Order Entry Systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey. https://app.cochat.ai/papers/10.1186/s12911-022-01783-z
- The paper
- Van De Sijpe, G., Quintens, C., Walgraeve, K., Van Laer, E., Penny, J., De Vlieger, G., Schrijvers, R., De Munter, P., Foulon, V., Casteels, M., Van der Linden, L., & Spriet, I. (2022). Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-022-01783-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.