Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 34 of the 36 references it cites were found in a registry or an index; 33 of the 45 sources its claims cite were checked for support.

Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey

Open access
Van De Sijpe, Greet, Quintens, Charlotte, Walgraeve, Karolien, Van Laer, Eva, Penny, Jens, De Vlieger, Greet, Schrijvers, Rik, De Munter, Paul, Foulon, Veerle, Casteels, Minne, Van der Linden, Lorenz, Spriet, Isabel · 2022 · BMC Medical Informatics and Decision Making

Abstract

Background: Clinical decision support systems are implemented in many hospitals to prevent medication errors and associated harm. They are however associated with a high burden of false positive alerts and alert fatigue. The aim of this study was to evaluate a drug–drug interaction (DDI) clinical decision support system in terms of its performance, uptake and user satisfaction and to identify barriers and opportunities for improvement. Methods: A quantitative evaluation and end-user survey were performed in a large teaching hospital. First, very severe DDI alerts generated between 2019 and 2021 were evaluated retrospectively. Data collection comprised alert burden, override rates, the number of alert overrides reviewed by pharmacists and the resulting pharmacist recommendations as well as their acceptance rate. Second, an e-survey was carried out among prescribers to assess satisfaction, usefulness and relevance of DDI alerts as well as reasons for overriding. Results: A total of 38,409 very severe DDI alerts were generated, of which 88.2% were overridden by the prescriber. In 3.2% of reviewed overrides, a recommendation by the pharmacist was provided, of which 79.2% was accepted. False positive alerts were caused by a too broad screening interval and lack of incorporation of patient-specific characteristics, such as QTc values. Co-prescribing of a non-vitamin K oral anticoagulant and a low molecular weight heparin accounted for 49.8% of alerts, of which 92.2% were overridden. In 88 (1.1%) of these overridden alerts, concurrent therapy was still present. Despite the high override rate, the e-survey revealed that the DDI clinical decision support system was found useful by prescribers. Conclusions: Identified barriers were the lack of DDI-specific screening intervals and inclusion of patient-specific characteristics, both leading to a high number of false positive alerts and risk for alert fatigue. Despite these barriers, the added value of the DDI clinical decision support system was recognized by prescribers. Hence, integration of DDI-specific screening intervals and patient-specific characteristics is warranted to improve the performance of the DDI software.

Overview

CoChat Summary

TL;DR

This study evaluated a drug–drug interaction (DDI) clinical decision support system (CDSS) in a large Belgian teaching hospital using retrospective alert data and a prescriber survey. It found high override rates and false-positive alerts driven by a broad screening interval and lack of patient-specific data, although prescribers still rated the system useful.

Key findings

  • Of 38,409 very severe DDI alerts across 64 DDI pairs, 33,888 (88.2%) were overridden by prescribers, giving an initial acceptance rate of 11.8%.
  • CMA pharmacists reviewed 14,320 overrides; 453 (3.2%) led to a pharmacist recommendation, of which 79.2% were accepted. NOAC plus another anticoagulant accounted for 19,143 (49.8%) alerts and was overridden in 92.2% of cases; in 88 (1.1%) overridden alerts, concurrent therapy was still present, and all recommendations for these were accepted (100%).
  • QTc-prolonging drug pairs accounted for 12,227 (31.8%) alerts and were overridden in 83.6% of cases; 146 (3.7%) reviewed QTc overrides led to a recommendation, with 76.2% acceptance.
  • The survey was completed by 175 physicians (10.6% response rate); 153 of 173 analyzed respondents (88.4%) rated the DDI module moderately to very useful. Main override reasons were that the DDI was not clinically relevant (n = 65, 37.6%) and that the DDI was no longer present (n = 61, 35.3%).

Methods

  • Retrospective quantitative evaluation of all CPOE medication orders for non-critically ill hospitalized patients and day care patients at UZ Leuven from January 2019 to January 2021, using very severe DDI alerts generated by a DelphiCare-based CDSS with a fixed 7-day before/after screening interval and secondary pharmacist review through the Check of Medication Appropriateness service.
  • Cross-sectional anonymous e-survey in February 2021 sent to all 1,631 physicians, with nine questions on satisfaction, usefulness, relevance, and reasons for overriding; quantitative data were analyzed descriptively and open-ended answers by thematic analysis.

Limitations

  • Stated by the authors: only very severe DDIs were analyzed; prescriber override motivations were not thoroughly evaluated; the survey had a limited 10% response rate, self-selection bias cannot be excluded, and the open-ended question was answered by only 34 prescribers, limiting generalisability.
  • Evident from the text but not listed as a limitation by the authors: the study was conducted at a single tertiary teaching hospital, so findings may not generalise to other settings.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12911-022-01783-z

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Actions that must be completed by prescribers and pharmacists when an alert for a very severe drug–drug interaction is generated. Panel A: general workflow diagram of very severe drug–drug interaction alerts. Panel B: workflow diagram using a specific example categorized as a very severe drug–drug interaction: co-prescription of a non-vitamin K oral anticoagulant and a low molecular weight heparin. Blue: actions that must be taken by the prescriber. Grey: actions that must be taken by the pharmacist. DDI, drug–drug interaction, NOAC, non-vitamin K oral anticoagulant; LMWH, low molecular weight heparin
Fig. 1 Actions that must be completed by prescribers and pharmacists when an alert for a very severe drug–drug interaction is generated. Panel A: general workflow diagram of very severe drug–drug interaction alerts. Panel B: workflow diagram using a specific example categorized as a very severe drug–drug interaction: co-prescription of a non-vitamin K oral anticoagulant and a low molecular weight heparin. Blue: actions that must be taken by the prescriber. Grey: actions that must be taken by the pharmacist. DDI, drug–drug interaction, NOAC, non-vitamin K oral anticoagulant; LMWH, low molecular weight heparinCC BY · PubMed Central
Fig. 2 General experience of the drug–drug interaction module
Fig. 2 General experience of the drug–drug interaction moduleCC BY · PubMed Central
Fig. 3 Main reasons for overruling a drug–drug interaction alert. DDI, drug–drug interaction
Fig. 3 Main reasons for overruling a drug–drug interaction alert. DDI, drug–drug interactionCC BY · PubMed Central
Fig. 4 Type of drug–drug interaction alerts considered least useful and most useful. DDI, drug–drug interaction; NOAC, non-vitamin K oral anticoagulant
Fig. 4 Type of drug–drug interaction alerts considered least useful and most useful. DDI, drug–drug interaction; NOAC, non-vitamin K oral anticoagulantCC BY · PubMed Central
Fig. 5 Main barriers of the current drug–drug interaction clinical decision support system. Panel A: a too broad screening interval leading to false positive alerts when switching correctly between anticoagulants. Panel B: lack of incorporation of patient-specific characteristics leading to false positive alerts when two QTc-prolonging drugs are prescribed for a patient whose QTc-interval is not prolonged
Fig. 5 Main barriers of the current drug–drug interaction clinical decision support system. Panel A: a too broad screening interval leading to false positive alerts when switching correctly between anticoagulants. Panel B: lack of incorporation of patient-specific characteristics leading to false positive alerts when two QTc-prolonging drugs are prescribed for a patient whose QTc-interval is not prolongedCC BY · PubMed Central

Verification

Claims the paper makes

17 Valid · 0 Not valid · 4 Not checked · 88 Own results and interpretations

50 more claims

Create a free account to see all 109 claims and to verify the 4 references still open.

See all claims — free account
  • Probably not a substantial claim. 27
  • It may state two separate claims as one. 32
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 36 references

Cited by

See all 54

Metadata

Authors
Van De Sijpe, G., Quintens, C., Walgraeve, K., Van Laer, E., Penny, J., De Vlieger, G., Schrijvers, R., De Munter, P., Foulon, V., Casteels, M., Van der Linden, L., & Spriet, I.
Affiliations
  • Universitair Ziekenhuis Leuven (BE)
  • KU Leuven (BE)
Venue
BMC Medical Informatics and Decision Making
Published
2022-02-22
Volume
22
Issue
1
Pages
48-48
ISSN
1472-6947 (ISSN-L)
OpenAlex
W4213099779
Cited by
54
References
36
Citations by year
2026: 12 · 2025: 15 · 2024: 16 · 2023: 7 · 2022: 4
FWCI
7.86
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Lirias (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Patient Safety and Medication Errors
Keywords
drug-drug interaction alerts, clinical decision support systems, alarm fatigue, false positive alerts, alert override, pharmacist recommendations, direct oral anticoagulants, low molecular weight heparin, screening frequency, medication errors, teaching hospitals
MeSH terms
Drug Interactions, Humans, Medication Errors, Retrospective Studies, Decision Support Systems, Clinical, Medical Order Entry Systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey. https://app.cochat.ai/papers/10.1186/s12911-022-01783-z
The paper
Van De Sijpe, G., Quintens, C., Walgraeve, K., Van Laer, E., Penny, J., De Vlieger, G., Schrijvers, R., De Munter, P., Foulon, V., Casteels, M., Van der Linden, L., & Spriet, I. (2022). Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-022-01783-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.