Designing and evaluating contextualized drug–drug interaction algorithms
Open accessAbstract
Objective: Alert fatigue is a common issue with off-the-shelf clinical decision support. Most warnings for drug–drug interactions (DDIs) are overridden or ignored, likely because they lack relevance to the patient’s clinical situation. Existing alerting systems for DDIs are often simplistic in nature or do not take the specific patient context into consideration, leading to overly sensitive alerts. The objective of this study is to develop, validate, and test DDI alert algorithms that take advantage of patient context available in electronic health records (EHRs) data. Methods: Data on the rate at which DDI alerts were triggered but for which no action was taken over a 3-month period (override rates) from a single tertiary care facility were used to identify DDIs that were considered a high-priority for contextualized alerting. A panel of DDI experts developed algorithms that incorporate drug and patient characteristics that affect the relevance of such warnings. The algorithms were then implemented as computable artifacts, validated using a synthetic health records data, and tested over retrospective data from a single urban hospital. Results: Algorithms and computable knowledge artifacts were developed and validated for a total of 8 high priority DDIs. Testing on retrospective real-world data showed the potential for the algorithms to reduce alerts that interrupt clinician workflow by more than 50%. Two algorithms (citalopram/QT interval prolonging agents, and fluconazole/opioid) showed potential to filter nearly all interruptive alerts for these combinations. Conclusion: The 8 DDI algorithms are a step toward addressing a critical need for DDI alerts that are more specific to patient context than current commercial alerting systems. Data commonly available in EHRs can improve DDI alert specificity.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Chou et al. showed a reduction in DDI alerts by more than 50%.
“Chou et al. showed a reduction in DDI alerts by more than 50% [17].”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [17]
The Results (Application of computable algorithms to real-world data; page unavailable) report that the eight algorithms would filter 1,584 of 3,020 interruptive alerts (52.4%). This supports the magnitude, but the reduction was projected from retrospective testing, not observed after deployment; the claim omits that qualification. (LLM, conf=0.86)
Experimental — accuracy not yet measuredDifferent studies have shown a substantial reduction in DDI alert burden by applying patient and context-specific factors.
“Different studies have shown a substantial reduction in DDI alert burden by applying patient and context-specific factors [14–18].”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [17]
Recent studies already showed the potential of including patient-specific and context-specific characteristics into DDI CDSS algorithms on reducing the DDI alert burden.
“Recent studies already showed the potential of including patient-specific and context-specific characteristics into DDI CDSS algorithms on reducing the DDI alert burden [17, 18].”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [17]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
George, H., Duke Jon, D., Shah Nigam, H., Reich Christian, G., Vojtech, H., Schuemie Martijn, J., Suchard Marc, A., Woong, P. R., Kei, W. I. C., Rijnbeek Peter, R., et al. (2015) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-564-7-574 · Cited by 1,107 · cited 1× · see citing works
“For the purpose of creating a sample implementation, and to test that the rules would run as expected, a synthetic patient population was created and loaded into the open source common data model provided by the Observational Health Data Science and Informatics (OHDSI) collaborative.(28) This common data model was cho…”
MethodImproving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · see citing works
Flagged first, then by citations
Research works printed but not linked: 33
- [1]Magro L, Moretti U, Leone R. Epidemiology and characteristics of adverse drug reactions caused by drug-drug interactions. Expert Opin Drug Saf 2012; 11 (1): 83– 94. 22022824
- [2]Hines LE, Murphy JE. Potentially harmful drug-drug interactions in the elderly: a review. Am J Geriatr Pharmacother 2011; 9 (6): 364– 77. 22078863
- [4]Peterson JF, Bates DW. Preventable medication errors: identifying and eliminating serious drug interactions. J Am Pharm Assoc (Wash) 2001; 41 (2): 159– 60. 11297326
- [5]Juurlink DN, Mamdani M, Kopp A, Laupacis A, Redelmeier DA. Drug-drug interactions among elderly patients hospitalized for drug toxicity. JAMA 2003; 289 (13): 1652– 8. 12672733
- [6]Chen Y-F, Avery AJ, Neil KE, Johnson C, Dewey ME, Stockley IH. Incidence and possible causes of prescribing potential hazardous/contraindicated drug combinations in general practice. Drug Saf 2005; 28 (1): 67– 80. 15649106
- [7]Hines LE, Malone DC, Murphy JE. Recommendations for generating, evaluating, and implementing drug–drug interaction evidence. Pharmacotherapy 2012; 32 (4): 304– 13. 22461120
- [8]Ko Y, Malone DC, D'Agostino JV, et al Potential determinants of prescribers’ drug-drug interaction knowledge. Res Social Adm Pharm 2008; 4 (4): 355– 66. 19064242
- [9]Weideman RA, Bernstein IH, McKinney WP. Pharmacist recognition of potential drug interactions. Am J Health Syst Pharm 1999; 56 (15): 1524– 9. 10478990
- [10]Nabovati E, Vakili-Arki H, Taherzadeh Z, Saberi MR, Abu-Hanna A, Eslami S. A survey of attitudes, practices, and knowledge regarding drug-drug interactions among medical residents in Iran. Int J Clin Pharm 2017; 39 (3): 560– 8. 28382584
- [11]van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (2): 138– 47. 16357358 PMC1447540
- [12]Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163 (21): 2625– 31. 14638563
- [13]Grizzle AJ, Mahmood MH, Ko Y, et al Reasons provided by prescribers when overriding drug-drug interaction alerts. Am J Manag Care 2007; 13 (10): 573– 8. 17927462
- [14]Wright A, McEvoy DS, Aaron S, et al Structured override reasons for drug-drug interaction alerts in electronic health records. J Am Med Inform Assoc 2019; 26 (10): 934– 42. 31329891 PMC6748816
- [15]Daniels CC, Burlison JD, Baker DK, et al Optimizing drug-drug interaction alerts using a multidimensional approach. Pediatrics 2019; 143 (3): e20174111. 30760508 PMC6398362
- [16]Edrees H, Amato MG, Wong A, Seger DL, Bates DW. High-priority drug-drug interaction clinical decision support overrides in a newly implemented commercial computerized provider order-entry system: override appropriateness and adverse drug events. J Am Med Inform Assoc 2020; 27 (6): 893– 900. 32337561 PMC7647273
- [17]Paterno MD, Maviglia SM, Gorman PN, et al Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009; 16 (1): 40– 6. 18952941 PMC2605599
- [19]Horn JR, Hansten PD, Osborn JD, Wareham P, Somani S. Customizing clinical decision support to prevent excessive drug-drug interaction alerts. Am J Health Syst Pharm 2011; 68 (8): 662– 4. 21460171
- [20]Bakker T, Klopotowska JE, de Keizer NF, SIMPLIFY Study Group, et al Improving medication safety in the intensive care by identifying relevant drug-drug interactions - results of a multicenter Delphi study. J Crit Care 2020; 57: 134– 40. 32145656
- [21]Pirnejad H, Amiri P, Niazkhani Z, et al Preventing potential drug-drug interactions through alerting decision support systems: a clinical context based methodology. Int J Med Inform 2019; 127: 18– 26. 31128828
- [22]McGreevey JD, Mallozzi CP, Perkins RM, Shelov E, Schreiber R. Reducing alert burden in electronic health records: state of the art recommendations from four health systems. Appl Clin Inform 2020; 11 (1): 1– 12. 31893559 PMC6938713
- [23]Seidling HM, Klein U, Schaier M, et al What, if all alerts were specific – estimating the potential impact on drug interaction alert burden. Int J Med Inform 2014; 83 (4): 285– 91. 24484781
- [24]Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012; 45 (6): 1202– 16. 22995208
- [25]Jung M, Riedmann D, Hackl WO, et al Physicians’ perceptions on the usefulness of contextual information for prioritizing and presenting alerts in Computerized Physician Order Entry systems. BMC Med Inform Decis Mak 2012; 12: 111. 23031275 PMC3522054
- [26]Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. Prescribers’ interactions with medication alerts at the point of prescribing: a multi-method, in situ investigation of the human–computer interaction. Int J Med Inform 2012; 81 (4): 232– 43. 22296761
- [27]Muylle KM, Gentens K, Dupont AG, Cornu P. Evaluation of context-specific alerts for potassium-increasing drug-drug interactions: a pre-post study. Int J Med Inform 2020; 133: 104013. 31698230
- [29]Bodenreider O, Nguyen D, Chiang P, et al The NLM value set authority center. Stud Health Technol Inform 2013; 192: 1224. 23920998 PMC4300102
- [32]Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med 2010; 363 (6): 501– 4. 20647183
- [33]Duke JD, Bolchini D. A successful model and visual design for creating context-aware drug-drug interaction alerts. AMIA Annu Symp Proc 2011; 2011: 339– 48. 22195086 PMC3243201
- [34]Duke JD, Li X, Dexter P. Adherence to drug-drug interaction alerts in high-risk patients: a trial of context-enhanced alerting. J Am Med Inform Assoc 2013; 20: 494– 8. 23161895 PMC3628050
- [35]Beeler PE, Eschmann E, Schneemann M, Blaser J. Negligible impact of highly patient-specific decision support for potassium-increasing drug-drug interactions - a cluster-randomised controlled trial. Swiss Med Wkly 2019; 149: w20035. 30957213
- [36]Riedmann D, Jung M, Hackl WO, Stühlinger W, van der Sijs H, Ammenwerth E. Development of a context model to prioritize drug safety alerts in CPOE systems. BMC Med Inform Decis Mak 2011; 11 (1): 35. 21612623 PMC3127742
- [37]Seidling HM, Storch CH, Bertsche T, et al Successful strategy to improve the specificity of electronic statin-drug interaction alerts. Eur J Clin Pharmacol 2009; 65 (11): 1149– 57. 19641913
- [38]Nguyen B-P, Reese T, Decker S, Malone D, Boyce RD, Beyan O. Implementation of clinical decision support services to detect potential drug-drug interaction using clinical quality language. Stud Health Technol Inform 2019; 264: 724– 8. 31438019
Cited by
Sorted by citations
Van De Sijpe, G., Quintens, C., Walgraeve, K., Van Laer, E., Penny, J., De Vlieger, G., Schrijvers, R., De Munter, P., Foulon, V., Casteels, M., et al. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01783-z · Cited by 54 · see citing works
“showed a reduction in DDI alerts by more than 50% [17].”
Background
- Authors
- Chou, E., Boyce, R. D., Balkan, B., Subbian, V., Romero, A., Hansten, P. D., Horn, J. R., Gephart, S., & Malone, D. C.
- Affiliations
- University of Pittsburgh (US)
- University of Arizona (US)
- Banner - University Medical Center Tucson (US)
- University of Washington (US)
- University of Utah (US)
- Corresponding author
- Boyce, Richard D
- ORCID
- Venue
- JAMIA Open
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-01-01
- Volume
- 4
- Issue
- 1
- Pages
- ooab023-ooab023
- ISSN
- 2574-2531 (ISSN-L)
- PMC
- PMC7976224
- OpenAlex
- W3138310088
- Cited by
- 34
- References
- 2
- Citations by year
- 2026: 5 · 2025: 1 · 2024: 6 · 2023: 7 · 2022: 14
- FWCI
- 5.88
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- JAMIA Open (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (R21 HS023826, R01 HS025984)
- National Science Foundation (1838745)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Keywords
- drug-drug interactions, clinical decision support, alarm fatigue, electronic health records, citalopram, QT prolongation, fluconazole, opioid interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
