Randomized clinical trial of a customized electronic alert requiring an affirmative response compared to a control group receiving a commercial passive CPOE alert: NSAID–warfarin co-prescribing as a test case
Open accessAbstract
BACKGROUND: Studies that have looked at the effectiveness of computerized decision support systems to prevent drug-drug interactions have reported modest results because of low response by the providers to the automated alerts. OBJECTIVE: To evaluate, within an inpatient computerized physician order entry (CPOE) system, the incremental effectiveness of an alert that required a response from the provider, intended as a stronger intervention to prevent concurrent orders of warfarin and non-steroidal anti-inflammatory drugs (NSAIDs). DESIGN: Randomized clinical trial of 1963 clinicians assigned to either an intervention group receiving a customized electronic alert requiring affirmative response or a control group receiving a commercially available passive alert as part of the CPOE. The study duration was 2 August 2006 to 15 December 2007. MEASUREMENTS: Alert adherence was compared between study groups. RESULTS: The proportion of desired ordering responses (ie, not reordering the alert-triggering drug after firing) was lower in the intervention group (114/464 (25%) customized alerts issued) than in the control group (154/560 (28%) passive alerts firing). The adjusted OR of inappropriate ordering was 1.22 (95% CI 0.69 to 2.16). CONCLUSION: A customized CPOE alert that required a provider response had no effect in reducing concomitant prescribing of NSAIDs and warfarin beyond that of the commercially available passive alert received by the control group. New CPOE alerts cannot be assumed to be effective in improving prescribing, and need evaluation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Evaluations of CDS have not consistently demonstrated improved patient outcomes.
“Despite initial promise of success and widespread CDS implementations, evaluations of CDS have not consistently demonstrated improved patient outcomes.9–12”
From Clinician collaboration to improve clinical decision support: the Clickbusters initiative (2022) · cites this paper as [10]
The Background (page not specified in our copy) says alerts are effective in some domains and ineffective in others, and that reviews find mixed effects of CPOE systems. The Results and Discussion report no advantage of the customized alert for prescribing behavior, but do not establish whether CDS evaluations consistently improve patient outcomes. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 38
- Kohn LT, Corrigan JM, Donaldson MS, eds. (Committee on Quality of Health Care in America, Institute of Medicine). To Err Is Human: Building a Safer Health System. Washington, DC, USA: National Academies Press, 1999.
- Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998;280:1311e16.
- Bates DW, Cohen M, Leape LL, et al. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc 2001;8:299e308.
- Teich JM, Merchia PR, Schmiz JL, et al. Effects of computerized physician order entry on prescribing practices. Arch Intern Med 2000;160:2741e7.
- Paterno MD, Maviglia SM, Gorman PN, et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009;16:40e6.
- Chrischilles EA, Fulda TR, Byrns PJ, et al. The role of pharmacy computer systems in preventing medication errors. J Am Pharm Assoc (Wash) 2002;42:439e48.
- Judge J, Field TS, DeFlorio M, et al. Prescribers' responses to alerts during medication ordering in the long term care setting. J Am Med Inform Assoc 2006;13:385e90.
- Feldstein AC, Smith DH, Perrin N, et al. Reducing warfarin medication interactions: an interrupted time series evaluation. Arch Intern Med 2006;166:1009e15.
- Tamblyn R, Huang A, Taylor L, et al. A randomized trial of the effectiveness of on- demand versus computer-triggered drug decision support in primary care. J Am Med Inform Assoc 2008;15:430e8.
- Palen TE, Raebel M, Lyons E, et al. Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care 2006;12:389e95.
- Eslami S, de Keizer NF, Abu-Hanna A. The impact of computerized physician medication order entry in hospitalized patientsda systematic review. Int J Med Inform 2008;77:365e76.
- Grizzle AJ, Mahmood MH, Ko Y, et al. Reasons provided by prescribers when overriding drug-drug interaction alerts. Am J Manag Care 2007;13:573e8.
- van der Sijs H, Lammers L, van den Tweel A, et al. Time-dependent drug-drug interaction alerts in care provider order entry: software may inhibit medication error reductions. J Am Med Inform Assoc 2009;16:864e8.
- Weingart SN, Toth M, Sands DZ, et al. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003;163:2625e31.
- van der Sijs H, Aarts J, Vulto A, et al. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006;13:138e47.
- Taylor LK, Tamblyn R. Reasons for physician non-adherence to electronic drug alerts. Stud Health Technol Inform 2004;107:1101e5.
- Isaac T, Weissman JS, Davis RB, et al. Overrides of medication alerts in ambulatory care. Arch Intern Med 2009;169:305e11.
- Weingart SN, Simchowitz B, Shiman L, et al. Clinicians' assessments of electronic medication safety alerts in ambulatory care. Arch Intern Med 2009;169:1627e32.
- Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999;6:313e21.
- Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial of a computer- based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999;106:144e50.
- Delaney JA, Opatrny L, Brophy JM, et al. Drug drug interactions between antithrombotic medications and the risk of gastrointestinal bleeding. CMAJ 2007;177:347e51.
- Chan TY. Adverse interactions between warfarin and nonsteroidal antiinflammatory drugs: mechanisms, clinical significance, and avoidance. Ann Pharmacother 1995;29:1274e83.
- Kucher N, Koo S, Quiroz R, et al. Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med 2005;352:969e77.
- Litzelman DK, Dittus RS, Miller ME, et al. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med 1993;8:311e17.
- Overhage JM, Tierney WM, Zhou XH, et al. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc 1997;4:364e75.
- Peterson JF, Kuperman GJ, Shek C, et al. Guided prescription of psychotropic medications for geriatric inpatients. Arch Intern Med 2005;165:802e7.
- Terrell KM, Perkins AJ, Dexter PR, et al. Computerized decision support to reduce potentially inappropriate prescribing to older emergency department patients: a randomized, controlled trial. J Am Geriatr Soc 2009;57:1388e94.
- Raebel MA, Carroll NM, Kelleher JA, et al. Randomized trial to improve prescribing safety during pregnancy. J Am Med Inform Assoc 2007;14:440e50.
- Simon SR, Smith DH, Feldstein AC, et al. Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people. J Am Geriatr Soc 2006;54:963e8.
- Smith DH, Perrin N, Feldstein A, et al. The impact of prescribing safety alerts for elderly persons in an electronic medical record: an interrupted time series evaluation. Arch Intern Med 2006;166:1098e104.
- Hsieh TC, Kuperman GJ, Jaggi T, et al. Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004;11:482e91.
- Eslami S, de Keizer NF, de Jonge E, et al. A systematic review on quality indicators for tight glycaemic control in critically ill patients: need for an unambiguous indicator reference subset. Crit Care 2008;12:R139.
- Conroy S, Sweis D, Planner C, et al. Interventions to reduce dosing errors in children: a systematic review of the literature. Drug Saf 2007;30:1111e25.
- Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163:1409e16.
- Thompson SG, Pyke SD, Hardy RJ. The design and analysis of paired cluster randomized trials: an application of meta-analysis techniques. Stat Med 1997;16:2063e79.
- Glassman PA, Belperio P, Simon B, et al. Exposure to automated drug alerts over time: effects on clinicians' knowledge and perceptions. Med Care 2006;44:250e6.
- Lapane KL, Waring ME, Schneider KL, et al. A mixed method study of the merits of e-prescribing drug alerts in primary care. J Gen Intern Med 2008;23:442e6.
- Strom BL, Schinnar R, Aberra F. Unintended effects of a CPOE nearly hard stop alert to prevent a drug interaction: A RCT. Arch Intern Med. In press.
Cited by
Sorted by citations
Effect of Clinical Decision-Support Systems: A Systematic Review
Bright, T. J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R. R., Samsa, G., Hasselblad, V., Williams, J. W., Musty, M. D., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-1-201207030-00450 · Cited by 1,200 · see citing works
A framework for evaluating the appropriateness of clinical decision support alerts and responses
McCoy, A. B., Waitman, L. R., Lewis, J. B., Wright, J. A., Choma, D. P., Miller, R. A., & Peterson, J. F. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000185 · Cited by 138 · see citing works
Miller, A. M., Boro, M. S., Korman, N. E., & Davoren, J. B. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000262 · Cited by 41 · see citing works
Clinician collaboration to improve clinical decision support: the Clickbusters initiative
McCoy, A. B., Russo, E. M., Johnson, K. B., Addison, B., Patel, N., Wanderer, J. P., Mize, D. E., Jackson, J. G., Reese, T. J., Littlejohn, S., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac027 · Cited by 41 · see citing works
- Authors
- Strom, B. L., Schinnar, R., Bilker, W., Hennessy, S., Leonard, C. E., & Pifer, E.
- Affiliations
- University of Pennsylvania (US)
- El Camino Hospital (US)
- Corresponding author
- Strom, B. L.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2010-07-01
- Volume
- 17
- Issue
- 4
- Pages
- 411-415
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC2995662
- OpenAlex
- W2151712959
- Cited by
- 51
- References
- 0
- Citations by year
- 2025: 3 · 2024: 3 · 2023: 1 · 2022: 4 · 2021: 2 · 2020: 2 · 2019: 7 · 2018: 1 · 2017: 3 · 2016: 3 · 2015: 6 · 2014: 2 · 2013: 5 · 2012: 5
- FWCI
- 7.47
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- University of Pennsylvania
- Agency for Healthcare Research and Quality (U18HS016946)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Keywords
- computerized physician order entry, electronic alerts, drug-drug interactions, nonsteroidal anti-inflammatory drugs, warfarin, prescribing safety, inpatient prescribing, clinical decision support
- MeSH terms
- Anti-Inflammatory Agents, Non-Steroidal, Anticoagulants, Drug Interactions, Humans, Medication Errors, Pennsylvania, Practice Patterns, Physicians', User-Computer Interface, Warfarin, Guideline Adherence, Medical Order Entry Systems, Practice Patterns, Nurses'
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
