Physicians' Decisions to Override Computerized Drug Alerts in Primary Care
Abstract
BACKGROUND: Although computerized physician order entry reduces medication errors among inpatients, little is known about the use of this system in primary care. METHODS: We calculated the override rate among 3481 consecutive alerts generated at 5 adult primary care practices that use a common computerized physician order entry system for prescription writing. For detailed review, we selected a random sample of 67 alerts in which physicians did not prescribe an alerted medication and 122 alerts that resulted in a written prescription. We identified factors associated with the physicians' decisions to override a medication alert, and determined whether an adverse drug event (ADE) occurred. RESULTS: Physicians overrode 91.2% of drug allergy and 89.4% of high-severity drug interaction alerts. In the multivariable analysis using the medical chart review sample (n = 189), physicians were less likely to prescribe an alerted medication if the prescriber was a house officer (odds ratio [OR], 0.26; 95% confidence interval [CI], 0.08-0.84) and if the patient had many drug allergies (OR, 0.70; 95% CI, 0.53-0.93). They were more likely to override alerts for renewals compared with new prescriptions (OR, 17.74; 95% CI, 5.60-56.18). We found no ADEs in cases where physicians observed the alert and 3 ADEs among patients with alert overrides, a nonsignificant difference (P =.55). Physician reviewers judged that 36.5% of the alerts were inappropriate. CONCLUSIONS: Few physicians changed their prescription in response to a drug allergy or interaction alert, and there were few ADEs, suggesting that the threshold for alerting was set too low. Computerized physician order entry systems should suppress alerts for renewals of medication combinations that patients currently tolerate.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It is widely accepted that alert fatigue explains high override rates.
“It is widely accepted that “alert fatigue” explains high override rates [7, 9–14].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [10]
Many overrides are known to be appropriate, as they are informed by clinical knowledge not captured in structured EHR data.
“Many overrides are known to be appropriate, as they are informed by clinical knowledge not captured in structured EHR data [10, 40].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [10]
Alert nonadherence, or clinician overrides, occurs for 49%-96% of alerts.
“Alert nonadherence, or clinician overrides, is one key barrier that occurs for 49%–96% of alerts.13–23”
From Clinician collaboration to improve clinical decision support: the Clickbusters initiative (2022) · cites this paper as [14]
As many as 96% of CDSS alerts are overridden or ignored by physicians.
“Importantly there is clear evidence that CDSS tools are not always used when available, 15 with up to 96% of alerts being overridden or ignored by physicians.[16][17][18][19]”
From Computerized clinical decision support for prescribing: provision does not guarantee uptake (2010) · cites this paper as [16]
The overall risk of ADEs is low, but very high override rates could increase the absolute number of inappropriate overrides, raising the ADE risk.
“While the overall risk of ADEs is low, our very high override rates could increase the absolute number of inappropriate overrides, raising the ADE risk [5,28].”
From Drug interaction alert override rates in the Meaningful Use era: No evidence of progress (2014) · cites this paper as [5]
Cites
Bates, D. W. (1998) · JAMA · doi:10.1001/jama.280.15.1311 · Cited by 2,107 · see citing works
Adverse Drug Events in Ambulatory Care
Gandhi, T. K., Weingart, S. N., Borus, J. S., Seger, A. C., Peterson, J. F., Burdick, E., Seger, D. L., Shu, K. M., Federico, F. A., Leape, L. L., et al. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa020703 · Cited by 1,371 · see citing works
Self-citationProtocol-Based Computer Reminders, the Quality of Care and the Non-Perfectability of Man
McDonald, C. J. (1976) · New England Journal of Medicine · doi:10.1056/nejm197612092952405 · Cited by 804 · see citing works
Reminders to Physicians from an Introspective Computer Medical Record
McDonald, C. J., Hui, S. L., Smith, D. M., Tierney, W. M., Cohen, S. J., Weinberger, M., & McCabe, G. P. (1984) · Annals of Internal Medicine · doi:10.7326/0003-4819-100-1-130 · Cited by 626 · see citing works
Physician inpatient order writing on microcomputer workstations. Effects on resource utilization
Tierney, W. M. (1993) · JAMA · doi:10.1001/jama.269.3.379 · Cited by 561 · see citing works
A Computer Alert System to Prevent Injury From Adverse Drug Events
Raschke, R. A., Gollihare, B., Wunderlich, T. A., Guidry, J. R., Leibowitz, A. I., Peirce, J. C., Lemelson, L., Heisler, M. A., & Susong, C. R. (1998) · JAMA · doi:10.1001/jama.280.15.1317 · Cited by 356 · see citing works
Tierney, W. M., Hui, S. L., & McDonald, C. J. (1986) · Medical Care · doi:10.1097/00005650-198608000-00001 · Cited by 337 · see citing works
Effect of Computer-Based Alerts on the Treatment and Outcomes of Hospitalized Patients
Rind, D. M. (1994) · Archives of Internal Medicine · doi:10.1001/archinte.1994.00420130107014 · Cited by 323 · see citing works
Kuperman, G. J., Teich, J. M., Tanasijevic, M. J., Ma'Luf, N., Rittenberg, E., Jha, A., Fiskio, J., Winkelman, J., & Bates, D. W. (1999) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1999.0060512 · Cited by 252 · see citing works
Overhage, J. M., Perkins, S., Tierney, W. M., & McDonald, C. J. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080361 · Cited by 232 · see citing works
Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics.
McDonald, C. J., Hui, S. L., & Tierney, W. M. (1992) · PubMed · Cited by 160 · see citing works
Improving allergy alerting in a computerized physician order entry system.
Abookire, S. A., Teich, J. M., Sandige, H. L., Paterno, M. D., Martin, M. T., Kuperman, G. J., & Bates, D. W. (2000) · PubMed Central · Cited by 120 · see citing works
A computer-based record and clinical monitoring system for ambulatory care.
McDonald, C. J., Murray, R. B., Jeris, D. W., et al. (1977) · American Journal of Public Health · doi:10.2105/ajph.67.3.240 · Cited by 88 · see citing works
Kuperman, G. J., Teich, J. M., Bates, D. W., Hiltz, F. L., Hurley, J. M., Lee, R. Y., & Paterno, M. D. (1996) · Europe PMC (PubMed Central) · Cited by 87 · see citing works
Schiff, G. D., Aggarwal, H. C., Kumar, S., & Mcnutt, R. (2000) · The American Journal of Medicine · doi:10.1016/s0002-9343(00)00546-5 · Cited by 52 · see citing works
Electronic Medical Records: A Decade of Experience
Safran, C. (2001) · JAMA · doi:10.1001/jama.285.13.1766-jms0404-4-1 · Cited by 31 · see citing works
A new knowledge structure for drug-drug interactions.
Kuperman, G. J., Bates, D. W., Teich, J. M., et al. (1994) · PubMed · Cited by 17 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Technology Acceptance Model: Its past and its future in health care
Holden, R. J., & Karsh, B.-T. (2010) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.07.002 · Cited by 2,669 · see citing works
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117 · see citing works
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763 · see citing works
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0430-8 · Cited by 709 · see citing works
“Many overrides are known to be appropriate, as they are informed by clinical knowledge not captured in structured EHR data [10, 40].”
BackgroundUsing AI-generated suggestions from ChatGPT to optimize clinical decision support
Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., & Wright, A. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad072 · Cited by 397 · see citing works
Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care
Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., Recklet, E. G., Bates, D. W., & Gandhi, T. K. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1868 · Cited by 353 · see citing works
Overrides of Medication Alerts in Ambulatory Care
Isaac, T., Weissman, J. S., Davis, R. B., Massagli, M., Cyrulik, A., Sands, D. Z., & Weingart, S. N. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2008.551 · Cited by 309 · see citing works
Self-citationComputerized clinical decision support for prescribing: provision does not guarantee uptake
Moxey, A., Robertson, J., Newby, D., Hains, I., Williamson, M., & Pearson, S.-A. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3170 · Cited by 305 · see citing works
Hsieh, T. C., Kuperman, G. J., Jaggi, T., Hojnowski-Diaz, P., Fiskio, J., Williams, D. H., Bates, D. W., & Gandhi, T. K. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1556 · Cited by 232 · see citing works
Phansalkar, S., Edworthy, J., Hellier, E., Seger, D. L., Schedlbauer, A., Avery, A. J., & Bates, D. W. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005264 · Cited by 226 · see citing works
Overrides of medication-related clinical decision support alerts in outpatients
Nanji, K. C., Slight, S. P., Seger, D. L., Cho, I., Fiskio, J. M., Redden, L. M., Volk, L. A., & Bates, D. W. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001813 · Cited by 219 · see citing works
Evaluation of Outpatient Computerized Physician Medication Order Entry Systems: A Systematic Review
Eslami, S., Abu-Hanna, A., & de Keizer, N. F. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2238 · Cited by 200 · see citing works
Embi, P. J., & Leonard, A. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000743 · Cited by 199 · see citing works
Medication-related clinical decision support alert overrides in inpatients
Nanji, K. C., Seger, D. L., Slight, S. P., Amato, M. G., Beeler, P. E., Her, Q. L., Dalleur, O., Eguale, T., Wong, A., Silvers, E. R., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx115 · Cited by 179 · see citing works
Drug interaction alert override rates in the Meaningful Use era: No evidence of progress
Bryant, A. D., Fletcher, G. S., & Payne, T. H. (2014) · Applied Clinical Informatics · doi:10.4338/aci-2013-12-ra-0103 · Cited by 172 · see citing works
A Clinical Case of Electronic Health Record Drug Alert Fatigue: Consequences for Patient Outcome
Carspecken, C. W., Sharek, P. J., Longhurst, C., & Pageler, N. M. (2013) · Pediatrics · doi:10.1542/peds.2012-3252 · Cited by 142 · see citing works
A framework for evaluating the appropriateness of clinical decision support alerts and responses
McCoy, A. B., Waitman, L. R., Lewis, J. B., Wright, J. A., Choma, D. P., Miller, R. A., & Peterson, J. F. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000185 · Cited by 138 · see citing works
Sittig, D. F., Ash, J. S., Zhang, J., Osheroff, J. A., & Shabot, M. M. (2006) · Pediatrics · doi:10.1542/peds.2005-3132 · Cited by 130 · see citing works
Appropriateness of Overridden Alerts in Computerized Physician Order Entry: Systematic Review
Poly, T. N., Islam, M. M., Yang, H.-C., & Li, Y.-C. (. (2020) · JMIR Medical Informatics · doi:10.2196/15653 · Cited by 118 · see citing works
Lin, C.-P., Payne, T. H., Nichol, W. P., Hoey, P. J., Anderson, C. L., & Gennari, J. H. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2453 · Cited by 110 · see citing works
Public health communications and alert fatigue
Baseman, J. G., Revere, D., Painter, I., Toyoji, M., Thiede, H., & Duchin, J. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-295 · Cited by 105 · see citing works
Simpao, A. F., Ahumada, L. M., Desai, B. R., Bonafide, C. P., Gálvez, J. A., Rehman, M. A., Jawad, A. F., Palma, K. L., & Shelov, E. D. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002538 · Cited by 94 · see citing works
Assessing the Value of ChatGPT for Clinical Decision Support Optimization
Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., & Wright, A. (2023) · doi:10.1101/2023.02.21.23286254 · Cited by 90 · see citing works
Emotional Aspects of Computer-based Provider Order Entry: A Qualitative Study
Sittig, D. F. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1711 · Cited by 87 · see citing works
Slight, S. P., Seger, D. L., Nanji, K. C., Cho, I., Maniam, N., Dykes, P. C., & Bates, D. W. (2013) · PLoS ONE · doi:10.1371/journal.pone.0085071 · Cited by 86 · see citing works
Clinical decision support systems in the pediatric intensive care unit*
Mack, E. H., Wheeler, D. S., & Embi, P. J. (2009) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0b013e3181936b23 · Cited by 76 · see citing works
Computerized Physician Order Entry and Online Decision Support
Handler, J. A., Feied, C. F., Coonan, K., Vozenilek, J., Gillam, M., Peacock, P. R., Sinert, R., & Smith, M. S. (2004) · Academic Emergency Medicine · doi:10.1197/j.aem.2004.08.007 · Cited by 58 · see citing works
Analysis of overridden alerts in a drug-drug interaction detection system
Mille, F., Schwartz, C., Brion, F., Fontan, J.-E., Bourdon, O., Degoulet, P., & Jaulent, M.-C. (2008) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzn038 · Cited by 47 · see citing works
Jani, Y. H., Barber, N., & Wong, I. C. K. (2011) · International Journal of Pharmacy Practice · doi:10.1111/j.2042-7174.2011.00132.x · Cited by 44 · see citing works
Miller, A. M., Boro, M. S., Korman, N. E., & Davoren, J. B. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000262 · Cited by 41 · see citing works
Clinician collaboration to improve clinical decision support: the Clickbusters initiative
McCoy, A. B., Russo, E. M., Johnson, K. B., Addison, B., Patel, N., Wanderer, J. P., Mize, D. E., Jackson, J. G., Reese, T. J., Littlejohn, S., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac027 · Cited by 41 · see citing works
Customizing clinical decision support to prevent excessive drug–drug interaction alerts
Horn, J. R., Hansten, P. D., Osborn, J. D., Wareham, P., & Somani, S. (2011) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp100465 · Cited by 26 · see citing works
Exploring the Utility of Automated Drug Alerts in Home Healthcare
Feldman, P. H., McDonald, M., Rosati, R. J., Murtaugh, C., Kovner, C., Goldberg, J. D., & King, L. (2006) · Journal for Healthcare Quality · doi:10.1111/j.1945-1474.2006.tb00592.x · Cited by 5 · see citing works
- Authors
- Weingart, S. N., Toth, M., Sands, D. Z., Aronson, M. D., Davis, R. B., & Phillips, R. S.
- Affiliations
- Beth Israel Deaconess Medical Center (US)
- Venue
- Archives of Internal Medicine
- Publisher
- American Medical Association (AMA)
- Published
- 2003-11-24
- Volume
- 163
- Issue
- 21
- Pages
- 2625
- ISSN
- 0003-9926 (ISSN-L)
- OpenAlex
- W2044859414
- Cited by
- 487
- References
- 17
- Citations by year
- 2026: 6 · 2025: 6 · 2024: 6 · 2023: 15 · 2022: 25 · 2021: 17 · 2020: 14 · 2019: 10 · 2018: 19 · 2017: 24 · 2016: 28 · 2015: 25 · 2014: 27 · 2013: 34 · 2012: 38
- FWCI
- 34.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Archives of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (1K08 HS 11644-01)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Health Sciences › Medicine › Geriatrics and Gerontology › Pharmaceutical Practices and Patient Outcomes
- Keywords
- alert override, computerized physician order entry, primary care, medication errors, drug allergy alerts, drug interaction alerts, adverse drug reactions, prescriber decision-making
- MeSH terms
- Adult, Aged, Boston, Drug Interactions, Drug Utilization, Female, Humans, Internal Medicine, Male, Medication Errors, Middle Aged, Practice Patterns, Physicians', Primary Health Care, Medical Records Systems, Computerized, Adverse Drug Reaction Reporting Systems, Safety Management, Decision Support Systems, Clinical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
