Overriding of Drug Safety Alerts in Computerized Physician Order Entry
Open accessAbstract
Many computerized physician order entry (CPOE) systems have integrated drug safety alerts. The authors reviewed the literature on physician response to drug safety alerts and interpreted the results using Reason's framework of accident causation. In total, 17 papers met the inclusion criteria. Drug safety alerts are overridden by clinicians in 49% to 96% of cases. Alert overriding may often be justified and adverse drug events due to overridden alerts are not always preventable. A distinction between appropriate and useful alerts should be made. The alerting system may contain error-producing conditions like low specificity, low sensitivity, unclear information content, unnecessary workflow disruptions, and unsafe and inefficient handling. These may result in active failures of the physician, like ignoring alerts, misinterpretation, and incorrect handling. Efforts to improve patient safety by increasing correct handling of drug safety alerts should focus on the error-producing conditions in software and organization. Studies on cognitive processes playing a role in overriding drug safety alerts are lacking.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CDSSs are often basic and associated with a high burden of false positive alerts and alert fatigue.
“Despite these benefits, CDSSs are often basic and associated with a high burden of false positive alerts and alert fatigue.”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [13]
The Abstract warns that a high burden of alerts can cause alert fatigue, and the Results describe irrelevant and false-positive alerts as reasons physicians ignore or override them. The Results also identify missing alerting features, but do not establish the broader claim that CDSSs are often basic. (LLM, conf=0.84)
Experimental — accuracy not yet measuredA substantial body of work suggests that repeated CDSS alerts are less likely to be informative.
“This was because a substantial body of work [7, 8, 32, 33] suggests that repeated CDSS alerts are less likely to be informative (that is, they are often equivalent to false alarms).”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [7]
The Results describe many alerts as irrelevant or false positive and recommend preventing some repeat alerts, including interaction alerts previously found not to cause problems for a particular patient. But the paper does not establish that repeated alerts generally become less informative or are often false alarms; it also cautions that an alert a clinician already knows can still flag a potentially unsafe situation. Page numbers were not available in our copy. (LLM, conf=0.82)
Experimental — accuracy not yet measuredDuplicate alerts have previously been identified as potentially uninformative.
“Duplicate alerts have previously been identified as potentially uninformative [7, 8, 32].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [7]
The Results (page not available) say that physicians find many alerts irrelevant and recommend preventing future alerts after an overriding decision. The paper does not specifically identify duplicate or repeated alerts as potentially uninformative; its references to “duplicate orders” concern alerts about duplicate medication orders, not duplicate alerts. (LLM, conf=0.82)
Experimental — accuracy not yet measuredThe inherent design of our site's EHR makes interruptive alerts one of the easiest and least expensive forms of CDS to implement, even though they are known to be disruptive and often ineffective.
“The inherent design of our site’s EHR makes interruptive alerts one of the easiest and least expensive forms of CDS to implement, even though they are known to be disruptive and often ineffective. 5 15 16”
From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [16]
The Results (page not available) describe unnecessary workflow interruptions and alert fatigue that can cause important alerts to be ignored, supporting the claim that alerts can be disruptive and ineffective. The paper does not assess the citing site's EHR or say that interruptive alerts are among the easiest or least expensive CDS tools to implement. (LLM, conf=0.86)
Experimental — accuracy not yet measuredAlert nonadherence, or clinician overrides, occurs for 49%-96% of alerts.
“Alert nonadherence, or clinician overrides, is one key barrier that occurs for 49%–96% of alerts.13–23”
From Clinician collaboration to improve clinical decision support: the Clickbusters initiative (2022) · cites this paper as [13]
The Results report that safety alerts were overridden in 49%–96% of cases, supporting the cited range. But the same passage excludes serious overdose alerts, which were overridden about one fourth of the time; the Discussion and Conclusion gives 27%. The citing sentence omits that qualification. (LLM, conf=0.86)
Experimental — accuracy not yet measured
An influential systematic review of CDSS override rates noted that across studies there was no evidence of a relationship between alert volume and percentage overridden.
“An influential systematic review of CDSS override rates noted that across studies there was no evidence of a relationship between alert volume and percentage overridden [7].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [7]
In practice, 49-96% of alerts are overridden.
“In practice, however, 49–96% of alerts are overridden [7], raising questions about the effectiveness of decision support.”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [7]
It is widely accepted that alert fatigue explains high override rates.
“It is widely accepted that “alert fatigue” explains high override rates [7, 9–14].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [7]
Alert fatigue might cause prescribers to override both irrelevant and clinically relevant alerts due to an overload of irrelevant alerts, which compromises the desired safety effect of CDSSs.
“Alert fatigue might cause prescribers to override both irrelevant and clinically relevant alerts due to an overload of irrelevant alerts, which compromises the desired safety effect of the CDSSs [13].”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [13]
Alert override rates ranging from 49 to 96% have been described.
“Alert override rates ranging from 49 to 96% have been described [13].”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [13]
High override rates of 56.3%-95.7% have already been described for DDI alerts.
“High override rates of 56.3%-95.7% have already been described for DDI alerts [13, 21, 27–32].”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [13]
The cognitive overload model holds that alert fatigue is caused by receipt of a large quantity of information along with insufficient time or cognitive resources to distinguish relevant from irrelevant information.
“One conceptual model, which we label cognitive overload, is that alert fatigue is caused by receipt of a large quantity of information along with insufficient time or cognitive resources to distinguish relevant from irrelevant information [7, 15].”
From Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system (2017) · cites this paper as [7]
Alerts fire so frequently that most are ignored, with prior research demonstrating rates of ignoring as high as 96%.
“Optimal alert volume is unknown, but alerts fire so frequently that most are ignored, with prior research demonstrating rates of ignoring as high as 96%, almost entirely defeating their purpose.”
From Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support (2026) · cites this paper as [16]
High alert burden causes alert fatigue and reduces the efficacy of CDS.
“Overrides may result from an excess of alerts that are repeated or deemed irrelevant (ie, a high alert burden), causing alert fatigue and reducing the efficacy of CDS.13,24”
From Clinician collaboration to improve clinical decision support: the Clickbusters initiative (2022) · cites this paper as [13]
Overrides may result from an excess of alerts that are repeated or deemed irrelevant (high alert burden).
“Overrides may result from an excess of alerts that are repeated or deemed irrelevant (ie, a high alert burden), causing alert fatigue and reducing the efficacy of CDS.13,24”
From Clinician collaboration to improve clinical decision support: the Clickbusters initiative (2022) · cites this paper as [13]
Our study results are in agreement with previous reports.
“Our study results are in agreement with previous reports.”
From Overall performance of a drug–drug interaction clinical decision support system: quantitative evaluation and end-user survey (2022) · cites this paper as [13]
The Results section reviews previous reports on drug-safety alert overrides, but the citing sentence does not identify which of its study results supposedly agree with them. The source alone cannot establish that agreement; no page number is available in our copy. (LLM, conf=0.70)
Experimental — accuracy not yet measuredAs many as 96% of CDSS alerts are overridden or ignored by physicians.
“Importantly there is clear evidence that CDSS tools are not always used when available, 15 with up to 96% of alerts being overridden or ignored by physicians.[16][17][18][19]”
From Computerized clinical decision support for prescribing: provision does not guarantee uptake (2010) · cites this paper as [17]
Excessive alerts, especially when procedural and cost information are added to safety alerts, may be counterproductive because they cause user fatigue.
“excessive alerts, exacerbated in contemporary practice when procedural information and cost advice are provided in addition to safety alerts, may be counterproductive as they lead to user fatigue.12 20”
From ‘Too much, too late’: mixed methods multi-channel video recording study of computerized decision support systems and GP prescribing (2013) · cites this paper as [20]
Alerts that give the prescriber a choice and require a reason for deviating from protocol may be more effective than alerts that can simply be over-ridden.
“Alerts that offer the prescriber choice, and require reasons for deviation from protocol may be more effective than those that can simply be over-ridden.20 26”
From ‘Too much, too late’: mixed methods multi-channel video recording study of computerized decision support systems and GP prescribing (2013) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Human error: models and management
Reason, J. (2000) · BMJ · doi:10.1136/bmj.320.7237.768 · Cited by 5,665 · see citing works
Rasmussen, J. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313160 · Cited by 3,434 · see citing works
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789 · see citing works
Hunt, D. L., Haynes, R. B., Hanna, S., & Smith, K. (1998) · JAMA · doi:10.1001/jama.280.15.1339 · Cited by 1,684 · see citing works
Systems Analysis of Adverse Drug Events
Leape, L. L. (1995) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1995.03530010049034 · Cited by 1,404 · see citing works
Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., Spurr, C., Khorasani, R., Tanasijevic, M., & Middleton, B. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1370 · Cited by 1,346 · see citing works
The contribution of latent human failures to the breakdown of complex systems
Reason, J. (1990) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1990.0090 · Cited by 762 · see citing works
Causes of prescribing errors in hospital inpatients: a prospective study
Franklin, B. D., Schachter, M., Vincent, C. A., & Barber, N. (2002) · The Lancet · doi:10.1016/s0140-6736(02)08350-2 · Cited by 674 · see citing works
Physicians' Decisions to Override Computerized Drug Alerts in Primary Care
Weingart, S. N., Toth, M., Sands, D. Z., Aronson, M. D., Davis, R. B., & Phillips, R. S. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.21.2625 · Cited by 487 · see citing works
Effects of Computerized Physician Order Entry on Prescribing Practices
Teich, J. M., Merchia, P., Schmiz, J. L., Kuperman, G. J., Spurr, C. D., & Bates, D. W. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.18.2741 · Cited by 475 · see citing works
Guided Medication Dosing for Inpatients With Renal Insufficiency
Chertow, G. M. (2001) · JAMA · doi:10.1001/jama.286.22.2839 · Cited by 447 · see citing works
A Randomized Trial of "Corollary Orders" to Prevent Errors of Omission
Overhage, J. M., Tierney, W. M., Zhou, X., & McDonald, C. J. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040364 · Cited by 375 · see citing works
A Computer Alert System to Prevent Injury From Adverse Drug Events
Raschke, R. A., Gollihare, B., Wunderlich, T. A., Guidry, J. R., Leibowitz, A. I., Peirce, J. C., Lemelson, L., Heisler, M. A., & Susong, C. R. (1998) · JAMA · doi:10.1001/jama.280.15.1317 · Cited by 356 · see citing works
Beyond the organisational accident: the need for "error wisdom" on the frontline
REASON, J. T. (2004) · BMJ Quality & Safety · doi:10.1136/qshc.2003.009548 · Cited by 334 · see citing works
Effect of Computer-Based Alerts on the Treatment and Outcomes of Hospitalized Patients
Rind, D. M. (1994) · Archives of Internal Medicine · doi:10.1001/archinte.1994.00420130107014 · Cited by 323 · see citing works
Lobach, D. F., & Hammond, W. E. (1997) · The American Journal of Medicine · doi:10.1016/s0002-9343(96)00382-8 · Cited by 303 · see citing works
Organizational controls and safety: The varieties of rule‐related behaviour
Reason, J., Parker, D., & Lawton, R. (1998) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1998.tb00678.x · Cited by 292 · see citing works
Improving Recognition of Drug Interactions
Glassman, P. A., Simon, B. F., Belperio, P. S., & Lanto, A. B. (2002) · Medical Care · doi:10.1097/00005650-200212000-00004 · Cited by 279 · see citing works
Hsieh, T. C., Kuperman, G. J., Jaggi, T., Hojnowski-Diaz, P., Fiskio, J., Williams, D. H., Bates, D. W., & Gandhi, T. K. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1556 · Cited by 232 · see citing works
Drug-Related Problems in Hospitalised Patients
van den Bemt, P. M. L. A., Egberts, T. C. G., den Berg, L. T. W. D. J., & Brouwers, J. R. B. J. (2000) · Drug Safety · doi:10.2165/00002018-200022040-00005 · Cited by 202 · see citing works
Nightingale, P. (2000) · BMJ · doi:10.1136/bmj.320.7237.750 · Cited by 196 · see citing works
Physician Response to Computer Reminders
McDonald, C. J. (1980) · JAMA · doi:10.1001/jama.1980.03310140037026 · Cited by 184 · see citing works
Clinical Relevance of Drug-Drug Interactions
van Roon, E. N., Flikweert, S., le Comte, M., et al. (2005) · Drug Safety · doi:10.2165/00002018-200528120-00007 · Cited by 182 · see citing works
GPs' views on computerized drug interaction alerts: questionnaire survey
Magnus, D., Rodgers, S., & Avery, A. J. (2002) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2002.00434.x · Cited by 157 · see citing works
Information technology and medication safety: what is the benefit?
Kaushal, R. (2002) · BMJ Quality & Safety · doi:10.1136/qhc.11.3.261 · Cited by 154 · see citing works
Linking Laboratory and Pharmacy
Schiff, G. D., Klass, D. B., Peterson, J. F., Shah, G., & Bates, D. W. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.8.893 · Cited by 152 · see citing works
Performance of Community Pharmacy Drug Interaction Software
Hazlet, T. K., Lee, T. A., Hansten, P. D., et al. (2001) · Journal of the American Pharmaceutical Association (1996) · doi:10.1016/s1086-5802(16)31230-x · Cited by 141 · see citing works
Characteristics and override rates of order checks in a practitioner order entry system.
Payne, T. H., Nichol, W. P., Hoey, P., & Savarino, J. E. (2002) · PubMed Central · Cited by 129 · see citing works
Improving allergy alerting in a computerized physician order entry system.
Abookire, S. A., Teich, J. M., Sandige, H. L., Paterno, M. D., Martin, M. T., Kuperman, G. J., & Bates, D. W. (2000) · PubMed Central · Cited by 120 · see citing works
Computer support for determining drug dose: systematic review and meta-analysis
Walton, R. T., Dovey, S. M., Harvey, E., & Freemantle, N. (1999) · BMJ · doi:10.1136/bmj.318.7189.984 · Cited by 112 · see citing works
Pharmacies and Prevention of Potentially Fatal Drug Interactions
Cavuto, N. J. (1996) · JAMA · doi:10.1001/jama.1996.03530380028022 · Cited by 107 · see citing works
Preventable Medication Errors: Identifying and Eliminating Serious Drug Interactions
Peterson, J. F., & Bates, D. O. (2001) · Journal of the American Pharmaceutical Association (1996) · doi:10.1016/s1086-5802(16)31243-8 · Cited by 106 · see citing works
Miller, R. A., Gardner, R. M., Johnson, K. B., & Hripcsak, G. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1830 · Cited by 99 · see citing works
Ahearn, M. D., & Kerr, S. J. (2003) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2003.tb05415.x · Cited by 89 · see citing works
Harpole, L. H., Khorasani, R., Fiskio, J. M., Kuperman, G. J., & Bates, D. W. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040511 · Cited by 87 · see citing works
Reasons for Physician Non-Adherence to Electronic Drug Alerts
Taylor, L. K., & Tamblyn, R. M. (2004) · Studies in health technology and informatics · doi:10.3233/978-1-60750-949-3-1101 · Cited by 86 · see citing works
How to Design Computerized Alerts to Ensure Safe Prescribing Practices
Feldstein, A. C., Simon, S. R., Schneider, J. P., Krall, M., Laferriere, D. S., Smith, D. H., Sittig, D. F., & Soumerai, S. B. (2004) · The Joint Commission Journal on Quality and Safety · doi:10.1016/s1549-3741(04)30071-7 · Cited by 81 · see citing works
Krall, M., & Sittig, D. F. (2002) · PubMed · Cited by 79 · see citing works
Kawamoto, K., & Lobach, D. F. (2003) · PubMed · Cited by 70 · see citing works
Clinical Relevance of Automated Drug Alerts From the Perspective of Medical Providers
Spina, J. R., Glassman, P. A., Belperio, P. S., Cader, R., Asch, S. M., & Primary Care Investigative Group of the VA Los Angeles Healthcare System (2005) · American Journal of Medical Quality · doi:10.1177/1062860604273777 · Cited by 64 · see citing works
(2002) · PubMed Central · Cited by 53 · see citing works
Effective drug-allergy checking: methodological and operational issues
Kuperman, G. J., Gandhi, T. K., & Bates, D. W. (2003) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00063-7 · Cited by 53 · see citing works
Beyond the organisational accident: the need for "error wisdom" on the frontline
REASON, J. T. (2004) · BMJ Quality & Safety · doi:10.1136/qhc.13.suppl_2.ii28 · Cited by 49 · see citing works
Patient Safety in Guideline-Based Decision Support for Hypertension Management: ATHENA DSS
Goldstein, M. K. (2002) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1218 · Cited by 46 · see citing works
Kalmeijer, M. D. (2003) · Pharmacy World & Science · doi:10.1023/a:1024008600247 · Cited by 42 · see citing works
Computerized physician order entry: challenges and opportunities
Shane, R. R. (2002) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/59.3.286 · Cited by 39 · see citing works
Krall, M., & Sittig, D. F. (2001) · PubMed · Cited by 34 · see citing works
Vicente, K. J. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.4.291 · Cited by 34 · see citing works
Kaushal, R., Shojania, K. G., & Bates, D. W. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.12.1409 · Cited by 30 · see citing works
Patient safety in guideline-based decision support for hypertension management: ATHENA DSS.
Goldstein, M. K., Hoffman, B. B., Coleman, R. W., Tu, S. W., Shankar, R., O’Connor, M. J., Martins, S. B., Martins, S. B., Advani, A. A., & Musen, M. A. (2001) · PubMed Central · Cited by 27 · see citing works
Using Computer Technology to Detect, Measure, and Prevent Adverse Drug Events
Gardner, R. M., & Evans, R. S. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1651 · Cited by 23 · see citing works
Comparison of two knowledge bases on the detection of drug-drug interactions.
Fiol, G. D., Rocha, B. H., Kuperman, G. J., et al. (2000) · PubMed · Cited by 22 · see citing works
Clinicians' Response to Computerized Detection of Infections
Rocha, B. H., Christenson, J. C., Evans, R. S., & Gardner, R. M. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080117 · Cited by 21 · see citing works
Designing information technology to support prescribing decision making
Barber, N. (2004) · BMJ Quality & Safety · doi:10.1136/qhc.13.6.450 · Cited by 21 · see citing works
Designing information technology to support prescribing decision making
Barber, N. (2004) · BMJ Quality & Safety · doi:10.1136/qshc.2004.012104 · Cited by 21 · see citing works
An information system to promote intravenous-to-oral medication conversion.
Teich, J. M., Petronzio, A. M., Gerner, J. R., et al. (1999) · PubMed · Cited by 19 · see citing works
Prescription errors occur despite computerized prescriber order entry
Ballentine, A. J., Kinnaird, D., & Wilson, J. P. (2003) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/60.7.708 · Cited by 17 · see citing works
AsthmaCritic: Issues in Designing a Noninquisitive Critiquing System for Daily Practice
Kuilboer, M. M., van Wijk, M. A. M., Mosseveld, M., & van der Lei, J. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1273 · Cited by 16 · see citing works
Informed Consent for Genetic Research-Reply
Clayton, E. W. (1996) · JAMA · doi:10.1001/jama.1996.03530380027021 · Cited by 13 · see citing works
Re: GPs' views on computerized drug interaction alerts
Ashworth, M. J. (2002) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2002.00433.x · Cited by 10 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [15]Peterson JF, Kuperman GJ, Shek C, Bates DW. Physician responses to life-threatening drug-drug interaction alerts. J Gen Intern Med. 2001;16:212. [Google Scholar]
- [26]Krall MA, Sittig DF. Clinicians' assessment of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proc AMIA Symp. 2002:400–4. [PMC free article] [PubMed]
Cited by
Sorted by citations
The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview
Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000387 · Cited by 1,442 · see citing works
Sittig, D. F., & Singh, H. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2010.042085 · Cited by 897 · see citing works
Medication-related clinical decision support in computerized provider order entry systems: a review
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., Classen, D. C., & Bates, D. W. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2170 · Cited by 763 · see citing works
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0430-8 · Cited by 709 · see citing works
“We considered alerts that were repeated the same year as potentially uninformative, following the work of previous researchers who have identified repeated alerts as duplicative and likely to have low relevance [7, 8, 32, 33].”
BackgroundInfluentialHarrison, M. I., Koppel, R., & Bar-Lev, S. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2384 · Cited by 691 · see citing works
A targeted real-time early warning score (TREWScore) for septic shock.
Henry, K. E., Hager, D. N., Pronovost, P. J., & Saria, S. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aab3719 · Cited by 598 · see citing works
Using AI-generated suggestions from ChatGPT to optimize clinical decision support
Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., & Wright, A. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad072 · Cited by 397 · see citing works
Computerized clinical decision support for prescribing: provision does not guarantee uptake
Moxey, A., Robertson, J., Newby, D., Hains, I., Williamson, M., & Pearson, S.-A. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3170 · Cited by 305 · see citing works
Kesselheim, A. S., Cresswell, K., Phansalkar, S., Bates, D. W., & Sheikh, A. (2011) · Health Affairs · doi:10.1377/hlthaff.2010.1111 · Cited by 277 · see citing works
Schedlbauer, A., Prasad, V., Mulvaney, C., Phansalkar, S., Stanton, W., Bates, D. W., & Avery, A. J. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2910 · Cited by 262 · see citing works
Phansalkar, S., van der Sijs, H., Tucker, A. D., Desai, A. A., Bell, D. S., Teich, J. M., Middleton, B., & Bates, D. W. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001089 · Cited by 239 · see citing works
Self-citationPhansalkar, S., Edworthy, J., Hellier, E., Seger, D. L., Schedlbauer, A., Avery, A. J., & Bates, D. W. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005264 · Cited by 226 · see citing works
Overrides of medication-related clinical decision support alerts in outpatients
Nanji, K. C., Slight, S. P., Seger, D. L., Cho, I., Fiskio, J. M., Redden, L. M., Volk, L. A., & Bates, D. W. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001813 · Cited by 219 · see citing works
Singh, H., Thomas, E. J., Mani, S., Sittig, D., Arora, H., Espadas, D., Khan, M. M., & Petersen, L. A. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.263 · Cited by 212 · see citing works
“Data on outpatient drug-related alerts have shown that HCPs ignore alerts because of information overload,(20-23) often resulting in failure to act on important as well as clinically unimportant alerts.(20) While no data exist to substantiate a similar problem for diagnostic test result alerts, HCPs often receive seve…”
BackgroundEvaluation of Outpatient Computerized Physician Medication Order Entry Systems: A Systematic Review
Eslami, S., Abu-Hanna, A., & de Keizer, N. F. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2238 · Cited by 200 · see citing works
Embi, P. J., & Leonard, A. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000743 · Cited by 199 · see citing works
Medication-related clinical decision support alert overrides in inpatients
Nanji, K. C., Seger, D. L., Slight, S. P., Amato, M. G., Beeler, P. E., Her, Q. L., Dalleur, O., Eguale, T., Wong, A., Silvers, E. R., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx115 · Cited by 179 · see citing works
Drug interaction alert override rates in the Meaningful Use era: No evidence of progress
Bryant, A. D., Fletcher, G. S., & Payne, T. H. (2014) · Applied Clinical Informatics · doi:10.4338/aci-2013-12-ra-0103 · Cited by 172 · see citing works
A framework for evaluating the appropriateness of clinical decision support alerts and responses
McCoy, A. B., Waitman, L. R., Lewis, J. B., Wright, J. A., Choma, D. P., Miller, R. A., & Peterson, J. F. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000185 · Cited by 138 · see citing works
Appropriateness of Overridden Alerts in Computerized Physician Order Entry: Systematic Review
Poly, T. N., Islam, M. M., Yang, H.-C., & Li, Y.-C. (. (2020) · JMIR Medical Informatics · doi:10.2196/15653 · Cited by 118 · see citing works
Scott, I. A., Pillans, P. I., Barras, M., & Morris, C. (2018) · Therapeutic Advances in Drug Safety · doi:10.1177/2042098618784809 · Cited by 114 · see citing works
Lin, C.-P., Payne, T. H., Nichol, W. P., Hoey, P. J., Anderson, C. L., & Gennari, J. H. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2453 · Cited by 110 · see citing works
Public health communications and alert fatigue
Baseman, J. G., Revere, D., Painter, I., Toyoji, M., Thiede, H., & Duchin, J. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-295 · Cited by 105 · see citing works
Assessing the Value of ChatGPT for Clinical Decision Support Optimization
Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., & Wright, A. (2023) · doi:10.1101/2023.02.21.23286254 · Cited by 90 · see citing works
Clinical decision support systems in the pediatric intensive care unit*
Mack, E. H., Wheeler, D. S., & Embi, P. J. (2009) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0b013e3181936b23 · Cited by 76 · see citing works
Leading the Way to Solutions to the Opioid Epidemic: AOA Critical Issues
Seymour, R. B., Ring, D., Higgins, T., & Hsu, J. R. (2017) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.17.00066 · Cited by 74 · see citing works
Edrees, H., Amato, M. G., Wong, A., Seger, D. L., & Bates, D. W. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa034 · Cited by 64 · see citing works
Scheepers-Hoeks, A.-M. J., Grouls, R. J., Neef, C., Ackerman, E. W., & Korsten, E. H. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2013.05.002 · Cited by 53 · see citing works
Van De Sijpe, G., Quintens, C., Walgraeve, K., Van Laer, E., Penny, J., De Vlieger, G., Schrijvers, R., De Munter, P., Foulon, V., Casteels, M., et al. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01783-z · Cited by 54 · see citing works
“Shah et al. compared two commercial CDSSs and showed a 94% reduction in alert burden, with much higher sensitivity and specificity [16].”
BackgroundInfluentialAlsolamy, S., Al Salamah, M., Al Thagafi, M., Al-Dorzi, H. M., Marini, A. M., Aljerian, N., Al-Enezi, F., Al-Hunaidi, F., Mahmoud, A. M., Alamry, A., et al. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-014-0105-7 · Cited by 52 · see citing works
Hayward, J., Thomson, F., Milne, H., Buckingham, S., Sheikh, A., Fernando, B., Cresswell, K., Williams, R., & Pinnock, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001484 · Cited by 48 · see citing works
Brown, S. M., Jones, J., Kuttler, K. G., Keddington, R. K., Allen, T. L., & Haug, P. (2016) · BMC Emergency Medicine · doi:10.1186/s12873-016-0095-0 · Cited by 45 · see citing works
Analysis of overridden alerts in a drug-drug interaction detection system
Mille, F., Schwartz, C., Brion, F., Fontan, J.-E., Bourdon, O., Degoulet, P., & Jaulent, M.-C. (2008) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzn038 · Cited by 47 · see citing works
Computerized Provider Order Entry and Patient Safety
Lehmann, C. U., & Kim, G. R. (2006) · Pediatric Clinics of North America · doi:10.1016/j.pcl.2006.09.006 · Cited by 45 · see citing works
Jani, Y. H., Barber, N., & Wong, I. C. K. (2011) · International Journal of Pharmacy Practice · doi:10.1111/j.2042-7174.2011.00132.x · Cited by 44 · see citing works
Miller, A. M., Boro, M. S., Korman, N. E., & Davoren, J. B. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000262 · Cited by 41 · see citing works
Clinician collaboration to improve clinical decision support: the Clickbusters initiative
McCoy, A. B., Russo, E. M., Johnson, K. B., Addison, B., Patel, N., Wanderer, J. P., Mize, D. E., Jackson, J. G., Reese, T. J., Littlejohn, S., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac027 · Cited by 41 · see citing works
Smith, M., Murphy, D., Laxmisan, A., Sittig, D., Reis, B., Esquivel, A., & Singh, H. (2013) · Applied Clinical Informatics · doi:10.4338/aci-2013-04-ra-0019 · Cited by 31 · see citing works
Customizing clinical decision support to prevent excessive drug–drug interaction alerts
Horn, J. R., Hansten, P. D., Osborn, J. D., Wareham, P., & Somani, S. (2011) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp100465 · Cited by 26 · see citing works
Horn, J., & Ueng, S. (2019) · Annals of Pharmacotherapy · doi:10.1177/1060028019863419 · Cited by 26 · see citing works
Kho, A. N., Doebbeling, B. N., Cashy, J. P., Rosenman, M. B., Dexter, P. R., Shepherd, D. C., Lemmon, L., Teal, E., Khokar, S., & Overhage, J. M. (2013) · Clinical Infectious Diseases · doi:10.1093/cid/cit229 · Cited by 23 · see citing works
Fallon, A., Haralambides, K., Mazzillo, J., & Gleber, C. (2024) · Applied Clinical Informatics · doi:10.1055/a-2226-8144 · Cited by 22 · see citing works
Wan, Y.-K. J., Wright, M. C., McFarland, M. M., Dishman, D., Nies, M. A., Rush, A., Madaras-Kelly, K., Jeppesen, A., & Del Fiol, G. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad203 · Cited by 11 · see citing works
Helwig, M. (2026) · arXiv · doi:10.48550/arxiv.2609.05510
“The gate’s contract borrows the SRE maxim that every page must be actionable [SRE 2016], and clinical alarm-fatigue research supplies the failure mode when it is violated: operators desensitize and override [DrugAlerts 2006; Ancker 2017].”
Qualitative Study of Inpatient Hospital Medicine Interruptive Alerts and Clinical Decision Support
Khan, A., & Schnipper, J. (2026) · Applied Clinical Informatics · doi:10.1055/a-2955-9555
- Authors
- van der Sijs, H., Aarts, J., Vulto, A., & Berg, M.
- Affiliations
- Erasmus MC (NL)
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2006-03-01
- Volume
- 13
- Issue
- 2
- Pages
- 138-147
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC1447540
- OpenAlex
- W2110922423
- Cited by
- 1,117
- References
- 60
- Citations by year
- 2026: 49 · 2025: 38 · 2024: 51 · 2023: 36 · 2022: 76 · 2021: 63 · 2020: 58 · 2019: 62 · 2018: 58 · 2017: 54 · 2016: 69 · 2015: 73 · 2014: 61 · 2013: 69 · 2012: 70
- FWCI
- 40.84
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Emergency Medical Services › Patient Safety and Medication Errors
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Keywords
- drug safety warnings, computerized physician order entry, adverse drug reactions, active failures, workflow disruption, patient safety, clinical decision support
- MeSH terms
- Drug Therapy, Computer-Assisted, Humans, Medication Errors, Reminder Systems, Safety Management, Medical Order Entry Systems, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
