Leveraging 10 Days of Alert Malfunction to Improve Mature Organizational Clinical Decision Support Processes
Open accessAbstract
Interruptive clinical decision support (CDS) alerts are intended to standardize patient care and prevent harm. However, failures can occur even in organizations with mature CDS governance and advanced analytics. These breakdowns, marked by excessive firings, workflow disruption, and clinician dissatisfaction, can provide insights into systemic weaknesses in CDS design, testing, and monitoring processes. This study aimed to examine a CDS alert malfunction as a lens for identifying system-level gaps and propose strategies to strengthen resilience in CDS operations. A retrospective analysis was conducted on an interruptive alert that was developed through a phased, multistakeholder, committee-driven process, but was removed within 10 days due to poor performance, revealing gaps that persisted despite established governance. The alert fired 1,866 times in 5 days, with a 91% dismissal rate and reports of workflow disruption. Feedback indicated provider frustration and concern for malfunction. Analysis revealed gaps in end-user engagement, testing rigor, committee reviews, and monitoring practices. CDS failures can serve as catalysts for system improvement. This case highlights actionable lessons, such as operationalizing user-centered design, clarifying testing expectations, and distributing monitoring responsibilities, to enhance CDS reliability. Even well-established governance structures must be continuously evaluated and adapted to keep pace with evolving CDS technologies, and such investments position organizations to maintain responsive, sustainable systems aligned with high-quality care.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
An overview of clinical decision support systems: Benefits, risks, and strategies for success
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0221-y · Cited by 3,019 · see citing works
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703 · see citing works
Effect of Clinical Decision-Support Systems: A Systematic Review
Bright, T. J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R. R., Samsa, G., Hasselblad, V., Williams, J. W., Musty, M. D., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-1-201207030-00450 · Cited by 1,200 · see citing works
Sittig, D. F., & Singh, H. (2010) · Quality and Safety in Health Care · doi:10.1136/qshc.2010.042085 · Cited by 897 · see citing works
with the HITEC Investigators, Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0430-8 · Cited by 709 · see citing works
Improving Outcomes with Clinical Decision Support: An Implementer's Guide, Second Edition
Osheroff, J. (2012) · doi:10.4324/9781498757461 · Cited by 517 · see citing works
Electronic Health Records and Clinical Decision Support Systems
Romano, M. J., & Stafford, R. S. (2011) · Archives of Internal Medicine · doi:10.1001/archinternmed.2010.527 · Cited by 296 · see citing works
Clinical Decision Support Systems
Berner, E. S. (2007) · Computers in health care · doi:10.1007/978-0-387-38319-4 · Cited by 276 · see citing works
Phansalkar, S., van der Sijs, H., Tucker, A. D., Desai, A. A., Bell, D. S., Teich, J. M., Middleton, B., & Bates, D. W. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001089 · Cited by 239 · see citing works
Cranky comments: detecting clinical decision support malfunctions through free-text override reasons
Aaron, S., McEvoy, D. S., Ray, S., Hickman, T.-T. T., & Wright, A. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy139 · Cited by 41 · see citing works
Clinician collaboration to improve clinical decision support: the Clickbusters initiative
McCoy, A. B., Russo, E. M., Johnson, K. B., Addison, B., Patel, N., Wanderer, J. P., Mize, D. E., Jackson, J. G., Reese, T. J., Littlejohn, S., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac027 · Cited by 41 · see citing works
Towards a Maturity Model for Clinical Decision Support Operations
Orenstein, E. W., Muthu, N., Weitkamp, A. O., et al. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-1697905 · Cited by 26 · see citing works
Kandaswamy, S., Williams, H., Thompson, S., Dawson, T. E., Muthu, N., & Orenstein, E. W. (2024) · Applied Clinical Informatics · doi:10.1055/a-2404-2129 · Cited by 12 · see citing works
Butler, J. M., Doubleday, A., Sattar, U., Nies, M. A., Jeppesen, A., Wright, M. C., Reese, T., Kawamoto, K., Fiol, G. D., & Madaras‐Kelly, K. (2025) · Applied Clinical Informatics · doi:10.1055/a-2505-7743 · Cited by 5 · see citing works
The Burden of a Highly Targeted Alert
Clarke, T., Kotarski, T., & Tobias, M. (2025) · Applied Clinical Informatics · doi:10.1055/a-2573-8067 · Cited by 4 · see citing works
Self-citationClinical decision support malfunctions related to medication routes: a case series
Wright, A. T., Nelson, S. D., Rubins, D. M., Schreiber, R., & Sittig, D. F. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac150 · Cited by 2 · see citing works
Newton, N., Bamgboje-Ayodele, A., Forsyth, R., Bruce, L., McPhail, S. M., Shaw, T., Naicker, S., Tariq, A., & Baysari, M. T. (2025) · Applied Clinical Informatics · doi:10.1055/a-2581-6236 · Cited by 2 · see citing works
Redesigning Clinical Decision Support for Retinopathy of Prematurity Screening After Alert Failure
Karlsson, M. G., Hess, L. M., Jeppesen, A., & Fortunov, R. M. (2025) · Applied Clinical Informatics · doi:10.1055/a-2594-3571 · Cited by 2 · see citing works
Plattner, A. S., Lockowitz, C. R., Same, R. G., Abdelnour, M., Chin, S., Cormier, M. J., Daugherty, M. S., Grier, A. E., Hampton, N. B., Hofford, M. R., et al. (2025) · Applied Clinical Informatics · doi:10.1055/a-2595-4849 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- A Wright, 2009, Governance for clinical decision support: Case studies and recommended practices from leading institutions, J Am Med Inform Assoc, 16, 232
- J D McGreevey III, 2020, Reducing alert burden in electronic health records: State of the art recommendations from four health systems, Appl Clin Inform, 11, 1
- F Magrabi, 2012, Patient safety problems associated with healthcare information technology: An analysis of adverse event reports, J Am Med Inform Assoc, 19, 1111
Cited by
Still being checked.
- Authors
- Ferro, D. F., Tobias, M., Carr, L. H., Wentz, P., Rodriguez, M., Pitts, C., Kane, E., & Shelov, E.
- Affiliations
- Children's Hospital of Philadelphia (US)
- Phrase Health, Inc., Philadelphia, Pennsylvania, United States
- Digital and Technology Services, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2026-01
- Volume
- 17
- Issue
- 01
- Pages
- 046-051
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC12875732
- OpenAlex
- W7125372730
- Semantic Scholar
- 8663dcf3eccb13e307eeae164274c2b59e88ab49
- Cited by
- 0
- References
- 19
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- green
- pmc.ncbi.nlm.nih.gov
- Locations
- Applied Clinical Informatics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Keywords
- clinical decision support alerts, interruptive alerts, workflow disruption, user-centered design, organizational resilience
- MeSH terms
- Humans, Time Factors, Decision Support Systems, Clinical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
