Opportunities and Challenges Associated with the Pilot Implementation of Clinical Decision Support Systems in a Rural Hospital: A Qualitative Study

Abstract

Despite their potential, Clinical Decision Support (CDS) systems often lack alignment with clinicians' needs and are underutilized in practice. Pilot implementations can help to improve the fit between systems and local needs by engaging users in real-world testing and refinement. Although pilot implementations of CDS have been reported, limited evidence has explored the factors contributing to pilot success. This study aimed to explore the opportunities and challenges associated with the pilot implementation of a CDS system that ultimately did not progress to full-scale implementation. We conducted interviews with clinicians, health service managers, and vendors involved in the pilot implementation and use of a mobile application-based CDS, and a dashboard-based CDS in two departments (Emergency and Patient Flow) of a rural Australian hospital. A semistructured interview guide was developed using the Non-adoption, Abandonment, Sustainability, Scale-up, and Spread (NASSS) framework. Interviews were audio-recorded, transcribed, and thematically analyzed. Analysis revealed four major themes: system performance and design, implementation processes, organizational support and resources, and perceived benefits of the CDS. The pilot implementation allowed for greater user input into the iterative design of CDS in practice, particularly in the Emergency Department (ED), where clinicians had both the capacity and willingness to engage. However, technical issues encountered early in the pilot deterred many users who did not re-engage even after issues were resolved. Although some users remained engaged, they became frustrated as organizational resource constraints meant that critical issues impacting the CDS's clinical utility went unresolved. Successful CDS pilots depend on the readiness of organizations, departments, and users to engage in pilot activities. Pilot implementations should be pursued in settings where users have both the capacity and willingness to participate in iterative feedback processes and where organizations have sufficient resources to address emerging needs.