CoChat
Sign Up
Cited by 131Electronic Health Records SystemsHealth Policy Implementation ScienceArtificial Intelligence in Healthcare and Education

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

Open access
Abell, Bridget, Naicker, Sundresan, Rodwell, David, Donovan, Thomasina, Tariq, Amina, Baysari, Melissa, Blythe, Robin, Parsons, Rex +1 more·2023·Implementation Science
DOI 10.1186/s13012-023-01287-yPMC PMC10373265
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim. Methods: Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework. Results: Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user’s role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the “Technology,” “Organization,” and “Adopters” domains most frequently reported. No determinants were assigned to the “Embedding and Adaptation Over Time” domain. Conclusions: This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13012-023-01287-y

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Increased workload and misalignment with clinical workflows are among the most common barriers to CDS adoption in clinical settings.

    “Increased workload and misalignment with clinical workflows are among the most common barriers to CDS adoption in clinical settings.12,13”

    From Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling (2025) · cites this paper as [12]

    The Results identify fit with workflows as the most commonly reported determinant of hospital CDSS implementation (19 of 44 studies). They also report that perceived time and effort and loss of productivity can deter use, but do not establish increased workload as one of the most common barriers. The review concerns hospital settings, rather than clinical settings generally; page numbers were not supplied. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Increased workload (either perceived or experienced) associated with CDS has been identified as a major barrier to CDS adoption.

    “In addition, increased workload (either perceived or experienced) associated with CDS has been identified as a major barrier to CDS adoption.12,13”

    From Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling (2025) · cites this paper as [12]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 62

  1. [2]Dissanayake PI Colicchio TK Cimino JJ Using clinical reasoning ontologies to make smarter clinical decision support systems: a systematic review and data synthesis J Am Med Inform Assoc 2020 27 1 159 174 10.1093/jamia/ocz169 31592534 PMC6913230 · DOI
  2. [3]Hak F Guimarães T Santos M Towards effective clinical decision support systems: a systematic review PLoS One 2022 17 8 e0272846 10.1371/journal.pone.0272846 35969526 PMC9377614 · DOI
  3. [4]World Health Organization (WHO) WHO guideline: recommendations on digital interventions for health system strengthening 2019 Geneva WHO 31162915
  4. [5]Laka M Milazzo A Merlin T Why provision of clinical decision support (CDS) is not enough? Factors influencing the CDS adoption Eur J Public Health 2020 30 Supplement_5 ckaa165.223 10.1093/eurpub/ckaa165.223 · DOI
  5. [6]Kouri A Yamada J Lam Shin Cheung J Van de Velde S Gupta S Do providers use computerized clinical decision support systems? A systematic review and meta-regression of clinical decision support uptake Implement Sci 2022 17 1 21 10.1186/s13012-022-01199-3 35272667 PMC8908582 · DOI
  6. [8]Petkus H Hoogewerf J Wyatt JC What do senior physicians think about AI and clinical decision support systems: quantitative and qualitative analysis of data from specialty societies Clin Med (Lond) 2020 20 3 324 328 10.7861/clinmed.2019-0317 32414724 PMC7354034 · DOI
  7. [9]Stone EG Unintended adverse consequences of a clinical decision support system: two cases J Am Med Inform Assoc 2018 25 5 564 567 10.1093/jamia/ocx096 29036296 PMC7646869 · DOI
  8. [10]Cresswell KM Lee L Mozaffar H Williams R Sheikh A Robertson A Sustained user engagement in health information technology: the long road from implementation to system optimization of computerized physician order entry and clinical decision support systems for prescribing in hospitals in England Health Serv Res 2017 52 5 1928 1957 10.1111/1475-6773.12581 27714800 PMC5583302 · DOI · Paper page
  9. [11]Perski O Short CE Acceptability of digital health interventions: embracing the complexity Transl Behav Med 2021 11 7 1473 1480 10.1093/tbm/ibab048 33963864 PMC8320880 · DOI
  10. [12]Liberati EG Ruggiero F Galuppo L Gorli M González-Lorenzo M Maraldi M What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation Implement Sci 2017 12 1 113 10.1186/s13012-017-0644-2 28915822 PMC5602839 · DOI · Paper page
  11. [13]Donovan T Abell B Fernando M McPhail SM Carter HE Implementation costs of hospital-based computerised decision support systems: a systematic review Implement Sci 2023 18 1 7 10.1186/s13012-023-01261-8 36829247 PMC9960445 · DOI
  12. [15]Bauer MS Kirchner J Implementation science: what is it and why should I care? Psychiatry Res 2020 283 112376 10.1016/j.psychres.2019.04.025 31036287 · DOI · Paper page
  13. [16]Peters DH Adam T Alonge O Agyepong IA Tran N Republished research: implementation research: what it is and how to do it Br J Sports Med 2014 48 8 731 10.1136/bmj.f6753 24659611 · DOI · Paper page
  14. [18]Greenhalgh T Wherton J Papoutsi C Lynch J Hughes G A’Court C Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework BMC Med 2018 16 1 66 10.1186/s12916-018-1050-6 29754584 PMC5950199 · DOI · Paper page
  15. [20]KadesjöBanck J Bernhardsson S Experiences from implementation of internet-delivered cognitive behaviour therapy for insomnia in psychiatric health care: a qualitative study applying the NASSS framework BMC Health Serv Res 2020 20 1 729 10.1186/s12913-020-05596-6 32771024 PMC7414663 · DOI · Paper page
  16. [23]Khong PCB Holroyd E Wang W A critical review of the theoretical frameworks and the conceptual factors in the adoption of clinical decision support systems Comput Inform Nurs 2015 33 12 555 70 10.1097/CIN.0000000000000196 26535769 · DOI
  17. [24]Khanijahani A Iezadi S Dudley S Goettler M Kroetsch P Wise J Organizational, professional, and patient characteristics associated with artificial intelligence adoption in healthcare: a systematic review Health Policy Technol 2022 11 1 100602 10.1016/j.hlpt.2022.100602 · DOI
  18. [27]Levac D Colquhoun H O’Brien KK Scoping studies: advancing the methodology Implement Sci 2010 5 1 69 10.1186/1748-5908-5-69 20854677 PMC2954944 · DOI · Paper page
  19. [29]Scells H, Zuccon G. searchrefiner: a query visualisation and understanding tool for systematic reviews. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management. Torino: Association for Computing Machinery; 2018. p. 1939–42.
  20. [30]Clark JM Sanders S Carter M Honeyman D Cleo G Auld Y Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial J Med Libr Assoc 2020 108 2 195 207 10.5195/jmla.2020.834 32256231 PMC7069833 · DOI
  21. [33]Sim I Gorman P Greenes RA Haynes RB Kaplan B Lehmann H Clinical decision support systems for the practice of evidence-based medicine J Am Med Inform Assoc 2001 8 6 527 534 10.1136/jamia.2001.0080527 11687560 PMC130063 · DOI · Paper page
  22. [34]Pollock D Peters MDJ Khalil H McInerney P Alexander L Tricco AC Recommendations for the extraction, analysis, and presentation of results in scoping reviews JBI Evid Synth. 2022 21 3 520 32 10.11124/JBIES-22-00123 36081365 · DOI · Paper page
  23. [35]Blanco N O’Hara LM Robinson GL Brown J Heil E Brown CH Health care worker perceptions toward computerized clinical decision support tools for Clostridium difficile infection reduction: a qualitative study at 2 hospitals Am J Infect Control 2018 46 10 1160 1166 10.1016/j.ajic.2018.04.204 29803592 · DOI
  24. [36]Flynn D Nesbitt DJ Ford GA McMeekin P Rodgers H Price C Development of a computerised decision aid for thrombolysis in acute stroke care BMC Med Inform Decis Mak 2015 15 1 1 15 10.1186/s12911-014-0127-1 25889696 PMC4326413 · DOI
  25. [37]Hasnie AA Kumbamu A Safarova MS Caraballo PJ Kullo IJ A clinical decision support tool for familial hypercholesterolemia based on physician input Mayo Clin Proc Innov Qual Outcomes 2018 2 2 103 112 10.1016/j.mayocpiqo.2018.03.006 30225440 PMC6124345 · DOI
  26. [38]Melnick ER Holland WC Ahmed OM Ma AK Michael SS Goldberg HS An integrated web application for decision support and automation of EHR workflow: a case study of current challenges to standards-based messaging and scalability from the EMBED trial JAMIA Open 2019 2 4 434 439 10.1093/jamiaopen/ooz053 32025639 PMC6994013 · DOI
  27. [39]Mugabe KV Barriers and facilitators to the adoption of artificial intelligence in radiation oncology: a New Zealand study Tech Innov Patient Support Radiat Oncol 2021 18 16 21 10.1016/j.tipsro.2021.03.004 33981867 PMC8085695 · DOI · Paper page
  28. [40]Ploegmakers KJ Medlock S Linn AJ Lin Y Seppälä LJ Petrovic M Barriers and facilitators in using a Clinical Decision Support System for fall risk management for older people: a European survey Eur Geriatr Med. 2022 13 2 395 405 10.1007/s41999-021-00599-w 35032323 · DOI
  29. [41]Westafer LM Kunz A Bugajska P Hughes A Mazor KM Schoenfeld EM Provider perspectives on the use of evidence-based risk stratification tools in the evaluation of pulmonary embolism: a qualitative study Acad Emerg Med 2020 27 6 447 456 10.1111/acem.13908 32220127 PMC7418048 · DOI
  30. [42]Yadav K Chamberlain JM Lewis VR Abts N Chawla S Hernandez A Designing real-time decision support for trauma resuscitations Acad Emerg Med 2015 22 9 1076 1084 10.1111/acem.12747 26300010 PMC5338692 · DOI
  31. [43]Ballard DW Vemula R Chettipally UK Kene MV Mark DG Elms AK Optimizing clinical decision support in the electronic health record: clinical characteristics associated with the use of a decision tool for disposition of ED patients with pulmonary embolism Appl Clin Inform 2016 7 3 883 98 10.4338/ACI-2016-05-RA-0073 27652375 PMC5052556 · DOI
  32. [44]Bersani K Fuller TE Garabedian P Espares J Mlaver E Businger A Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR Appl Clin Inform 2020 11 1 034 45 10.1055/s-0039-3402756 PMC6962088 31940670 · DOI
  33. [45]Bowen SP Johnson KMPA Reed MHF Zhang L Curry LP The effect of incorporating guidelines into a computerized order entry system for diagnostic imaging J Am Coll Radiol 2011 8 4 251 258 10.1016/j.jacr.2010.11.020 21458763 · DOI
  34. [46]Campion TR Waitman LR Lorenzi NM May AK Gadd CS Barriers and facilitators to the use of computer-based intensive insulin therapy Int J Med Inform (Shannon, Ireland) 2011 80 12 863 871 10.1016/j.ijmedinf.2011.10.003 PMC3226863 22019280 · DOI
  35. [47]Chow A Lye DCB Arah OA Psychosocial determinants of physicians’ acceptance of recommendations by antibiotic computerised decision support systems: a mixed methods study Int J Antimicrob Agents 2014 45 3 295 304 10.1016/j.ijantimicag.2014.10.009 25434998 · DOI
  36. [48]Chua AQ Tang SSL Lee LW Yii DYC Kong ST Lee W Psychosocial determinants of physician acceptance toward an antimicrobial stewardship program and its computerized decision support system in an acute care tertiary hospital J Am Coll Clin Pharm 2018 1 1 e1 e8 10.1002/jac5.1028 · DOI
  37. [49]Chung P Scandlyn J Dayan PS Mistry RD Working at the intersection of context, culture, and technology: provider perspectives on antimicrobial stewardship in the emergency department using electronic health record clinical decision support Am J Infect Control 2017 45 11 1198 1202 10.1016/j.ajic.2017.06.005 28711345 · DOI
  38. [50]Collins I Breathnach O Felle P Electronic clinical decision support systems attitudes and barriers to use in the oncology setting Ir J Med Sci 2012 181 4 521 525 10.1007/s11845-012-0809-6 22382578 · DOI
  39. [51]de Vries AE van der Wal MHL Nieuwenhuis MMW de Jong RM van Dijk RB Jaarsma T Perceived barriers of heart failure nurses and cardiologists in using clinical decision support systems in the treatment of heart failure patients BMC Med Inform Decis Mak 2013 13 1 54 10.1186/1472-6947-13-54 23622342 PMC3651365 · DOI
  40. [52]English D Ankem K English K Acceptance of clinical decision support surveillance technology in the clinical pharmacy Inform Health Soc Care 2017 42 2 135 152 10.3109/17538157.2015.1113415 26890621 · DOI
  41. [53]Giuliano CA Binienda J Kale-Pradhan PB Fakih MG “I never would have caught that before”: pharmacist perceptions of using clinical decision support for antimicrobial stewardship in the United States Qual Health Res 2018 28 5 745 755 10.1177/1049732317750863 29334865 · DOI
  42. [54]Glassman PA Simon B Belperio P Lanto A Improving recognition of drug interactions: benefits and barriers to using automated drug alerts Med Care 2002 40 12 1161 1171 10.1097/00005650-200212000-00004 12458299 · DOI
  43. [55]Goud R van Engen-Verheul M de Keizer NF Bal R Hasman A Hellemans IM The effect of computerized decision support on barriers to guideline implementation: a qualitative study in outpatient cardiac rehabilitation Int J Med Inform (Shannon, Ireland) 2010 79 6 430 437 10.1016/j.ijmedinf.2010.03.001 20378396 · DOI
  44. [56]Grau LE Weiss J O’Leary TK Camenga D Bernstein SL Electronic decision support for treatment of hospitalized smokers: a qualitative analysis of physicians’ knowledge, attitudes, and practices Drug Alcohol Depend 2019 194 296 301 10.1016/j.drugalcdep.2018.10.006 30469101 PMC7720717 · DOI
  45. [57]Green TA Whitt S Belden JL Erdelez S Shyu C-R Medical calculators: prevalence, and barriers to use Comput Methods Programs Biomed 2019 179 105002 10.1016/j.cmpb.2019.105002 31443857 · DOI
  46. [58]Gutenstein M Pickering JW Than M Development of a digital clinical pathway for emergency medicine: lessons from usability testing and implementation failure Health Informatics J 2019 25 4 1563 1571 10.1177/1460458218779099 29905094 · DOI
  47. [59]Jacobs J Weir C Evans RS Staes C Assessment of readiness for clinical decision support to aid laboratory monitoring of immunosuppressive care at U.S. liver transplant centers Appl Clin Inform 2014 5 4 988 1004 10.4338/ACI-2014-08-RA-0060 25589912 PMC4287676 · DOI
  48. [60]Johnson R Evans M Cramer H Bennert K Morris R Eldridge S Feasibility and impact of a computerised clinical decision support system on investigation and initial management of new onset chest pain: a mixed methods study BMC Med Inform Decis Mak 2015 15 1 71 10.1186/s12911-015-0189-8 26307007 PMC4550063 · DOI
  49. [61]Lai F Macmillan J Daudelin DH Kent DM The potential of training to increase acceptance and use of computerized decision support systems for medical diagnosis Hum Factors 2006 48 1 95 108 10.1518/001872006776412306 16696260 · DOI
  50. [62]Lesselroth BJ Yang J McConnachie J Brenk T Winterbottom L Addressing the sociotechnical drivers of quality improvement: a case study of post-operative DVT prophylaxis computerised decision support BMJ Qual Saf 2011 20 5 381 389 10.1136/bmjqs.2010.042689 21209144 PMC3088464 · DOI · Paper page
  51. [63]Masterson Creber RM Dayan PS Kuppermann N Ballard DW Tzimenatos L Alessandrini E Applying the RE-AIM Framework for the evaluation of a clinical decision support tool for pediatric head trauma: a mixed-methods study Appl Clin Inform 2018 9 3 693 703 10.1055/s-0038-1669460 30184559 PMC6125135 · DOI
  52. [64]Miller MK Mollen C Behr K Dowd MD Miller E Satterwhite CL Development of a novel computerized clinical decision support system to improve adolescent sexual health care provision Acad Emerg Med 2019 26 4 420 433 10.1111/acem.13570 30240032 PMC6625349 · DOI
  53. [66]Salwei ME Carayon P Hoonakker PL Hundt AS Wiegmann D Pulia M Workflow integration analysis of a human factors-based clinical decision support in the emergency department Appl Ergon 2021 97 103498 10.1016/j.apergo.2021.103498 34182430 PMC8474147 · DOI
  54. [67]Santucci W Day RO Baysari MT Evaluation of hospital-wide computerised decision support in an intensive care unit: an observational study Anaesth Intensive Care 2016 44 4 507 512 10.1177/0310057X1604400403 27456183 · DOI
  55. [68]Sheehan B Nigrovic LE Dayan PS Kuppermann N Ballard DW Alessandrini E Informing the design of clinical decision support services for evaluation of children with minor blunt head trauma in the emergency department: a sociotechnical analysis J Biomed Inform 2013 46 5 905 913 10.1016/j.jbi.2013.07.005 23892207 · DOI
  56. [70]van der Stap L, de Heij AH, van der Heide A, Reyners AK, van der Linden YM. Clinical decision support system to optimise symptom management in palliative medicine: focus group study. BMJ Support Palliat Care. 2021. 10.1136/bmjspcare-2021-002940. 34272271 · DOI
  57. [71]Vandenberg AE Vaughan CP Stevens M Hastings SN Powers J Markland A Improving geriatric prescribing in the ED: a qualitative study of facilitators and barriers to clinical decision support tool use Int J Qual Health Care 2017 29 1 117 123 27852639
  58. [72]Weber S Crago EA Sherwood PR Smith T Practitioner approaches to the integration of clinical decision support system technology in critical care J Nurs Adm 2009 39 11 465 469 10.1097/NNA.0b013e3181bd5fc2 19898097 · DOI
  59. [73]Yılmaz AA Ozdemir L Development and Implementation of the clinical decision support system for patients with cancer and nurses’ experiences regarding the system Int J Nurs Knowl 2017 28 1 4 12 10.1111/2047-3095.12099 26011435 · DOI
  60. [74]Zaidi STR Marriott JL Barriers and facilitators to adoption of a web-based antibiotic decision support system South Med Rev 2012 5 2 42 49 23532767 PMC3606938
  61. [75]Zaidi STR Thursky KA Using formative evaluation to improve uptake of a web-based tool to support antimicrobial stewardship J Clin Pharm Ther 2013 38 6 490 497 10.1111/jcpt.12093 23992301 · DOI
  62. [77]Ronan CE Crable EL Drainoni ML Walkey AJ The impact of clinical decision support systems on provider behavior in the inpatient setting: a systematic review and meta-analysis J Hosp Med 2022 17 5 368 383 10.1002/jhm.12825 35514024 · DOI

Cited by

Sorted by citations

Authors
Abell, B., Naicker, S., Rodwell, D., Donovan, T., Tariq, A., Baysari, M., Blythe, R., Parsons, R., & McPhail, S. M.
Affiliations
  • Queensland University of Technology (AU)
  • The University of Sydney (AU)
Corresponding author
Naicker, Sundresan
ORCID
Venue
Implementation Science
Publisher
Springer Science and Business Media LLC
Published
2023-07-26
Volume
18
Issue
1
Pages
32-32
ISSN
1748-5908 (ISSN-L)
DOI
10.1186/s13012-023-01287-y
PMC
PMC10373265
OpenAlex
W4385281466
Cited by
131
References
15
Citations by year
2026: 52 · 2025: 49 · 2024: 26 · 2023: 4
FWCI
5.13
Type
journal article
Language
en
Open access
Licence
Locations
  • Implementation Science (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Digital Health Cooperative Research Centre (DHCRC-0058)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
computerized clinical decision support systems, implementation barriers, implementation facilitators, NASSS framework, hospital implementation, clinician attitudes, technology adoption, implementation theory
MeSH terms
Health Personnel, Hospitals, Humans, Technology, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review