CoChat
Sign Up
Cited by 156Electronic Health Records SystemsArtificial Intelligence in Healthcare and EducationClinical practice guidelines implementation

A systematic review of trials evaluating success factors of interventions with computerised clinical decision support

Open access
Van de Velde, Stijn, Heselmans, Annemie, Delvaux, Nicolas, Brandt, Linn, Marco-Ruiz, Luis, Spitaels, David, Cloetens, Hanne, Kortteisto, Tiina +5 more·2018·Implementation Science
DOI 10.1186/s13012-018-0790-1PMC PMC6102833
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Computerised clinical decision support (CDS) can potentially better inform decisions, and it can help with the management of information overload. It is perceived to be a key component of a learning health care system. Despite its increasing implementation worldwide, it remains uncertain why the effect of CDS varies and which factors make CDS more effective. OBJECTIVE: To examine which factors make CDS strategies more effective on a number of outcomes, including adherence to recommended practice, patient outcome measures, economic measures, provider or patient satisfaction, and medical decision quality. METHODS: We identified randomised controlled trials, non-randomised trials, and controlled before-and-after studies that directly compared CDS implementation with a given factor to CDS without that factor by searching CENTRAL, MEDLINE, EMBASE, and CINAHL and checking reference lists of relevant studies. We considered CDS with any objective for any condition in any healthcare setting. We included CDS interventions that were either displayed on screen or provided on paper and that were directed at healthcare professionals or targeted at both professionals and patients. The reviewers screened the potentially relevant studies in duplicate. They extracted data and assessed risk of bias in independent pairs or individually followed by a double check by another reviewer. We summarised results using medians and interquartile ranges and rated our certainty in the evidence using the GRADE system. RESULTS: We identified 66 head-to-head trials that we synthesised across 14 comparisons of CDS intervention factors. Providing CDS automatically versus on demand led to large improvements in adherence. Displaying CDS on-screen versus on paper led to moderate improvements and making CDS more versus less patient-specific improved adherence modestly. When CDS interventions were combined with professional-oriented strategies, combined with patient-oriented strategies, or combined with staff-oriented strategies, then adherence improved slightly. Providing CDS to patients slightly increased adherence versus CDS aimed at the healthcare provider only. Making CDS advice more explicit and requiring users to respond to the advice made little or no difference. The CDS intervention factors made little or no difference to patient outcomes. The results for economic outcomes and satisfaction outcomes were sparse. CONCLUSION: Multiple factors may affect the success of CDS interventions. CDS may be more effective when the advice is provided automatically and displayed on-screen and when the suggestions are more patient-specific. CDS interventions combined with other strategies probably also improves adherence. Providing CDS directly to patients may also positively affect adherence. The certainty of the evidence was low to moderate for all factors. TRIAL REGISTRATION: PROSPERO, CRD42016033738.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13012-018-0790-1

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The use of black box models in health care settings presents unique challenges and opportunities related to trust and transparency.

    “Although previous studies have studied factors affecting the implementation of CDS tools [48], the use of black box models in health care settings presents unique challenges and opportunities related to trust and transparency.”

    From Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study (2020) · cites this paper as [48]

  • The WPA in common with WHO advocates for a pragmatic approach to implementing viable alternatives to coercion, one of which is adopting a community-based collaborative care approach that includes shared or supported decision-making.

    “The WPA in common with WHO advocates for a pragmatic approach to implementing viable alternatives to coercion, 601 one of which is adopting a community-based collaborative care approach that includes shared or supported decisionmaking.”

    From Time for united action on depression: a Lancet–World Psychiatric Association Commission (2022) · cites this paper as [601]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [4]Fretheim A, Flottorp S, Oxman A. Effect of interventions for implementing clinical practice guidelines. Oslo: Knowledge Centre for the Health Services at The Norwegian Institute of Publich Health (NIPH); 2015. Report from Norwegian Knowledge Centre for the Health Services (NOKC) No. 10-2015. Available from: https://www.ncbi.nlm.nih.gov/books/NBK390571/. 28510383
  2. [19]EPOC resources for review authors. http://epoc.cochrane.org/resources/epoc-resources-review-authors. Accessed Jan 2016.

Cited by

Sorted by citations

Related

Authors
Van de Velde, S., Heselmans, A., Delvaux, N., Brandt, L., Marco-Ruiz, L., Spitaels, D., Cloetens, H., Kortteisto, T., Roshanov, P., Kunnamo, I., Aertgeerts, B., Vandvik, P. O., & Flottorp, S.
Affiliations
  • KU Leuven (BE)
  • University of Oslo (NO)
  • Tampere University Hospital (FI)
  • McMaster University (CA)
  • Finnish Medical Society Duodecim (FI)
  • Norwegian Institute of Public Health (NO)
ORCID
Venue
Implementation Science
Publisher
Springer Science and Business Media LLC
Published
2018-08-20
Volume
13
Issue
1
Pages
114-114
ISSN
1748-5908 (ISSN-L)
DOI
10.1186/s13012-018-0790-1
PMC
PMC6102833
OpenAlex
W2888668484
Cited by
156
References
40
Citations by year
2026: 11 · 2025: 14 · 2024: 14 · 2023: 18 · 2022: 26 · 2021: 20 · 2020: 35 · 2019: 15 · 2018: 1
FWCI
3.28
Type
journal article
Language
en
Open access
Licence
Locations
  • Implementation Science (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • BIBSYS Brage (BIBSYS (Norway)) (acceptedVersion, cc-by-nc-sa)
  • Lirias (KU Leuven) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • NORA - Norwegian Open Research Archives (submittedVersion, other-oa)
  • NORA - Norwegian Open Research Archives (submittedVersion, cc-by-nc-sa)
  • NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
Funders
Horizon 2020 (654981)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
computerized clinical decision support, success factors, patient satisfaction, economic outcomes, learning healthcare systems
MeSH terms
Decision Support Techniques, Humans, Quality of Life, Randomized Controlled Trials as Topic, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review