CoChat
Sign Up
Cited by 2,703Electronic Health Records SystemsClinical practice guidelines implementationArtificial Intelligence in Healthcare and Education

Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success

Open access
Kawamoto, Kensaku, Houlihan, Caitlin A, Balas, E Andrew, Lobach, David F·2005·BMJ
DOI 10.1136/bmj.38398.500764.8fPMC PMC555881
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: To identify features of clinical decision support systems critical for improving clinical practice. DESIGN: Systematic review of randomised controlled trials. DATA SOURCES: Literature searches via Medline, CINAHL, and the Cochrane Controlled Trials Register up to 2003; and searches of reference lists of included studies and relevant reviews. STUDY SELECTION: Studies had to evaluate the ability of decision support systems to improve clinical practice. DATA EXTRACTION: Studies were assessed for statistically and clinically significant improvement in clinical practice and for the presence of 15 decision support system features whose importance had been repeatedly suggested in the literature. RESULTS: Seventy studies were included. Decision support systems significantly improved clinical practice in 68% of trials. Univariate analyses revealed that, for five of the system features, interventions possessing the feature were significantly more likely to improve clinical practice than interventions lacking the feature. Multiple logistic regression analysis identified four features as independent predictors of improved clinical practice: automatic provision of decision support as part of clinician workflow (P < 0.00001), provision of recommendations rather than just assessments (P = 0.0187), provision of decision support at the time and location of decision making (P = 0.0263), and computer based decision support (P = 0.0294). Of 32 systems possessing all four features, 30 (94%) significantly improved clinical practice. Furthermore, direct experimental justification was found for providing periodic performance feedback, sharing recommendations with patients, and requesting documentation of reasons for not following recommendations. CONCLUSIONS: Several features were closely correlated with decision support systems' ability to improve patient care significantly. Clinicians and other stakeholders should implement clinical decision support systems that incorporate these features whenever feasible and appropriate.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmj.38398.500764.8f

Cites

69 of 69 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • When the fit between the intervention and existing workflow and systems was good, work process was improved.

    “When the fit was good [14, 73, 74], for example, when e-prescribing was sufficiently integrated as part of clinician workflow, work process was improved [14].”

    From Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews (2015) · cites this paper as [73]

    The Description of studies (page not available) reports that decision support integrated with charting or order-entry systems was more likely to improve clinical practice than stand-alone systems. It does not establish that e-prescribing integration improved the work process itself; the claim changes both the specific intervention and the outcome. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Designed tools, while promising in laboratories, often fail in practice in a variety of ways.

    “This chimes with a common observation from clinical decision support-that designed tools, while promising in laboratories, often fail in practice in a variety of ways [47].”

    From Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making (2018) · cites this paper as [47]

    The Introduction says clinical decision support systems do not always improve practice and reports that 34% of systems in a prior review did not significantly improve it. The Additional information section (p. 7) describes the systems as promising but not always effective in clinical practice. Neither passage establishes that they were promising in laboratories before failing in practice, and “often” overstates the reported failure rate. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • A variety of randomized controlled trials clearly demonstrate that CDS can improve care.

    “However, a variety of randomized controlled trials clearly demonstrate that CDS can improve care.5 6”

    From Overrides of medication-related clinical decision support alerts in outpatients (2014) · cites this paper as [6]

  • CDSS success is linked to greater integration into clinical workflow.

    “CDSS success has also been shown to be associated with higher levels of integration into the clinical workflow and advice presented at the time and location of decision making.12”

    From Computerized clinical decision support for prescribing: provision does not guarantee uptake (2010) · cites this paper as [12]

  • CDSS success is linked to advice being presented at the time and place of decision making.

    “CDSS success has also been shown to be associated with higher levels of integration into the clinical workflow and advice presented at the time and location of decision making.12”

    From Computerized clinical decision support for prescribing: provision does not guarantee uptake (2010) · cites this paper as [12]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

2 retracted

Flagged first, then by citations

Research works printed but not linked: 2

  1. Wyatt JR. Lessons learnt from the field trial of ACORN, an expert system to advise on chest pain. Proceedings of the Sixth World Conference on Medical Informatics, Singapore 1989:111-5.
  2. McPhee SJ, Detmer WM. Office-based interventions to improve delivery of cancer pre- vention services by primary care physicians. Cancer 1993;72:1100-12.

Cited by

Sorted by citations

Related

Authors
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F.
Affiliations
  • Duke University (US)
  • Old Dominion University (US)
Corresponding author
Lobach, David F
ORCID
Venue
BMJ
Publisher
BMJ
Published
2005-03-14
Volume
330
Issue
7494
Pages
765
ISSN
  • 0007-1447 (ISSN-L)
  • 0959-8138
  • 1468-5833
DOI
10.1136/bmj.38398.500764.8f
PMC
PMC555881
OpenAlex
W2115441252
Cited by
2,703
References
69
Citations by year
2026: 149 · 2025: 105 · 2024: 111 · 2023: 133 · 2022: 117 · 2021: 120 · 2020: 140 · 2019: 146 · 2018: 145 · 2017: 143 · 2016: 135 · 2015: 152 · 2014: 160 · 2013: 151 · 2012: 163
FWCI
12.82
Type
journal article
Language
en
Open access
Licence
www.bmj.org/licenses/tdm/1.0/terms-and-conditions.html (tdm)
Locations
  • BMJ (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • ODU Digital Commons (Old Dominion University) (submittedVersion)
  • HighWire Press Open Archive (submittedVersion)
Funders
  • Agency for Healthcare Research and Quality (R01-HS10472, R03-HS10814)
  • National Institute of General Medical Sciences (T32GM007171)
  • U.S. National Library of Medicine (F37LM008161, F37-LM008161-01)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
clinical decision support systems, clinical practice improvement, computer-based decision support, performance feedback, quality of care
MeSH terms
Decision Making, Professional Practice, Regression Analysis, Randomized Controlled Trials as Topic, Decision Support Systems, Clinical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review