CoChat

The impact of Cochrane Reviews: a mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research

Open access
Bunn, Frances, Trivedi, Daksha, Alderson, Phil, Hamilton, Laura, Martin, Alice, Pinkney, Emma, Iliffe, Steve · 2015 · Health Technology Assessment
Cited by 42

The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.

Abstract

Background: The last few decades have seen a growing emphasis on evidence-informed decision-making in health care. Systematic reviews, such as those produced by Cochrane, have been a key component of this movement. The National Institute for Health Research (NIHR) Systematic Review Programme currently supports 20 Cochrane Review Groups (CRGs) in the UK and it is important that this funding represents value for money. Aims and objectives: The overall aim was to identify the impacts and likely impacts on health care, patient outcomes and value for money of Cochrane Reviews published by 20 NIHR-funded CRGs during the years 2007–11. Design: We sent questionnaires to CRGs and review authors, undertook interviews with guideline developers (GDs) and used bibliometrics and documentary review to get an overview of CRG impact and to evaluate the impact of a sample of 60 Cochrane Reviews. The evaluation was guided by a framework with four categories (knowledge production, research targeting, informing policy development and impact on practice/services). Results: A total of 3187 new and updated reviews were published on the Cochrane Database of Systematic Reviews between 2007 and 2011, 1502 (47%) of which were produced by the 20 CRGs funded by the NIHR. We found 40 examples where reviews appeared to have influenced primary research and reviews had contributed to the creation of new knowledge and stimulated debate. Twenty-seven of the 60 reviews had 100 or more citations in Google Scholar™ (Google, CA, USA). Overall, 483 systematic reviews had been cited in 247 sets of guidance. This included 62 sets of international guidance, 175 sets of national guidance (87 from the UK) and 10 examples of local guidance. Evidence from the interviews suggested that Cochrane Reviews often play an instrumental role in informing guidance, although reviews being a poor fit with guideline scope or methods, reviews being out of date and a lack of communication between CRGs and GDs were barriers to their use. Cochrane Reviews appeared to have led to a number of benefits to the health service including safer or more appropriate use of medication or other health technologies or the identification of new effective drugs or treatments. However, whether or not these changes were directly as a result of the Cochrane Review and not the result of subsequent clinical guidance was difficult to judge. Potential benefits of Cochrane Reviews included economic benefits through budget savings or the release of funds, improvements in clinical quality, the reduction in the use of unproven or unnecessary procedures and improvements in patient and carer experiences. Conclusions: This study identified a number of impacts and likely impacts of Cochrane Reviews. The clearest impacts of Cochrane Reviews are on research targeting and health-care policy, with less evidence of a direct impact on clinical practice and the organisation and delivery of NHS services. Although it is important for researchers to consider how they might increase the influence of their work, such impacts are difficult to measure. More work is required to develop suitable methods for defining and quantifying the impact of research. Funding: The NIHR Health Technology Assessment programme.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3310/hta19280

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 42

Metadata

Authors
Bunn, F., Trivedi, D., Alderson, P., Hamilton, L., Martin, A., Pinkney, E., & Iliffe, S.
Venue
Health Technology Assessment
Published
2015-04
Volume
19
Issue
28
Pages
1-100
ISSN
  • 1366-5278 (ISSN-L)
  • 2046-4924
OpenAlex
W3020979115
Cited by
42
References
14
FWCI
2.39
Type
journal article
Language
en
Open access
diamond

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.