The impact of Cochrane Reviews: a mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Background: The last few decades have seen a growing emphasis on evidence-informed decision-making in health care. Systematic reviews, such as those produced by Cochrane, have been a key component of this movement. The National Institute for Health Research (NIHR) Systematic Review Programme currently supports 20 Cochrane Review Groups (CRGs) in the UK and it is important that this funding represents value for money. Aims and objectives: The overall aim was to identify the impacts and likely impacts on health care, patient outcomes and value for money of Cochrane Reviews published by 20 NIHR-funded CRGs during the years 2007–11. Design: We sent questionnaires to CRGs and review authors, undertook interviews with guideline developers (GDs) and used bibliometrics and documentary review to get an overview of CRG impact and to evaluate the impact of a sample of 60 Cochrane Reviews. The evaluation was guided by a framework with four categories (knowledge production, research targeting, informing policy development and impact on practice/services). Results: A total of 3187 new and updated reviews were published on the Cochrane Database of Systematic Reviews between 2007 and 2011, 1502 (47%) of which were produced by the 20 CRGs funded by the NIHR. We found 40 examples where reviews appeared to have influenced primary research and reviews had contributed to the creation of new knowledge and stimulated debate. Twenty-seven of the 60 reviews had 100 or more citations in Google Scholar™ (Google, CA, USA). Overall, 483 systematic reviews had been cited in 247 sets of guidance. This included 62 sets of international guidance, 175 sets of national guidance (87 from the UK) and 10 examples of local guidance. Evidence from the interviews suggested that Cochrane Reviews often play an instrumental role in informing guidance, although reviews being a poor fit with guideline scope or methods, reviews being out of date and a lack of communication between CRGs and GDs were barriers to their use. Cochrane Reviews appeared to have led to a number of benefits to the health service including safer or more appropriate use of medication or other health technologies or the identification of new effective drugs or treatments. However, whether or not these changes were directly as a result of the Cochrane Review and not the result of subsequent clinical guidance was difficult to judge. Potential benefits of Cochrane Reviews included economic benefits through budget savings or the release of funds, improvements in clinical quality, the reduction in the use of unproven or unnecessary procedures and improvements in patient and carer experiences. Conclusions: This study identified a number of impacts and likely impacts of Cochrane Reviews. The clearest impacts of Cochrane Reviews are on research targeting and health-care policy, with less evidence of a direct impact on clinical practice and the organisation and delivery of NHS services. Although it is important for researchers to consider how they might increase the influence of their work, such impacts are difficult to measure. More work is required to develop suitable methods for defining and quantifying the impact of research. Funding: The NIHR Health Technology Assessment programme.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses
Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008) · The FASEB Journal · doi:10.1096/fj.07-9492lsf · Cited by 5,039
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044
Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
Barbour, R. S. (2001) · BMJ · doi:10.1136/bmj.322.7294.1115 · Cited by 2,613
Interventions for preventing obesity in children
Waters, E., de Silva-Sanigorski, A., Burford, B. J., Brown, T., Campbell, K. J., Gao, Y., Armstrong, R., Prosser, L., & Summerbell, C. D. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001871.pub3 · Cited by 1,455
Three options for citation tracking: Google Scholar, Scopus and Web of Science
Bakkalbasi, N., Bauer, K., Glover, J., & Wang, L. (2006) · Biomedical Digital Libraries · doi:10.1186/1742-5581-3-7 · Cited by 752
Evidence-based policy: the promise of ‘realist synthesis’
Pawson, R. (2002) · Evaluation · doi:10.1177/135638902401462448 · Cited by 729
Structured telephone support or telemonitoring programmes for patients with chronic heart failure
Inglis, S. C., Clark, R. A., McAlister, F. A., Ball, J., Lewinter, C., Cullington, D., Stewart, S., & Cleland, J. G. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007228.pub2 · Cited by 614
Have We Learned Anything New About the Use of Evaluation?
Weiss, C. H. (1998) · American Journal of Evaluation · doi:10.1177/109821409801900103 · Cited by 384
Google Scholar: the pros and the cons
Jacsó, P. (2005) · Online Information Review · doi:10.1108/14684520510598066 · Cited by 383
The rise and rise of citation analysis
Meho, L. I. (2007) · Physics World · doi:10.1088/2058-7058/20/1/33 · Cited by 371
Cited by
Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study
Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.008 · Cited by 1,082
“Systematic reviews are considered among the best available sources of research evidence and are increasingly relied on to inform decision-making [1].”
BackgroundHong, Q. N., Pluye, P., Bujold, M., & Wassef, M. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0454-2 · Cited by 754
“researchers, and health service providers to inform decision-making [1].”
BackgroundDechartres, A., Trinquart, L., Atal, I., Moher, D., Dickersin, K., Boutron, I., Perrodeau, E., Altman, D. G., & Ravaud, P. (2017) · BMJ · doi:10.1136/bmj.j2490 · Cited by 171
A Conceptual Framework for Critical Appraisal in Systematic Mixed Studies Reviews
Hong, Q. N., & Pluye, P. (2019) · Journal of Mixed Methods Research · doi:10.1177/1558689818770058 · Cited by 139
“Systematic reviews are considered among the best sources of research evidence, are used for decision making, and are helpful for coping with the rapidly increasing volume of scientific literature (Bunn et al., 2015; Moynihan, 2004).”
Background
Metadata
- Authors
- Bunn, F., Trivedi, D., Alderson, P., Hamilton, L., Martin, A., Pinkney, E., & Iliffe, S.
- Venue
- Health Technology Assessment
- Published
- 2015-04
- Volume
- 19
- Issue
- 28
- Pages
- 1-100
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W3020979115
- Cited by
- 42
- References
- 14
- FWCI
- 2.39
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The impact of Cochrane Reviews: a mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research. https://app.cochat.ai/papers/10.3310/hta19280
- The paper
- Bunn, F., Trivedi, D., Alderson, P., Hamilton, L., Martin, A., Pinkney, E., & Iliffe, S. (2015). The impact of Cochrane Reviews: a mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research. Health Technology Assessment. https://doi.org/10.3310/hta19280
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.