CoChat
Sign Up
Cited by 159Meta-analysis and systematic reviewsClinical practice guidelines implementationDelphi Technique in Research

The DOI matches its Crossref record; 19 of the 20 references it cites were found in a registry or an index.

Quality of Cochrane reviews: Assessment of sample from 1998.

Open access
Olsen, Ole, Middleton, Philippa, Ezzo, Jeanette, Gøtzsche, Peter C, Hadhazy, Victoria, Herxheimer, Andrew, Kleijnen, Jos, McIntosh, Heather·2001·BMJ
DOI 10.1136/bmj.323.7317.829PMC PMC57800
Open this paper in CoChat — free accountOpen PDF

Abstract

Abstract Objective: To assess the quality of Cochrane reviews. Design: Ten methodologists affiliated with the Cochrane Collaboration independently examined, in a semistructured way, the quality of reviews first published in 1998. Each review was assessed by two people; if one of them noted any major problems, they agreed on a common assessment. Predominant types of problem were categorised. Setting: Cyberspace collaboration coordinated from the Nordic Cochrane Centre. Studies: All 53 reviews first published in issue 4 of the Cochrane Library in 1998. Main outcome measure: Proportion of reviews with various types of major problem. Results: No problems or only minor ones were found in most reviews. Major problems were identified in 15 reviews (29%). The evidence did not fully support the conclusion in nine reviews (17%), the conduct or reporting was unsatisfactory in 12 reviews (23%), and stylistic problems were identified in 12 reviews (23%). The problematic conclusions all gave too favourable a picture of the experimental intervention. Conclusions: Cochrane reviews have previously been shown to be of higher quality and less biased on average than other systematic reviews, but improvement is always possible. The Cochrane Collaboration has taken steps to improve editorial processes and the quality of its reviews. Meanwhile, the Cochrane Library remains a key source of evidence about the effects of healthcare interventions. Its users should interpret reviews cautiously, particularly those with conclusions favouring experimental interventions and those with many typographical errors. What is already known on this topic Cochrane reviews are, on average, more systematic and less biased than systematic reviews published in paper journals Errors and biases also occur in Cochrane reviews What this study adds Too often, reviewers' conclusions over-rated the benefits of new interventions Readers of Cochrane reviews should remain cautious, especially regarding conclusions that favour new interventions The Cochrane Collaboration has taken steps to improve the quality of reviews

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmj.323.7317.829

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 20 references

Research works printed but not linked: 4

  1. 1987, Proceedings of “Methodologic issues in overviews of randomized clinical trials.”, Stat Med, 6, 217
  2. Shea B Dubé C Moher D .Assessing the quality of reports of systematic reviews and meta-analyses: A systematic review of checklists and scales [abstract]. VII Cochrane Colloquium, Rome, 5–9 October 1999: A40.
  3. Clarke M Oxman AD eds.Common errors in reaching conclusions. Cochrane reviewers' handbook 4.1.3, section 9.7. In:Cochrane Library. Issue 3.Oxford: Update Software, 2001.
  4. EBH calendar of events .(accessed 17 July 2001). · DOI

Cited by

Sorted by citations

Items 101–159 of 159

Related

See all

Metadata

Authors
Olsen, O., Middleton, P., Ezzo, J., Gøtzsche, P. C., Hadhazy, V., Herxheimer, A., Kleijnen, J., & McIntosh, H.
Affiliations
  • Rigshospitalet (DK)
  • Flinders Medical Centre (AU)
  • University of Maryland, Baltimore (US)
  • Cochrane (GB)
  • University of York (GB)
Corresponding author
Olsen, Ole
ORCID
Venue
BMJ
Publisher
BMJ
Published
2001-10-13
Volume
323
Issue
7317
Pages
829-832
ISSN
  • 0007-1447 (ISSN-L)
  • 0959-8138
  • 1468-5833
DOI
10.1136/bmj.323.7317.829
PMC
PMC57800
OpenAlex
W2154762721
Cited by
159
References
20
Citations by year
2025: 2 · 2024: 2 · 2023: 2 · 2022: 3 · 2021: 4 · 2020: 2 · 2019: 4 · 2018: 1 · 2017: 6 · 2016: 2 · 2015: 6 · 2014: 10 · 2013: 13 · 2012: 6
FWCI
17.73
Type
journal article
Language
en
Open access
Licence
www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
Locations
  • BMJ (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
  • Research at the University of Copenhagen (University of Copenhagen) (submittedVersion)
  • DigitalGeorgetown (Georgetown University Library) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
review quality, systematic review methodology, reporting quality, editorial quality, Cochrane Library, health interventions
MeSH terms
Humans, Review Literature as Topic, Meta-Analysis as Topic, Bias, Evidence-Based Medicine, Databases as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review