CoChat
Sign Up
Cited by 534Meta-analysis and systematic reviewsClinical practice guidelines implementationDelphi Technique in Research

Methodology and Reports of Systematic Reviews and Meta-analyses: A Comparison of Cochrane Reviews With Articles Published in Paper-Based Journals

Jadad, Alejandro R., Cook, Deborah J., Jones, Alison, Klassen, Terry P., Tugwell, Peter, Moher, Michael, Moher, David·1998·JAMA
DOI 10.1001/jama.280.3.278
Open this paper in CoChat — free account

Abstract

CONTEXT: Review articles are important sources of information to help guide decisions by clinicians, patients, and other decision makers. Ideally, reviews should include strategies to minimize bias and to maximize precision and be reported so explicitly that any interested reader would be able to replicate them. OBJECTIVE: To compare the methodological and reporting aspects of systematic reviews and meta-analyses published by the Cochrane Collaboration with those published in paper-based journals indexed in MEDLINE. DATA SOURCES: The Cochrane Library, issue 2 of 1995, and a search of MEDLINE restricted to 1995. STUDY SELECTION: All 36 completed reviews published in the Cochrane Database of Systematic Reviews and a randomly selected sample of 39 meta-analyses or systematic reviews published in journals indexed by MEDLINE in 1995. DATA EXTRACTION: Number of authors, trials, and patients; trial sources; inclusion and exclusion criteria; language restrictions; primary outcome; trial quality assessment; heterogeneity testing; and effect estimates. Updating by 1997 was evaluated. RESULTS: Reviews found in MEDLINE included more authors (median, 3 vs 2; P<.001), more trials (median, 13.5 vs 5; P<.001), and more patients (median, 1280 vs 528; P<.001) than Cochrane reviews. More Cochrane reviews, however, included a description of the inclusion and exclusion criteria (35/36 vs 18/39; P<.001) and assessed trial quality (36/36 vs 12/39; P<.001). No Cochrane reviews had language restrictions (0/36 vs 7/39; P<.01). There were no differences in sources of trials, heterogeneity testing, or description of effect estimates. By June 1997, 18 of 36 Cochrane reviews had been updated vs 1 of 39 reviews listed in MEDLINE. CONCLUSIONS: Cochrane reviews appear to have greater methodological rigor and are more frequently updated than systematic reviews or meta-analyses published in paper-based journals.

Integrity signals

Registry

Crossref · 10.1001/jama.280.3.278

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A comparative analysis between paper-based journal meta-analyses and Cochrane reviews demonstrated significant heterogeneity in the quality of published systematic reviews, with relatively poor adherence to sound methodological principles in the majority of non-Cochrane reviews.

    “A comparative analysis between paper-based journal meta-analyses and the more rigorous Cochrane reviews demonstrated significant heterogeneity in the quality of published systematic reviews with relatively poor adherence to sound methodological principles with the majority of non-Cochrane reviews.140”

    From Challenges in Testosterone Measurement, Data Interpretation, and Methodological Appraisal of Interventional Trials (2016) · cites this paper as [140]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Jadad, A. R., Cook, D. J., Jones, A., Klassen, T. P., Tugwell, P., Moher, M., & Moher, D.
Affiliations
McMaster University (CA)
ORCID
Venue
JAMA
Publisher
American Medical Association (AMA)
Published
1998-07-15
Volume
280
Issue
3
Pages
278
ISSN
  • 0002-9955 (ISSN-L)
  • 0098-7484
DOI
10.1001/jama.280.3.278
OpenAlex
W1982130175
Cited by
534
References
6
Citations by year
2026: 5 · 2025: 3 · 2024: 9 · 2023: 8 · 2022: 10 · 2021: 8 · 2020: 15 · 2019: 22 · 2018: 9 · 2017: 8 · 2016: 12 · 2015: 24 · 2014: 25 · 2013: 23 · 2012: 16
FWCI
38.16
Type
journal article
Language
en
Open access
closed
Locations
  • JAMA (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
systematic reviews, meta-analysis, Cochrane Collaboration, review methodology, reporting quality, trial quality assessment, heterogeneity testing, review updating, methodological quality
MeSH terms
Peer Review, Periodicals as Topic, Publishing, Review Literature as Topic, Meta-Analysis as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review