The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
Epidemiology and reporting characteristics of systematic reviews.
Open accessAbstract
BACKGROUND: Systematic reviews (SRs) have become increasingly popular to a wide range of stakeholders. We set out to capture a representative cross-sectional sample of published SRs and examine them in terms of a broad range of epidemiological, descriptive, and reporting characteristics, including emerging aspects not previously examined. METHODS AND FINDINGS: We searched Medline for SRs indexed during November 2004 and written in English. Citations were screened and those meeting our inclusion criteria were retained. Data were collected using a 51-item data collection form designed to assess the epidemiological and reporting details and the bias-related aspects of the reviews. The data were analyzed descriptively. In total 300 SRs were identified, suggesting a current annual publication rate of about 2,500, involving more than 33,700 separate studies including one-third of a million participants. The majority (272 [90.7%]) of SRs were reported in specialty journals. Most reviews (213 [71.0%]) were categorized as therapeutic, and included a median of 16 studies involving 1,112 participants. Funding sources were not reported in more than one-third (122 [40.7%]) of the reviews. Reviews typically searched a median of three electronic databases and two other sources, although only about two-thirds (208 [69.3%]) of them reported the years searched. Most (197/295 [66.8%]) reviews reported information about quality assessment, while few (68/294 [23.1%]) reported assessing for publication bias. A little over half (161/300 [53.7%]) of the SRs reported combining their results statistically, of which most (147/161 [91.3%]) assessed for consistency across studies. Few (53 [17.7%]) SRs reported being updates of previously completed reviews. No review had a registration number. Only half (150 [50.0%]) of the reviews used the term "systematic review" or "meta-analysis" in the title or abstract. There were large differences between Cochrane reviews and non-Cochrane reviews in the quality of reporting several characteristics. CONCLUSIONS: SRs are now produced in large numbers, and our data suggest that the quality of their reporting is inconsistent. This situation might be improved if more widely agreed upon evidence-based reporting guidelines were endorsed and adhered to by authors and journals. These results substantiate the view that readers should not accept SRs uncritically.
Overview
CoChat Summary
TL;DR
This cross-sectional study sampled 300 systematic reviews (SRs) indexed in Medline in November 2004 and found that SRs were being published in large numbers, but reporting quality was inconsistent. Cochrane reviews generally reported methods more completely than non-Cochrane reviews, and the authors argue that stronger reporting guidelines are needed.
Key findings
- The search identified 1,046 records; 300 eligible SR reports were included after screening, with excellent inter-reviewer agreement across screening phases (kappa = 0.86).
- The 300 SRs were indexed in Medline in November 2004 and corresponded to an estimated annual publication rate of about 2,500 SRs; Cochrane reviews made up 125/300 (41.7%) of the sample.
- Reporting was often incomplete: 122/300 (40.7%) did not report funding sources, 68/294 (23.1%) reported assessing publication bias, and 0 reviews had a registration number.
- Only 150/300 (50.0%) used the term “systematic review” or “meta-analysis” in the title or abstract; 161/300 (53.7%) reported statistical combination of results, and 197/295 (66.8%) reported quality assessment.
Methods
- Cross-sectional descriptive study of SR reports indexed in Medline during November 2004 and written in English.
- SRs were defined broadly as reports whose stated objective was to summarize evidence from multiple studies and that described explicit methods; 300 eligible reports were retained.
- Data were collected with a 51-item form and analyzed descriptively in SPSS; one a priori subgroup analysis compared Cochrane versus non-Cochrane reviews.
Limitations
- The paper states that it examined only one month from a single database and only English-language publications, so the annual publication estimate is an underestimate of total non-Cochrane SRs.
- The authors note that most of the sample was therapeutic, so findings for other review categories should be interpreted cautiously.
- The study relied on what authors reported in the papers, so incomplete reporting may reflect omission rather than poor conduct; the text says this explicitly.
- The broad SR definition may have included some literature reviews with detailed search methods rather than clearly intended SRs; this is an evident limitation from the text.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
0 Valid · 0 Not valid · 5 Not checked · 41 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There were large differences between Cochrane reviews and non-Cochrane reviews in the quality of reporting several characteristics.”
§ Epidemiology and Reporting Characteristics of Systematic Reviews > Abstract > Methods and Findings We searched Medline for SRs indexed during November 2004 and written in English. Citations were screened and those meeting our inclusion criteria were retained. Data were collected using a 51-item data collection form designed to assess the epidemiological and reporting details and the bias-related aspects of the reviews. The data were analyzed descriptively. In total 300 SRs were identified, suggesting a current annual publication rate of about 2,500, involving more than 33,700 separate studies including one-third of a million participants. The majority (272 [90.7%]) of SRs were reported in specialty journals. Most reviews (213 [71.0%]) were categorized as therapeutic, and included a median of 16 studies involving 1,112 participants. Funding sources were not reported in more than one-third (122 [40.7%]) of the reviews. Reviews typically searched a median of three electronic databases and two other sources, although only about two-thirds (208 [69.3%]) of them reported the years searched. Most (197/295 [66.8%]) reviews reported information about quality assessment, while few (68/294 [23.1%]) reported assessing for publication bias. A little over half (161/300 [53.7%]) of the SRs reported combining their results statistically, of which most (147/161 [91.3%]) assessed for consistency across studies. Few (53 [17.7%]) SRs reported being updates of previously completed reviews. No review had a registration number. Only half (150 [50.0%]) of the reviews used the term “systematic review” or “meta-analysis” in the title or abstract. There were large differences between Cochrane reviews and non-Cochrane reviews in the quality of reporting several characteristics.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nearly half (134 [44.7%]) of the SRs did not report whether they had eligibility criteria based on language of reports.”
§ Epidemiology and Reporting Characteristics of Systematic Reviews > Results > Reporting Characteristics Related to Potential for Bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The 300 reviews were published in 132 journals”
§ Epidemiology and Reporting Characteristics of Systematic Reviews > Results > Overall Epidemiology Characteristics
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Cochrane reviews contributed nearly half of these reviews (125 [41.7%]).”
§ Epidemiology and Reporting Characteristics of Systematic Reviews > Results > Prevalence of Publishing SRs
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“publication bias was considered or assessed in only a minority of reports despite much evidence for its existence and potential influence on the results of reviews [20,21].”
§ Epidemiology and Reporting Characteristics of Systematic Reviews > Discussion
What the sources say
- Not checked yetcites [20] Dickersin, 2005, Publication Bias in Meta-Analysis—Prevention, Assessment and Adjustments
- Not checked yetcites [21] Sutton, 2005, Publication Bias in Meta-Analysis—Prevention, Assessment and Adjustments
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
26 more claims
Create a free account to see all 46 claims and to verify the 9 references still open.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 4
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, conducting a SR is still a complex, challenging and time-consuming process [20, 21]”
From Machine learning computational tools to assist the performance of systematic reviews: A mapping review. (2022) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement published in 2009 (hereafter referred to as PRISMA 2009)[4–10] is a reporting guideline designed to address poor reporting of systematic reviews.[11]”
From The PRISMA 2020 statement: An updated guideline for reporting systematic reviews (2021) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A recent survey found that 50% of 300 authors did not mention the terms “systematic review” or “meta-analysis” in the title or abstract of their systematic review [3].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, despite advances, the quality of the conduct and reporting of systematic reviews remains well short of ideal [3],[4],[5],[6].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In a survey of 300 systematic reviews published in November 2004, funding sources were not reported in 41% of the reviews [3].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [3]
Experimental — accuracy not yet measured
20 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Self-citationMoher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
Self-citationMoher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Self-citationPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147 · cited 1×
“Similarly, very few reviews reported being funded by commercial sources, for which such funding has been associated with bias in the results of clinical trials [19].”
BackgroundChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 1×
“Strong evidence of outcome reporting bias was recently reported within clinical trials [ 23 ,24].”
ResultSelf-citationEvaluation of the Quality of Prognosis Studies in Systematic Reviews
Hayden, J. A., Côté, P., & Bombardier, C. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-6-200603210-00010 · Cited by 1,452 · cited 1×
“Typically the focus of such examinations is in specific health care areas [8–11] or type of SR [12,13].”
BackgroundMoher, D., Schulz, K. F., Altman, D. G., & for the CONSORT Group* (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00011 · Cited by 1,426
Self-citationMeta-Analyses of Randomized Controlled Trials
Sacks, H. S., Berrier, J., Reitman, D., Ancona-Berk, V. A., & Chalmers, T. C. (1987) · New England Journal of Medicine · doi:10.1056/nejm198702193160806 · Cited by 1,250
Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241
Research works printed but not linked: 33
- Young, 2005, Lancet, 366, 107
- Sacks, 1987, N Engl J Med, 316, 450
- McAlister, 1999, Ann Intern Med, 131, 947
- Mulrow, 1987, Ann Intern Med, 106, 485
- Jadad, 1996, J Clin Epidemiol, 49, 235
- Shea, 2002, Eval Health Prof, 25, 116
- Jadad, 2000, BMJ, 320, 537
- Delaney, 2005, Crit Care, 9, R575
- Bhandari, 2001, J Bone Joint Surg Am, 83A, 15
- Glenny, 2003, Eur J Oral Sci, 111, 85
- Hayden, 2006, Ann Inter Med, 144, 427
- Whiting, 2005, J Clin Epidemiol, 58, 1
- Montori, 2005, BMJ, 330, 68
- Fritsche, 2004, Am J Transplant, 4, 738
- Mallett, 2002, Int J Technol Assess Health Care, 18, 820
- Chan, 2005, Lancet, 365, 1159
- Moja, 2005, BMJ, 330, 1053
- Lexchin, 2003, BMJ, 326, 1167
- Sutton, 2005, Publication Bias in Meta-Analysis—Prevention, Assessment and Adjustments
- Silagy, 2002, JAMA, 287, 2831
- Chan, 2004, JAMA, 291, 2457
- Chan, 2004, CMAJ, 171, 735
- Moher, 2006, Lancet, 367, 881
- Simes, 1986, J Clin Oncol, 4, 1529
- Whittington, 2004, Lancet, 363, 1341
- De Angelis, 2004, CMAJ, 171, 606
- Plint, 2006, Med J Aust, 185, 263
- Moher, 2001, Ann Intern Med, 134, 657
- Rennie, 1997, JAMA, 276, 649
- Moher, 1999, Lancet, 354, 1896
- Delaney, 2005, Crit Care, 9, R575
- Montori, 2005, BMJ, 330, 68
- Schulman, 2005, BMJ eLetters, 330
Cited by
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
Self-citationPreferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Self-citationPreferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Self-citationPreferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Self-citationPreferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Self-citation
Metadata
- Authors
- Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G.
- Affiliations
- University of Ottawa (CA)
- Children's Hospital of Eastern Ontario (CA)
- University of Oxford (GB)
- Corresponding author
- Moher, David
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2007-03-27
- Volume
- 4
- Issue
- 3
- Pages
- e78
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC1831728
- OpenAlex
- W2066896354
- Semantic Scholar
- e545753c909f6720973dfbda956821e441c3a38a
- Cited by
- 891
- References
- 36
- Citations by year
- 2026: 14 · 2025: 22 · 2024: 41 · 2023: 27 · 2022: 42 · 2021: 65 · 2020: 50 · 2019: 35 · 2018: 35 · 2017: 57 · 2016: 56 · 2015: 58 · 2014: 62 · 2013: 58 · 2012: 67
- FWCI
- 10.29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- University of Ottawa
- Cancer Research UK
- National Institutes of Health
- U.S. Public Health Service (MZ402054)
- Canadian Institutes of Health Research
- U.S. National Library of Medicine
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- systematic reviews, reporting quality, reporting guidelines, publication bias, funding disclosure, quality assessment, meta-analysis, research registration, evidence-based medicine, research transparency, publication rates, study heterogeneity
- MeSH terms
- Epidemiology, Health Services, Health Services Research, Humans, Periodicals as Topic, Research Design, Review Literature as Topic, Bias, Outcome Assessment, Health Care, Evidence-Based Medicine, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Epidemiology and reporting characteristics of systematic reviews. https://app.cochat.ai/papers/10.1371/journal.pmed.0040078
- The paper
- Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G. (2007). Epidemiology and reporting characteristics of systematic reviews. PLoS Medicine. https://doi.org/10.1371/journal.pmed.0040078
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
