Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
The aims of this study were to assess and compare the methodological quality of Cochrane and non-Cochrane systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) using AMSTAR and to compare the prevalence of meta-analysis in both review types. A literature search was undertaken to identify SRs that consisted of hand-searching five major orthodontic journals [American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics and Orthodontics and Craniofacial Research (February 2002 to July 2011)] and the Cochrane Database of Systematic Reviews from January 2000 to July 2011. Methodological quality of the included reviews was gauged using the AMSTAR tool involving 11 key methodological criteria with a score of 0 or 1 given for each criterion. A cumulative grade was given for the paper overall (0-11); an overall score of 4 or less represented poor methodological quality, 5-8 was considered fair and 9 or greater was deemed to be good. In total, 109 SRs were identified in the five major journals and on the CDSR. Of these, 26 (23.9%) were in the CDSR. The mean overall AMSTAR score was 6.2 with 21.1% of reviews satisfying 9 or more of the 11 criteria; a similar prevalence of poor reviews (22%) was also noted. Multiple linear regression indicated that reviews published in the CDSR (P < 0.01); and involving meta-analysis (β = 0.50, 95% confidence interval 0.72, 2.07, P < 0.001) showed greater concordance with AMSTAR.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The other two were recent systematic reviews and were selected from the Cochrane Database of Systematic Reviews as Cochrane Reviews tend to be of higher quality than those published in peer review journals [18, 19, 20, 21, 22, 23].”
From Comparison of Elicit AI and Traditional Literature Searching in Evidence Syntheses Using Four Case Studies (2025) · cites this paper as [20]
In the citing paper
“The rationale for prioritising inclusion of Cochrane reviews over non-Cochrane reviews was that Cochrane reviews are regularly updated and are of higher quality [52–55].”
From Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews (2021) · cites this paper as [52]
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517
Meta-Analyses of Randomized Controlled Trials
Sacks, H. S., Berrier, J., Reitman, D., Ancona-Berk, V. A., & Chalmers, T. C. (1987) · New England Journal of Medicine · doi:10.1056/nejm198702193160806 · Cited by 1,250
Epidemiology and reporting characteristics of systematic reviews.
Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040078 · Cited by 891
Systematic Reviews: Checklists for review articles
Oxman, A. D. (1994) · BMJ · doi:10.1136/bmj.309.6955.648 · Cited by 480
Systematic Reviews: Critical Links in the Great Chain of Evidence
Mulrow, C. D., Cook, D. J., & Davidoff, F. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-5-199703010-00008 · Cited by 260
Systematic reviews : synthesis of best evidence for health care decisions
Mulrow, C. D., & Cook, D. J. (1998) · American College of Physicians eBooks · Cited by 172
Research works printed but not linked: 1
- 2007, BMC MEDICAL RESEARCH METHODOLOGY, 15, 7
Cited by
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Systematic review adherence to methodological or reporting quality
Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L. M., Hróbjartsson, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0527-2 · Cited by 322
Hoffmann, F., Allers, K., Rombey, T., Helbach, J., Hoffmann, A., Mathes, T., & Pieper, D. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.022 · Cited by 278
Lunny, C., Pieper, D., Thabet, P., & Kanji, S. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01269-y · Cited by 252
Stevens, A., Shamseer, L., Weinstein, E., Yazdi, F., Turner, L., Thielman, J., Altman, D. G., Hirst, A., Hoey, J., Palepu, A., et al. (2014) · BMJ · doi:10.1136/bmj.g3804 · Cited by 242
Metadata
- Authors
- Fleming, P. S., Seehra, J., Polychronopoulou, A., Fedorowicz, Z., & Pandis, N.
- Affiliations
- Queen Mary University of London (GB)
- Guy's Hospital (GB)
- National and Kapodistrian University of Athens (GR)
- University of Bern (CH)
- Venue
- The European Journal of Orthodontics
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-04-16
- Volume
- 35
- Issue
- 2
- Pages
- 244-248
- ISSN
- 0141-5387 (ISSN-L)
- 1460-2210
- OpenAlex
- W2112824763
- Cited by
- 95
- References
- 18
- Citations by year
- 2025: 2 · 2024: 7 · 2023: 5 · 2022: 9 · 2021: 8 · 2020: 13 · 2019: 9 · 2018: 8 · 2017: 11 · 2016: 6 · 2015: 8 · 2014: 5 · 2013: 2 · 2012: 2
- FWCI
- 1.51
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- European Journal of Orthodontics (publishedVersion)
- PubMed (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion, other-oa)
- Open Access CRIS of the University of Bern
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- systematic reviews, Cochrane Database of Systematic Reviews, methodological quality, AMSTAR, orthodontic journals, meta-analysis
- MeSH terms
- Algorithms, Humans, Orthodontics, Periodicals as Topic, Review Literature as Topic, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.