Blinded by PRISMA: Are Systematic Reviewers Focusing on PRISMA and Ignoring Other Guidelines?
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
BACKGROUND: PRISMA guidelines have been developed to improve the reporting of systematic reviews (SRs). Other reporting guidelines and techniques to assess methodological quality of SRs have been developed. We aimed to assess the frequency of the use of reporting and other guidelines in SRs to assess whether PRISMA is being used inappropriately as a substitute for other relevant guidelines. METHODS: Web of Knowledge was searched to identify articles citing the PRISMA guidelines over a 12-month period. The use of reporting guidelines (including PRISMA and MOOSE) and tools for assessing methodological quality (including QUADAS) was assessed. Factors associated with appropriate use of guidelines including review type, field of publication and involvement of a methodologist were investigated. RESULTS: Over the 12-month period, 701 SRs were identified. MOOSE guidelines were cited in just 17% of epidemiologic reviews; QUADAS or QUADAS-2 was referred to in just 40% of diagnostic SRs. In the multivariable analysis, medical field of publication and methodologist involvement (OR = 1.97, 95% CI: 1.37, 2.83) were significant predictors of appropriate use of guidelines. Inclusion of a meta-analysis resulted in 73% higher odds of appropriate usage of systematic review guidelines (OR = 1.73, 95% CI: 1.22, 2.35). CONCLUSIONS: Usage of SR reporting guidelines and tools for assessment of methodological quality other than PRISMA may be under-utilized with negative implications both for the reporting and methodological quality of systematic reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Elm, E. V., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · BMJ · doi:10.1136/bmj.39335.541782.ad · Cited by 10,766
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Whiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003) · BMC Medical Research Methodology · doi:10.1186/1471-2288-3-25 · Cited by 3,592
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · Trials · doi:10.1186/1745-6215-11-32 · Cited by 1,871
Bias and causal associations in observational research
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07451-2 · Cited by 1,553
Panic, N., Leoncini, E., de Belvis, G., Ricciardi, W., & Boccia, S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0083138 · Cited by 883 · cited 2×
“In recent times efforts have been made to improve compliance with PRISMA guidelines using active editorial involvement [6]; these efforts have not been replicated in respect of other guidelines, including those pertaining to observational research.”
BackgroundTao, K.-M., Li, X.-Q., Zhou, Q.-H., Moher, D., Ling, C.-Q., & Yu, W.-F. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027611 · Cited by 101 · cited 1×
“This is certainly a positive development, which has led to enhanced reporting of SRs [6], although further work is required [7,8].”
Background
Research works printed but not linked: 8
- JN Lavis, 2009, SUPPORT Tools for evidence-informed Policymaking (STP), Health Res Policy Syst, 7, 11
- D Moher, 2009, Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement, PLoS Med, 6, e1000097, 10.1371/journal.pmed.1000097 · DOI · Paper page
- PS Fleming, 2012, A PRISMA assessment of the reporting quality of systematic reviews in orthodontics, Angle Orthod, 83, 158, 10.2319/032612-251.1 · DOI · Paper page
- D Moher, 1999, Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement, Lancet, 354, 1896, 10.1016/S0140-6736(99)04149-5 · DOI · Paper page
- DF Stroup, 2000, Meta-analysis of observational studies in epidemiology, JAMA, 283, 2008, 10.1001/jama.283.15.2008 · DOI · Paper page
- M Leeflang, 2008, Systematic reviews of diagnostic test accuracy, Ann Intern Med, 149, 889, 10.7326/0003-4819-149-12-200812160-00008 · DOI · Paper page
- Egger M, Dickersin K, Smith GD (2001) Chapter 3, Problems and limitations in conducting systematic reviews. In Systematic reviews in healthcare, BMJ books. 2nd edition. Edited by Egger M, Smith GD, Altman DG. London: 43–68. · DOI · Paper page
- BH Willis, 2011, The assessment of the quality of reporting of meta-analyses in diagnostic research: a systematic review, BMC Med Res Methodol, 11, 163, 10.1186/1471-2288-11-163 · DOI · Paper page
Cited by
Page, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769
“Frequency of inappropriate citation of the PRISMA Statement by authors [125] 1”
BackgroundThe benefits of health information exchange: an updated systematic review
Menachemi, N., Rahurkar, S., Harle, C. A., & Vest, J. R. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy035 · Cited by 226
Koffel, J. B., & Rethlefsen, M. L. (2016) · PLOS ONE · doi:10.1371/journal.pone.0163309 · Cited by 107
Use of guidelines to improve the quality and transparency of reporting oral health research
Sarkis-Onofre, R., Cenci, M. S., Demarco, F. F., Lynch, C. D., Fleming, P. S., Pereira-Cenci, T., & Moher, D. (2015) · Journal of Dentistry · doi:10.1016/j.jdent.2015.01.006 · Cited by 73
Self-citationSeo, C., Kim, B., & Kruis, N. E. (2021) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2020.101752 · Cited by 47
Metadata
- Authors
- Fleming, P. S., Koletsi, D., & Pandis, N.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-05-01
- Volume
- 9
- Issue
- 5
- Pages
- e96407
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2073533288
- Cited by
- 115
- References
- 8
- FWCI
- 2.13
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Blinded by PRISMA: Are Systematic Reviewers Focusing on PRISMA and Ignoring Other Guidelines? https://app.cochat.ai/papers/10.1371/journal.pone.0096407
- The paper
- Fleming, P. S., Koletsi, D., & Pandis, N. (2014). Blinded by PRISMA: Are Systematic Reviewers Focusing on PRISMA and Ignoring Other Guidelines? PLoS ONE. https://doi.org/10.1371/journal.pone.0096407
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.