Are Peer Reviewers Encouraged to Use Reporting Guidelines? A Survey of 116 Health Research Journals
Open accessThe DOI matches its Crossref record; 55 of the 57 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Pre-publication peer review of manuscripts should enhance the value of research publications to readers who may wish to utilize findings in clinical care or health policy-making. Much published research across all medical specialties is not useful, may be misleading, wasteful and even harmful. Reporting guidelines are tools that in addition to helping authors prepare better manuscripts may help peer reviewers in assessing them. We examined journals' instructions to peer reviewers to see if and how reviewers are encouraged to use them. METHODS: We surveyed websites of 116 journals from the McMaster list. Main outcomes were 1) identification of online instructions to peer reviewers and 2) presence or absence of key domains within instructions: on journal logistics, reviewer etiquette and addressing manuscript content (11 domains). FINDINGS: Only 41/116 journals (35%) provided online instructions. All 41 guided reviewers about the logistics of their review processes, 38 (93%) outlined standards of behaviour expected and 39 (95%) contained instruction about evaluating the manuscript content. There was great variation in explicit instruction for reviewers about how to evaluate manuscript content. Almost half of the online instructions 19/41 (46%) mentioned reporting guidelines usually as general statements suggesting they may be useful or asking whether authors had followed them rather than clear instructions about how to use them. All 19 named CONSORT for reporting randomized trials but there was little mention of CONSORT extensions. PRISMA, QUOROM (forerunner of PRISMA), STARD, STROBE and MOOSE were mentioned by several journals. No other reporting guideline was mentioned by more than two journals. CONCLUSIONS: Although almost half of instructions mentioned reporting guidelines, their value in improving research publications is not being fully realised. Journals have a responsibility to support peer reviewers. We make several recommendations including wider reference to the EQUATOR Network online library (www.equator-network.org/).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Furthermore, policies around how journal editors and peer reviewers should ensure and/or enforce adherence to reporting checklists are even less clear, if they exist at all [34].”
From Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement (2015) · cites this paper as [34]
In the citing paper
“Most of the instructions to reviewers from medical journals in another study [21] emphasized issues about general presentation, but varied a lot in how explicit and detailed they were.”
From Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature (2020) · cites this paper as [21]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Self-citationPreferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 6×
“Such guidelines include CONSORT (CONsolidated Standards Of Reporting Trials) [3] and PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [4].”
MethodInfluentialSelf-citationPreferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Self-citationTong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509 · cited 1×
“Other reporting guidelines received only one or two mentions: TREND [21], SQUIRE [22], RATS [23], COREQ [24], QUALRES [25], biomedical images [26].”
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Self-citationMeta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Self-citationCONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
Self-citation
Research works printed but not linked: 1
- M Ware, 2011, Peer Review: Recent Experience and Future Directions New Review of Information Networking, 16, 23
Cited by
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
“Furthermore, policies around how journal editors and peer reviewers should ensure and/or enforce adherence to reporting checklists are even less clear, if they exist at all [34].”
BackgroundBest practice guidance and reporting items for the development of scoping review protocols
Peters, M. D. J., Godfrey, C., McInerney, P., Khalil, H., Larsen, P., Marnie, C., Pollock, D., Tricco, A. C., & Munn, Z. (2022) · JBI Evidence Synthesis · doi:10.11124/jbies-21-00242 · Cited by 1,444
“16 Reporting guidelines help authors to develop better manuscripts; 16 therefore, as noted above, our intention is to support authors to write high-quality scoping review protocols to enable more transparently conducted and reported scoping reviews.”
The CARE guidelines: consensus-based clinical case reporting guideline development
Gagnier, J. J., Kienle, G., Altman, D. G., Moher, D., Sox, H., Riley, D., & the CARE Group (2013) · Journal of Medical Case Reports · doi:10.1186/1752-1947-7-223 · Cited by 1,309
“Of those, only 19 (46%) mentioned or referred to reporting guidelines as a useful resource [26].”
Self-citationThe CARE Guidelines: Consensus‐Based Clinical Case Reporting Guideline Development
Gagnier, J. J., Kienle, G., Altman, D. G., Moher, D., Sox, H., Riley, D., & CARE Group (2013) · Headache: The Journal of Head and Face Pain · doi:10.1111/head.12246 · Cited by 1,342
“In response, the authors provide several recommendations for editors to improve the peer review of submitted manuscripts, suggesting that journals have a responsibility to support peer reviewers.(26) The developers of reporting guidelines have a responsibility to plan a dissemination and implementation strategy that s…”
Self-citationThe CARE guidelines: consensus-based clinical case reporting guideline development
Gagnier, J. J., Kienle, G., Altman, D. G., Moher, D., Sox, H., & Riley, D. (2013) · Global Advances in Health and Medicine · doi:10.7453/gahmj.2013.008 · Cited by 1,305
Self-citation
Metadata
- Authors
- Hirst, A., & Altman, D. G.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Hirst, Allison
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-04-27
- Volume
- 7
- Issue
- 4
- Pages
- e35621
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3338712
- OpenAlex
- W2041188266
- Semantic Scholar
- 57c34533b44ea0a5963f4048af302178300e820e
- Cited by
- 200
- References
- 57
- Citations by year
- 2026: 3 · 2025: 19 · 2024: 9 · 2023: 13 · 2022: 12 · 2021: 12 · 2020: 16 · 2019: 13 · 2018: 9 · 2017: 13 · 2016: 15 · 2015: 22 · 2014: 18 · 2013: 20 · 2012: 6
- FWCI
- 24.08
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Cancer Research UK
- University of Oxford
- Medical Research Council (MC_qA137931, MR/J00488X/1)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- peer review, reporting guidelines, CONSORT, PRISMA, STROBE, STARD, moose, EQUATOR Network, systematic reviews
- MeSH terms
- Humans, Peer Review, Publishing, Guidelines as Topic, Editorial Policies, Biomedical Research, Research Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Are Peer Reviewers Encouraged to Use Reporting Guidelines? A Survey of 116 Health Research Journals. https://app.cochat.ai/papers/10.1371/journal.pone.0035621
- The paper
- Hirst, A., & Altman, D. G. (2012). Are Peer Reviewers Encouraged to Use Reporting Guidelines? A Survey of 116 Health Research Journals. PLoS ONE. https://doi.org/10.1371/journal.pone.0035621
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
