Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
Open accessThe DOI matches its Crossref record; 44 of the 46 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both. METHODOLOGY AND PRINCIPAL FINDINGS: Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with "no statistical expert" and "no checklist" as controls. The two interventions were crossed in a 2x2 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6-24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: -0.3-+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3-6.7), showing a significant improvement in quality. CONCLUSIONS AND SIGNIFICANCE: This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous studies have also found that providing additional specialized review based on reporting guidelines led to small improvements to manuscripts, while suggesting reporting checklists to regular reviewers had no effect [10, 11].”
From Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature (2020) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · cited 1×
“The two main objectives of our investigation were to assess the effects of (1) adding a statistical peer reviewer and (2) suggesting reporting guidelines [17–22] to reviewers, by taking into account a direct measurement of the final quality of papers instead of relying on surrogate variables.”
BackgroundConsort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.361 · Cited by 2,817
CONSORT statement: extension to cluster randomised trials
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7441.702 · Cited by 2,557
Moher, D. (2001) · JAMA · doi:10.1001/jama.285.15.1987 · Cited by 2,545
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124 · cited 1×
“The two main objectives of our investigation were to assess the effects of (1) adding a statistical peer reviewer and (2) suggesting reporting guidelines [17–22] to reviewers, by taking into account a direct measurement of the final quality of papers instead of relying on surrogate variables.”
BackgroundOxley, J. A. (1995) · Medical Entomology and Zoology · Cited by 761
Research works printed but not linked: 4
- 2002, The fourth international congress on biomedical peer review, JAMA, 287, 2749
- D Altman, 1999, Statistical peer review., 157
- M Mora-Ripoll, 1999, Cómo mejorar la calidad estadística de los artículos presentados a revistas biomédicas: lista de comprobación para los autores., Med Clin (Barc), 113, 138
- C Arnau, 2003, Effect of statistical review on manuscript quality in Medicina Clinica (Barcelona): a randomized study., Med Clin (Barc), 121, 690
Cited by
Non-publication of large randomized clinical trials: cross sectional analysis
Jones, C. W., Handler, L., Crowell, K. E., Keil, L. G., Weaver, M. A., & Platts-Mills, T. F. (2013) · BMJ · doi:10.1136/bmj.f6104 · Cited by 350
Cobo, E., Cortes, J., Ribera, J. M., Cardellach, F., Selva-O'Callaghan, A., Kostov, B., Garcia, L., Cirugeda, L., Altman, D. G., Gonzalez, J. A., et al. (2011) · BMJ · doi:10.1136/bmj.d6783 · Cited by 290
“22 This second trial also investigated the effect of suggesting the use of reporting guidelines to reviewers, but did not observe any benefit.”
Self-citationBruce, R., Chauvin, A., Trinquart, L., Ravaud, P., & Boutron, I. (2016) · BMC Medicine · doi:10.1186/s12916-016-0631-5 · Cited by 218
“Two RCTs examined the impact of adding a statistical peer reviewer on the final manuscript quality [39, 40].”
BackgroundMethodInfluentialAre Peer Reviewers Encouraged to Use Reporting Guidelines? A Survey of 116 Health Research Journals
Hirst, A., & Altman, D. G. (2012) · PLoS ONE · doi:10.1371/journal.pone.0035621 · Cited by 200
“Third, there is the issue of exactly how to use reporting guidelines in the peer review process. A few randomised trials have been conducted comparing strategies for peer review which have included reporting guidelines. Cobo and colleagues”
MethodScoping review on interventions to improve adherence to reporting guidelines in health research
Blanco, D., Altman, D., Moher, D., Boutron, I., Kirkham, J. J., & Cobo, E. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026589 · Cited by 154
“Finally, editors may suggest that peer reviewers use RGs.(107) In addition, by asking peer reviewers questions about whether the author has followed RGs, this might be an indirect way to encourage them.”
BackgroundSelf-citation
Metadata
- Authors
- Cobo, E., Selva-O'Callagham, A., Ribera, J.-M., Cardellach, F., Dominguez, R., & Vilardell, M.
- Affiliations
- Universitat Politècnica de Catalunya (ES)
- Universitat de Barcelona (ES)
- Vall d'Hebron Hospital Universitari (ES)
- Corresponding author
- Cobo, Erik
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2007-03-28
- Volume
- 2
- Issue
- 3
- Pages
- e332
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC1824709
- OpenAlex
- W2076272198
- Cited by
- 112
- References
- 46
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 8 · 2023: 3 · 2022: 6 · 2021: 10 · 2020: 11 · 2019: 11 · 2018: 4 · 2017: 6 · 2016: 3 · 2015: 6 · 2014: 4 · 2013: 6 · 2012: 3
- FWCI
- 4.05
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona) (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (submittedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- peer review, reporting quality, reporting guidelines, CONSORT, STARD, manuscript quality, editorial process, clinical research articles, journal editorial policy, manuscript rejection, publication bias, bias in clinical trials
- MeSH terms
- Manuscripts as Topic, Manuscripts, Medical as Topic, Peer Review, Periodicals as Topic, Research, Peer Review, Research, Checklist
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial. https://app.cochat.ai/papers/10.1371/journal.pone.0000332
- The paper
- Cobo, E., Selva-O'Callagham, A., Ribera, J.-M., Cardellach, F., Dominguez, R., & Vilardell, M. (2007). Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial. PLoS ONE. https://doi.org/10.1371/journal.pone.0000332
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.