Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
Open accessThe DOI matches its Crossref record; 44 of the 46 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both. METHODOLOGY AND PRINCIPAL FINDINGS: Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with "no statistical expert" and "no checklist" as controls. The two interventions were crossed in a 2x2 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6-24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: -0.3-+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3-6.7), showing a significant improvement in quality. CONCLUSIONS AND SIGNIFICANCE: This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous studies have also found that providing additional specialized review based on reporting guidelines led to small improvements to manuscripts, while suggesting reporting checklists to regular reviewers had no effect [10, 11].”
From Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature (2020) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · cited 1×
“The two main objectives of our investigation were to assess the effects of (1) adding a statistical peer reviewer and (2) suggesting reporting guidelines [17–22] to reviewers, by taking into account a direct measurement of the final quality of papers instead of relying on surrogate variables.”
BackgroundConsort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.361 · Cited by 2,817
CONSORT statement: extension to cluster randomised trials
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7441.702 · Cited by 2,557
Moher, D. (2001) · JAMA · doi:10.1001/jama.285.15.1987 · Cited by 2,545
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124 · cited 1×
“The two main objectives of our investigation were to assess the effects of (1) adding a statistical peer reviewer and (2) suggesting reporting guidelines [17–22] to reviewers, by taking into account a direct measurement of the final quality of papers instead of relying on surrogate variables.”
BackgroundOxley, J. A. (1995) · Medical Entomology and Zoology · Cited by 761
Research works printed but not linked: 2
- Mora-Ripoll M 1999 Cómo mejorar la calidad estadística de los artículos presentados a revistas biomédicas: lista de comprobación para los autores. Med Clin (Barc) 113 138 149 10472600
- Arnau C Cobo E Ribera JM Cardellach F Selva A 2003 Effect of statistical review on manuscript quality in Medicina Clinica (Barcelona): a randomized study. Med Clin (Barc) 121 690 694 14651815
Cited by
Items 101–112 of 112
Rethlefsen, M. L. (2024) · Research Publications (Maastricht University) · doi:10.26481/dis.20250121mr
Collins, G., Logullo, P., MacCarthy, A., Kirtley, S., Bullock, G. S., Dhiman, P., & Ma, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5197113
Carneiro, C. F. D., Queiroz, V. G. S., Moulin, T. C., Carvalho, C. A. M., Haas, C. B., Rayêe, D., Henshall, D. E., De‐Souza, E. A., Amorim, F. E., Boos, F. Z., et al. (2020) · Figshare · doi:10.6084/m9.figshare.13310563
Carneiro, C. F. D., Queiroz, V. G. S., Moulin, T. C., Carvalho, C. A. M., Haas, C. B., Rayêe, D., Henshall, D. E., De‐Souza, E. A., Amorim, F. E., Boos, F. Z., et al. (2020) · Figshare · doi:10.6084/m9.figshare.13310584
Boudreau, K. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6025114 · Cited by 1
Good publication practices: declaration, approval, and now enforcement of reporting standards
Saygitov, R. T., & Sinitsyn, V. E. (2025) · Digital Diagnostics · doi:10.17816/dd680847
Stan, J. H. V. (2026) · Current Opinion in Otolaryngology & Head & Neck Surgery · doi:10.1097/moo.0000000000001121
Rivoirard, R. (2019) · theses.fr (ABES) · doi:10.70675/bbd245c8z3efdz4412z99a9z55363bd4c4bf
Accountability in evidence syntheses: On the need for rigorous peer review and reporting guidelines
Bakker, C., Rethlefsen, M. L., White, W., & Schneider, J. (2026) · Accountability in Research · doi:10.1080/08989621.2026.2634744 · Cited by 1
“Including peer reviews with specific methodological expertise, such as statistical reviewers, has previously been shown to improve the overall quality of manuscripts (Cobo et al. 2007).”
Launching the Annals of Emergency Medicine Peer Review Academy
Klein, A. L., Schenkel, S. M., Cooper, R. J., Barrett, T. W., Schriger, D. L., & Yealy, D. (2026) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2026.05.013
Statistical Competence and Editorial Policies in Biomedical Publications
Zimba, O., Oralbek, A., Mukhamediyarov, M., Yessirkepov, M., & Koçyiğit, B. F. (2026) · Journal of Korean Medical Science · doi:10.3346/jkms.2026.41.e286 · Cited by 1
Metadata
- Authors
- Cobo, E., Selva-O'Callagham, A., Ribera, J.-M., Cardellach, F., Dominguez, R., & Vilardell, M.
- Affiliations
- Universitat Politècnica de Catalunya (ES)
- Universitat de Barcelona (ES)
- Vall d'Hebron Hospital Universitari (ES)
- Corresponding author
- Cobo, Erik
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2007-03-28
- Volume
- 2
- Issue
- 3
- Pages
- e332
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC1824709
- OpenAlex
- W2076272198
- Cited by
- 112
- References
- 46
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 8 · 2023: 3 · 2022: 6 · 2021: 10 · 2020: 11 · 2019: 11 · 2018: 4 · 2017: 6 · 2016: 3 · 2015: 6 · 2014: 4 · 2013: 6 · 2012: 3
- FWCI
- 4.05
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona) (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (submittedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- peer review, reporting quality, reporting guidelines, CONSORT, STARD, manuscript quality, editorial process, clinical research articles, journal editorial policy, manuscript rejection, publication bias, bias in clinical trials
- MeSH terms
- Manuscripts as Topic, Manuscripts, Medical as Topic, Peer Review, Periodicals as Topic, Research, Peer Review, Research, Checklist
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.