The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The peer review process is a cornerstone of biomedical research publications. However, it may fail to allow the publication of high-quality articles. We aimed to identify and sort, according to their importance, all tasks that are expected from peer reviewers when evaluating a manuscript reporting the results of a randomized controlled trial (RCT) and to determine which of these tasks are clearly requested by editors in their recommendations to peer reviewers. METHODS: We identified the tasks expected of peer reviewers from 1) a systematic review of the published literature and 2) recommendations to peer reviewers for 171 journals (i.e., 10 journals with the highest impact factor for 14 different medical areas and all journals indexed in PubMed that published more than 15 RCTs over 3 months regardless of the medical area). Participants who had peer-reviewed at least one report of an RCT had to classify the importance of each task relative to other tasks using a Q-sort technique. Finally, we evaluated editors' recommendations to authors to determine which tasks were clearly requested by editors in their recommendations to peer reviewers. RESULTS: The Q-sort survey was completed by 203 participants, 93 (46 %) with clinical expertise, 72 (36 %) with methodological/statistical expertise, 17 (8 %) with expertise in both areas, and 21 (10 %) with other expertise. The task rated most important by participants (evaluating the risk of bias) was clearly requested by only 5 % of editors. In contrast, the task most frequently requested by editors (provide recommendations for publication), was rated in the first tertile only by 21 % of all participants. CONCLUSIONS: The most important tasks for peer reviewers were not congruent with the tasks most often requested by journal editors in their guidelines to reviewers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Bohannon, J. (2013) · Science · doi:10.1126/science.342.6154.60 · Cited by 934
Boutron, I. (2010) · JAMA · doi:10.1001/jama.2010.651 · Cited by 842 · cited 1×
“important flaws in the quality of manuscripts and published articles [7, 8].”
BackgroundSelf-citationCORRELATING PERSONS INSTEAD OF TESTS
STEPHENSON, W. (1935) · Journal of Personality · doi:10.1111/j.1467-6494.1935.tb02022.x · Cited by 475 · cited 1×
“Sorting tasks With Q-sort, participants classify the importance of each task relative to other tasks [23].”
MethodEditorial peer review for improving the quality of reports of biomedical studies
Jefferson, T., Rudin, M., Brodney Folse, S., & Davidoff, F. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000016.pub3 · Cited by 312 · cited 1×
“The editorial peer review process is described as journal editors relying on the views of independent experts in making decisions on, for example, the publication of submitted manuscripts or presentation of reports at meetings [2].”
MethodWhat errors do peer reviewers detect, and does training improve their ability to detect them?
Schroter, S., Black, N., Evans, S., Godlee, F., Osorio, L., & Smith, R. (2008) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2008.080062 · Cited by 226 · cited 1×
“Finally, peer reviewers are not able to detect fraud [15] and mistakes [16].”
BackgroundMills, E. J., Wu, P., Gagnier, J., & Devereaux, P. J. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.02.008 · Cited by 216 · cited 1×
“The ranking of tasks concerning the CONSORT Statement could explain in part why reporting according to the CONSORT guidelines is poor [7, 32].”
MethodAre Peer Reviewers Encouraged to Use Reporting Guidelines? A Survey of 116 Health Research Journals
Hirst, A., & Altman, D. G. (2012) · PLoS ONE · doi:10.1371/journal.pone.0035621 · Cited by 200 · cited 1×
“Duplicates were deleted and tasks were classified into the following categories used in a previous work [17]: 1) logistics (i.”
MethodKravitz, R. L., Franks, P., Feldman, M. D., Gerrity, M., Byrne, C., & Tierney, W. M. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010072 · Cited by 178
Wang, C.-Y., Kim, H.-H., Hiroi, Y., Sawada, N., Salomone, S., Benjamin, L. E., Walsh, K., Moskowitz, M. A., & Liao, J. K. (2009) · Science Signaling · doi:10.1126/scisignal.2000143 · Cited by 177
Cited by
A multi-disciplinary perspective on emergent and future innovations in peer review
Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., Elkhatib, Y., B. Collister, L., Pikas, C. K., Crick, T., et al. (2017) · F1000Research · doi:10.12688/f1000research.12037.1 · Cited by 289
Comparison of registered and published outcomes in randomized controlled trials: a systematic review
Jones, C. W., Keil, L. G., Holland, W. C., Caughey, M. C., & Platts-Mills, T. F. (2015) · BMC Medicine · doi:10.1186/s12916-015-0520-3 · Cited by 237
Bruce, R., Chauvin, A., Trinquart, L., Ravaud, P., & Boutron, I. (2016) · BMC Medicine · doi:10.1186/s12916-016-0631-5 · Cited by 218
“For example, a study showed a discrepancy between editors and peer reviewers regarding items considered most important when assessing the report of an RCT [58].”
BackgroundSelf-citationGlonti, K., Cauchi, D., Cobo, E., Boutron, I., Moher, D., & Hren, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1347-0 · Cited by 80
Self-citationChauvin, A., Ravaud, P., Moher, D., Schriger, D., Hopewell, S., Shanahan, D., Alam, S., Baron, G., Regnaux, J.-P., Crequit, P., et al. (2019) · BMC Medicine · doi:10.1186/s12916-019-1436-0 · Cited by 72
Self-citation
Metadata
- Authors
- Chauvin, A., Ravaud, P., Baron, G., Barnes, C., & Boutron, I.
- Venue
- BMC Medicine
- Published
- 2015-07-03
- Volume
- 13
- Issue
- 1
- Pages
- 158-158
- ISSN
- 1741-7015 (ISSN-L)
- OpenAlex
- W1508724808
- Cited by
- 62
- References
- 14
- FWCI
- 5.38
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors. https://app.cochat.ai/papers/10.1186/s12916-015-0395-3
- The paper
- Chauvin, A., Ravaud, P., Baron, G., Barnes, C., & Boutron, I. (2015). The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors. BMC Medicine. https://doi.org/10.1186/s12916-015-0395-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.