Differences in Review Quality and Recommendations for Publication Between Peer Reviewers Suggested by Authors or by Editors
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
CONTEXT: Many journals give authors who submit papers the opportunity to suggest reviewers. Use of these reviewers varies by journal and little is known about the quality of the reviews they produce. OBJECTIVE: To compare author- and editor-suggested reviewers to investigate differences in review quality and recommendations for publication. DESIGN, SETTING, AND PARTICIPANTS: Observational study of original research papers sent for external review at 10 biomedical journals. Editors were instructed to make decisions about their choice of reviewers in their usual manner. Journal administrators then requested additional reviews from the author's list of suggestions according to a strict protocol. MAIN OUTCOME MEASURE: Review quality using the Review Quality Instrument and the proportion of reviewers recommending acceptance (including minor revision), revision, or rejection. RESULTS: There were 788 reviews for 329 manuscripts. Review quality (mean difference in Review Quality Instrument score, -0.05; P = .27) did not differ significantly between author- and editor-suggested reviewers. The author-suggested reviewers were more likely to recommend acceptance (odds ratio, 1.64; 95% confidence interval, 1.02-2.66) or revise (odds ratio, 2.66; 95% confidence interval, 1.43-4.97). This difference was larger in the open reviews of BMJ than among the blinded reviews of other journals for acceptance (P = .02). Where author- and editor-suggested reviewers differed in their recommendations, the final editorial decision to accept or reject a study was evenly balanced (50.9% of decisions consistent with the preferences of the author-suggested reviewers). CONCLUSIONS: Author- and editor-suggested reviewers did not differ in the quality of their reviews, but author-suggested reviewers tended to make more favorable recommendations for publication. Editors can be confident that reviewers suggested by authors will complete adequate reviews of manuscripts, but should be cautious about relying on their recommendations for publication.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Development of the Review Quality Instrument (RQI) for Assessing Peer Reviews of Manuscripts
van Rooyen, S., Black, N., & Godlee, F. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00047-5 · Cited by 183 · cited 1×
“05; power, 90%) on the Review Quality Instrument (RQI).(4,5)”
The characteristics of peer reviewers who produce good-quality reviews
Evans, A. T., Mcnutt, R. A., Fletcher, S. W., & Fletcher, R. H. (1993) · Journal of General Internal Medicine · doi:10.1007/bf02599618 · Cited by 150 · cited 1×
“Several factors are associated with review quality (reviewer age, being a current investigator, and postgraduate training in epidemiology or statistics).(1,2) Many journals give authors the opportunity to suggest reviewers for their own paper, but editors’ decisions to select these reviewers vary because some are conc…”
Cited by
Marsh, H. W., Jayasinghe, U. W., & Bond, N. W. (2008) · American Psychologist · doi:10.1037/0003-066x.63.3.160 · Cited by 299
“Seeking ways to improve peer reviews, Schroter et al. (2004) found that a short-term peer-review training package had a small effect on the quality of reviews but that the effect was no longer significant in a six-month follow-up.”
BackgroundA multi-disciplinary perspective on emergent and future innovations in peer review
Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., Elkhatib, Y., B. Collister, L., Pikas, C. K., Crick, T., et al. (2017) · F1000Research · doi:10.12688/f1000research.12037.1 · Cited by 289
Reed, D. A., Beckman, T. J., Wright, S. M., Levine, R. B., Kern, D. E., & Cook, D. A. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0664-3 · Cited by 282
Kravitz, R. L., Franks, P., Feldman, M. D., Gerrity, M., Byrne, C., & Tierney, W. M. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010072 · Cited by 178
“) but does not monitor website traffic nor formally certify reviewer competence. More formal attempts at reviewer training have met with mixed results.”
BackgroundTeixeira da Silva, J. A., & Dobránszki, J. (2015) · Accountability in Research · doi:10.1080/08989621.2014.899909 · Cited by 158
Metadata
- Authors
- Schroter, S.
- Venue
- JAMA
- Publisher
- American Medical Association (AMA)
- Published
- 2006-01-18
- Volume
- 295
- Issue
- 3
- Pages
- 314
- ISSN
- 0002-9955 (ISSN-L)
- 0098-7484
- OpenAlex
- W1967379013
- Cited by
- 147
- References
- 2
- FWCI
- 57.68
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Differences in Review Quality and Recommendations for Publication Between Peer Reviewers Suggested by Authors or by Editors. https://app.cochat.ai/papers/10.1001/jama.295.3.314
- The paper
- Schroter, S. (2006). Differences in Review Quality and Recommendations for Publication Between Peer Reviewers Suggested by Authors or by Editors. JAMA. https://doi.org/10.1001/jama.295.3.314
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.