Is the quality of reviews reflected in editors' and authors' satisfaction with peer review? A cross‐sectional study in 12 journals across four research fields
Abstract
Perception of review quality by authors and editors may play a vital role in helping to keep the peer review process constructive. Comprehensive studies examining author and editor perceptions of reviews of manuscripts from different disciplines are rare. We assessed satisfaction of corresponding authors and opinions of editors with reviewer‐generated reports and reviewers' recommendations and checked whether there was association between authors' and editors' perceptions and recommendations in 12 Elsevier journals across four disciplines. We used a modified Review Quality Instrument (RQI) to measure review quality for 809 unique manuscripts from which we accessed 1,313 reviews and recommendations, 331 authors' perception of the review's helpfulness before editor's decision, and 541 editor's opinions regarding both review timeliness and impact on decision. Authors were most satisfied with reviews that recommended acceptance compared to revision or rejection. Reviews that recommended revisions had highest quality as reflected by the RQI. Authors highly rated their satisfaction with review constructiveness from natural sciences, and editors for the same subject also highly rated timeliness and reviews' influence on publication. Editors' opinion regarding the impact of review on their publication decision and RQI were associated. Our findings suggest that more constructive reviews may better guide the editorial decision‐making process.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior papers [19-23] all find that in authors' evaluations of reviews on their own papers, the reviewer's accept-or-reject decision is highly correlated with the evaluation rating given by the authors.
“The papers [19–23] all find that in authors’ evaluations of reviews on their own papers, the decision of accept or reject given by the reviewer is highly correlated with evaluation rating given by the authors.”
From Peer reviews of peer reviews: A randomized controlled trial and other experiments (2025) · cites this paper as [23]
In Elsevier journal reviews from 2014 across medicine, science, and computer science, there is a correlation between author satisfaction with the review and whether the review recommended acceptance.
“In a cross-sectional study of journals in multiple disciplines, the study [23] analyzed authors’ and editors’ evaluations of review quality in Elsevier journal reviews from 2014 across medicine, science, and computer science. They found correlation between author satisfaction with the review and whether the review recommended acceptance.”
From Peer reviews of peer reviews: A randomized controlled trial and other experiments (2025) · cites this paper as [23]
Cites
Duration and quality of the peer review process: the author’s perspective
Huisman, J., & Smits, J. (2017) · Scientometrics · doi:10.1007/s11192-017-2310-5 · Cited by 303 · see citing works
Does Masking Author Identity Improve Peer Review Quality?: A Randomized Controlled Trial
Justice, A. C., Cho, M. K., Winker, M. A., Berlin, J. A., Rennie, D., & and the PEER Investigators (1998) · JAMA · doi:10.1001/jama.280.3.240 · Cited by 278 · see citing works
Bornmann, L., Mutz, R., & Daniel, H.-D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0014331 · Cited by 231 · cited 1× · see citing works
“…ICC (and low absolute agreement between two reviewers) were also shown previously and may reflect different reviewers’ abilities to recognize or emphasize areas that need improvement in manuscripts (Bornmann et al., 2010; Huisman & Smits, 2017; Jackson, Srinivasan, Rea, Fletcher, & Kravitz, 2011).”
ResultAre Peer Reviewers Encouraged to Use Reporting Guidelines? A Survey of 116 Health Research Journals
Hirst, A., & Altman, D. G. (2012) · PLoS ONE · doi:10.1371/journal.pone.0035621 · Cited by 200 · see citing works
Development of the Review Quality Instrument (RQI) for Assessing Peer Reviews of Manuscripts
van Rooyen, S., Black, N., & Godlee, F. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00047-5 · Cited by 177 · see citing works
Reliability of Editors' Subjective Quality Ratings of Peer Reviews of Manuscripts
Callaham, M. L. (1998) · JAMA · doi:10.1001/jama.280.3.229 · Cited by 122 · see citing works
Tools used to assess the quality of peer review reports: A methodological systematic review
Superchi, C., González, J. A., Solà, I., Cobo, E., Hren, D., & Boutron, I. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0688-x · Cited by 110 · cited 4× · see citing works
“The authors showed that existing instruments lacked objective rigour to assess the quality of peer review reports, leaving room for the future development of such a tool (Superchi et al., 2019).”
BackgroundInfluential
Flagged first, then by citations
Cited by
Sorted by citations
Peer reviews of peer reviews: A randomized controlled trial and other experiments
Goldberg, A., Stelmakh, I., Cho, K., Oh, A., Agarwal, A., Belgrave, D., & Shah, N. B. (2025) · PLOS ONE · doi:10.1371/journal.pone.0320444 · Cited by 18 · see citing works
“The papers [KBY10, KHB13, Pap07, WKWC02, PMM + 21] all find that in authors’ evaluations of reviews on their own papers, the decision of accept or reject given by the reviewer is highly correlated with evaluation rating given by the authors.”
- Authors
- Pranić, S. M., Malički, M., Marušić, S. L., Mehmani, B., & Marušić, A.
- Affiliations
- University of Split (HR)
- Language Editing Services Rogor Zagreb Croatia
- Elsevier BV (Netherlands) (NL)
- Corresponding author
- Pranić, Shelly M.
- Venue
- Learned Publishing
- Publisher
- Wiley
- Published
- 2020-11-08
- Volume
- 34
- Issue
- 2
- Pages
- 187-197
- ISSN
- 0953-1513 (ISSN-L)
- 1741-4857
- OpenAlex
- W3098050497
- Cited by
- 19
- References
- 7
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 1 · 2023: 2 · 2022: 6 · 2021: 4
- FWCI
- 11.18
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Learned Publishing (publishedVersion)
- Funders
- European Cooperation in Science and Technology (TD1306)
- Topics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Publishing and Scholarly Communication
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- peer review quality, reviewer reports, editorial decision-making, review timeliness, manuscript revision, disciplinary differences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
