Peer review in scholarly journals: Perspective of the scholarly community – Results from an international study
Open accessThe DOI matches its Crossref record.
Abstract
This summary is extracted from the report of the same title published by the Publishing Research Council (PRC) and reproduced here by kind permission of the PRC. The full report and a shorter edited version can be found on the PRC website at http://www.publishingresearch.net/PeerReview.htm. Peer review is seen as an essential component of scholarly communication, the mechanism that facilitates the publication of primary research in academic journals. Although sometimes thought of as an essential part of the journal, it is only since the second world war that peer review has been institutionalised in the form we know it today. More recently it has come under criticism on a number of fronts: it has been said that it is unreliable, unfair and fails to validate or authenticate; that it is unstandardised and idiosyncratic; that its secrecy leads to irresponsibility on the part of reviewers; that it stifles innovation; that it causes delay in publication; and so on. Perhaps the strongest criticism is that there is a lack of evidence that peer review actually works, and a lack of evidence to indicate whether the documented failings are rare exceptions or the tip of an iceberg. The survey reported here does not attempt directly to address the question of whether or not peer review works, but instead looks in detail at the experiences and perceptions of a large group of mostly senior authors, reviewers and editors (there is of course considerable overlap between these groups). Respondents were spread by region and by field of research broadly in line with the universe of authors publishing in the journals in the Thomson Scientific database, which covers the leading peer reviewed journals. The survey presents its findings in two broad areas: attitudes to peer review and current practices in peer review.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Peer review in a changing world: An international study measuring the attitudes of researchers
Mulligan, A., Hall, L., & Raphael, E. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22798 · Cited by 393
“It is worth noting that the percentage agreeing that double-blind peer review is effective is 5% higher than in the 2007 peer review study (t = 4.7, p 0.05) by Ware and Monkman (2008).”
BackgroundInfluentialGender bias in scholarly peer review
Helmer, M., Schottdorf, M., Neef, A., & Battaglia, D. (2017) · eLife · doi:10.7554/elife.21718 · Cited by 380
“Ambiguous reports exist whether open-peer review (as opposed to single- or double-blind peer-review) affects potential reviewers’ willingness to assess a paper (Nature Neuroscience, 1999, van Rooyen et al., 1999; Ware, 2008; Baggs et al., 2008).”
BackgroundA multi-disciplinary perspective on emergent and future innovations in peer review
Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., Elkhatib, Y., B. Collister, L., Pikas, C. K., Crick, T., et al. (2017) · F1000Research · doi:10.12688/f1000research.12037.1 · Cited by 289
Validating online reference managers for scholarly impact measurement
Li, X., Thelwall, M., & Giustini, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0580-x · Cited by 255
Adoption and use of Web 2.0 in scholarly communications
Procter, R., Williams, R., Stewart, J., Poschen, M., Snee, H., Voss, A., & Asgari-Targhi, M. (2010) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2010.0155 · Cited by 190
Overburdening of peer reviewers: a multi-stakeholder perspective on causes and effects
Severin, A., & Chataway, J. (2021) · Learned Publishing · doi:10.1002/leap.1392 · Cited by 73
“…overall sustainability of the peer review system (Kovanis et al., 2016) and standardized surveys that gauge overburdening of reviewers as one reason why scholars decline to review (Brannon et al., 2016; Djupe, 2015; Mulligan et al., 2013; Publons, 2018; Tite & Schroter, 2007; Ware, 2008a, 2008b).”
BackgroundGenerative AI and Learning Analytics
Khosravi, H., Viberg, O., Kovanovic, V., & Ferguson, R. (2023) · Journal of Learning Analytics · doi:10.18608/jla.2023.8333 · Cited by 25
Examining uncertainty in journal peer reviewers’ recommendations: a cross-sectional study
Barnett, A., Allen, L., Aldcroft, A., Lash, T. L., & McCreanor, V. (2024) · Royal Society Open Science · doi:10.1098/rsos.240612 · Cited by 10
“The median time taken to review was 3 hours, which is less than the median of 5 hours from an international survey [47].”
Background
Metadata
- Authors
- Ware, M.
- Venue
- Information Services & Use
- Publisher
- SAGE Publications
- Published
- 2008-10-02
- Volume
- 28
- Issue
- 2
- Pages
- 109-112
- ISSN
- 0167-5265 (ISSN-L)
- 1875-8789
- OpenAlex
- W1594458610
- Cited by
- 102
- References
- 0
- FWCI
- 4.22
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.