Fragments of peer review: A quantitative analysis of the literature (1969-2015)
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
This paper examines research on peer review between 1969 and 2015 by looking at records indexed from the Scopus database. Although it is often argued that peer review has been poorly investigated, we found that the number of publications in this field doubled from 2005. A half of this work was indexed as research articles, a third as editorial notes and literature reviews and the rest were book chapters or letters. We identified the most prolific and influential scholars, the most cited publications and the most important journals in the field. Co-authorship network analysis showed that research on peer review is fragmented, with the largest group of co-authors including only 2.1% of the whole community. Co-citation network analysis indicated a fragmented structure also in terms of knowledge. This shows that despite its central role in research, peer review has been examined only through small-scale research projects. Our findings would suggest that there is need to encourage collaboration and knowledge sharing across different research communities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22784 · Cited by 1,000
Bohannon, J. (2013) · Science · doi:10.1126/science.342.6154.60 · Cited by 934
Peer Review: A Flawed Process at the Heart of Science and Journals
Smith, R. (2006) · Journal of the Royal Society of Medicine · doi:10.1177/014107680609900414 · Cited by 880
Bornmann, L. (2011) · Annual Review of Information Science and Technology · doi:10.1002/aris.2011.1440450112 · Cited by 567
Peer review: still king in the digital age
NICHOLAS, D., WATKINSON, A., JAMALI, H. R., HERMAN, E., TENOPIR, C., VOLENTINE, R., ALLARD, S., & LEVINE, K. (2015) · Learned Publishing · doi:10.1087/20150104 · Cited by 242
Bruce, R., Chauvin, A., Trinquart, L., Ravaud, P., & Boutron, I. (2016) · BMC Medicine · doi:10.1186/s12916-016-0631-5 · Cited by 218
Let’s make peer review scientific
Rennie, D. (2016) · Nature · doi:10.1038/535031a · Cited by 111
Cited by
The limitations to our understanding of peer review
Tennant, J. P., & Ross-Hellauer, T. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00092-1 · Cited by 384
The effect of publishing peer review reports on referee behavior in five scholarly journals
Bravo, G., Grimaldo, F., López-Iñesta, E., Mehmani, B., & Squazzoni, F. (2019) · Nature Communications · doi:10.1038/s41467-018-08250-2 · Cited by 273
Self-citationDondio, P., Casnici, N., Grimaldo, F., Gilbert, N., & Squazzoni, F. (2019) · Journal of Informetrics · doi:10.1016/j.joi.2019.03.018 · Cited by 48
Self-citationComparing human and AI expertise in the academic peer review process: towards a hybrid approach
Farber, S. (2025) · Higher Education Research & Development · doi:10.1080/07294360.2024.2445575 · Cited by 33
The impact of peer review on the contribution potential of scientific papers
Matsui, A., Chen, E., Wang, Y., & Ferrara, E. (2021) · PeerJ · doi:10.7717/peerj.11999 · Cited by 21
Metadata
- Authors
- Grimaldo, F., Marušić, A., & Squazzoni, F.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-02-21
- Volume
- 13
- Issue
- 2
- Pages
- e0193148
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2788415229
- Cited by
- 55
- References
- 7
- FWCI
- 5.18
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Fragments of peer review: A quantitative analysis of the literature (1969-2015). https://app.cochat.ai/papers/10.1371/journal.pone.0193148
- The paper
- Grimaldo, F., Marušić, A., & Squazzoni, F. (2018). Fragments of peer review: A quantitative analysis of the literature (1969-2015). PLOS ONE. https://doi.org/10.1371/journal.pone.0193148
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.