The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
What is open peer review? A systematic review
Open accessAbstract
Background : “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with numerous overlapping and contradictory definitions. While for some the term refers to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods : Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are systematically analysed to build a coherent typology of the various innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results : This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase “open peer review” has been used thus far, for the literature offers 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature reviewed. Conclusions : I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Retracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405
RetractedMisconduct accounts for the majority of retracted scientific publications
Fang, F. C., Steen, R. G., & Casadevall, A. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1212247109 · Cited by 1,232
A method for taxonomy development and its application in information systems
Nickerson, R. C., Varshney, U., & Muntermann, J. (2013) · European Journal of Information Systems · doi:10.1057/ejis.2012.26 · Cited by 1,143
Peer-review practices of psychological journals: The fate of published articles, submitted again
Peters, D. P., & Ceci, S. J. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011183 · Cited by 1,079
Publication prejudices: An experimental study of confirmatory bias in the peer review system
Mahoney, M. J. (1977) · Cognitive Therapy and Research · doi:10.1007/bf01173636 · Cited by 1,013
Peer Review: A Flawed Process at the Heart of Science and Journals
Smith, R. (2006) · Journal of the Royal Society of Medicine · doi:10.1177/014107680609900414 · Cited by 880
Dickersin, K. (1992) · JAMA · doi:10.1001/jama.1992.03480030052036 · Cited by 854
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
Open Science: One Term, Five Schools of Thought
Fecher, B., & Friesike, S. (2014) · Opening Science · doi:10.1007/978-3-319-00026-8_2 · Cited by 654
Ioannidis, J. P. A. (1998) · JAMA · doi:10.1001/jama.279.4.281 · Cited by 632
Research works printed but not linked: 7
- T Bloom, 2017, Referee Report For: What is open peer review? A systematic review [version 1; referees: 1 approved, 3 approved with reservations]., F1000Res., 6, 588, 10.5256/f1000research.12273.r22301 · doi:10.5256/f1000research.12273.r22301
- D Chubin, 1990, Peerless Science: Peer Review and US Science Policy.
- 2016, About | The EMBO Journal [WWW Document]
- K Fitzpatrick, 2012, Open Review, A Study of Contexts and Practices
- S Kaplan, 2015, Major Publisher Retracts 64 Scientific Papers in Fake Peer Review Outbreak.
- M Ware, Peer Review: Benefits, Perceptions and Alternatives, 2008
- M Ware, 2016, Peer Review Survey 2015
Cited by
The limitations to our understanding of peer review
Tennant, J. P., & Ross-Hellauer, T. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00092-1 · Cited by 384
Self-citationThe evaluation of scholarship in academic promotion and tenure processes: Past, present, and future
Schimanski, L. A., & Alperin, J. P. (2018) · F1000Research · doi:10.12688/f1000research.16493.1 · Cited by 342
Checco, A., Bracciale, L., Loreti, P., Pinfield, S., & Bianchi, G. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-020-00703-8 · Cited by 286
The effect of publishing peer review reports on referee behavior in five scholarly journals
Bravo, G., Grimaldo, F., López-Iñesta, E., Mehmani, B., & Squazzoni, F. (2019) · Nature Communications · doi:10.1038/s41467-018-08250-2 · Cited by 273
The changing forms and expectations of peer review
Horbach, S. P. J. M., & Halffman, W. (. W. (2018) · Research Integrity and Peer Review · doi:10.1186/s41073-018-0051-5 · Cited by 179
Metadata
- Authors
- Ross-Hellauer, T.
- Venue
- F1000Research
- Publisher
- F1000 Research Ltd
- Published
- 2017-08-31
- Volume
- 6
- Pages
- 588
- ISSN
- 2046-1402 (ISSN-L)
- OpenAlex
- W2609499779
- Cited by
- 449
- References
- 73
- FWCI
- 9.62
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- European Commission (643410)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What is open peer review? A systematic review. https://app.cochat.ai/papers/10.12688/f1000research.11369.2
- The paper
- Ross-Hellauer, T. (2017). What is open peer review? A systematic review. F1000Research. https://doi.org/10.12688/f1000research.11369.2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.