Peer Review: Recent Experience and Future Directions
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
What are the challenges and opportunities facing peer review in a networked world? We review the state of journal peer review at the end of 2010, beset by ever-increasing volumes of journal articles demanding to be reviewed in ever-shorter times and by a range of criticisms of its effectiveness and value. We consider the opportunities for improvement and review alternative approaches including open and post-publication review and the possible use of article-level metrics as a replacement. We conclude that far from being in crisis, peer review has remains widely supported and diversely innovative.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 1×
“The reasons given for preferring double-blind review were principally to do with objectiveness and fairness, reducing the scope for various kinds of bias.”
Godlee, F., Gale, C. R., & Martyn, C. N. (1998) · JAMA · doi:10.1001/jama.280.3.237 · Cited by 333
Editorial peer review for improving the quality of reports of biomedical studies
Jefferson, T., Rudin, M., Brodney Folse, S., & Davidoff, F. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000016.pub3 · Cited by 312 · cited 2×
“The same study reported that peer-reviewed journal articles were considered “very important” information sources by 92% of researchers compared to 4% giving that rating to “un-refereed articles”.”
Bornmann, L., Mutz, R., & Daniel, H.-D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0014331 · Cited by 290 · cited 1×
“More encouragingly, although there has been tentative evidence for gender bias in the past, more recent studies (Marsh et al. 2008; Marsh and Bornmann 2010) have shown gender to have no effect on outcome.”
Is open peer review the fairest system? No
Khan, K. (2010) · BMJ · doi:10.1136/bmj.c6425 · Cited by 77
The luck of the referee draw: the effect of exchanging reviews
BORNMANN, L., & DANIEL, H. (2009) · Learned Publishing · doi:10.1087/2009207 · Cited by 51
Reliability of reviewers' ratings when using public peer review: a case study
BORNMANN, L., & DANIEL, H. ‐. (2010) · Learned Publishing · doi:10.1087/20100207 · Cited by 49
Cited by
What is open peer review? A systematic review
Ross-Hellauer, T. (2017) · F1000Research · doi:10.12688/f1000research.11369.2 · Cited by 449
Peer review in a changing world: An international study measuring the attitudes of researchers
Mulligan, A., Hall, L., & Raphael, E. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22798 · Cited by 393
“05) by Ware and Monkman (2008). When these scores are broken out by subject area of the respondent, there are some distinct differences between subgroup and overall group.”
BackgroundInfluentialA multi-disciplinary perspective on emergent and future innovations in peer review
Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., Elkhatib, Y., B. Collister, L., Pikas, C. K., Crick, T., et al. (2017) · F1000Research · doi:10.12688/f1000research.12037.1 · Cited by 289
What is open peer review? A systematic review
Ross-Hellauer, T. (2017) · F1000Research · doi:10.12688/f1000research.11369.1 · Cited by 232
“Reviewers can come from the wider research community, as well as those traditionally under-represented in scientific assessment, including representatives from industry or members of special-interest groups, for example patients in the case of medical journals (Ware, 2011).”
BackgroundInfluentialCan editors save peer review from peer reviewers?
D’Andrea, R., & O’Dwyer, J. P. (2017) · PLOS ONE · doi:10.1371/journal.pone.0186111 · Cited by 80
“While some say peer review is broken [7, 8] and in need of replacement, the majority of scholars still hold it in high regard, and, while acknowledging the system’s imperfections, advocate evidencebased efforts to improve it [18, 35].”
Background
Metadata
- Authors
- Ware, M.
- Venue
- New Review of Information Networking
- Publisher
- Informa UK Limited
- Published
- 2011-05
- Volume
- 16
- Issue
- 1
- Pages
- 23-53
- ISSN
- 1361-4576 (ISSN-L)
- 1740-7869
- OpenAlex
- W1993066822
- Cited by
- 94
- References
- 7
- FWCI
- 38.05
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.