CoChat

Peer Review: A System under Stress

Gropp, Robert E., Glisson, Scott, Gallo, Stephen, Thompson, Lisa · 2017 · BioScience
Cited by 49

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Abstract

The American Institute of Biological Sciences (AIBS) convened a meeting in Washington, DC, on 6 December 2016 to explore The Role of Peer Review in Informed Decision-making. The participants included representatives from AIBS member organizations, government agencies, research funders, scholarly publishers, and scholars of peer review. In recent years, there have been a number of high-profile examples of flawed or fraudulent research being published. Simultaneously, many funders have been pressed to limit the amount of funds expended on peer review. Adding to these challenges have been an increase in the number of for-profit journals, the frequency of requests of scientists to review manuscripts, and interdisciplinary research that can strain the capacity of peer-review panel members. Peer-review managers have also been challenged to reduce bias and maintain the integrity of the process. To further strain the system, policymakers and the public increasingly demand transparency and open access to data and publications. Despite some flaws, the peer-review system has served us well. Of significant concern, however, is a growing belief that the system is under stress. Agencies are struggling to review increasing numbers of research applications, and journals are struggling to find a third reviewer for articles. The scientific community must allocate more attention to these issues, particularly to the need for improved training, new expectations for scientific community members, and greater research into the factors that inform and influence peer review. The scientific community regularly discusses the importance of research and education for improving the human condition through the creation of new knowledge; informed decision­making; and the development of new applications that enhance human health, environmental management, and food security. It is with less frequency that we consider the elements of the scientific process that stimulate or hinder these outcomes. The peer-, or merit-, review process is considered by the overwhelming majority of the scientific community to be an important tool for identifying quality science. It is through our community policing and review practices that we identify important work, sleuth out the rare—but present—huckster, and fuel the engines that power the creation of new knowledge. Despite our reliance on peer review, the factors that influence the process and its outcomes are not well understood (Rennie 2016). When it comes to scientific peer review, relatively few scholarly reviews of the factors that influence decision-making have been published. Importantly, this lack of information has led some lawmakers to question the validity of peer review. On 6 December 2016, AIBS convened a meeting of leaders from scientific societies and research organizations, journal publishers and editors, funding organizations, and government for an open dialog intended to improve our collective understanding of the status of peer review. The workshop-style meeting considered the following four overarching questions: (1) Do we know what we need to know about peer review? (2) Does peer review meet the needs of the modern publication system? (3) How effectively does peer review identify the best research proposals? (4) Is peer review properly incentivized, and are reviewers being asked to evaluate the right things? To set the stage for the day's deliberations, Dr. Drummond Rennie provided a historical overview of peer review in the United States. In addition to drawing attention to situations where peer review of both funding applications and publications has worked less than optimally, he also highlighted efforts by funders, editors, and scholars to conduct rigorous investigations into peer review. The balance of the meeting featured panel and group discussions. Because the purpose of this meeting was to foster an open dialogue, this report does not attribute specific remarks to individual meeting participants or sponsoring organizations. Rather, the following section provides a general synthesis of the discussions. A general recognition shared by scientists and managers of peer-review processes alike is that the system is under stress. If the strain is not ameliorated, there is a real concern about how much longer it can function before its foundation is significantly damaged. AIBS President Dr. Joseph Travis offering opening remarks. Photograph: Julie Palakovich Carr. Notably, journal editors are increasingly challenged to secure an adequate number of reviewers for publications, particularly in smaller, more specialized fields. Some hypothesize that the proliferation of journals and demands for rapid publication are taxing the community's capacity to provide reviews. As scientists increasingly turn down invitations to review because of limited time, many journal editors are also beginning to worry that the reviewers who are conducting these evaluations may not be allocating the requisite ti

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/biosci/bix034

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 5 references

Cited by

Metadata

Authors
Gropp, R. E., Glisson, S., Gallo, S., & Thompson, L.
Venue
BioScience
Published
2017-05
Volume
67
Issue
5
Pages
407-410
ISSN
  • 0006-3568 (ISSN-L)
  • 1525-3244
OpenAlex
W2618665414
Cited by
49
References
5
FWCI
18.13
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.