CoChat

Peer review for improving the quality of grant applications

Open access
Demicheli, Vittorio, Di Pietrantonj, Carlo · 2007 · Cochrane Database of Systematic Reviews
Cited by 99

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Grant giving relies heavily on peer review for the assessment of the quality of proposals but the evidence of effects of these procedures is scarce. OBJECTIVES: To estimate the effect of grant giving peer review processes on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research. SEARCH STRATEGY: Electronic database searches and citation searches; researchers in the field were contacted. SELECTION CRITERIA: Prospective or retrospective comparative studies with two or more comparison groups assessing different interventions or one intervention against doing nothing. Interventions may regard different ways of screening, assigning or masking submissions, different ways of eliciting opinions or different decision making procedures. Only original research proposals and quality outcome measures were considered. DATA COLLECTION AND ANALYSIS: Studies were read, classified and described according to their design and study question. No quantitative analysis was performed. MAIN RESULTS: Ten studies were included. Two studies assessed the effect of different ways of screening submissions, one study compared open versus blinded peer review and three studies assessed the effect of different decision making procedures. Four studies considered agreement of the results of peer review processes as the outcome measure. Screening procedures appear to have little effect on the result of the peer review process. Open peer reviewers behave differently from blinded ones. Studies on decision-making procedures gave conflicting results. Agreement among reviewers and between different ways of assigning proposals or eliciting opinions was usually high. AUTHORS' CONCLUSIONS: There is little empirical evidence on the effects of grant giving peer review. No studies assessing the impact of peer review on the quality of funded research are presently available. Experimental studies assessing the effects of grant giving peer review on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research are urgently needed. Practices aimed to control and evaluate the potentially negative effects of peer review should be implemented meanwhile.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/14651858.mr000003.pub2

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 99

Metadata

Authors
Demicheli, V., & Di Pietrantonj, C.
Venue
Cochrane Database of Systematic Reviews
Publisher
Wiley
Published
2007-04-18
Volume
2010
Issue
1
Pages
MR000003-MR000003
ISSN
  • 1361-6137 (ISSN-L)
  • 1465-1858
OpenAlex
W1504922255
Cited by
99
References
2
FWCI
7.53
Type
journal article
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.