Peer review for improving the quality of grant applications
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Grant giving relies heavily on peer review for the assessment of the quality of proposals but the evidence of effects of these procedures is scarce. OBJECTIVES: To estimate the effect of grant giving peer review processes on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research. SEARCH STRATEGY: Electronic database searches and citation searches; researchers in the field were contacted. SELECTION CRITERIA: Prospective or retrospective comparative studies with two or more comparison groups assessing different interventions or one intervention against doing nothing. Interventions may regard different ways of screening, assigning or masking submissions, different ways of eliciting opinions or different decision making procedures. Only original research proposals and quality outcome measures were considered. DATA COLLECTION AND ANALYSIS: Studies were read, classified and described according to their design and study question. No quantitative analysis was performed. MAIN RESULTS: Ten studies were included. Two studies assessed the effect of different ways of screening submissions, one study compared open versus blinded peer review and three studies assessed the effect of different decision making procedures. Four studies considered agreement of the results of peer review processes as the outcome measure. Screening procedures appear to have little effect on the result of the peer review process. Open peer reviewers behave differently from blinded ones. Studies on decision-making procedures gave conflicting results. Agreement among reviewers and between different ways of assigning proposals or eliciting opinions was usually high. AUTHORS' CONCLUSIONS: There is little empirical evidence on the effects of grant giving peer review. No studies assessing the impact of peer review on the quality of funded research are presently available. Experimental studies assessing the effects of grant giving peer review on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research are urgently needed. Practices aimed to control and evaluate the potentially negative effects of peer review should be implemented meanwhile.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Chance and Consensus in Peer Review
Cole, S., Cole, J. R., & Simon, G. A. (1981) · Science · doi:10.1126/science.7302566 · Cited by 623
Cicchetti, D. V. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065675 · Cited by 496 · cited 2×
“Cicchetti 1991 {published data only} Cicchetti D.”
Background
Cited by
Understanding current causes of women's underrepresentation in science
Ceci, S. J., & Williams, W. M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1014871108 · Cited by 1,114
An evidence-based practice guideline for the peer review of electronic search strategies
Sampson, M., McGowan, J., Cogo, E., Grimshaw, J., Moher, D., & Lefebvre, C. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.012 · Cited by 608
Bornmann, L. (2011) · Annual Review of Information Science and Technology · doi:10.1002/aris.2011.1440450112 · Cited by 567
The limitations to our understanding of peer review
Tennant, J. P., & Ross-Hellauer, T. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00092-1 · Cited by 384
“As such, there is scope for extending what we have done to other forms of peer review, including for grants and clinical trials [111, 112].”
BackgroundMarsh, H. W., Bornmann, L., Mutz, R., Daniel, H.-D., & O’Mara, A. (2009) · Review of Educational Research · doi:10.3102/0034654309334143 · Cited by 174
Metadata
- Authors
- Demicheli, V., & Di Pietrantonj, C.
- Venue
- Cochrane Database of Systematic Reviews
- Publisher
- Wiley
- Published
- 2007-04-18
- Volume
- 2010
- Issue
- 1
- Pages
- MR000003-MR000003
- ISSN
- 1361-6137 (ISSN-L)
- 1465-1858
- OpenAlex
- W1504922255
- Cited by
- 99
- References
- 2
- FWCI
- 7.53
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Peer review for improving the quality of grant applications. https://app.cochat.ai/papers/10.1002/14651858.mr000003.pub2
- The paper
- Demicheli, V., & Di Pietrantonj, C. (2007). Peer review for improving the quality of grant applications. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.mr000003.pub2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.