Grant Peer Review Scoring Data with Criteria Scores

Abstract

The available data come from a scientific peer review of biomedical applications from a collaborative biomedical research funding program (2014-2017). Three assigned individual reviewers were asked to provide scores and commentary for the following application criteria: Innovation, Approach/Feasibility, Investigator, and Significance (Impact added as scored criteria in 2014). Each of these criteria is scored on a scale from 1.0 (best) to 5.0 (worst) with a 0.1 gradation, as well as an Overall Score (1-5). Asynchronous discussion was allowed, although few changed scores post-discussion. The data includes reviewers’ self-reported expertise scores (1 is high expertise) relative to each proposal reviewed, and reviewer/PI demographics. A total of 72 applications (“Standard” or “Pilot”) were reviewed in 3 review cycles. The success rate was 34-38%. Standard projects received a maximum of $450k over 3 years, Pilot projects received $200k max direct costs over 2 years. Application scores indicate where each application falls among all practically possible applications in comparison with the ideal standard of quality from a perfect application. No live panel discussion takes place in this review mechanism; the final proposal score is the average of the three reviewers’ scientific merit scores and is the primary factor in deter-mining AIBS funding decisions.