The oracles of science: On grant peer review and competitive funding

“…this essay by highlighting something that sociologists of science and science and technology studies (STS) scholars have been aware of for quite some time, namely that experts within a certain research field rely on their developed sense of taste (Merton, 1973) or ‘skilled academic intuition’.”

“In my view, there is no point in trying to convince ourselves that evaluation of grant proposals is a rational, fair and reliable practice; there is ample empirical evidence showing exactly the opposite (e.g. Jerrim and De Vries, 2020; Pier et al., 2018; Van den Besselaar et al., 2018).”

“…may nonetheless offer an important source of intellectual exchange between researchers – an ongoing ‘organized skepticism’ is taking place, for example, during seminars, conferences and workshops, and in journal peer review (cf. Brezis and Birukou, 2020; Fyfe et al., 2020; Teplitskiy et al., 2018).”

“…may nonetheless offer an important source of intellectual exchange between researchers – an ongoing ‘organized skepticism’ is taking place, for example, during seminars, conferences and workshops, and in journal peer review (cf. Brezis and Birukou, 2020; Fyfe et al., 2020; Teplitskiy et al., 2018).”

“And many original and idiosyncratic scholars who do not fit into the ‘entrepreneurial culture’, and have difficulty learning the art of ‘grantsmanship’ (Roumbanis, 2019a), will often struggle in vain.”

“The problematic consequences of the excessive use of quantitative metrics and performance indicators have already been thoroughly examined by many scholars, including Olof Hallonsten (2021).”

“In my view, there is no point in trying to convince ourselves that evaluation of grant proposals is a rational, fair and reliable practice; there is ample empirical evidence showing exactly the opposite (e.g. Jerrim and De Vries, 2020; Pier et al., 2018; Van den Besselaar et al., 2018).”

“…may nonetheless offer an important source of intellectual exchange between researchers – an ongoing ‘organized skepticism’ is taking place, for example, during seminars, conferences and workshops, and in journal peer review (cf. Brezis and Birukou, 2020; Fyfe et al., 2020; Teplitskiy et al., 2018).”

“It has been argued that ambiguity, and the uncertainty that follows from ambiguity, can sometimes play a distinct role in the creative development of science by creating zones of intellectual engagement (McMahan and Evans, 2018).”

“But the true quality of a research project can never be fully observed in a proposal because it is impossible to predict future discoveries and breakthroughs in science (see, e.g., Roumbanis 2021).”

“The limitation lies in the subjective nature of evaluation, where experts may not always grasp a project’s full scope or potential solely from its written description [28].”

“There is widespread acknowledgement that resource allocation processes are flawed, idiosyncratic or opaque (Fang & Casadevall, 2016; Herbert et al., 2013; Moore et al., 2017; Roumbanis, 2021).”

“In other cases, a return to substantial public funding ‘with no strings attached’ is explicitly advocated as a solution for the problems plaguing contemporary academia – including those stemming from CPBF itself (Roumbanis, 2021; see also Hallonsten, 2021).”

“There have been ethical criticisms of the status quo allocation of academic capital (Conix et al., 2021; Roumbanis, 2021) and alternative ways in which capital can be distributed have been created, largely attempting to restore fairness into the process (Chawla, 2021).”

“Academic research fundamentally involves uncertainty, so it is not possible through any methods to predict the most successful research to fund in advance (Roumbanis 2021).”

“Within competitive funding schemes, funding organizations and agencies, as noted in [2], expect the expert community to address complex issues re-lated to assessing project prospects and forecas ting the future success of their outcomes.”