What is open peer review? A systematic review

“” (Suber, 2012) Page 4 of 32 Within just these two examples, there are already a multitude of factors at play, including the removal of anonymity, the publishing of review reports, interaction between participants, crowdsourcing of reviews, and making manuscripts public pre-review, amongst others.”

“Reviewers often fail to detect major methodological failings (Schroter et al., 2004), with eminent journals (whose higher rejection rates might suggest more stringent peer review processes) seeming to perform no better than others (Fang et al., 2012).”

“Nickerson et al. (2013) advise that the development of a taxonomy should begin by identifying the appropriate meta-characteristic – in this case distinct individual innovations to the traditional peer review system.”

“Published reports can enable readers to consider these criticisms themselves, and “have a chance to examine and appraise this process of ”creative disagreement” and form their own opinions” (Peters & Ceci, 1982).”

“Next, “confirmatory bias” is theorized to lead to conservatism, biasing reviewers against innovative methods or results contrary to dominant theoretical perspectives (Chubin & Hackett, 1990; Garcia et al., 2016; Mahoney, 1977).”

“, 2017) mean that editors and reviewers seem primed to prefer positive results over negative or neutral ones (Bardy, 1998; Dickersin et al., 1992; Fanelli, 2010; Ioannidis, 1998), and to disfavour replication studies (Campanario, 1998; Kerr et al.”

“Smith, an experienced editor, for example, reports reviewers stealing ideas and passing them off as their own, or intentional blocking or delaying publication of competitors’ ideas through harsh reviews (Smith, 2006).”

“, 2017) mean that editors and reviewers seem primed to prefer positive results over negative or neutral ones (Bardy, 1998; Dickersin et al., 1992; Fanelli, 2010; Ioannidis, 1998), and to disfavour replication studies (Campanario, 1998; Kerr et al.”

“…of “impact” and “excellence” (Moore et al., 2017) mean that editors and reviewers seem primed to prefer positive results over negative or neutral ones (Bardy, 1998; Dickersin et al., 1992; Fanelli, 2010; Ioannidis, 1998), and to disfavour replication studies (Campanario, 1998; Kerr et al., 1977).”

“Ross-Hellauer (2017) apresenta uma sistematização dessas modalidades que podem ser adotadas de forma isolada ou combinada: Identidades abertas - autores e revisores conhecem a identidade uns dos outros; Relatórios abertos - os pareceres são publicados junto ao artigo avaliado; Participação aberta -…”

“…common complaints about academia, including those focused on lack of reproducibility (Open Science Collaboration, 2015), problems with peer review (Ross-Hellauer, 2017; Smith, 2006; Tennant et al., 2017), and the lack of access to research, could conceivably be reduced by building mechanisms that…”

“Most notably, open peer review is being more widely introduced as an alternative paradigm of interaction between authors and reviewers (Ford, 2013; Ross-Hellauer, 2017).”

“This study did not include journals that implemented only one of the following OPR elements in Ross-Hellauer (2017): open participation, open interaction, open pre-review manuscripts, open final-version commenting and open platforms/decoupled review.”

“Following [16], we can summarise these proposed benefits and drawbacks thus: • Unreliability and inconsistency: Open identities and open reports are theorized to lead to better reviews, as the thought of having their name publicly connected to a work or seeing their review published encourages…”

“Greater transparency in processes of evaluation might eliminate bias in selection procedures [50,51].”

“This is sometimes referred to as “ open identities ” peer review, or “ no-blinding ” [30, 31].”

“Published and OA review reports are one of a few simple solutions to allow any reader or potential author to directly assess both quality and efficiency of the review system of any given journal, and the value for money of the requested APCs; thus evaluating whether a journal has ‘deceptive’ or predatory practices [42…”

“Equally long as the litany of debated issues is the list of proposed solutions, including alternatives to traditional peer review such as preprint repositories [11], double-blind peer review [12], and open peer review [13, 14, 15].”

“One of these is open peer review, for which there is, however, yet no commonly accepted and clear definition nor an agreed schema as elaborated in a secondary study by Ross-Hellauer (2017).”

“A further strand of innovation is described as ‘open peer review’, which encompasses different ways in which peer review can be opened (Ross-Hellauer, 2017).”

“Similar to the anonymity of the author, some discussions regarding peer review procedures have centred around the anonymity of the reviewer (Amsen 2014; Ross-Hellauer 2017).”

“Open peer review, in which reviewer identities and review reports are published, is a clear example (Ross-Hellauer, 2017).”

“Most notably, these models include diversity in the level of anonymity of both authors and reviewers (Pontille and Torny 2014), leading to systems ranging from doubleor even triple-blind review to radically open peer-review models (Ross-Hellauer 2017).”

“One of these is open peer review, for which there is, however, yet no commonly accepted and clear definition nor an agreed schema as elaborated in a secondary study by Ross-Hellauer (2017).”

“Open Peer-Review (OPR) mainly refers to ‘open identities’ of authors and reviewers (Ford 2013), but can also refer to ‘open reports’ (making peer-review reports publicly available), ‘open participation’ (crowdsourced peer-review) and other open review practices (Ross-Hellauer 2017).”

“For example, they could implement open peer review (pre- and post-publication), which tends to encourage more respectful and constructive feedback as well as clearer communication between reviewers, editors, and authors (Ross-Hellauer, 2017).”

“It might be argued that open and/or transparent peer review provides an alternative mechanism for reducing reviewer bias (Ross-Hellauer, 2017).”

“This variant addresses the problems many authors face when receiving contradictory reviews of their manuscripts and has already been implemented by the journal eLife (Schekman et al., 2013).”

“2017), the common characteristics of OPR are as follows (Ross-Hellauer 2017): open identities (both reviewers and authors are known to each other), open peer reports, open participation, open interaction, open prereview manuscripts, open final-version commenting and open platforms.”

“Ross-Hellauer [82] discusses a variety of problems with peer review, most of which affect the realms of quality and credibility fundamentally.”

“The space of possibilities for such reforms is large, and includes outsourcing peer-review to highly-trained experts [70], harnessing the wisdom of crowds via various forms of open peer-review [71], allowing scientists to evaluate each other’s peer-review quality [72], and supplementing peer review with "red teams" of…”

“, [6, 7]), we will use “open review” to refer to all reviews that are publicly available, whether anonymous or signed.”

“Lastly, to increase the transparency of the review process itself, journals and publishers are increasingly trialing “open peer review models,” including making reviewer and author identities known, publishing review reports alongside articles, and crowdsourcing participation in the peer review process (Ross-Hellauer,…”

“Of note, Ross-Hellauer [69], who conducted an assessment of definitions of OPR in the literature, found as many as 22 separate definitions, although ultimately, “optimal” OPR involved the following seven characteristics: (1) open identities in which “authors and reviewers are aware of each other’s…”

“8 Many definitions and understandings of the notion of open peer review have been offered (Ross-Hellauer 2017).”

“Indeed, as Ross-Hellauer (2017) notes, as reviewer comments often add context to the scholarly article, point to areas for future work, and even expose areas of tension in a theory or argument, they are considered potentially useful and re-usable scholarly information.”

“…accessible, interoperable and re-usable) sharing of data and code [5–8], transparency of processes and methods [9–12], transparent models of peer review [13,14] and engaging publics and others beyond academia (e.g. civil society, policymakers, industry) in participatory research processes [15–17].”

“Current reform initiatives concern open review (in which the reviews are publicly accessible; Ross-Hellauer, 2017), double-blind review (in which the reviewers do not know the identity of the authors; Snodgrass, 2006), or noncommercial peer review that is independent from journals.”

“The open peer review is based on the open participation of the wider scientific community (Ross-Hellauer, 2017).”

“In some cases, open peer-review is used to denote the practice of publishing peer-review reports alongside a publication (Ross-Hellauer 2017).”

“It is important to note that the adoption of the practice of publishing the peer-review reports is a positive development (Ross-Hellauer, 2017), leading to more openness and transparency, without which the detection of peer-review plagiarism reported here would not be possible.”

“open peer review (Ross-Hellauer, 2017), open grant writing and open evaluation.”

“In addition to biases, many other supposed, corroborated, or obvious phenomena can be found in the literature, such as overburdening (Kovanis, Porcher et al., 2016) or lack of transparency (Horbach, Hepkema, & Halffman, 2020; Ross-Hellauer, 2017).”

“Although the term ‘open peer review’ has no standardised definition, scholars use the term to indicate that the identities of the authors and reviewers are disclosed and that reviewer reports are openly available [13].”

“A number of journals have shifted to processes that unmask reviewers’ identity as either an optional or required part of peer review [19].”

“There is already a rich literature about this model, 67 and it has been tested experimentally by different initiatives (e.g. F1000Research, ScienceOpen, The Winnower).”

“The lack of accountability for reviewers (and the possibility of biased reviews) in the current peer-review system remains a major problem in academia [43–47] and taxonomy is no exception.”

“According to Ross-Hellauer (2017), OPR refers to the openness in one or several of the elements in the peer review process, including authors’ or reviewers’ identities, review content, participation of other actors, preview manuscripts, final-version commenting, the interaction between authors and…”

“…to accept requests to review for journals that have adopted open review practices, such as unblinded review, publication of reviews alongside the final article, or direct interaction between authors and reviewers during the review process (see Ross-Hellauer, 2017; Walker & Rocha da Silva, 2015).”

“…identities of reviewers and authors in the review of a scientific article, but it can also mean open sharing of reviewer comments and even allowing the public to participate in the review process (Ross-Hellauer, 2017; Ross-Hellauer & Görögh, 2019; Teixeira da Silva, 2018; Thelwall et al., 2021).”

“Open-identities peer review is not currently widespread [47], but is commonly promoted as a means to increase transparency and accountability in the peer-review process.”

“Besides a blind peer‐review policy (in which reviewers do not know the authors and their affiliation), journals are also experimenting with more open approaches to peer reviews, such as making the identity of peer reviewers and even the content of reviews openly available (Ross‐Hellauer, 2017).”

“Ross-Hellauer (2017) considered open peer review as an inclusive academic keyword.”

“…discrepancies and differences about what open peer review means and the different Quantitative Science Studies 939 Wikinformetrics models proposed (Ross-Hellauer, 2017), the three basic principles (open identities, reports, and participation) are clearly recognizable in Wikipedia (Table S2 in the…”

“Thus, the development of the different interpretations of what has been called ‘openness’ with varying levels of ‘transparency’ (Ross-Hellauer, 2017).”

“…today, approaches to author identification can run the gamut, from overtly emphasizing anonymity of authors (as in the “ Journal of Controversial Ideas ”) [70, 77],valuing the traditional double blind review process [64],to models of more radical transparency (such as Open Peer Review 12 ) [71].”

“Preprint servers facilitate more public discussion and criticism of works, and allow findings to become immediately available to other researchers (Ross-Hellauer 2017, Elmore 2018).”

“Supervisors can reinforce these principles by training graduate students and postdocs in practices such as contributorship (Holcombe, 2019) and encouraging publications in RRs or diamond/green OA journals with transparent peer review (Ross-Hellauer, 2017).”