Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review

Probably not a substantial claim: may not pass S3 (load-bearing).

A lack of incentives for reviewers is a hurdle to rigorous testing or validation of studies under review.

“a lack of incentives for reviewers has been introduced as a hurdle to rigorous testing or validation of studies under review [5]”

• cites [5] Tennant JP, Ross-Hellauer T. The limitations to our understanding of peer review. Res Integr Peer Rev. 2020;5(1):6. sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Partly, automated screening and compliance checking are already done, but future LLMs could enhance these applications.

“Partly, this is already done [3, 4] but future LLMs could enhance these applications.”

• cites [3] Schulz R, Barnett A, Bernard R, Brown NJL, Byrne JA, Eckmann P, et al. Is the future of peer review automated? BMC Res Notes. 2022;15(1):203. sourceSupported
• cites [4] Weissgerber T, Riedel N, Kilicoglu H, Labbé C, Eckmann P, ter Riet G, et al. Automated screening of COVID-19 preprints: can we help authors to improve transparency and reproducibility? Nat Med. 2021;27(1):6–7. sourceSupported

May combine two claims.

The feasibility and usefulness of using LLMs to improve writing in academic contexts has already been demonstrated.

“The feasibility and usefulness of this approach has already been demonstrated in academic contexts [7].”

• cites [7] Pividori M, Greene CS. A publishing infrastructure for AI-assisted academic authoring. bioRxiv; 2023 [cited 2023 Jan 31]. p. 2023.01.21.525030. Available from: https://www.biorxiv.org/content/10.1101/2023.01.21.525030v1 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Currently, several journals have attempted to develop guidelines.

“Currently, several journals have attempted to develop guidelines [28, 29].”

• cites [28] Hosseini M, Rasmussen LM, Resnik DB. Using AI to write scholarly publications. Account Res. 2023;0(0):1–9. sourceSupported
• cites [29] Thorp HH. ChatGPT is fun, but not an author. Science. 2023;379(6630):313–313. sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Peer review contributes to the quality of manuscripts, filters out poor or problematic science, and improves arguments and gaps in logic in a collegial and constructive manner.

“Apart from contributing to the quality of manuscripts and filtering out poor or problematic science and improving “arguments and gaps in logic” in a collegial and constructive manner [22], peer review also has important social functions.”

• cites [22] Woods HB, Brumberg J, Kaltenbrunner W, Pinfield S, Waltman L. An overview of innovations in the external peer review of journal manuscripts. Wellcome Open Res. 2023;7:82. https://doi.org/10.12688/wellcomeopenres.17715.2. sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Currently, ChatGPT does not seem very capable of performing the task of finding suitable reviewers.

“Currently, ChatGPT does not seem very capable of performing this task”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The system is very sensitive to minor differences in prompts.

“In addition, we note that the system is very sensitive to minor differences in prompts.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The introduction of novel LLMs has the potential to move automation of the peer-review process much further.

“the introduction of novel LLMs has the potential to move automation much further”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLMs are likely to have a profound impact on academia and scholarly communication.

“We believe that LLMs are likely to have a profound impact on academia and scholarly communication.”

May combine two claims.

Using LLMs to review unpublished manuscripts may inadvertently incorporate data or ideas into training data or generate responses that may disclose authors' identity in closed peer-review processes.

“When using LLMs to review unpublished manuscripts, however, the LLM may inadvertently incorporate data or ideas to its training data or generate responses that may disclose authors' identity (in closed peer-review processes).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Freeing up reviewers' time can improve productivity.

“freeing up reviewers’ time can improve productivity”

Probably not a substantial claim: may not pass S4 (based).

The quality of reviewed work could be improved when more feedback is delivered in a timely fashion.

“The quality of reviewed work could be improved when more feedback is delivered in a timely fashion”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Since LLMs can support reviewers to write better reviews and submit their report more quickly, editors would likely have access to a larger and potentially more diverse and efficient pool of candidate reviewers.

“Since LLMs can support reviewers to write better reviews and submit their report more quickly, editors would likely have access to a larger and potentially more diverse and efficient pool of candidate reviewers.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

LLMs could help researchers conduct repetitive or tedious writing tasks more efficiently, thereby improving the likelihood of having more time for testing or assessing studies more rigorously.

“It is possible that LLMs could help researchers conduct repetitive or tedious writing tasks more efficiently (e.g., editing manuscripts to minimize the use of jargon, correcting grammar and spelling mistakes, or changing the voice from passive to active) [8] thereby improving the likelihood to have more time for testing or assessing studies more rigorously.”

• cites [8] Stokel-Walker C, Van Noorden R. What ChatGPT and generative AI mean for science. Nature. 2023;614(7947):214–6. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

LLMs have the potential to substantially alter the role of both peer reviewers and editors.

“LLMs have the potential to substantially alter the role of both peer reviewers and editors.”

Probably not a substantial claim: may not pass S4 (based).

Because of certain shortcomings of AI, it can provide biased views.

“but because of certain shortcomings of AI, we know that it can provide biased views”

Probably not a substantial claim: may not pass S4 (based).

LLMs can facilitate higher quality review.

“LLMs can facilitate higher quality review”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Even if a basic version remains freely available, more sophisticated versions with better functionality may become available to researchers/universities who can afford it.

“Even if a basic version would remain freely available, it is possible that more sophisticated versions with better functionality would become available to researchers/universities who can/will afford it.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

These systems are necessarily conservative, favoring the status-quo and potentially skewed approaches already present in current and past discourse.

“In essence, these systems are necessarily conservative, favoring the status-quo and potentially skewed approaches already present in current and past discourse.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

While ChatGPT is currently freely available, it is unclear what business model will be chosen by its future investors, thereby introducing further accessibility inequalities.

“In addition, while ChatGPT is currently freely available, it is unclear what business model will be chosen by its future investors, thereby introducing further accessibility inequalities”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

LLMs could be used in initial screening of manuscripts, for instance to assess fit with journal scope or general quality.

“First, LLMs could be used in initial screening of manuscripts, for instance to assess fit with journal scope or general quality.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

As the final stage of editorial work arguably involves little original contribution from the editor, it is an obvious part of the editorial process that LLMs, even in their current state, can already contribute to.

“As this stage arguably involves little original contribution from the side of the editor, it is an obvious part of the editorial process that LLMs, even in their current state, can already contribute to.”

• AssumptionRests on an assumption: The final stage of editorial work is a core task of editors and potentially takes up a significant amount of their time.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

LLMs could either automatically perform triage or support editorial staff to perform inspections more efficiently.

“LLMs could either automatically perform triage (e.g., initial quality checks to filter or flag problematic research), or support editorial staff to perform such inspections more efficiently.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can support both peer reviewers and editors in efficiently writing constructive reports or decision letters.

“Through supporting both actors in efficiently writing constructive reports or decision letters”

Probably not a substantial claim: may not pass S2 (checkable).

Partly outsourcing editorial work to LLMs might have unforeseen consequences for social and epistemic relations within academia.

“partly outsourcing this work to LLMs might have unforeseen consequences for social and epistemic relations within academia”

• AssumptionRests on an assumption: Editorial work has a prominent function in defining and shaping epistemic communities and negotiating normative frameworks within such communities.

Probably not a substantial claim: may not pass S4 (based).

Using LLMs in various writing tasks, including writing peer review reports, could result in improved productivity.

“Since LLMs generate grammatically correct and mostly relevant (yet sometimes outright wrong, irrelevant or biased) outputs in response to provided prompts, using them in various writing tasks including writing peer review reports could result in improved productivity.”

• AssumptionRests on an assumption: LLMs generate grammatically correct and mostly relevant, but sometimes wrong, irrelevant or biased, outputs in response to provided prompts.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

As LLMs are trained on existing data by mostly biased humans, it is unclear whether or how these systems can mitigate existing biases, with the risk of reproducing or amplifying them unless human agents are educated and aware of possible ways of addressing these biases.

“As LLMs are trained on existing data by (mostly) biased humans, it is unclear whether or how these systems can mitigate existing biases, with the risk of in fact reproducing or amplifying them unless human agents are educated and aware of possible ways of addressing these biases.”

• AssumptionRests on an assumption: LLMs are trained on existing data by mostly biased humans.

May combine two claims.

Editors struggle to find sufficiently qualified reviewers and maintain reasonable turnaround times for their journals.

“Editors struggle to find sufficiently qualified reviewers and maintain reasonable turnaround times for their journals [14].”

• cites [14] Dance A. Stop the peer-review treadmill. I want to get off. Nature. 2023;614(7948):581–3. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

LLMs could assist editors in writing final decision letters and summarizing individual review reports.

“Second, LLMs could assist editors in writing final decision letters and summarizing individual review reports.”

Classified as probably not a substantial claim.

Using LLMs to review manuscripts that contain sensitive data could raise concerns about data privacy and unauthorized sharing of confidential information.

“Since it is possible that LLMs may store inputted text, using them to review manuscripts that contain sensitive data could raise concerns about data privacy and unauthorized sharing of confidential information.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Systems such as ChatGPT seem to have been trained well in avoiding judgmental views and, for the most part, do a good job in refraining from using abusive, racist or homophobic language.

“It might be true that systems such as ChatGPT seem to have been trained well in avoiding judgmental views and, for the most part, do a good job in refraining from using abusive, racist or homophobic language [9]”

• cites [9] Gault M. Conservatives Are Panicking About AI Bias, Think ChatGPT Has Gone “Woke”. Vice. 2023 [cited 2023 Feb 14]. Available from: https://www.vice.com/en/article/93a4qe/conservatives-panicking-about-ai-bias-years-too-late-think-chatgpt-has-gone-wokeNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

LLMs' ability to adopt unique styles or constructs could provide a chance to write reasonably consistent yet context-tailored reviews.

“Another advantage of using LLMs pertains to their ability to adopt unique styles or constructs (e.g., literary, philosophical, temporal, etc.). When writing reviews this could provide a chance to write reasonably consistent (not to be mistaken with accurate, useful or relevant) yet context-tailored reviews.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

LLMs could contribute to editors' tasks in peer review by supporting the search for suitable reviewers, the initial screening of manuscripts, and the write-up of final decision letters.

“We believe LLMs could contribute to editors’ tasks in peer review by supporting the search for suitable reviewers, the initial screening of manuscripts, and the write-up of final decision letters from individual review reports.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Given the significance of providing a solution in peer review reports, and the fact that human reviewers might not always be motivated to do this, LLMs could complement human skills to improve the usefulness of review reports.

“For example, in terms of usefulness, given the significance of providing a solution (on how to resolve highlighted problems) in peer review reports [18, 19], and the fact that human reviewers might not always be motivated to do this, LLMs could complement human skills to improve the usefulness of review reports.”

• cites [18] Waggoner DA. Improving the quality of constructive peer feedback. Coll Teach. 2018;66(1):22–3. sourceNot checked — no full text of the source
• cites [19] Horner RD, Lines LM. Anatomy of constructive peer review. Med Care. 2019;57(6):399. sourceNot checked — no full text of the source

• AssumptionRests on an assumption: Human reviewers might not always be motivated to provide solutions in peer review reports.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Even in preprint servers where there are practically no editors, LLMs could enhance automated reviews to address the concern that unreviewed papers may disseminate substandard quality research or unvetted knowledge.

“Even in preprint servers where there are practically no editors, LLMs could enhance automated reviews to address the concern commonly voiced regarding preprints, i.e., that such unreviewed papers may disseminate substandard quality research or unvetted knowledge.”

Probably not a substantial claim: may not pass S4 (based).

The fundamental opacity of LLMs' training data, inner workings, data handling, and development processes raises concerns about potential biases, confidentiality, and the reproducibility of review reports.

“However, the fundamental opacity of LLMs’ training data, inner workings, data handling, and development processes raise concerns about potential biases, confidentiality and the reproducibility of review reports.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Continuous improvements in LLMs in responding to prompts and avoiding pejorative or disrespectful words could help make review reports more constructive and reduce outright harsh or jarring reviews.

“In addition, continuous improvements in LLMs in responding to prompts and avoiding the use of pejorative or disrespectful words could help make review reports more constructive and result in reduced outright harsh or jarring reviews.”

Probably not a substantial claim: may not pass S4 (based).

Unlike researchers who might only be fluent in a handful of languages, LLMs are likely to access sources of knowledge regardless of language.

“Furthermore, unlike researchers who might only be fluent in a handful of languages, LLMs are likely to access sources of knowledge regardless of language.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

In principle, LLMs can improve researchers' capabilities to provide more constructive feedback.

“but in principle, LLMs can improve researchers’ capabilities to provide more constructive feedback”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Collective publication outlets and the peer review process are prime mechanisms that define and help shape epistemic communities.

“Collective publication outlets in general, and the peer review process in particular, are prime mechanisms that define and help shape epistemic communities [23, 24].”

• cites [23] Cronin B. The hand of science: academic writing and its rewards. Lanham: Scarecrow Press; 2005.Not checked yet

  Not checked in the free run.

• cites [24] Hinz A. Exploring the Human Dimension of Peer Review. De Gruyter Conversations. 2021 [cited 2023 Feb 14]. Available from: https://blog.degruyter.com/exploring-the-human-dimension-of-peer-review/Not checked yet

  Not checked in the free run.

May combine two claims.

When a system is developing this fast, its results cannot be reproducible and the basis on which to responsibly rely on the system becomes fragile.

“Clearly, when a system is developing this fast, its results cannot be reproducible and the basis on which to responsibly rely on the system becomes fragile.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The peer review process is a way to shape and negotiate normative frameworks within epistemic communities.

“The peer review process is also a way to shape and negotiate normative frameworks within such communities, for example regarding what is to be considered ‘good’ science, what methods and questions are appropriate and relevant, and what means of communication are most suitable [25].”

• cites [25] Strang D, Siler K. From ‘just the facts’ to ‘more theory and methods, please’: The evolution of the research article in Administrative Science Quarterly, 1956–2008. Soc Stud Sci. 2017;47(4):528–55. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

It is unclear how exactly LLMs learn, how they have been trained and using what data.

“Partly due to the inherent nature of self-learning AI systems and partly due to the commercial interests of their developers, it is unclear how exactly the LLMs learn, how they have been trained and using what data.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

We are likely at the very beginning of an era in which LLMs and future models will have a significant impact on many parts of society, including academia and scholarly communication.

“We are likely at the very beginning of an era in which LLMs and future models will have a significant impact on many parts of society, including academia and scholarly communication.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The tension between keeping LLMs up to date and ensuring reproducibility is likely to confront metascience experts with major challenges.

“This tension between keeping LLMs up to date and ensuring reproducibility is likely to confront metascience experts with major challenges.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

When integrated in search engines, AI tools are likely capable of detecting undisclosed conflicts of interest or related issues.

“Particularly when integrated in search engines, to the extent that they could find information about various involved actors, such tools are likely to be capable of detecting (undisclosed) conflicts of interest or related issues.”

Probably not a substantial claim: may not pass S2 (checkable).

If used responsibly, such capabilities could improve the validity and rigor of reviews.

“If used responsibly, such capabilities could improve the validity and rigor of reviews.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLMs could improve rigor and validity of peer-reviews because they can access and have the capacity to analyze a larger pool of previously published articles and review reports.

“LLMs could improve rigor and validity of peer-reviews because they can access and have the capacity to analyze a larger pool of previously published articles and review reports.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The introduction of LLMs might further exacerbate challenges around skills development and enhancement in peer review.

“Even though the quality of human-written review reports has often been critiqued [30] and calls for more training in peer review have been voiced [31], the introduction of LLMs might further exacerbate challenges around skills development and enhancement.”

• AssumptionRests on an assumption: Writing good-quality and useful reviews is a skill that researchers acquire by practice.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

LLMs learn rapidly.

“It is clear though, that they learn rapidly (although this feature too, could be a double-edged sword because they may learn impulses that humans have not had time to reflect on, or learn insights that are not deemed desirable or correct in hindsight).”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Due to opacity, it is unclear how LLMs will develop, why they perform in the way they do, and what they do with inputted texts in terms of storing, sharing or using these for training purposes.

“Due to this opacity, it is unclear how LLMs will develop, why they perform in the way they do and what they do with inputted texts in terms of storing, sharing or using these for training purposes.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

LLMs are still in early stages of their development and for the moment seem only suitable to improve the first draft of a review instead of writing a review from scratch.

“Indeed, LLMs are still in early stages of their development and for the moment seem only suitable to improve the first draft of a review instead of writing a review from scratch.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Selective feeding of content, or not providing offered limitations or used references, might result in biased or outright erroneous reviews.

“one can imagine that a selective feeding of content, or not providing offered limitations or used references might result in biased or outright erroneous reviews.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Peer review is fundamentally built around the notion of the scientific 'peer' and derives its legitimacy from this notion.

“peer review is fundamentally built around the notion of the scientific ‘peer’ and it derives its legitimacy from this notion [26, 27].”

• cites [26] Biagioli M. From book censorship to academic peer review. Emerg J Study Media Compos Cult. 2002;12(1):11–45.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [27] Shapin S. A social history of truth: civility and science in seventeenth-century England. 1st ed. Chicago: University of Chicago Press; 1995. p. 512.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

If these systems would not develop this quickly, their analysis might be out of date.

“On the other hand though, if these systems would not develop this quickly, their analysis might be out of date.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Given the recent exponential expansion of the corpus of scholarly publications and human limitations to read and analyze these to remain up to date, LLMs that are trained to be unbiased and neutral could significantly enhance researchers' capabilities to write better reviews.

“Given the recent exponential expansion of the corpus of scholarly publications and human limitations to read and analyze these in order to remain up to date, LLMs that are trained to be unbiased and neutral could significantly enhance researchers' capabilities to write better reviews.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

While LLMs could develop fast, it is unclear why they developed in ways they did and how they will develop in the future.

“Furthermore, this example shows that while LLMs could develop fast, it is unclear why they developed in ways they did and how they will develop in the future.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs could pose threats to post-publication peer-review systems by enabling ill-intentioned actors to flood the scene and start smear campaigns against competitors.

“Furthermore, LLMs could pose threats to post-publication peer-review systems (such as PubPeer) that allow anonymous comments by enabling ill-intentioned actors to, for example, flood the scene and start smear campaigns against competitors.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

LLMs might exacerbate existing challenges of the peer review system such as fake peer reviews by allowing fraudsters to create more unique and well-written reviews.

“On the flipside, LLMs might exacerbate existing challenges of the peer review system such as fake peer reviews as they allow fraudsters to create more unique and well-written reviews.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Using LLMs to write review reports could impact initiatives to give credit for performing reviews.

“Using LLMs to write review reports, either partly or in full, could obviously impact such initiatives, necessitating strict regulations on the acknowledgement of the use of LLMs in review, similar to the use of LLMs in original articles.”