CoChat
Sign Up
Cited by 44Publishing and Scholarly CommunicationEthics and Social Impacts of AIAcademic Writing and Publishing

Exploring the Impact of Generative AI on Peer Review: Insights from Journal Reviewers

Open access
Ebadi, Saman, Nejadghanbar, Hassan, Salman, Ahmed Rawdhan, Khosravi, Hassan·2025·Journal of Academic Ethics
DOI 10.1007/s10805-025-09604-4
Open this paper in CoChat — free accountOpen PDF

Abstract

This study investigates the perspectives of 12 journal reviewers from diverse academic disciplines on using large language models (LLMs) in the peer review process. We identified key themes regarding integrating LLMs through qualitative data analysis of verbatim responses to an open-ended questionnaire. Reviewers noted that LLMs can automate tasks such as preliminary screening, plagiarism detection, and language verification, thereby reducing workload and enhancing consistency in applying review standards. However, significant ethical concerns were raised, including potential biases, lack of transparency, and risks to privacy and confidentiality. Reviewers emphasized that LLMs should not replace human judgment but rather complement it with human oversight, which is essential to ensure the relevance and accuracy of AI-generated feedback. This study underscores the need for clear guidelines and policies, as well as their proper dissemination among researchers, to address the ethical and practical challenges of using LLMs in academic publishing.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10805-025-09604-4

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Current studies mainly investigate either generic AI reviewers or AI-human collaboration.

    “Current studies mainly investigate either generic AI reviewers or AI-human collaboration (Sarker et al. 2024;Chen et al. 2025;Ebadi et al. 2025).”

    From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Ebadi et al. 2025

    The Discussion (page unavailable) examines how reviewers might use AI alongside their own judgment, so this paper is an example of work on AI–human collaboration in peer review. It does not establish the broader claim that current studies mainly investigate either that topic or generic AI reviewers. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Additional qualitative investigations involving reviewers across multiple disciplines concluded that LLMs should complement rather than replace expert reviewers.

    “Additional qualitative investigations involving reviewers across multiple disciplines reached similar conclusions, emphasizing that LLMs should complement rather than replace expert reviewers and that clearer editorial policies are urgently needed (Ebadi et al. 2025).”

    From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Ebadi et al. 2025

  • Additional qualitative investigations concluded that clearer editorial policies are urgently needed.

    “Additional qualitative investigations involving reviewers across multiple disciplines reached similar conclusions, emphasizing that LLMs should complement rather than replace expert reviewers and that clearer editorial policies are urgently needed (Ebadi et al. 2025).”

    From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Ebadi et al. 2025

  • This finding is consistent with research emphasizing collaborative human-AI review models and with evidence that AI-assisted reviewing may influence review scores and reproduce existing biases.

    “This finding is consistent with research emphasizing collaborative human-AI review models (Sarker et al. 2024;Ebadi et al. 2025) and with evidence that AIassisted reviewing may influence review scores and reproduce existing biases (Russo et al. 2025;Vasu et al. 2025).”

    From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Ebadi et al. 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 28

  1. Alqahtani, T., Badreldin, H. A., Alrashed, M., Alshaya, A. I., Alghamdi, S. S., bin, Saleh, K., & Albekairy, A. M. (2023). The emergent role of artificial intelligence, natural learning processing, and large language models in higher education and research. Research in Social and Administrative Pharmacy. American Journal Experts. (2013). Peer review: How we found 15 million hours of lost time. AJE. h t t p s : / / w w w . . / /
  2. Andersson, E., Dellkvist, H., Johansson, U., & Skär, L. (2018). Relatives' experiences of sharing a written life story about a close family member with dementia who has moved to residential care: An interview study. Nursing Open, 6(2), 276-282. https://doi.org/10.1002/nop2.208 · DOI
  3. Biswas, S., Dobaria, D., & Cohen, H. L. (2023). Focus: Big Data: ChatGPT and the future of Journal reviews: A feasibility study. The Yale Journal of Biology and Medicine, 96(3), 415.
  4. Boddy, C. (2016). Sample size for qualitative research. Qualitative Market Research an International Jour- nal, 19(4), 426-432. https://doi.org/10.1108/qmr-06-2016-0053 · DOI · Paper page
  5. Checco, A., Bracciale, L., Loreti, P., Pinfield, S., & Bianchi, G. (2021). AI-assisted peer review. Humanities and Social Sciences Communications, 8(1), 1-11.
  6. Crawford, J., Cowling, M., Ashton-Hay, S., Kelder, J. A., Middleton, R., & Wilson, G. S. (2023). Artificial Intelligence and authorship editor policy: ChatGPT, Bard Bing AI, and beyond. Journal of University Teaching & Learning Practice, 20(5), 1.
  7. Creswell, J. W., & Creswell, L. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage.
  8. Darvishi, A., Khosravi, H., Sadiq, S., & Gašević, D. (2022). Incorporating Ai and learning analytics to build trustworthy peer assessment systems. British Journal of Educational Technology, 53(4), 844-875. https://doi.org/10.1111/bjet.13233 · DOI · Paper page
  9. Dergaa, I., Chamari, K., Żmijewski, P., & Saad, H. (2023). From human writing to artificial intelligence generated text: Examining the prospects and potential threats of ChatGPT in academic writing. Biology of Sport, 40(2), 615-622. h t t p s : / / d o i . o r g / 1 0 . 5 1 1 4 / b i o l s p o r t . 2 0 2 3 . 1 2 5 6 2 · DOI · Paper page
  10. Donker, T. (2023). The dangers of using large language models for peer review. The Lancet Infectious Dis- eases, 23(7), 781. h t t p s : / / d o i . o r g / 1 0 . 1 0 1 6 / S 1 4 7 3 -3 0 9 9 ( 2 3 ) 0 0 2 9 0 -6 · DOI
  11. Flanagin, A., Kendall-Taylor, J., & Bibbins-Domingo, K. (2023). Guidance for authors, peer reviewers, and editors on use of AI, language models, and chatbots. Jama, 330(8), 702. h t t p s : / / d o i . o r g / 1 0 . 1 0 0 1 / j a m a . 2 0 2 3 . 1 2 5 0 · DOI · Paper page
  12. Garcia, M. B. (2023). Using AI tools in writing peer review reports: Should academic journals embrace the use of ChatGPT? Annals of Biomedical Engineering (in press). https://doi.org/10.1007/s10439-023-03299-7 · DOI
  13. Gatrell, C. (2024). Here, there and everywhere: On the responsible use of artificial intelligence (Ai) in man- agement research and the peer-review process. Journal of Management Studies, 61(3), 739-751. h t t p s : / / d o i . o r g / 1 0 . 1 1 1 1 / j o m s . 1 3 0 4 · DOI
  14. Holmes, A. G. D. (2023). The design and use of questionnaires in educational research: A new (student) researcher guide. Innovare Journal of Education, 1-5. h t t p s : / / d o i . o r g / 1 0 . 2 2 1 5 9 / i j o e . 2 0 2 3 v 1 1 i 3 . 4 7 5 · DOI
  15. Hu, G. (2023). Challenges for enforcing editorial policies on AI-generated papers. Accountability in Research, 1-3. h t t p s : / / d o i . o r g / 1 0 . 1 0 8 0 / 0 8 9 8 9 6 2 1 . 2 0 2 3 . 2 1 8 4 2 · DOI
  16. Kousha, K., & Thelwall, M. (2023). Artificial intelligence to support publishing and peer review: A summary and review. Learned Publishing, 37(1), 4-12. https://doi.org/10.1002/leap.1570 · DOI
  17. Larsen, P., & Von Ins, M. (2010). The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics, 84(3), 575-603.
  18. Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2-17.
  19. Leonard, C. (2024). Is AI the answer to peer review problems, or the Problem itself?> The Scholarly Kitchen.
  20. Miao, J. (2023). Ethical dilemmas in using Ai for academic writing and an example framework for peer review in nephrology academia: A narrative review. Clinics and Practice, 14(1), 89-105. h t t p s : / / d o i . o r / 1 0 . 3 3 9 0 / c l i n · DOI · Paper page
  21. Reja, U., Manfreda, K. L., Hlebec, V., & Vehovar, V. (2003). Open-ended vs. close-ended questions in web questionnaires. Developments in Applied Statistics, 19(1), 159-177.
  22. Salah, M., Abdelfattah, F., & Halbusi, H. A. (2023). Debate: Peer reviews at the crossroads-'To AI or not to AI?'. Public Money & Management, 43(8), 781-782.
  23. Tang, A. (2023). The importance of transparency: Declaring the use of generative artificial intelligence (Ai) in academic writing. Journal of Nursing Scholarship, 56(2), 314-318. https://doi.org/10.1111/jnu.12938 · DOI · Paper page
  24. Van Noorden, R. (2022). Journals adopt AI to spot duplicated images in manuscripts. Nature, 601(7891), 14-15.
  25. Ware, M., & Mabe, M. (2015). The STM report: An overview of scientific and scholarly journal publishing. University of Nebraska.
  26. Wright, C., Lartey, M., Khomsi, K., Peres, F., Yilma, D., Kigera, J., & Muula, S., A (2024). The African jour- nal partnership program's guidance on the use of AI in scholarly publishing. Ghana Medical Journal, 58(1), 1-4. https://doi.org/10.4314/gmj.v58i1. · DOI
  27. Farangi, M. R., Nejadghanbar, H., & Hu, G. (2024). Use of generative AI in research: Ethical considerations and emotional experiences. Ethics & Behavior, 1-17. h t t p s : / / d o i . o r g / 1 0 . 1 0 8 0 / 1 0 5 0 8 4 2 2 . 2 0 2 4 . 2 4 2 0 1 · DOI
  28. Farangi, M. R., & Nejadghanbar, H. (2024). Investigating questionable research practices among Iranian applied linguists: Prevalence, severity, and the role of artificial intelligence tools. System, 125, 103427. https://doi.org/10.1016/j.system.2024.103427 · DOI

Cited by

Sorted by citations

Authors
Ebadi, S., Nejadghanbar, H., Salman, A. R., & Khosravi, H.
Affiliations
  • Razi University (IR)
  • Hong Kong Polytechnic University (HK)
  • Al-Mustaqbal University (IQ)
  • The University of Queensland (AU)
Corresponding author
Nejadghanbar, Hassan
ORCID
Venue
Journal of Academic Ethics
Publisher
Springer Science and Business Media LLC
Published
2025-02-11
Volume
23
Issue
3
Pages
1383-1397
ISSN
  • 1570-1727 (ISSN-L)
  • 1572-8544
DOI
10.1007/s10805-025-09604-4
OpenAlex
W4407373980
Cited by
44
References
12
Citations by year
2026: 30 · 2025: 14
FWCI
167.94
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Academic Ethics (publishedVersion, cc-by)
  • PolyU Institutional Research Archive (Hong Kong Polytechnic University) (publishedVersion, cc-by)
Funders
The Hong Kong Polytechnic University
Topics
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Publishing and Scholarly Communication
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
Keywords
generative AI, peer review, large language models, journal reviewers, plagiarism detection, AI ethics, human oversight, transparency, privacy and confidentiality, scholarly publishing, reviewer workload

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review