Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews
Open accessAbstract
The adoption of large language models (LLMs) is transforming the peer review process, from assisting reviewers in writing detailed evaluations to generating entire reviews automatically. While these capabilities offer new opportunities, they also raise concerns about fairness and reliability. In this paper, we investigate bias in LLM-generated peer reviews through controlled interventions on author metadata, including affiliation, gender, seniority, and publication history. Our analysis consistently shows a strong affiliation bias favoring authors from highly ranked institutions. We also identify directional preferences associated with seniority and prior publication record, which can influence acceptance decisions for borderline papers. Gender effects are smaller but present in several models. Notably, implicit biases become more pronounced when examining token-level soft ratings, suggesting that alignment may mask but not fully eliminate underlying preferences
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Concerns regarding hallucinations, bias, overconfidence and accountability remain substantial.
“However, concerns regarding hallucinations, bias, overconfidence and accountability remain substantial (Vasu et al. 2025;Alami et al. 2025;Doskaliuk et al. 2025;Seghier 2025).”
From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Vasu et al. 2025
The Introduction and Discussion report systematic bias in LLM-assisted peer reviews and describe it as a serious fairness concern. The cited paper does not establish the other concerns in the sentence—hallucinations, overconfidence, or accountability—so it supports only the bias component. (LLM, conf=0.88)
Experimental — accuracy not yet measuredLLM-generated reviews exhibit affiliation bias favouring highly ranked institutions.
“More recent investigations confirmed that LLM-generated reviews exhibit affiliation bias favouring highly ranked institutions and may also demonstrate subtle gender-related preferences (Vasu et al. 2025).”
From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Vasu et al. 2025
LLM-generated reviews may demonstrate subtle gender-related preferences.
“More recent investigations confirmed that LLM-generated reviews exhibit affiliation bias favouring highly ranked institutions and may also demonstrate subtle gender-related preferences (Vasu et al. 2025).”
From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Vasu et al. 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 31
- AAAI. 2026. Aaai launches ai-powered peer review assessment system. Web page. Accessed: 2025-07- 29.
- Jiafu An, Difang Huang, Chen Lin, and Mingzhu Tai. 2025. Measuring gender and racial biases in large language models: Intersectional evidence from auto- mated resume evaluation. · DOI · Paper page
- Xuechunzi Bai, Angelina Wang, Ilia Sucholutsky, and Thomas L Griffiths. 2025. Explicitly unbiased large language models still form biased associations. Proceedings of the National Academy of Sciences, 122(8):e2416228122.
- Juhwan Choi, JungMin Yun, Changhun Kim, and YoungBin Kim. 2026. Position paper: How should we responsibly adopt LLMs in the peer review pro- cess? In Findings of the Association for Compu- tational Linguistics: EACL 2026, pages 151-165, Rabat, Morocco. Association for Computational Lin- guistics. · DOI
- Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhen- hua Dong, and Jun Xu. 2024. Bias and unfairness in information retrieval systems: New challenges in the llm era. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Min- ing, pages 6437-6447.
- Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Peng Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Sri- nath, Haoran Ranran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, and 21 others. 2024. LLMs assist NLP researchers: Critique paper (meta-)reviewing. In Proceedings of the 2024 Conference on Empiri- cal Methods in Natural Language Processing, pages 5081-5099, Miami, Florida, USA. Association for Computational Linguistics. · DOI · Paper page
- Shurui Du. 2025. TitleTrap: Probing presentation bias in LLM-based scientific reviewing. In Proceedings of the 5th Workshop on Evaluation and Compari- son of NLP Systems, pages 119-125, Mumbai, India. Association for Computational Linguistics. · DOI
- Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernon- court, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. Bias and fairness in large language models: A survey. Computational Linguistics, 50(3):1097- 1179.
- Federico Germani and Giovanni Spitale. 2025. Source framing triggers systematic bias in large language models. Science Advances, 11(45):eadz2924. · DOI
- Mohammad Hosseini and Serge P. J. M. Horbach. 2023. Fighting reviewer fatigue or amplifying bias? consid- erations and recommendations for use of chatgpt and other large language models in scholarly peer review. Research Integrity and Peer Review, 8(1):4. · DOI · Paper page
- Anthony Howell, Jieshu Wang, Luyu Du, Julia Melk- ers, and Varshil Shah. 2025. Prestige over merit: An adapted audit of llm bias in peer review. arXiv preprint arXiv:2509.15122. · arXiv
- ICLR. 2025. Leveraging llm feedback to enhance re- view quality. Web page. Accessed: 2025-07-29.
- ICML. 2026. Icml 2026 policy for llm use in reviewing. Web page.
- Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kai- jie Zhu, Yijia Xiao, and Jindong Wang. 2024. Agen- tReview: Exploring peer review dynamics with LLM agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1208-1226.
- Ilia Kuznetsov, Osama Mohammed Afzal, Koen Der- cksen, Nils Dycke, Alexander Goldberg, Tom Hope, Dirk Hovy, Jonathan K. Kummerfeld, Anne Lauscher, Kevin Leyton-Brown, Sheng Lu, Mausam, Margot Mieskes, Aurélie Névéol, Danish Pruthi, Lizhen Qu, Roy Schwartz, Noah A. Smith, Thamar Solorio, and 5 others. 2024. What can natural language processing do for peer review? Preprint, arXiv:2405.06563. · arXiv
- Weixin Liang, Zachary Izzo, Yaohui Zhang, Haley Lepp, Hancheng Cao, Xuandong Zhao, Lingjiao Chen, Hao- tian Ye, Sheng Liu, Zhi Huang, and 1 others. 2024a. Monitoring ai-modified content at scale: A case study on the impact of chatgpt on ai conference peer re- views. arXiv preprint arXiv:2403.07183. · arXiv
- Dave Mbiazi, Meghana Bhange, Maryam Babaei, Ivaxi Sheth, Patrik Kenfack, and Samira Ebrahimi Kahou. 2025. Survey on ai ethics: A socio-technical perspec- tive. Computational Intelligence, 41(6):e70149.
- Mathias Wullum Nielsen, Christine Friis Baker, Emer Brady, Michael Bang Petersen, and Jens Peter Ander- sen. 2021. Weak evidence of country-and institution- related status bias in the peer review of abstracts. Elife, 10:e64561.
- Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow in- structions with human feedback. Advances in neural information processing systems, 35:27730-27744.
- Pat Pataranutaporn, Nattavudh Powdthavee, and Pattie Maes. 2025. Can ai solve the peer review crisis? a large scale experiment on llm's performance and bi- ases in evaluating economics papers. arXiv preprint arXiv:2502.00070. · arXiv
- Giuseppe Russo, Manoel Horta Ribeiro, Tim Ruben Davidson, Veniamin Veselovsky, and Robert West. 2025. The ai review lottery: Widespread ai-assisted peer reviews boost paper scores and acceptance rates. Proc. ACM Hum.-Comput. Interact., 9(7). · DOI · Paper page
- Selena Ming Shen, Zifu Wang, Krittika Paul, Meng-Hao Li, Xiao Huang, and Naoru Koizumi. 2025. Eval- uation of large language models for peer review in transplantation research: Algorithm validation study. JMIR AI, 5:e84322.
- Hyungyu Shin, Jingyu Tang, Yoonjoo Lee, Nayoung Kim, Hyunseung Lim, Ji Yong Cho, Hwajung Hong, Moontae Lee, and Juho Kim. 2025. Mind the blind spots: A focus-level evaluation framework for llm reviews. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 35618-35644.
- Sarath Sivaprasad, Pramod Kaushik, Sahar Abdelnabi, and Mario Fritz. 2025. A theory of response sam- pling in llms: Part descriptive and part prescriptive. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 30091-30135.
- Nitya Thakkar, Mert Yuksekgonul, Jake Silberg, Ani- mesh Garg, Nanyun Peng, Fei Sha, Rose Yu, Carl Vondrick, and James Zou. 2026. A large-scale ran- domized study of large language model feedback in peer review. Nature Machine Intelligence, 8:326- 336. · DOI · Paper page
- Daniel von Wedel, R. A. Schmitt, M. Thiele, R. Leuner, D. Shay, S. Redaelli, and M. S. Schaefer. 2024. Af- filiation bias in peer review of abstracts by a large language model. JAMA, 331(3):252-253. · DOI · Paper page
- Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, and Nanyun Peng. 2023. " kelly is a warm person, joseph is a role model": Gender biases in llm-generated reference letters. arXiv preprint arXiv:2310.09219. · arXiv
- Rui Ye, Xianghe Pang, Jingyi Chai, Jiaao Chen, Zhenfei Yin, Zhen Xiang, Xiaowen Dong, Jing Shao, and Siheng Chen. 2024. Are we there yet? revealing the risks of utilizing large language models in scholarly peer review. arXiv preprint arXiv:2412.01708. · arXiv
- Yaohui Zhang, Haijing Zhang, Wenlong Ji, Tianyu Hua, Nick Haber, Hancheng Cao, and Weixin Liang. 2025. From replication to redesign: Exploring pair- wise comparisons for llm-based peer review. arXiv preprint arXiv:2506.11343. · arXiv
- Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. 2025. DeepReview: Improving LLM-based paper review with human-like deep thinking process. In Proceedings of the 63rd Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 29330-29355, Vienna, Austria. Association for Computational Linguistics. · DOI
- Zhenzhen Zhuang, Jiandong Chen, Hongfeng Xu, Yuwen Jiang, and Jialiang Lin. 2025. Large lan- guage models for automated scholarly paper review: A survey. Inf. Fusion, 124(C). · DOI · Paper page
Cited by
Sorted by citations
Ali, Ș., Oprea, S., & Bâra, A. (2026) · Systems Research and Behavioral Science · doi:10.1002/sres.70163
- Authors
- Vasu, S. S. M., Sheth, I., Wang, H.-P., Binkyte, R., & Fritz, M.
- Affiliations
- Helmholtz Center for Information Security (DE)
- ORCID
- Fritz, Mario
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- Pages
- 307-330
- arXiv
- 2509.13400
- OpenAlex
- W4416259212
- Cited by
- 6
- References
- 0
- Citations by year
- 2026: 6
- FWCI
- 56.56
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- CISPA Helmholtz Center (cc-by)
- Funders
- European Commission (101070617, 101214398)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Publishing and Scholarly Communication
- Keywords
- large language models, peer review bias, institutional prestige, gender bias, publishing history, acceptance decisions, implicit bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
