The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
Open accessAbstract
Large language models (LLMs) are increasingly deployed as automatic judges to evaluate system outputs in tasks such as summarization, dialogue, and creative writing. A faithful judge should base its verdicts solely on response quality and explicitly acknowledge the factors shaping its decision. We show that current LLM judges fail on both counts by relying on shortcuts introduced in the prompt. Our study uses two evaluation datasets: ELI5, a benchmark for long-form question answering, and LitBench, a recent benchmark for creative writing. Both datasets provide pairwise comparisons, where the evaluator must choose which of two responses is better. From each dataset we construct 100 pairwise judgment tasks and employ two widely used models, GPT-4o and Gemini-2.5-Flash, as evaluators in the role of LLM-as-a-judge. For each pair, we assign superficial cues to the responses, provenance cues indicating source identity (Human, Expert, LLM, or Unknown) and recency cues indicating temporal origin (Old, 1950 vs. New, 2025), while keeping the rest of the prompt fixed. Results reveal consistent verdict shifts: both models exhibit a strong recency bias, systematically favoring new responses over old, as well as a clear provenance hierarchy (Expert > Human > LLM > Unknown). These biases are especially pronounced in GPT-4o and in the more subjective and open-ended LitBench domain. Crucially, cue acknowledgment is rare: justifications almost never reference the injected cues, instead rationalizing decisions in terms of content qualities. These findings demonstrate that current LLM-as-a-judge systems are shortcut-prone and unfaithful, undermining their reliability as evaluators in both research and deployment.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs as judges still struggle to perform well on certain complex and open-ended tasks.
“However, research has shown that LLMs, as judges, still struggle to perform well on certain complex and open-ended tasks [Li et al., 2025, Chakrabarty et al., 2024, Marioriyad et al., 2025, Thakur et al., 2025].”
From DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports (2026) · cites this paper as Marioriyad et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
Research works printed but not linked: 15
- R. Arcuschin, X. Geng, H. Xu, T. Goldstein, V. Raina, and I. Shafran. Implicit rationalizations: When chain-of-thought explanations do not reflect the true reasoning of llms. arXiv preprint arXiv:2503.08679, 2025. URL https://arxiv.org/abs/2503.08679. · arXiv
- Y. Chen, F. F. Xu, S. R. Bowman, et al. Reasoning models don't always say what they think. arXiv preprint arXiv:2505.05410, 2025. URL https://arxiv.org/abs/2505.05410. · arXiv
- Y. Dubois, B. Galambosi, P. Liang, and T. B. Hashimoto. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475, 2024. URL https://arxiv.org/abs/ 2404.04475. · arXiv
- A. R. Fabbri, W. Krysciński, B. McCann, C. Xiong, R. Socher, and D. Radev. Summeval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics, 9:391-409, 2021. doi: 10.1162/tacl_a_00373. URL https://aclanthology.org/2021.tacl-1.24/. · DOI · Paper page
- A. Fan, Y. Jernite, E. Perez, D. Grangier, J. Weston, and M. Auli. ELI5: Long form question answering. In ACL 2019, pages 3558-3567, 2019. doi: 10.18653/v1/P19-1346. · DOI · Paper page
- D. Fein, S. Russo, V. Xiang, K. Jolly, R. Rafailov, and N. Haber. LitBench: A benchmark and dataset for reliable evaluation of creative writing. arXiv preprint arXiv:2507.00769, 2025. URL https://arxiv. org/abs/2507.00769. · arXiv
- X. Liu et al. G-eval: General evaluation of language models. In EMNLP 2023, 2023. URL https: //aclanthology.org/2023.emnlp-main.153/.
- S. Mehri and M. Eskenazi. Human-bot comparison as an evaluation framework for dialogue. In Proceedings of the 23rd Annual SIGdial Meeting on Discourse and Dialogue, pages 62-74, 2022. URL https: //aclanthology.org/2022.sigdial-1.11/.
- A. Panickssery, S. R. Bowman, and S. Feng. Llm evaluators recognize and favor their own generations. In NeurIPS 2024, 2024. URL https://arxiv.org/abs/2404.13076. · arXiv
- K. Saito. Verbosity bias in preference labeling by large language models. In arXiv preprint arXiv:2310.10076, 2023. URL https://arxiv.org/abs/2310.10076. · arXiv
- L. Shi, C. Ma, W. Liang, W. Ma, and S. Vosoughi. Judging the judges: A systematic study of position bias in llm-as-a-judge. arXiv preprint arXiv:2406.07791, 2024. URL https://arxiv.org/abs/2406.07791. · arXiv
- M. Turpin, J. Michael, E. Perez, and S. R. Bowman. Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. In Advances in Neural Information Processing Systems (NeurIPS), 2023. URL https://arxiv.org/abs/2305.04388. · arXiv
- K. Wataoka, T. Takahashi, and R. Ri. Self-preference bias in llm-as-a-judge. arXiv preprint arXiv:2410.21819, 2024. URL https://arxiv.org/abs/2410.21819. · arXiv
- L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Li, D. Li, E. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica. Judging LLM/as-a-judge with MT-bench and chatbot arena. arXiv preprint arXiv:2306.05685, 2023. URL https://arxiv.org/abs/2306.05685. · arXiv
- L. Zheng et al. Chatbot arena: A human-ai comparative judgment platform. In NeurIPS 2023 Datasets and Benchmarks Track, 2023. URL https://openreview.net/forum?id=uccHPGDlao.
- Authors
- Marioriyad, A., Rohban, M. H., & Baghshah, M. S.
- Affiliations
- Sharif University of Technology (IR)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2509.26072
- OpenAlex
- W4415338432
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- LLM-as-a-judge, automated assessment, recency bias, ELI5, GPT-4o, Gemini 2.5 Flash, assessment reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
