Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge

LLaMA: Open and Efficient Foundation Language Models

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works

Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · see citing works

Gemini: A Family of Highly Capable Multimodal Models

Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990 · see citing works

Bias and Fairness in Large Language Models: A Survey

Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 665 · see citing works

JudgeLM: Fine-tuned Large Language Models are Scalable JudgesPreprint

Zhu, L.-H., Wang, X.-G., & Wang, X.-L. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2310.17631 · Cited by 441 · see citing works

Can Large Language Models Be an Alternative to Human Evaluations?

Chiang, C.-H., & Lee, H.-Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.870 · Cited by 331 · see citing works

Large Language Models are Zero-Shot Rankers for Recommender Systems

Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J., & Zhao, W. X. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-56060-6_24 · Cited by 327 · see citing works

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-JudgePreprint

Wu, T., Yuan, W.-Z., Golovneva, O., Xu, J., Tian, Y.-D., Jiao, J.-T., Weston, J. E., & Sukhbaatar, S. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2407.19594 · Cited by 236 · see citing works

Large Language Models are not Fair Evaluators

Wang, P., Li, L., Chen, L., Cai, Z., Zhu, D., Lin, B., Cao, Y., Kong, L., Liu, Q., Liu, T., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.511 · Cited by 138 · see citing works

Large Language Models Are State-of-the-Art Evaluators of Translation Quality

Kocmi, T., & Federmann, C. (2023) · arXiv · doi:10.48550/arxiv.2302.14520 · Cited by 122 · see citing works

Bias in Large Language Models: Origin, Evaluation, and Mitigation

Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2026) · Electronics · doi:10.3390/electronics15091824 · Cited by 115 · see citing works

Eliminating Position Bias of Language Models: A Mechanistic Approach

Wang, Z.-Q., Zhang, H., Li, X.-E., et al. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2407.01100 · Cited by 88

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge

Wu, T., Yuan, W., Golovneva, O., Xu, J., Tian, Y., Jiao, J., Weston, J., & Sukhbaatar, S. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.583 · Cited by 67 · see citing works

Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment

Raina, V., Liusie, A., & Gales, M. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.427 · Cited by 59 · see citing works

The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation

Karpinska, M., Akoury, N., & Iyyer, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.97 · Cited by 56 · see citing works

ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate

Chan, C.-M., Chen, W., Su, Y., Yu, J., Xue, W., Zhang, S., Fu, J., & Liu, Z. (2023) · arXiv · doi:10.48550/arxiv.2308.07201 · Cited by 55 · see citing works

LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models

Liusie, A., Manakul, P., & Gales, M. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.8 · Cited by 45 · see citing works

Split and Merge: Aligning Position Biases in LLM-based Evaluators

Li, Z., Wang, C., Ma, P., Wu, D., Wang, S., Gao, C., & Liu, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.621 · Cited by 43 · see citing works

Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment

Raina, V., Liusie, A., & Gales, M. (2024) · arXiv · doi:10.48550/arxiv.2402.14016 · Cited by 40 · see citing works

Large Language Models are not Fair Evaluators

Wang, P., Li, L., Chen, L., Cai, Z., Zhu, D., Lin, B., Cao, Y., Liu, Q., Liu, T., & Sui, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.17926 · Cited by 34 · see citing works

Large Language Models Are Not Robust Multiple Choice Selectors

Zheng, C., Zhou, H., Meng, F., Zhou, J., & Huang, M. (2023) · arXiv · doi:10.48550/arxiv.2309.03882 · Cited by 29 · see citing works

Split and Merge: Aligning Position Biases in LLM-based Evaluators

Li, Z., Wang, C., Ma, P., Wu, D., Wang, S., Gao, C., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.01432 · Cited by 26

Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case StudyPreprint

Li, B., Wu, W., Tang, Z., Shi, L., Yang, J. J., Li, J., Yao, S., Chen, Q., Hui, B., Zhang, Q., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.08604 · Cited by 24 · see citing works

Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting

Qin, Z., Jagerman, R., Hui, K., Zhuang, H., Wu, J., Yan, L., Shen, J., Liu, T., Liu, J., Metzler, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.17563 · Cited by 16 · see citing works

Debating with More Persuasive LLMs Leads to More Truthful Answers

Khan, A., Hughes, J., Valentine, D., Ruis, L., Sachan, K., Radhakrishnan, A., Grefenstette, E., Bowman, S. R., Rocktäschel, T., & Perez, E. (2024) · arXiv · doi:10.48550/arxiv.2402.06782 · Cited by 13 · see citing works

Evaluating Large Language Models at Evaluating Instruction Following

Zeng, Z., Yu, J., Gao, T., Meng, Y., Goyal, T., & Chen, D. (2023) · arXiv · doi:10.48550/arxiv.2310.07641 · Cited by 13 · see citing works

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge

Ye, J., Wang, Y., Huang, Y., Chen, D., Zhang, Q., Moniz, N., Gao, T., Geyer, W., Huang, C., Chen, P.-Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.02736 · Cited by 11 · see citing works

Likelihood-based Mitigation of Evaluation Bias in Large Language Models

Ohi, M., Kaneko, M., Koike, R., Loem, M., & Okazaki, N. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.193 · Cited by 9 · see citing works

Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM JudgmentsPreprint

Li, Y., Mohamud, J. H., Sun, C., Wu, D., & Boulet, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.17087 · Cited by 9 · see citing works

Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplification and Resistance in Multi-Agent Based LLM-as-Judge

Ma, C., Zhang, E., Zhao, Y., Liu, W., Jia, Y., Qing, P., Shi, L., Cohan, A., Yan, Y., & Vosoughi, S. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.941 · Cited by 6 · see citing works

MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark

Chen, D., Chen, R., Zhang, S., Liu, Y., Wang, Y., Zhou, H., Zhang, Q., Wan, Y., Zhou, P., & Sun, L. (2024) · arXiv · doi:10.48550/arxiv.2402.04788 · Cited by 6 · see citing works

Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought

Chua, J., Rees, E., Batra, H., Bowman, S. R., Michael, J., Perez, E., & Turpin, M. (2024) · arXiv · doi:10.48550/arxiv.2403.05518

Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-JudgePreprint

Ma, C., Zhang, E., Zhao, Y., Liu, W., Jia, Y., Qing, P., Shi, L., Cohan, A., Yan, Y., & Vosoughi, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.19477

Likelihood-based Mitigation of Evaluation Bias in Large Language ModelsPreprint

Ohi, M., Kaneko, M., Koike, R., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.15987

PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations

Li, R., Patel, T., & Du, X. (2023) · arXiv · doi:10.48550/arxiv.2307.02762

GPT-4 Technical Report

OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774

Bias in Large Language Models: Origin, Evaluation, and Mitigation

Guo, Y., Guo, M., Su, J., Yang, Z., Zhu, M., Li, H., Qiu, M., & Liu, S. S. (2024) · arXiv · doi:10.48550/arxiv.2411.10915

GradeAgentOps: A Verification-First Framework for Evidence-Anchored LLM Exam Grading

Anghel, C., Anghel, A. A., Crăciun, M., Cocu, A., Vulpe, D. E., Andrei, C. A., Maier, C., Scheau, C., Dragosloveanu, Ş., & Cergan, R. (2026) · AI · doi:10.3390/ai7060198 · Cited by 8 · see citing works

Human Evaluation of Large Language Models: A Review and Protocol Selection Framework

Brunyé, T. T. (2026) · AI · doi:10.3390/ai7050174 · Cited by 4 · see citing works

Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges

Eiras, F., Zemour, E., Lin, E., & Mugunthan, V. (2025) · arXiv · doi:10.48550/arxiv.2503.04474 · Cited by 1 · see citing works

LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment

Li, L., Li, D., Chen, C., Ma, R., Yu, R., Lin, M., Yin, R., Fan, L., Shyr, C., Ma, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.25273 · Cited by 1 · see citing works

A Scoping Review of LLM-as-a-Judge in Healthcare and the MedJUDGE Framework

Li, C., Akhtar, Z., Kwak, M., Ji, Y., Zhang, H., Obi, T., Ren, Y., Wu, X., Sivarajkumar, S., Lehmann, H. P., et al. (2026) · arXiv · doi:10.48550/arxiv.2604.25933 · Cited by 1 · see citing works

Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias

Norman, J. D., Rivera, M. U., & Hughes, D. A. (2026) · arXiv · doi:10.48550/arxiv.2606.19544 · Cited by 1 · see citing works

Quantifying and Mitigating Self-Preference Bias of LLM Judges

Yang, J., Hu, Z., Qiu, C., Deng, Z., Jiao, X., & Zhou, T. (2026) · arXiv · doi:10.48550/arxiv.2604.22891

How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review

Li, M., Wang, C., Li, X., Zeng, X., Li, D., Shi, P., Zhou, D., & Zhou, T. (2026) · arXiv · doi:10.48550/arxiv.2608.08975

Beyond Consensus: Downward Bias and Role Asymmetry in Multi-Agent LLM Judges for Subjective Evaluation

Song, M., Kim, C., Eo, S., & Park, C. (2026) · arXiv · doi:10.48550/arxiv.2608.30373

Question–Answer Readability Alignment in Patient Education: A Comparative Evaluation of Prompting Strategies in Large Language Models (Preprint)Preprint

Bulla, M. C., Lavinsky, D., Rosa, A. A. M., & Carricondo, P. C. (2026) · doi:10.2196/preprints.97408

Confounder Balance in Next Basket Prediction

Deng, Z., Li, J., Liu, W., & Zhao, J. (2026) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2026.3682294

Pre-Registered Causal Audit of Multi-Provider LLM Validator Consensus on an EHR-Derived Medication Benchmark: Answer-Position Effects, Pool-Level Biases, and a Two-Physician Pilot (Preprint)Preprint

Sharma, A. K. (2026) · doi:10.2196/preprints.99322

AI-based hardware trojan generation and detection in post-quantum cryptographyPreprint

Hayashi, V. T., & Ruggiero, W. V. (2026) · Research Square · doi:10.21203/rs.3.rs-9849559/v1

GenAI-Supported strategy development in VUCA scenarios and Multi-GenAI evaluation

Yurdasever, E., & Yıldıran, C. (2026) · Management · doi:10.58691/man/221173

Test-Time Improvement for Large Models: A Survey of Inference-Stage Mechanisms, Resource Control, and Research OpportunitiesPreprint

Tang, Y., Liu, Z., Chen, Z., Yang, J., Xiang, L., Xu, Z., & He, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-9875246/v1

Prompt Perturbation for reliable LLM evaluation over comparison graphs

Huang, D., Sun, J., & Yang, P. (2026) · Statistical Learning and Data Science · doi:10.1016/j.slads.2026.100022

Toward Reliable LLM-Integrated Web Architectures for Teacher-Aligned Automatic Student Grading

Gwozdz, J., & Both, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29372-5_8

Automated bias blind spots: Examining stereotype associations in VLM-as-a-judge paradigms

Hong, R. S., Ovalle, A., Ung, M., Spiliopoulou, E., Sagun, L., Williams, A., & Ross, C. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806459

Examining the Performance of Large Language Models in Health Information Quality Evaluation Tasks: An AI Post-Alignment Perspective

Zhang, S., Zhang, J., Li, P., Zuo, M., & Feng, L. (2026) · Information · doi:10.3390/info17070649

Safeguarding LLM Agents from Misalignment through Provenance Analysis

She, Y., Liang, Y., & Kang, E. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837535

Highway Landscape Preference Along Malaysia’s North–South Expressway: A Comparison of Multimodal Large Language Models and Human Judgments

Gao, H., Smardon, R., Bakar, S. A., Maulan, S., & Yang, J. (2026) · Land · doi:10.3390/land15071240

SafetyJudge-LLM: Auditing Local Open-Weight LLMs as Semantic Safety Judges for Boundary-Failure Detection

Anghel, C., Anghel, A. A., Crăciun, M., Cocu, A., Moldovanu, S., & Sandu, C. (2026) · AI · doi:10.3390/ai7080286

WHEN FAIR AI BECOMES UNFAIR: A COUNTERFACTUAL AUDIT OF POSITIONAL BIAS IN LARGE LANGUAGE MODELS FOR HIRING DECISIONS

Camargo, A. M., & Camargo, R. S. F. (2026) · Revista de Geopolítica · doi:10.56238/revgeov17n7-181

RMEval: Bridging Reward Model Evaluation and Policy Evaluation through Ranking Consistency

Qiu, W., Zhang, X., Zhang, Y., Wang, A., Zhang, Z., & Yu, Y. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817593

Auditing Implicit Sycophancy in Mental-Health Dialogue: Structured Clinical-State Diagnostics and Clean Matched Benchmarks

Han, T., Xu, B., Zhang, H., & Lu, Y. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-4529-1_14

HGARena: Budgeted Test-Driven Multi-Agent Repository Issue Resolution with Heterogeneous Graph-Augmented Retrieval

Cao, Y., Keung, J., Wang, Y., Sun, Y., & Mao, Z. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00062

Visual LLM-guided consensus spatial domain detection with L-STAR

Zhao, C., & Ji, Z. (2026) · Nature Communications · doi:10.1038/s41467-026-78923-w

MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models

Cunha, R. D., Lozano, A., Sun, X., Jarquin, D. V., Sun, M., Aklilu, J. G., Burgess, J., Zhang, Y., Robayo, P. A., Ye, J., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37556-8_32

Candidate information structure reduces over-alignment in LLM- based concept mappingPreprint

Gao, W., Zhou, S., Lyu, X., Angeluci, A. C. B., & Dong, J. (2026) · Research Square · doi:10.21203/rs.3.rs-10813874/v1

From Hand-crafted to LLM-based Variation Operators in Metaheuristics: A TutorialPreprint

Sartori, C. C., Corominas, G. R., & Blum, C. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7450238

IDV: Customer-Context-Aware Multi-Engine Architecture for AI-Agent Intent Drift Detection in Finance

Zhou, M., & Xue, Y. (2026) · IEEE Conference on Computational Intelligence for Financial Engineering & Economics · doi:10.1109/cifer67845.2026.11692374

Intent-Description Anchoring Bias in LLM-as-a-Judge Evaluation of Recommender Systems

Abdollahpouri, H., Kretschman, K., & Lalmas, M. (2026) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3773078.3841257

Measuring Position Bias in LLM-as-a-Judge for Hiring

Aminou, L., Daaif, A., Soulami, M., & Youssfi, M. (2026) · International Conference on Advanced Computer Information Technologies (ACIT) · doi:10.1109/acit69831.2026.11699959

An Evidence-Based LLM Framework for Expert-Validated Clean Architecture Conformance Assessment in Enterprise Software Systems

Bravo, E. J. G., Chavira, G., Álvarez-Navarro, E., Montoto-González, A., & Juarez, J. L. D. (2026) · Computers · doi:10.3390/computers15100657

Disassembly-Augmented Root Cause Analysis for C/C++ Segmentation Faults in SAP HANA

Jung, J., You, C., Hong, J., Bach, T., & An, G. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834461

Toward Ludic-Aware Narrative Generation: A Neuro-Symbolic Framework for Evaluating Playability in LLM-Generated Backstories

Pena, L., Peña, J.-M., & Leal, R. (2026) · Applied Sciences · doi:10.3390/app16199966

ClaimArena-LLM: Auditing Adversarial Propagation of False Causal Attributions in Multi-Agent LLM Debates

Anghel, C., Anghel, A. A., Balau, A. S., Crăciun, M., Cocu, A., Moldovanu, S., & Sandu, C. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8100327

Same Passages, Different Readings: Comparing Complete Human and LLM Interpretations of a Story

Li, B., & Feng, Y. (2026) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3805696.3847293

Auditing LLM Grading for Short-Answer Responses: Confidence Gating and Cross-Model Agreement

Gwozdz, J., & Both, A. (2026) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt68022.2026.00102

Fair-MAD:For FGKB-Guided Adjudication of Stereotype-Prone Questions

Dong, Z., Yu, Z., Zuo, J., & Zhu, E. (2026) · International Conference on Computer Science, Machine Learning and Agents (CSMLA) · doi:10.1109/csmla70906.2026.11724473

Prompting Strategies for Readability Alignment Between User-Generated Ophthalmology Questions and Large Language Model Answers: Comparative Evaluation Study (Preprint)

Bulla, M. C., Lavinsky, D., Rosa, A. A. M., & Carricondo, P. C. (2026) · JMIR Formative Research · doi:10.2196/97408

ProtoPure: Literature Mining and Analysis of Protein Purification Protocols at Proteome ScalePreprint

Almada‐Monter, R., Martinez-Lomeli, J., Garay, E., Ana, J. T. S., Spencer, A., Siordia, A., Chordiya, P., Armijo, S. L., Herrera, J. M., Roethler, N., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.26.762056

Triage of Completed Evidence-Label Tuples with an LLM: Controlled Stress Testing and a Source-Clean Extension

Mou, N., Han, S., Wu, F., & Ye, X.-F. (2026) · International Conference on Digital Media, Communication and Information Systems (DMCIS) · doi:10.1109/dmcis71493.2026.11749491

When Is Debateability a Model Property? Cross-Interlocutor Validation in Multi-Agent Debate

Ju, C.-Y., Kim, J.-Y., & Lee, D. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840079

An Empirical Analysis of Formatting Bias in Automated LLM-Based Code Assessment

Gunawan, V. V., Welen, C. J., Gunawan, A. A. S., & Pradana, R. C. (2026) · International Conference on Information Technology Research and Innovation (ICITRI) · doi:10.1109/icitri71999.2026.11751339

SimRAG as Traceable Governance for Multilingual Thesaurus Localization: Evidence From AAT

Chen, S., & Wang, J.-S. (2026) · KNOWLEDGE ORGANIZATION · doi:10.31083/ko52642

Measuring Position Bias in LLM-as-a-Judge for Hiring

Aminou, L., Daaif, A., Soulami, M., & Youssfi, M. (2026) · International Conference on Advanced Computer Information Technologies (ACIT) · doi:10.1109/acit69831.2026.11699959

From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge

Li, D., Jiang, B., Huang, L., Beigi, A., Zhao, C., Tan, Z. Y. A., Bhattacharjee, A., Jiang, Y., Chen, C., Wu, T., et al. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.138 · Cited by 269 · see citing works

Am I More Pointwise or Pairwise? Revealing Position Bias in Rubric-Based LLM-as-a-Judge

Xu, Y., Hirasawa, T., Kozuno, T., & Ushiku, Y. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1444

Judging the Judges: A Systematic Evaluation of Bias Mitigation Strategies in LLM-as-a-Judge PipelinesPreprint

Soumik, S. K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.23178 · Cited by 1 · see citing works

The First Token Is Not the Verdict: Hidden Costs of Reading LLM Judges Without GeneratingPreprint

Villuri, G., Shaik, H., & Doboli, A. (2026) · arXiv (Cornell University)