RewardBench: Evaluating Reward Models for Language Modeling
Open accessAbstract
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Judge validation in practice relies heavily on exact match, and established judge benchmarks privilege raw agreement as the headline validation metric.
“Despite these challenges, judge validation continues to rely heavily on exact match.Established judge benchmarks-MT-Bench (Zheng et al., 2023), RewardBench (Lambert et al., 2025), JudgeBench (Tan et al., 2025)-and recent successors (Malik et al., 2026;Whitehouse et al., 2025;Jiang et al., 2025) privilege raw agreement as the headline validation metric.”
From Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Bias (2026) · cites this paper as Lambert et al., 2025
Section 4.2, “REWARDBENCH Scoring” (page unavailable), says RewardBench scores models by whether they rank the chosen response above the rejected one, then aggregates accuracy into a final score. That supports the claim that RewardBench makes agreement with preference labels its headline metric. The paper does not establish that judge validation generally relies heavily on exact match, or that the other named benchmarks use the same metric. (LLM, conf=0.84)
Experimental — accuracy not yet measuredExisting LLM-as-a-Judge benchmarks (RewardBench, RM-Bench, JudgeBench, AgentRewardBench) are multi-domain but not environment-aware and do not allow agentic interaction.
“| RewardBench (Lambert et al., 2025) | LLM-as-a-Judge | ✓ | ✗ | ✗ |”
From AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026) · cites this paper as Lambert et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Part of this paper’s reference list could not be split into single entries.
Research works printed but not linked: 4
- Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. 2022. Understanding dataset difficulty with V-usable information. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 5988-6008.
- PMLR.
- Nathan Lambert, Thomas Krendl Gilbert, and Tom Zick. 2023. The history and risks of reinforcement learning and human feedback. arXiv e-prints, pages arXiv- 2310.
- Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learn- ing to summarize with human feedback. In Ad- vances in Neural Information Processing Systems, volume 33, pages 3008-3021.
Cited by
Sorted by citations
A Survey of Progress in LLM Alignment From the Perspective of Reward Design
Ji, M., Wu, Y., Wu, Z., Wang, S., Yang, J., Dras, M., & Naseem, U. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3654921 · Cited by 10 · see citing works
A Survey of Post-Training Scaling in Large Language Models
Lai, H., Liu, X., Gao, J., Cheng, J., Qi, Z., Xu, Y., Yao, S., Zhang, D., Du, J., Hou, Z., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.140 · Cited by 8 · see citing works
REACT: REinforcement Learning-Based Adaptive ECG Anonymization and Privacy Threat Mitigation*
Datta, A., Bhattacharyya, T., Khatibi, E., Seth, A., Wang, Z., Moosavi, S. R., Rahmani, A. M., Firouzi, F., & Chakrabarty, K. (2025) · IEEE International Conference on Omni-layer Intelligent Systems (COINS) · doi:10.1109/coins65080.2025.11125728 · Cited by 3 · see citing works
RLHF-Driven Intrusion Detection and Reward Analysis in Industrial IoT
Hanlin, C. H. E. N., Qi, E., Zuo, M., & Shi, X. (2025) · International Conference on Blockchain Technology and Information Security (ICBCTIS) · doi:10.1109/icbctis66509.2025.11387352 · Cited by 2 · see citing works
No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding
Krumdick, M., Lovering, C., Reddy, V., Ebner, S., & Tanner, C. (2025) · arXiv · doi:10.48550/arxiv.2503.05061 · Cited by 1 · see citing works
Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)
Jiang, L., Chai, Y., Li, M., Liu, M., Fok, R., Dziri, N., Tsvetkov, Y., Sap, M., Albalak, A., & Choi, Y. (2025) · arXiv · doi:10.48550/arxiv.2510.22954 · Cited by 1 · see citing works
Self-citationCamelEval: Advancing Benchmarks for Arabic Language Models in Generative Tasks
Qian, Z., AL-Tam, F., Al‐Qurishi, M., Iacobacci, I., Aldawsari, A., & Souissi, R. (2025) · IEEE Access · doi:10.1109/access.2025.3646336 · Cited by 1 · see citing works
CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling
Liu, D., Yang, F., Wang, X., Yan, S., Chai, J., Li, J., Ban, Y., Mao, Z., Lin, W., & Yin, G. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817703 · Cited by 1 · see citing works
Can Textual Reasoning Improve the Performance of MLLMs on Fine-Grained Visual Classification?
Zhu, J., Su, Y., & Liu, X. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00901 · Cited by 1 · see citing works
AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation
Shi, W., Wang, Y., Zhao, Y., Chen, Y., Feng, F., Hao, X., Su, X., Gu, Q., Su, H., Cai, X., et al. (2026) · arXiv · doi:10.48550/arxiv.2604.18240
“Comprehensive benchmarks like RewardBench (Lambert et al., 2025) and RM-Bench (Liu et al., 2025b) further examine judges across diverse domains including safety, dialogue, and reasoning.”
InfluentialEvaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis
Goo, T., Kim, J., Han, K., Jo, G., Kim, J.-S., & Kim, T.-H. (2026) · arXiv · doi:10.48550/arxiv.2607.20527
"Give a Positive Review Only": Can Hidden PDF Prompts Sway LLMs?Preprint
Guo, X., Huang, Z., Song, Z., & Zhang, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5368453
Think-to-Detect: Rationale-Driven Vision–Language Anomaly Detection
Abdalla, M. I., Kasem, M. S., Mahmoud, M., Senussi, M. F., Abdallah, A., & Kang, H. (2025) · Mathematics · doi:10.3390/math13243920
What Makes a Good Response? Learning Personal Preferences from Interpretable Features
Afsharrad, A., Soroka, E., O’Neill, D., & Lall, S. (2026) · ICON · doi:10.1109/icnlp69856.2026.11527497
Tang, Y., Liu, Z., Chen, Z., Yang, J., Xiang, L., Xu, Z., & He, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-9875246/v1
Automated bias blind spots: Examining stereotype associations in VLM-as-a-judge paradigms
Hong, R. S., Ovalle, A., Ung, M., Spiliopoulou, E., Sagun, L., Williams, A., & Ross, C. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806459
AutoQABench: A Three-Level UX Benchmark for Automated Evaluation of Open-Ended LLM Responses
Wang, T., Yuan, F., Miao, C., Kaniyanoor, M. S., & Matsumoto, K. (2026) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest70988.2026.00011
Arbiter: Unifying Executable Verification and Semantic Assessment
Lu, Y., Huang, Y., Ying, Y., Li, L., Yang, T., & Tie, Y. (2026) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr70517.2026.00067
- Authors
- Lambert, N., Pyatkin, V., Morrison, J., Miranda, L., Lin, B. Y., Chandu, K., Dziri, N., Kumar, S., Zick, T., Choi, Y., Smith, N. A., & Hajishirzi, H.
- Affiliations
- University of Washington (US)
- ORCID
- Venue
- Findings of the Association for Computational Linguistics: NAACL 2025
- Published
- 2025
- Pages
- 1755-1797
- OpenAlex
- W4411113083
- Cited by
- 13
- References
- 0
- Citations by year
- 2026: 8 · 2025: 5
- FWCI
- 12.33
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: NAACL (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- reward models, language modeling, benchmark, large language models, reinforcement learning from human feedback
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
