Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agents
Open accessAbstract
As Multimodal Large Language Models (MLLMs) advance, multimodal agents show promise in real-world tasks like web navigation and embodied intelligence. However, due to limitations in a lack of external feedback, these agents struggle with self-correction and generalization. A promising approach is to use reward models as external feedback, but there is no clear on how to select reward models for agents. Thus, there is an urgent need to build a reward bench targeted at agents. To address these challenges, we propose Agent-RewardBench, a benchmark designed to evaluate reward modeling ability in MLLMs. The benchmark is characterized by three key features: (1) Multiple dimensions and real-world agent scenarios evaluation. It covers perception, planning, and safety with 7 scenarios; (2) Step-level reward evaluation. It allows for the assessment of agent capabilities at the individual steps of a task, providing a more granular view of performance during the planning process; and (3) Appropriately difficulty and high-quality. We carefully sample from 10 diverse models, difficulty control to maintain task challenges, and manual verification to ensure the integrity of the data. Experiments demonstrate that even state-of-the-art multimodal models show limited performance, highlighting the need for specialized training in agent reward modeling. Code is available at github.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing LLM-as-a-Judge benchmarks (RewardBench, RM-Bench, JudgeBench, AgentRewardBench) are multi-domain but not environment-aware and do not allow agentic interaction.
“| RewardBench (Lambert et al., 2025) | LLM-as-a-Judge | ✓ | ✗ | ✗ |”
From AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026) · cites this paper as Men et al., 2025
Section 3 describes Agent-RewardBench as evaluating collected responses across perception, planning, and safety, which supports the multi-domain characterization and suggests an offline evaluation rather than live agent interaction. But the Related Work section says it tests web grounding and embodied spatial perception, so the claim that Agent-RewardBench is not environment-aware is not supported. The supplied table row names RewardBench, not Agent-RewardBench; this paper does not establish the stated properties of every benchmark in the claim. (LLM, conf=0.85)
Experimental — accuracy not yet measuredOGRBench is the first holistic cross-platform ORM benchmark spanning Mobile, Web, and Desktop environments.
“We construct OGRBench, the first holistic crossplatform ORM benchmark spanning Mobile, Web, and Desktop environments”
From OS-Themis: A Scalable Critic Framework for Generalist GUI Rewards (2026) · cites this paper as Men et al., 2025
The Introduction (page unavailable) says Agent-RewardBench covers mobile, web, and desktop scenarios, but it does not identify OGRBench or establish that OGRBench is the first holistic cross-platform ORM benchmark. Searches for OGRBench and the claimed benchmark description found no passage addressing that claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Still being checked.
Research works printed but not linked: 45
- Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly ca- pable language model locally on your phone. arXiv preprint arXiv:2404.14219. · arXiv
- Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
- Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chud- novsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. 2024. Pixtral 12b. arXiv preprint arXiv:2410.07073. · arXiv
- Rizhao Cai, Zirui Song, Dayan Guan, Zhenhao Chen, Yaohang Li, Xing Luo, Chenyu Yi, and Alex Kot. 2024. Benchlmm: Benchmarking cross-style visual capability of large multimodal models. In Euro- pean Conference on Computer Vision, pages 340- 358. Springer.
- Pengfei Cao, Tianyi Men, Wencan Liu, Jingwen Zhang, Xuzhao Li, Xixun Lin, Dianbo Sui, Yanan Cao, Kang Liu, and Jun Zhao. 2025. Large language models for planning: A comprehensive and systematic survey. arXiv preprint arXiv:2505.19683. · arXiv
- Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Col- lier, Karthik Narasimhan, and Shunyu Yao. 2023a. Fireact: Toward language agent fine-tuning. arXiv preprint arXiv:2310.05915. · arXiv
- Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai, Yuchi Wang, Peiyi Wang, Tianyu Liu, and Baobao Chang. 2023b. Towards end-to-end embod- ied decision making via multi-modal large language model: Explorations with gpt4-vision and beyond. arXiv preprint arXiv:2310.02071. · arXiv
- Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai, Yuchi Wang, Peiyi Wang, Xiangdi Meng, Tianyu Liu, and Baobao Chang. 2024a. Pca-bench: Evaluating multimodal large language models in perception-cognition-action chain. arXiv preprint arXiv:2402.15527. · arXiv
- Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024b. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67(12):220101.
- Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024. Seeclick: Harnessing gui grounding for advanced visual gui agents. arXiv preprint arXiv:2401.10935. · arXiv
- Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. 2024a. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems, 36.
- Zehang Deng, Yongjian Guo, Changzhou Han, Wan- lun Ma, Junwu Xiong, Sheng Wen, and Yang Xiang. 2024b. Ai agents under threat: A survey of key security challenges and future pathways. ACM Com- puting Surveys.
- Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. · arXiv
- Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei- Chiu Ma, and Ranjay Krishna. 2024. Blink: Multi- modal large language models can see but not perceive. In European Conference on Computer Vision, pages 148-166. Springer.
- Yu Gu, Boyuan Zheng, Boyu Gou, Kai Zhang, Cheng Chang, Sanjari Srivastava, Yanan Xie, Peng Qi, Huan Sun, and Yu Su. 2024. Is your llm secretly a world model of the internet? model-based planning for web agents. arXiv preprint arXiv:2411.06559. · arXiv
- Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. · arXiv
- Zhuoran Jin, Hongbang Yuan, Tianyi Men, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2024. Rag- rewardbench: Benchmarking reward models in re- trieval augmented generation for preference align- ment. arXiv preprint arXiv:2412.13746. · arXiv
- Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. 2024a. Visualwebarena: Evaluat- ing multimodal agents on realistic visual web tasks. arXiv preprint arXiv:2401.13649. · arXiv
- Jing Yu Koh, Stephen McAleer, Daniel Fried, and Rus- lan Salakhutdinov. 2024b. Tree search for language model agents. arXiv preprint arXiv:2407.01476. · arXiv
- Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Effi- cient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
- Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. 2024a. Llava- onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326. · arXiv
- Lei Li, Yuancheng Wei, Zhihui Xie, Xuqing Yang, Yi- fan Song, Peiyi Wang, Chenxin An, Tianyu Liu, Sujian Li, Bill Yuchen Lin, et al. 2024b. Vlre- wardbench: A challenging benchmark for vision- language generative reward models. arXiv preprint arXiv:2411.17451. · arXiv
- Zongyu Lin, Yao Tang, Xingcheng Yao, Da Yin, Ziniu Hu, Yizhou Sun, and Kai-Wei Chang. 2025. Qlass: Boosting language agent inference via q-guided step- wise search. arXiv preprint arXiv:2502.02584. · arXiv
- Tianyi Men, Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, and Jun Zhao. 2024. Unlocking the future: Exploring look-ahead planning mechanistic inter- pretability in large language models. arXiv preprint arXiv:2406.16033. · arXiv
- Lu Qiu, Yuying Ge, Yi Chen, Yixiao Ge, Ying Shan, and Xihui Liu. 2024. Egoplan-bench2: A benchmark for multimodal large language model planning in real- world scenarios. arXiv preprint arXiv:2412.04447. · arXiv
- Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell- Ajala, et al. 2024. Androidworld: A dynamic bench- marking environment for autonomous agents. arXiv preprint arXiv:2405.14573. · arXiv
- Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. 2024. Trial and error: Exploration-based trajectory optimization for llm agents. arXiv preprint arXiv:2403.02502. · arXiv
- Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, et al. 2024. Os-genesis: Automating gui agent trajectory con- struction via reverse task synthesis. arXiv preprint arXiv:2412.19723. · arXiv
- Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal under- standing across millions of tokens of context. arXiv preprint arXiv:2403.05530. · arXiv
- Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhi- hao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhanc- ing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191. · arXiv
- Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xi- aojian Ma, and Yitao Liang. 2023. Describe, explain, plan and select: Interactive planning with large lan- guage models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560. · arXiv
- Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transform- ers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
- Qiucheng Wu, Handong Zhao, Michael Saxon, Trung Bui, William Yang Wang, Yang Zhang, and Shiyu Chang. 2024. Vsp: Assessing the dual challenges of perception and reasoning in spatial planning tasks for vlms. arXiv preprint arXiv:2407.01863. · arXiv
- Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024a. Travelplanner: A benchmark for real- world planning with language agents. arXiv preprint arXiv:2402.01622. · arXiv
- Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhou- jun Cheng, Dongchan Shin, Fangyu Lei, et al. 2024b. Osworld: Benchmarking multimodal agents for open- ended tasks in real computer environments. arXiv preprint arXiv:2404.07972. · arXiv
- Da Yin, Faeze Brahman, Abhilasha Ravichander, Khy- athi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. 2024. Agent lumos: Unified and modular training for open-source language agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12380-12403.
- Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. 2020. Bdd100k: A diverse driv- ing dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 2636- 2645.
- Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. 2023. Agenttuning: Enabling generalized agent abilities for llms. arXiv preprint arXiv:2310.12823. · arXiv
- Yuanzhao Zhai, Tingkai Yang, Kele Xu, Feng Dawei, Cheng Yang, Bo Ding, and Huaimin Wang. 2024. En- hancing decision-making for llm agents via step-level q-value models. arXiv preprint arXiv:2409.09345. · arXiv
- Min Zhang, Xian Fu, Jianye Hao, Peilong Han, Hao Zhang, Lei Shi, Hongyao Tang, and Yan Zheng. 2024a. Mfe-etp: A comprehensive evaluation bench- mark for multi-modal foundation models on embod- ied task planning. arXiv preprint arXiv:2407.05047. · arXiv
- Yanzhe Zhang, Tao Yu, and Diyi Yang. 2024b. Attack- ing vision-language computer agents via pop-ups. arXiv preprint arXiv:2411.02391. · arXiv
- Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. 2024c. Webpilot: A versatile and autonomous multi-agent system for web task exe- cution with strategic exploration. arXiv preprint arXiv:2408.15978. · arXiv
- Enyu Zhou, Guodong Zheng, Binghai Wang, Zhiheng Xi, Shihan Dou, Rong Bao, Wei Shen, Limao Xiong, Jessica Fan, Yurong Mou, et al. 2024a. Rmb: Com- prehensively benchmarking reward models in llm alignment. arXiv preprint arXiv:2410.09893. · arXiv
- Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, An- derson Compalas, Dawn Song, and Xin Eric Wang. 2024b. Multimodal situational safety. arXiv preprint arXiv:2410.06172. · arXiv
- Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. 2023. We- barena: A realistic web environment for building au- tonomous agents. arXiv preprint arXiv:2307.13854. · arXiv
Cited by
Sorted by citations
AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents
Fan, S., Ye, X., Huo, Y., Chen, Z.-Y., Guo, Y., Yang, S., Yang, W., Ye, S., Chen, J., Chen, H., et al. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817494 · Cited by 1 · see citing works
OS-Themis: A Scalable Critic Framework for Generalist GUI Rewards
Li, Z., Wu, Z., Zhao, Y., Yang, B., Xie, J., Liu, Z., Liu, Z., Jin, K., Liang, J., Li, Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2603.19191
“Similarly, recent works like OS-Critic Bench (Wu et al., 2025b), AgentRe-wardBench (Men et al., 2025), and CUAReward-Bench (OpenAI, 2025) are restricted to isolated domains (e.g., Web or Desktop) or prioritize step-level supervision.”
AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation
Shi, W., Wang, Y., Zhao, Y., Chen, Y., Feng, F., Hao, X., Su, X., Gu, Q., Su, H., Cai, X., et al. (2026) · arXiv · doi:10.48550/arxiv.2604.18240
“…a natural progression is to endow RewardBench (Lambert et al., 2025) LLM-as-a-Judge ✓ ✗ ✗ RM-Bench (Liu et al., 2025b) LLM-as-a-Judge ✓ ✗ ✗ JudgeBench (Tan et al., 2025) LLM-as-a-Judge ✓ ✗ ✗ AgentRewardBench (Men et al., 2025) LLM-as-a-Judge ✓ ✗ ✗ DevAI (Zhuge et al., 2025) Agent-as-a-Judge ✗ ✓ ✓”
Self-citationLLM-based agents for generalized web automation and schema-guided data extraction: a survey
Khayoub, I., Chadi, M.-A., Mousannif, H., & Mohamed, F. A. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01335-y
- Authors
- Men, T., Jin, Z., Cao, P., Chen, Y., Liu, K., & Zhao, J.
- Affiliations
- Chinese Academy of Sciences (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- Pages
- 17521-17541
- ISSN
- 0736-587X (ISSN-L)
- arXiv
- 2506.21252
- OpenAlex
- W4412945066
- Cited by
- 2
- References
- 0
- Citations by year
- 2026: 2
- FWCI
- 1.49
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (U24A20335)
- China Postdoctoral Science Foundation
- Youth Innovation Promotion Association of the Chinese Academy of Sciences
- Youth Innovation Promotion Association
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- reward modeling, multimodal large language models, multimodal agents, benchmark, perception, planning, safety, self-correction, web navigation, embodied agents, external feedback
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
