Evaluation and Benchmarking of LLM Agents: A Survey
Open accessPublished version of a preprint
The DOI matches its Crossref record; 136 of the 136 references it cites were found in a registry or an index; 52 of the 80 sources its claims cite were checked for support.
Abstract
The rise of LLM-based agents has opened new frontiers in AI applications, yet evaluating these agents remains a complex and underdeveloped area.This survey provides an in-depth overview of the emerging field of LLM agent evaluation, introducing a twodimensional taxonomy that organizes existing work along (1) evaluation objectives-what to evaluate, such as agent behavior, capabilities, reliability, and safety-and (2) evaluation process-how to evaluate, including interaction modes, datasets and benchmarks, metric computation methods, and tooling.In addition to taxonomy, we highlight enterprise-specific challenges, such as role-based access to data, the need for reliability guarantees, dynamic and longhorizon interactions, and compliance, which are often overlooked in current research.We also identify the future research directions, including holistic, more realistic, and scalable evaluation.This work aims to bring clarity to the fragmented landscape of agent evaluation and provide a framework for systematic assessment, enabling researchers and practitioners to evaluate LLM agents for real-world deployment.
Overview
CoChat Summary
TL;DR
This survey proposes a two-dimensional taxonomy for LLM agent evaluation along evaluation objectives (what to evaluate) and evaluation process (how to evaluate), and highlights enterprise-specific challenges and future directions. It aims to organize fragmented work and guide systematic assessment for real-world deployment.
Key findings
- The taxonomy separates evaluation objectives—agent behavior, capabilities, reliability, and safety/alignment—from evaluation process—interaction mode, evaluation data, metrics computation, tooling, and evaluation contexts.
- Enterprise-specific challenges include role-based access control, reliability guarantees, dynamic and long-horizon interactions, and adherence to domain-specific policies and compliance requirements such as GDPR or HIPAA.
- Reported metrics and benchmarks include Success Rate, pass@k, pass^k, Invocation Accuracy, Tool Selection Accuracy, MRR, NDCG, parameter name F1, Factual Recall Accuracy, and Consistency Score; memory evaluations are described with dialogues of 40+ turns and 600+ turns.
- Future directions are more realistic evaluation settings, holistic evaluation frameworks, automated and scalable evaluation techniques, and time- and cost-bounded evaluation protocols.
Methods
- Survey/position paper that proposes a two-dimensional taxonomy and reviews prior work on LLM agent evaluation; no primary experiments are reported.
- Organizes existing literature by evaluation objectives and evaluation process, then discusses enterprise challenges and open research directions.
- Draws on cited benchmarks, datasets, metrics, and tooling, including SWE-bench, WebArena, HELM, τ-bench, LangSmith, Arize AI, and OpenAI Evals.
Limitations
- The paper states that existing surveys focus narrowly on LLM evaluation or specific agent capabilities without a holistic perspective, and that enterprise requirements are rarely addressed in existing literature.
- The paper states that current benchmarks often focus on short episodes or single interactions, and that measuring reliability requires repeated trials, introducing significant evaluation overhead.
- Evident from the text: no systematic review protocol, search strategy, or inclusion criteria are described; the taxonomy is conceptual and not empirically validated; the survey reports no primary data or quantitative results of its own.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
35 Valid · 3 Not valid · 13 Not checked · 17 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“To address this limitation, recent work, such as the Gorilla paper, has proposed execution-based evaluation, in which the system runs the tool calls and assesses their outcomes, offering a more comprehensive and grounded assessment of tool use capability [75].”
What the sources say
- Disputedcites [75] [75] Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. Gorilla: Large Language Model Connected with Massive APIs. doi:10.48550/arXiv.2305. 15334 arXiv:2305.15334 [cs]. - sourceExperimental — accuracy not yet measured
Section 3.3, “Verifying APIs” (page unavailable), says test cases fall short for evaluating API calls and describes AST tree matching as the paper’s evaluation strategy. Section 4.1 reports AST accuracy. The paper therefore describes the opposite approach from the execution-based evaluation attributed to it. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Others, such as FlowBench [96], ToolBench [38], and API-Bank [47], focus on tool use and function-calling across large API repositories.”
What the sources say
- Partly supportedcites [96] [96] Ruixuan Xiao, Wentao Ma, Ke Wang, Yuchuan Wu, Junbo Zhao, Haobo Wang, Fei Huang, and Yongbin Li. 2024. FlowBench: revisiting and benchmarking workflow-guided planning for LLM-based agents. doi:10.48550/arXiv.2406.14884 - sourceExperimental — accuracy not yet measured
Section 4.2 says FlowBench includes tool-invocation information in GPT-4 function-calling format, so the tool-use and function-calling part applies. The Introduction identifies its focus as workflow-guided agent planning across 6 domains and 51 scenarios; our copy does not describe it as a benchmark across large API repositories. The FlowBench citation is assessed here, not the separate citations to ToolBench and API-Bank. (LLM, conf=0.88)
- Disputedcites [38] [38] Yue Huang, Jiawen Shi, Yuan Li, Chenrui Fan, Siyuan Wu, Qihui Zhang, Yixin Liu, Pan Zhou, Yao Wan, Neil Zhenqiang Gong, and Lichao Sun. 2024. MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use. arXiv:2310.03128 [cs.SE] https://arxiv.org/abs/2310.03128 - sourceExperimental — accuracy not yet measured
In the “Dimension APIBank GPT4Tool” comparison table (page unavailable), the paper lists ToolBench and MetaTool as separate benchmarks and labels MetaTool “Ours.” It therefore contradicts the identification implied by citing this paper as ToolBench; the passage also does not establish the claim about function-calling across large API repositories. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
- Supportedcites [47] [47] Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. API-bank: a comprehensive benchmark for tool-augmented LLMs. doi:10.48550/arXiv.2304.08244 - source
The Introduction describes API-Bank as a benchmark for tool-augmented LLMs that evaluates planning, retrieving, and calling API tools. Benchmark Analysis reports 2,211 APIs across 1,008 domains, supporting the characterization of a large API repository. The citation to API-Bank supports the API-Bank part of the sentence; the other datasets have separate citations. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“This crucial objective is applied across a wide range of LLM agent evaluation domains and benchmarks [93].”
What the sources say
- Supportedcites [93] [93] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. 2024. A Survey on Large Language Model based Autonomous Agents. Frontiers of Computer Science 18, 6 (Dec. 2024), 186345. doi:10.1007/ s11704-024-40231-1 - source
The Metrics section identifies task-success metrics as a way to measure how well agents complete tasks and lists success rate among common measures. Its Protocols and Benchmarks discussions describe their use in varied evaluation settings and list benchmarks across multiple domains. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although sometimes noted for providing limited fine-grained insight into failures, especially when most models achieve low success rates [64], task completion remains a predominant and essential measure of overall agent performance [64].”
What the sources say
- Partly supportedcites [64] [64] Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. 2024. AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents. doi:10.48550/arXiv.2401.13178 - sourceExperimental — accuracy not yet measured
Section 2.2 describes success rate—the proportion of tasks completed—as the predominant metric for agent evaluation and uses it alongside progress rate. It does not establish that task completion is essential; Section 4.2 instead says progress rate is a more effective measure for distinguishing overall model performance. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Beyond avoiding overt toxicity, many deployments require agents to comply with specific regulatory or policy constraints [6, 123].”
What the sources say
- Supportedcites [6] [6] Nik Bear Brown. 2024. Enhancing trust in llms: Algorithms for comparing and interpreting llms. arXiv preprint arXiv:2406.01943 (2024). - source
Section 10.2, “Application in Evaluating LLMs” (page unavailable), says LLM evaluation is critical for meeting regulatory requirements and ethical standards. The Introduction also identifies adherence to regulations and, in medical applications, regulatory standards as considerations for real-world use. These passages support the broad claim that deployments can require compliance with such constraints. (LLM, conf=0.86)
- Could not verifycites [123] [123] Zhiping Zhang, Michelle Jia, B Yao, S Das, A Lerner, D Wang, and T Li. 2023. It'sa fair game, or is it? examining how users navigate disclosure risks and benefits when using llm-based conversational agents. arXiv preprint arXiv:2309.11653 (2023). - sourceExperimental — accuracy not yet measured
Section 5.2.6 (p. 10) reports that users avoid sharing confidential work data because of company policies or NDAs. The passages found do not address whether deployments require agents themselves to comply with regulatory or policy constraints; searches for agent compliance and regulatory requirements found no such statement. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
26 more claims · 17 with their cited sources checked
Create a free account to see all 68 claims and to verify the 12 references still open.
- It may state two separate claims as one. 16
- Probably not a substantial claim. 30
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Bansal et al.'s evaluation-benchmarking survey of LLM agents [86] catalogs evaluation methodology but does not address IoT-specific deployment concerns.”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [86]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“All tested production models exhibit between 26% and 41% successful prompt-injection rates (CyberSecEval 3 against GPT-4 and Llama-3-70B, respectively, as reported in [19,86])”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [86]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“CyberSecEval (versions 1-3) and CyberSOCEval, summarized in [19,86], benchmark LLM safety properties: insecure coding suggestions, cyberattack helpfulness, prompt-injection resistance, codeinterpreter abuse, and autonomous offensive capability.”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [86]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“12 patterns are derived from the public CyberSecEval 3 indirect-prompt-injection split [19,86] (rephrased to fit IoT-telemetry observation shapes); 6 patterns adapt indirect-prompt-injection vectors from Greshake et al. [94]; and 7 patterns are author-generated to cover IoT-specific surfaces not present in either source”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [86]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.2 Agent Capabilities · p. 4
“LongEval [43] and SocialBench [9] are benchmarks that test an agent's context retention in long dialogues (40+ turns).”
§ References · p. 9
“[43] Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, and Kyle Lo. 2023. LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization.”
Citations
Cites
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 1×
“…MATSA [66], GAMEBENCH [15], BALROG [72], TheAgentCompany [ CASA [77], R-Judge [112], SimuCourt [35], MATSA [66], FinCon [111], AutoGuide [24] Harm Adversarial Robustness, Prompt Injection Resistance, Harmfulness, Bias Detection Agent Security Bench(ASB) [118], AgentPoison [13], AgentDojo [17],…”
Know What You Don't Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · arXiv · doi:10.48550/arxiv.1806.03822 · Cited by 2,242
Know What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240 · cited 2×
“Factual Accuracy Recall, Consistency Scores LongEval [43], SocialBench [9], LoCoMo [65], Optimus-1 [50] Multi-Agent Collaboration Information Sharing Effectiveness, Adaptive Role Switching, Reasoning Rating AgentSims [52], WebArena [126], MATSA [66], GAMEBENCH [15], BALROG [72], TheAgentCompany [ CASA [77], R-Judge [1…”
InfluentialGenerative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941
A Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888
Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y., Ma, S., Liu, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.15594 · Cited by 1,781
Large Language Model based Multi-Agents: A Survey of Progress and Challenges
Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., & Zhang, X. (2024) · arXiv · doi:10.48550/arxiv.2402.01680 · Cited by 1,713
ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
Chan, C.-M., Chen, W., Su, Y., Yu, J., Xue, W., Zhang, S., Fu, J., & Liu, Z. (2023) · arXiv · doi:10.48550/arxiv.2308.07201 · Cited by 725
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · arXiv · doi:10.48550/arxiv.1606.05250 · Cited by 678
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024) · arXiv · doi:10.48550/arxiv.2406.13352 · Cited by 616
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608 · cited 1×
“This pattern is illustrated by the ReAct paradigm, where agents alternate between reasoning steps and tool usage [106].”
A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges
Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y. (2024) · Vicinagearth · doi:10.1007/s44336-024-00009-2 · Cited by 404
PaperBench: Evaluating AI's Ability to Replicate AI Research
Starace, G., Jaffe, O., Sherburn, D., Aung, J., Chan, J. S., Maksin, L., Dias, R., Mays, E., Kinsella, B., Thompson, W., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.01848 · Cited by 310
Measuring Massive Multitask Language Understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2009.03300 · Cited by 291
Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
Zhang, A. K., Perry, N., Dulepet, R., Ji, J., Menders, C., Lin, J. W., Jones, E. K., Hussein, G., Liu, S., Jasper, D. J., et al. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2408.08926 · Cited by 268
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255
MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
Tao, W., Zhou, Y., Wang, Y., Zhang, W., Zhang, H., & Cheng, Y. (2024) · neural information processing systems · doi:10.52202/079017-1647 · Cited by 255
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
Chen, Z., Xiang, Z., Xiao, C., Song, D., & Li, B. (2024) · arXiv · doi:10.48550/arxiv.2407.12784 · Cited by 246
MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue ResolutionPreprint
Tao, W., Zhou, Y., Wang, Y.-L., Zhang, W., Zhang, H.-Y., & Cheng, Y. (2024) · Neural Information Processing Systems · doi:10.48550/arxiv.2403.17927 · Cited by 232
Gorilla: Large Language Model Connected with Massive APIs
Patil, S. G., Zhang, T., Wang, X., & Gonzalez, J. E. (2023) · arXiv · doi:10.48550/arxiv.2305.15334 · Cited by 226 · cited 2×
“To address this limitation, recent work, such as the Gorilla paper, has proposed execution-based evaluation, in which the system runs the tool calls and assesses their outcomes, offering a more comprehensive and grounded assessment of tool use capability [75].”
Gorilla: Large Language Model Connected with Massive APIs
Patil, S., Zhang, T., Wang, X., & Gonzalez, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-4020 · Cited by 226 · cited 2×
“To address this limitation, recent work, such as the Gorilla paper, has proposed execution-based evaluation, in which the system runs the tool calls and assesses their outcomes, offering a more comprehensive and grounded assessment of tool use capability [75].”
AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
Yoran, O., Amouyal, S. J., Malaviya, C., Bogin, B., Press, O., & Berant, J. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.505 · Cited by 212
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3680384 · Cited by 202
AutoCodeRover: Autonomous Program Improvement
Zhang, Y., Ruan, H., Fan, Z., & Roychoudhury, A. (2024) · arXiv · doi:10.48550/arxiv.2404.05427 · Cited by 202
Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM AgentsPreprint
Zhan, Q., Fang, R., Panchal, H., & Kang, D. (2025) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2503.00061 · Cited by 196
The BrowserGym Ecosystem for Web Agent Research
Chezelles, T. L. S. D., Gasse, M., Drouin, A., Caccia, M., Boisvert, L., Thakkar, M., Marty, T., Assouel, R., Shayegan, S. O., Jang, L., et al. (2025) · Transactions on machine learning research · doi:10.48550/arxiv.2412.05467 · Cited by 186
A Survey on the Memory Mechanism of Large Language Model based Agents
Zhang, Z., Bo, X., Ma, C., Li, R., Chen, X., Dai, Q., Zhu, J., Dong, Z., & Wen, J.-R. (2024) · arXiv · doi:10.48550/arxiv.2404.13501 · Cited by 149 · cited 1×
“Despiteincreasing interest in this space, existing surveys focus narrowly on LLM evaluation or cover specific agent capabilities without a holistic perspective [121].”
SocialBench: Sociality Evaluation of Role-Playing Conversational Agents
Chen, H., Chen, H., Yan, M., Xu, W., Xing, G., Shen, W., Quan, X., Li, C., Zhang, J., & Huang, F. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.125 · Cited by 137
GameBench: Evaluating Strategic Reasoning Abilities of LLM AgentsPreprint
Costarelli, A., Allen, M., Hauksson, R., Sodunke, G., Hariharan, S., Cheng, C., Li, W., Clymer, J., & Yadav, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.06613 · Cited by 131
API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
Li, M., Zhao, Y., Yu, B., Song, F., Li, H., Yu, H., Li, Z., Huang, F., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2304.08244 · Cited by 124 · cited 2×
“They struggle with evaluating open-ended or qualitative responses, such as natural language generation or creative problem-solving, where correctness is subjective.”
Self-citationAPI-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
Li, M., Zhao, Y., Yu, B., Song, F., Li, H., Yu, H., Li, Z., Huang, F., & Li, Y. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.187 · Cited by 124 · cited 2×
“They struggle with evaluating open-ended or qualitative responses, such as natural language generation or creative problem-solving, where correctness is subjective.”
Self-citationAdaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents
Zhan, Q., Fang, R., Panchal, H. S., & Kang, D. (2025) · Findings of the Association for Computational Linguistics: NAACL 2025 · doi:10.18653/v1/2025.findings-naacl.395 · Cited by 124
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
Lu, C., Lu, C., Lange, R. T., Foerster, J., Clune, J., & Ha, D. (2024) · arXiv · doi:10.48550/arxiv.2408.06292 · Cited by 118
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Yao, S., Chen, H., Yang, J., & Narasimhan, K. (2022) · arXiv · doi:10.48550/arxiv.2207.01206 · Cited by 107
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Yao, S., Chen, H., Yang, J., & Narasimhan, K. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1508 · Cited by 107
Evaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.747 · Cited by 96
Zhang, Z., Jia, M., Lee, H.-P. (., Yao, B., Das, S., Lerner, A., Wang, D., & Li, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642385 · Cited by 95
Xu, L., Hu, Z.-Y., Zhou, D.-Q., et al. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2311.08562 · Cited by 92
Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI TestingPreprint
Liu, Z., Chen, C., Wang, J., Chen, M., Wu, B., Che, X., Wang, D., & Wang, Q. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.09434 · Cited by 85
AgentBench: Evaluating LLMs as Agents
Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.03688 · Cited by 79
RAGAs: Automated Evaluation of Retrieval Augmented Generation
Es, S., James, J., Espinosa-Anke, L., & Schockaert, S. (2023) · arXiv · doi:10.48550/arxiv.2309.15217 · Cited by 79
MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM AgentsPreprint
Wang, L., Deng, Y., Zha, Y., Mao, G., Wang, Q., Min, T., Chen, W., & Chen, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.08184 · Cited by 79
Agent AI: Surveying the Horizons of Multimodal Interaction
Durante, Z., Huang, Q., Wake, N., Gong, R., Park, J. S., Sarkar, B., Taori, R., Noda, Y., Terzopoulos, D., Choi, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.03568 · Cited by 75 · cited 1×
“They reason and make plans, execute tools, leverage memory, and even collaborate with humans or other agents [20].”
Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile AgentsPreprint
Deng, S., Xu, W.-K., Sun, H., Liu, W., Tan, T., Liu, J.-F., Li, A., Luan, J., Wang, B., Yan, R., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2407.00993 · Cited by 75
Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacy
Guan, Z., Kong, X., Zhong, F., & Wang, Y. (2024) · neural information processing systems · doi:10.52202/079017-3925 · Cited by 75
FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents
Xiao, R., Ma, W., Wang, K., Wu, Y., Zhao, J., Wang, H., Huang, F., & Li, Y. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.638 · Cited by 75
Self-citationSPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
Chen, J., Yuen, D., Xie, B., Yang, Y., Chen, G., Wu, Z., Yixing, L., Zhou, X., Liu, W., Wang, S., et al. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.15164 · Cited by 73
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2310.06770 · Cited by 72 · cited 3×
“Factual Accuracy Recall, Consistency Scores LongEval [43], SocialBench [9], LoCoMo [65], Optimus-1 [50] Multi-Agent Collaboration Information Sharing Effectiveness, Adaptive Role Switching, Reasoning Rating AgentSims [52], WebArena [126], MATSA [66], GAMEBENCH [15], BALROG [72], TheAgentCompany [ CASA [77], R-Judge [1…”
InfluentialMMInA: Benchmarking Multihop Multimodal Internet Agents
Tian, S., Zhang, Z., Chen, L., & Liu, Z. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.703 · Cited by 72
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.16789 · Cited by 71 · cited 1×
“Other related metrics include Task Goal Completion (TGC) [91] and Pass Rate [76].”
Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents
Talebirad, Y., & Nadiri, A. (2023) · arXiv · doi:10.48550/arxiv.2306.03314 · Cited by 65 · cited 1×
“Evaluating multi-agent collaboration in LLM-based systems requires different methodologies compared to traditional reinforcement learning-driven coordination [7, 48, 89].”
The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies
He, F., Zhu, T., Ye, D., Liu, B., Zhou, W., & Yu, P. S. (2024) · arXiv · doi:10.48550/arxiv.2407.19354 · Cited by 60
Integrity-based Explanations for Fostering Appropriate Trust in AI Agents
Mehrotra, S., Jorge, C. C., Jonker, C. M., & Tielman, M. L. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3610578 · Cited by 51
Evaluating Cultural and Social Awareness of LLM Web AgentsPreprint
Qiu, H., Fabbri, A. R., Agarwal, D., et al. (2024) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2410.23252 · Cited by 48
Understanding the planning of LLM agents: A survey
Huang, X., Liu, W., Chen, X., Wang, X., Wang, H., Lian, D., Wang, Y., Tang, R., & Chen, E. (2024) · arXiv · doi:10.48550/arxiv.2402.02716 · Cited by 47 · cited 2×
“At the same time, reasoning enables agents to make context-aware decisions, either ahead of time or dynamically during task execution [37].”
Protecting Your LLMs with Information Bottleneck
Liu, Z., Wang, Z., Xu, L., Wang, J., Song, L., Wang, J., Chen, C., Cheng, W., & Bian, J. (2024) · neural information processing systems · doi:10.52202/079017-0935 · Cited by 46
Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security
Li, Y., Wen, H., Wang, W., Li, X., Yuan, Y., Liu, G., Liu, J., Xu, W., Wang, X., Sun, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.05459 · Cited by 43
Self-citationLLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
Li, H., Dong, Q., Chen, J., Su, H., Zhou, Y., Ai, Q., Ye, Z., & Liu, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.05579 · Cited by 40 · cited 1×
“Xia et al. [95] further propose an AgentOps architecture to continuously monitor deployed agents, closing the loop between development and deployment through real-time feedback and quality control.”
LLM Multi-Agent Systems: Challenges and Open Problems
Han, S., Zhang, Q., Jin, W., & Xu, Z. (2024) · arXiv · doi:10.48550/arxiv.2402.03578 · Cited by 33
GUI Agents with Foundation Models: A Comprehensive Survey
Wang, S., Liu, W., Chen, J., Zhou, Y., Gan, W., Zeng, X., Che, Y., Yu, S., Hao, X., Shao, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.04890 · Cited by 32
T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step
Chen, Z., Du, W., Zhang, W., Liu, K., Liu, J., Zheng, M., Zhuo, J., Zhang, S., Lin, D., Chen, K., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.515 · Cited by 32
WebArena: A Realistic Web Environment for Building Autonomous Agents
Zhou, S., Xu, F. F., Zhu, H., Zhou, X., Lo, R., Sridhar, A., Cheng, X., Ou, T., Bisk, Y., Fried, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.13854 · Cited by 31
T-Eval: Evaluating the Tool Utilization Capability Step by StepPreprint
Chen, Z., Du, W., Zhang, W.-W., Liu, K.-K., Liu, J.-N., Zheng, M., Zhuo, J.-M., Zhang, S.-Y., Lin, D.-H., Chen, K., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2312.14033 · Cited by 30 · cited 2×
“The T-Eval framework [12] addresses this by introducing a reasoning metric that assesses how closely an agent’s predicted next tool call aligns with the expected one at each step.”
Kang, H., & Xiong, C. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.303 · Cited by 30
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Krishna, K., Bransom, E., Kuehl, B., Iyyer, M., Dasigi, P., Cohan, A., & Lo, K. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.121 · Cited by 29
Evaluating Cultural and Social Awareness of LLM Web Agents
Qiu, H., Fabbri, A. R., Agarwal, D., Huang, K.-H., Tan, S., Peng, N., & Wu, C.-S. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.222 · Cited by 29
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
Ma, C., Zhang, J., Zhu, Z., Yang, C., Yang, Y., Jin, Y., Lan, Z., Kong, L., & He, J. (2024) · arXiv · doi:10.48550/arxiv.2401.13178 · Cited by 26
Survey on Evaluation of LLM-based Agents
Yehudai, A., Eden, L., Li, A., Uziel, G., Zhao, Y., Bar-Haim, R., Cohan, A., & Shmueli-Scheuer, M. (2025) · arXiv · doi:10.48550/arxiv.2503.16416 · Cited by 26
LLM Agents can Autonomously Exploit One-day Vulnerabilities
Fang, R., Bindu, R., Gupta, A., & Kang, D. (2024) · arXiv · doi:10.48550/arxiv.2404.08144 · Cited by 26
Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents
Yang, W., Bi, X., Lin, Y., Chen, S., Zhou, J., & Sun, X. (2024) · arXiv · doi:10.48550/arxiv.2402.11208 · Cited by 26
AgentBoard: An analytical evaluation board of multi-turn LLM agents
Ma, C., Zhang, J., Zhu, Z., Yang, C., Yang, Y., Jin, Y., Lan, Z., Kong, L., & He, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2365 · Cited by 26
TaskBench: Benchmarking Large Language Models for Task Automation
Shen, Y., Song, K., Tan, X., Zhang, W., Ren, K., Yuan, S., Lu, W., Li, D., & Zhuang, Y. (2023) · arXiv · doi:10.48550/arxiv.2311.18760 · Cited by 24 · cited 4×
“The parameter name F1 score [83] measures the agent’s ability to identify the parameter names required for a given function correctly and then correctly assign values to them.”
InfluentialTaskBench: Benchmarking Large Language Models for Task Automation
Shen, Y., Song, K., Tan, X., Zhang, W., Ren, K., Yuan, S., Lu, W., Li, D., & Zhuang, Y. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-0148 · Cited by 24
EnDex: Evaluation of Dialogue Engagingness at Scale
Xu, G., Liu, R., Harel-Canada, F., Chandra, N. R., & Peng, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.359 · Cited by 20
Identifying the Risks of LM Agents with an LM-Emulated Sandbox
Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., Dubois, Y., Maddison, C. J., & Hashimoto, T. (2023) · arXiv · doi:10.48550/arxiv.2309.15817 · Cited by 19
Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
Fourney, A., Bansal, G., Mozannar, H., Tan, C., Salinas, E., Erkang, Zhu, Niedtner, F., Proebsting, G., Bassman, G., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.04468 · Cited by 17 · cited 2×
“As these agents move from research prototypes to real-world applications [23, 62], the question of how to rigorously evaluate them becomes both pressing and complex.”
Autonomous Agents for Collaborative Task under Information Asymmetry
Liu, W., Wang, C., Wang, Y.-F., et al. (2024) · Neural Information Processing Systems · doi:10.48550/arxiv.2406.14928 · Cited by 17
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
Xu, F. F., Song, Y., Li, B., Tang, Y., Jain, K., Bao, M., Wang, Z. Z., Zhou, X., Guo, Z., Cao, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.14161 · Cited by 17 · cited 3×
“Factual Accuracy Recall, Consistency Scores LongEval [43], SocialBench [9], LoCoMo [65], Optimus-1 [50] Multi-Agent Collaboration Information Sharing Effectiveness, Adaptive Role Switching, Reasoning Rating AgentSims [52], WebArena [126], MATSA [66], GAMEBENCH [15], BALROG [72], TheAgentCompany [ CASA [77], R-Judge [1…”
InfluentialLiu, B., Li, X., Zhang, J., Wang, J., He, T., Hong, S., Liu, H., Zhang, S., Song, K., Zhu, K., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.01990 · Cited by 16 · cited 3×
“Invocation Accuracy [54] evaluates whether the agent makes the correct decision about whether to call a tool at all.”
InfluentialFlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents
Xiao, R., Ma, W., Wang, K., Wu, Y., Zhao, J., Wang, H., Huang, F., & Li, Y. (2024) · arXiv · doi:10.48550/arxiv.2406.14884 · Cited by 15
Self-citationJia, Z., Wang, M., Tong, B., Zhu, S. C., & Zheng, Z. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.879 · Cited by 14
AgentSims: An Open-Source Sandbox for Large Language Model Evaluation
Lin, J., Zhao, H., Zhang, A., Wu, Y., Ping, H., & Chen, Q. (2023) · arXiv · doi:10.48550/arxiv.2308.04026 · Cited by 14 · cited 1×
“Instead of relying on synthetic, historical, or manually crafted data, online evaluations leverage simulations or fundamental user interactions.”
Enhancing Trust in LLMs: Algorithms for Comparing and Interpreting LLMs
Brown, N. B. (2024) · arXiv · doi:10.48550/arxiv.2406.01943 · Cited by 14
LLM Agents for Psychology: A Study on Gamified AssessmentsPreprint
Yang, Q.-S., Wang, Z., Chen, H.-H., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2402.12326 · Cited by 12
AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
Andriushchenko, M., Souly, A., Dziemian, M., Duenas, D., Lin, M., Wang, J., Hendrycks, D., Zou, A., Kolter, Z., Fredrikson, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.09024 · Cited by 11 · cited 2×
“For example, in the assessment of web agents, researchers built web simulators (MiniWoB [85], WebShop [103], WebArena [126], etc.) where the behavior of an agent (clicking links, filling forms) can be programmed to verify the correct sequence.”
AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
Yoran, O., Amouyal, S. J., Malaviya, C., Bogin, B., Press, O., & Berant, J. (2024) · arXiv · doi:10.48550/arxiv.2407.15711 · Cited by 11 · cited 2×
“It is also extensively used for agents interacting with web environments, including general web navigation (BrowserGym [14], WebArena [126], WebCanvas [73]), multimodal web tasks (Visual-WebArena [42], MMInA [124]), and realistic time-consuming web tasks (ASSISTANTBENCH [109]).”
Jia, Z.-X., Wang, M.-M., Tong, B.-C., Zhu, S.-C., & Zheng, Z.-L. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2406.16294 · Cited by 11
Large Language Model-Brained GUI Agents: A Survey
Zhang, C., He, S., Qian, J., Li, B., Li, L., Qin, S., Kang, Y., Ma, M., Liu, G., Lin, Q., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.18279 · Cited by 10 · cited 2×
“Task completion is commonly quantified using metrics such as Success Rate (SR) [11, 57, 64], which can also be referred to as Task Success Rate [115] or Overall Success Rate [57].”
$τ$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Yao, S., Shinn, N., Razavi, P., & Narasimhan, K. (2024) · arXiv · doi:10.48550/arxiv.2406.12045 · Cited by 9 · cited 2×
“For example, the 𝜏 -benchmark [104] explicitly incorporates the pass^ 𝑘 metric to evaluate the consistency of an agent.”
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents
Zhang, H., Huang, J., Mei, K., Yao, Y., Wang, Z., Zhan, C., Wang, H., & Zhang, Y. (2024) · arXiv · doi:10.48550/arxiv.2410.02644 · Cited by 9 · cited 1×
“Dynamic evaluations may use proxies to simulate users or environments in reactive, real-time response to agent behavior.”
Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents
Gan, Y., Yang, Y., Ma, Z., He, P., Zeng, R., Wang, Y., Li, Q., Zhou, C., Li, S., Wang, T., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.09523 · Cited by 8
Shi, H., Cheng, L. K., Wu, W., Wang, Y., Liu, X., Nie, S., Wang, W., Min, X., Men, C., & Lin, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.05349 · Cited by 8
Chen, Z., Chen, S., Ning, Y., Zhang, Q., Wang, B., Yu, B., Li, Y., Liao, Z., Wei, C., Lu, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.05080 · Cited by 7
Self-citationMetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
Huang, Y., Shi, J., Li, Y., Fan, C., Wu, S., Zhang, Q., Liu, Y., Zhou, P., Wan, Y., Gong, N. Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.03128 · Cited by 7
Self-citationCoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference
Yu, E., Li, J., Liao, M., Wang, S., Gao, Z., Mi, F., & Hong, L. (2024) · arXiv · doi:10.48550/arxiv.2406.17626 · Cited by 7 · cited 1×
“For example, the HELM benchmark [51] for enterprises was proposed to include domain-specific prompts and metrics (such as accuracy on financial jargon or compliance in responses) for fields like finance and law.”
Agent-SafetyBench: Evaluating the Safety of LLM Agents
Zhang, Z., Cui, S., Lu, Y., Zhou, J., Yang, J., Wang, H., & Huang, M. (2024) · arXiv · doi:10.48550/arxiv.2412.14470 · Cited by 6 · cited 1×
“For example, in the assessment of web agents, researchers built web simulators (MiniWoB [85], WebShop [103], WebArena [126], etc.) where the behavior of an agent (clicking links, filling forms) can be programmed to verify the correct sequence.”
Xia, B., Lu, Q., Zhu, L., Xing, Z., Zhao, D., & Zhang, H. (2024) · arXiv · doi:10.48550/arxiv.2411.13768 · Cited by 6
Re-ReST: Reflection-Reinforced Self-Training for Language Agents
Dou, Z.-Y., Yang, C.-F., Wu, X., Chang, K.-W., & Peng, N. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.861 · Cited by 6 · cited 1×
“…AutoGuide [24] Harm Adversarial Robustness, Prompt Injection Resistance, Harmfulness, Bias Detection Agent Security Bench(ASB) [118], AgentPoison [13], AgentDojo [17], Backdoor Attacks [102], SafeAgentBench [108], Agent-Safety Bench [122], AgentHarm [5], Adaptive Attacks [113],…”
Liu, Y., Zhong, T., Lu, Y., Lin, H., He, B., Zhou, S., Zhu, H., Wang, W., Liu, Z., Han, X., et al. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.336 · Cited by 5
Agent-as-a-Judge: Evaluate Agents with Agents
Zhuge, M., Zhao, C., Ashley, D., Wang, W., Khizbullin, D., Xiong, Y., Liu, Z., Chang, E., Krishnamoorthi, R., Tian, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.10934 · Cited by 5 · cited 1×
“Xia et al. [95] further propose an AgentOps architecture to continuously monitor deployed agents, closing the loop between development and deployment through real-time feedback and quality control.”
An Evaluation Mechanism of LLM-based Agents on Manipulating APIs
Liu, B., Jianxiang, Z., Meng, D., & Lu, H. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.267 · Cited by 4
Siegel, Z. S., Kapoor, S., Nadgir, N., Stroebl, B., & Narayanan, A. (2024) · arXiv · doi:10.48550/arxiv.2409.11363 · Cited by 4 · cited 1×
“This includes tasks re-lated to coding and software engineering, such as resolving GitHub issues (SWE-bench [40]), scientific data analysis programming (Sci-enceAgentBench [11]), reproducing research (CORE-Bench [86], PaperBench [87]), and interactive coding in apps (AppWorld [91]).”
Predicting Interaction Quality Aspects Using Level-Based Scores for Conversational Agents
Reimann, M. M., Oertel, C., Kunneman, F. A., & Hindriks, K. V. (2023) · Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents · doi:10.1145/3570945.3607332 · Cited by 4
WebCanvas: Benchmarking Web Agents in Online Environments
Pan, Y., Kong, D., Zhou, S., Cui, C., Leng, Y., Jiang, B., Liu, H., Shang, Y., Zhou, S., Wu, T., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.12373 · Cited by 4 · cited 1×
“It is also extensively used for agents interacting with web environments, including general web navigation (BrowserGym [14], WebArena [126], WebCanvas [73]), multimodal web tasks (Visual-WebArena [42], MMInA [124]), and realistic time-consuming web tasks (ASSISTANTBENCH [109]).”
Can We Trust AI Agents? A Case Study of an LLM-Based Multi-Agent System for Ethical AI
de Cerqueira, J. A. S., Agbese, M., Rousi, R., Xi, N., Hamari, J., & Abrahamsson, P. (2024) · arXiv · doi:10.48550/arxiv.2411.08881 · Cited by 4
SocialBench: Sociality Evaluation of Role-Playing Conversational AgentsPreprint
Chen, H., Chen, H., Yan, M., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.13679 · Cited by 4
AgentQuest: a modular benchmark framework to measure progress and improve LLM agents
Gioacchini, L., Siracusano, G., Sanvito, D., Gashteovski, K., Friede, D., Bifulco, R., & Lawrence, C. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: System Demonstrations) · doi:10.18653/v1/2024.naacl-demo.19 · Cited by 4
MATSA: Multi-Agent Table Structure Attribution
Mathur, P., Siu, A. F., Lipka, N., et al. (2024) · doi:10.18653/v1/2024.emnlp-demo.26 · Cited by 3
AAAR-1.0: Assessing AI's Potential to Assist Research
Lou, R., Xu, H., Wang, S., Du, J., Kamoi, R., Lu, X., Xie, J., Sun, Y., Zhang, Y., Ahn, J. J., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.22394 · Cited by 3
Evaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.17753 · Cited by 3
He, Z.-T., Cao, P., Wang, C., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.02959 · Cited by 3
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Krishna, K., Bransom, E., Kuehl, B., Iyyer, M., Dasigi, P., Cohan, A., & Lo, K. (2023) · arXiv · doi:10.48550/arxiv.2301.13298 · Cited by 3 · cited 2×
“Factual Accuracy Recall, Consistency Scores LongEval [43], SocialBench [9], LoCoMo [65], Optimus-1 [50] Multi-Agent Collaboration Information Sharing Effectiveness, Adaptive Role Switching, Reasoning Rating AgentSims [52], WebArena [126], MATSA [66], GAMEBENCH [15], BALROG [72], TheAgentCompany […”
Kang, H., & Xiong, C. (2024) · arXiv · doi:10.48550/arxiv.2406.10291 · Cited by 3
AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents
Gioacchini, L., Siracusano, G., Sanvito, D., Gashteovski, K., Friede, D., Bifulco, R., & Lawrence, C. (2024) · arXiv · doi:10.48550/arxiv.2404.06411 · Cited by 2 · cited 1×
“Additionally, the Step Success Rate has been proposed to measure the percentage of steps in the generated plan that are successfully executed, providing a holistic view of planning quality during execution [28].”
AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
Trivedi, H., Khot, T., Hartmann, M., Manku, R., Dong, V., Li, E., Gupta, S., Sabharwal, A., & Balasubramanian, N. (2024) · arXiv · doi:10.48550/arxiv.2407.18901 · Cited by 2
IntellAgent: A Multi-Agent Framework for Evaluating Conversational AI Systems
Levi, E., & Kadar, I. (2025) · arXiv · doi:10.48550/arxiv.2501.11067 · Cited by 2 · cited 1×
“This introduces significant evaluation overhead: running multiple trials per input can be computationally expensive, especially when testing complex tasks involving tools, memory, or multi-agent coordination.”
Autonomous Agents for Collaborative Task under Information Asymmetry
Liu, W., Wang, C., Wang, Y., Xie, Z., Qiu, R., Dang, Y., Du, Z., Chen, W., Yang, C., & Qian, C. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-0090 · Cited by 2
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
Paglieri, D., Cupiał, B., Coward, S., Piterbarg, U., Wolczyk, M., Khan, A., Pignatelli, E., Kuciński, Ł., Pinto, L., Fergus, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.13543 · Cited by 2
R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
Yuan, T., He, Z., Dong, L., Wang, Y., Zhao, R., Xia, T., Xu, L., Zhou, B., Li, F., Zhang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.10019 · Cited by 2 · cited 3×
“Meanwhile, Rjudge [112] analyzes how agents perceive risk when making autonomous decisions, emphasizing transparency and trustworthiness in AI interactions.”
Re-ReST: Reflection-Reinforced Self-Training for Language AgentsPreprint
Dou, Z.-Y., Yang, C., Wu, X., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.01495 · Cited by 2
SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents
Yin, S., Pang, X., Ding, Y., Chen, M., Bi, Y., Xiong, Y., Huang, W., Xiang, Z., Shao, J., & Chen, S. (2024) · arXiv · doi:10.48550/arxiv.2412.13178 · Cited by 2
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue
Lù, X. H., Kasner, Z., & Reddy, S. (2024) · arXiv · doi:10.48550/arxiv.2402.05930 · Cited by 2
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
Koh, J. Y., Lo, R., Jang, L., Duvvur, V., Lim, M. C., Huang, P.-Y., Neubig, G., Zhou, S., Salakhutdinov, R., & Fried, D. (2024) · arXiv · doi:10.48550/arxiv.2401.13649 · Cited by 1
MMInA: Benchmarking Multihop Multimodal Internet AgentsPreprint
Tian, S., Zhang, Z., Chen, L., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.09992 · Cited by 1 · cited 1×
“It is also extensively used for agents interacting with web environments, including general web navigation (BrowserGym [14], WebArena [126], WebCanvas [73]), multimodal web tasks (Visual-WebArena [42], MMInA [124]), and realistic time-consuming web tasks (ASSISTANTBENCH [109]).”
Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interaction
Zhang, D., Shen, Z., Xie, R., Zhang, S., Xie, T., Zhao, Z., Chen, S., Chen, L., Xu, H., Cao, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.08144 · Cited by 1 · cited 1×
“…Risk Awareness, Task Completion Under Constraints R-Judge [112], Cybench [114], TheAgentCompany [97] 4.1.1 Static & Offline Evaluation Often performed as a baseline, offline evaluations typically rely on datasets and static test cases: collections of tasks, prompts, or conversations that…”
InfluentialLLMEval: A Preliminary Study on How to Evaluate Large Language ModelsPreprint
Zhang, Y., Zhang, M., Yuan, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.07398
Inspect AI: Framework for Large Language Model Evaluations
AI Security Institute, U. (2024) · Zenodo · doi:10.5281/zenodo.18434278
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · arXiv · doi:10.48550/arxiv.2009.11462
Zhang, Z., Jia, M., Lee, H.-P., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.11653
Research works printed but not linked: 7
- Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, and Liqiang Nie. 2024d. Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang(Eds.), Vol. 37. Curran Associates, Inc., 49881-49913.
- Xiao Liu Hao Yu Hanchen Zhang Yifan Xu Xuanyu Lei Hanyu Lai Yu Gu Hangliang Ding Kaiwen Men Kejuan Yang Shudan Zhang Xiang Deng Aohan Zeng Zhengxiao Du Chenhui Zhang Sheng Shen Tianjun Zhang Yu Su Huan Sun Minlie Huang Yuxiao Dong and Jie Tang. 2023c. AgentBench: Evaluating LLMs as Agents. arXiv:2308.03688 [cs.AI] https://arxiv.org/abs/2308.03688 · arXiv
- Wenjie Ruan, Wei Huang, Xiaowei Huang, Gaojie Jin, Yi Dong, Changshun Wu, Saddek Bensalem, Ronghui Mu, Yi Qi, Xingyu Zhao, et al., 2024. ToolEmu. In Proceedings of t.
- Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. 2017. World of bits: An open-domain platform for web-based agents. In International Conference on Machine Learning. PMLR, 3135-3144.
- Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023a. ReAct: Synergizing Reasoning and Acting in Language Models. In Proceedings of the International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2210.03629v3 · arXiv
- Yangyang Yu, Zhiyuan Yao, Haohang Li, Zhiyang Deng, Yuechen Jiang, Yupeng Cao, Zhi Chen, Jordan Suchow, Zhenyu Cui, Rong Liu, Zhaozhuo Xu, Denghui Zhang, Koduvayur (Suba) Subbalakshmi, GUOJUN XIONG, Yueru He, Jimin Huang, Dong Li, and Qianqian Xie. 2024b. FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Making. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang(Eds.), Vol. 37. Curran Associates, Inc., 137010-137045.
- Danyang Zhang, Lu Chen, and Kai Yu. 2023a. Mobile-env: A universal platform for training and evaluation of mobile interaction. CoRR(2023).
Cited by
Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
Chhabra, A., Datta, S., Nahin, S. K., & Mohapatra, P. (2026) · IEEE Access · doi:10.1109/access.2026.3675554 · Cited by 28
Codepori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
Rasheed, Z., Malik, S., Kemell, K.-K., Muhammad, W., Saari, M., Systä, K., & Abrahamsson, P. (2024) · doi:10.2139/ssrn.4979510 · Cited by 27
Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents
Khanal, A., Tao, Y., & Zhou, J. (2026) · arXiv · doi:10.48550/arxiv.2603.29231 · Cited by 10
Nageshwaran, V., & Ezekiel, S. (2026) · doi:10.20944/preprints202606.0142.v1 · Cited by 4
From benchmarks to deployment: a comprehensive review of agentic AI evaluation
Kehkashan, T., Abdullah, M. I., Al‐Shamayleh, A. S., Ivković, N., Ismail, N. A., Ahmad, S. S. S., Rehman, A., & Akhunzada, A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11571-0 · Cited by 19
Metadata
- Authors
- Mohammadi, M., Li, Y., Lo, J., & Yip, W.
- Affiliations
- SAP Labs, Bellevue, Washington, USA
- Systems, Applications & Products in Data Processing (United States) (US)
- Venue
- Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2
- Publisher
- ACM
- Published
- 2025-08-03
- Pages
- 6129-6139
- OpenAlex
- W4412877164
- Semantic Scholar
- a56efef88a8eb94d9c9704f279c254c1bf4a88ab
- Cited by
- 74
- References
- 61
- Citations by year
- 2026: 68 · 2025: 5
- FWCI
- 68.73
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- LLM agents, agent evaluation, benchmarking, evaluation taxonomy, agent behavior, agent reliability, agent safety, evaluation dataset, evaluation metrics, enterprise AI deployment, compliance, long-horizon interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.