CoChat
Sign Up
PreprintCited by 16Multi-Agent Systems and NegotiationAI-based Problem Solving and PlanningArtificial Intelligence Applications

Large Language Model Agent: A Survey on Methodology, Applications and Challenges

Open access
Luo, Junyu, Zhang, Weizhi, Yuan, Ye, Zhao, Yusheng, Yang, Junwei, Gu, Yiyang, Wu, Bohan, Chen, Binqi +18 more·2025·arXiv
DOI 10.48550/arxiv.2503.21460arXiv 2503.21460
Open this paper in CoChat — free accountOpen PDF

Abstract

The era of intelligent agents is upon us, driven by revolutionary advancements in large language models. Large Language Model (LLM) agents, with goal-driven behaviors and dynamic adaptation capabilities, potentially represent a critical pathway toward artificial general intelligence. This survey systematically deconstructs LLM agent systems through a methodology-centered taxonomy, linking architectural foundations, collaboration mechanisms, and evolutionary pathways. We unify fragmented research threads by revealing fundamental connections between agent design principles and their emergent behaviors in complex environments. Our work provides a unified architectural perspective, examining how agents are constructed, how they collaborate, and how they evolve over time, while also addressing evaluation methodologies, tool applications, practical challenges, and diverse application domains. By surveying the latest developments in this rapidly evolving field, we offer researchers a structured taxonomy for understanding LLM agents and identify promising directions for future research. The collection is available at https://github.com/luo-junyu/Awesome-Agent-Papers.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2503.21460

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Multi-domain agents engage with tools and/or APIs, manage complex workflows such as coding, healthcare, finance, and crime analysis, and provide memory-based learning that allows them to continue a task by absorbing adaptive information from external cues.

    “Multi-domain agents engage with tools and/or APIs, manage workflows that will inevitably be very complex, such as coding, healthcare, finance, crime analysis, etc., and provide memory-based learning that allows them to continue a task by absorbing adaptive information from external cues [31,67,75-77,136,139].”

    From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [75]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • BrainBody-LLM and RASC utilize feedback in self-reinforcement to continue learning and optimizing over time.

    “BrainBody-LLM and RASC [75] utilize feedback in self-reinforcement to continue learning and optimizing over time.”

    From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [75]

    Section 2.3.1 (p. 7) describes self-feedback and self-rewarding approaches that help LLM agents refine their outputs and improve performance. Section 2.3.3 (p. 8) also describes iterative improvement from external feedback. Neither passage identifies BrainBody-LLM or RASC or establishes that those particular systems use these mechanisms. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Adaptability encompasses ongoing interpersonal learning and agile feedback and tuned goals and measures via conceptual multi-agent models, formal policy operationalizations such as Petri Nets, and applied second burst benchmark measures that simulate unpredictable, complex, and dynamic environments.

    “Adaptability: It encompasses ongoing interpersonal learning and agile feedback and tuned goals and measures via conceptual multi-agent models, formal policy operationalizations such as Petri Nets [154], and applied second burst benchmark measures that simulate unpredictable, complex, and dynamic environments [75].”

    From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [75]

    Section 3.1.1 (p. 8) describes benchmarks that test agents in interactive environments and assess their ability to handle a complex, dynamic visual world; Section 3.1.2 (p. 9) also describes simulation-based benchmarks in dynamic environments. These passages support the benchmark portion of the claim, but do not identify “second burst” measures or say the environments are unpredictable. The earlier conceptual-model and Petri Net clauses are attached to a different citation. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • In mature agent systems, reliability increasingly depends on external memory stores, tool registries, protocol definitions, sandboxes, sub-agent orchestration, compression pipelines, evaluators, test harnesses, and approval loops.

    “In mature agent systems, reliability increasingly depends on external memory stores, tool registries, protocol definitions, sandboxes, sub-agent orchestration, compression pipelines, evaluators, test harnesses, and approval loops [Wang et al., 2024a, Li, 2025, Luo et al., 2025, Xi et al., 2023].”

    From Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering (2026) · cites this paper as Luo et al., 2025

    The survey discusses long-term memory stores and context compression (Section 2.1.2, p. 4), system-level evaluation (Section 3.1.3, p. 9), coordination protocols (Section 6.1, p. 18), and human verification loops for reliability (Section 6.3, p. 18). It does not establish that reliability in mature agent systems depends on the entire listed stack: in particular, its sandbox example is a benchmark environment, not a reliability requirement, and it does not identify tool registries or test harnesses as such requirements. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Agentic AI use cases include virtual assistants, autonomous systems, adaptive planning, and scientific discovery.

    “Virtual assistants, autonomous systems, adaptive planning, scientific discovery”

    From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [75]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 302

  1. [1][1] Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou et al. , 'The rise and potential of large language model based agents: A survey,' Science China Information Sciences , vol. 68, no. 2, p. 121101, 2025. -
  2. [2][2] M. Wooldridge and N. R. Jennings, 'Intelligent agents: Theory and practice,' The knowledge engineering review , vol. 10, no. 2, pp. 115-152, 1995. -
  3. [3][3] D. Zheng, M. Lapata, and J. Z. Pan, 'Large language models as reliable knowledge bases?' arXiv preprint arXiv:2407.13578 , 2024. - · arXiv
  4. [4][4] S. Lotfi, M. Finzi, Y. Kuang, T. G. Rudner, M. Goldblum, and A. G. Wilson, 'Non-vacuous generalization bounds for large language models,' arXiv preprint arXiv:2312.17173 , 2023. - · arXiv
  5. [5][5] H. Fei, Y. Yao, Z. Zhang, F. Liu, A. Zhang, and T.-S. Chua, 'From multimodal llm to human-level ai: Modality, instruction, reasoning, efficiency and beyond,' in COLING , 2024, pp. 1-8. -
  6. [6][6] J. Huang and K. C.-C. Chang, 'Towards reasoning in large language models: A survey,' arXiv preprint arXiv:2212.10403 , 2022. - · arXiv
  7. [7][7] C. Wang, W. Luo, Q. Chen, H. Mai, J. Guo, S. Dong, Z. Li, L. Ma, S. Gao et al. , 'Tool-lmm: A large multi-modal model for tool agent learning,' arXiv e-prints , pp. arXiv-2401, 2024. -
  8. [8][8] Z. Zhang, X. Bo, C. Ma, R. Li, X. Chen, Q. Dai, J. Zhu, Z. Dong, and J.-R. Wen, 'A survey on the memory mechanism of large language model based agents,' arXiv preprint arXiv:2404.13501 , 2024. - · arXiv
  9. [9][9] P. Zhao, Z. Jin, and N. Cheng, 'An in-depth survey of large language model-based artificial intelligence agents,' arXiv preprint arXiv:2309.14365 , 2023. - · arXiv
  10. [11][11] S. Hu, T. Huang, F. Ilhan, S. Tekin, G. Liu, R. Kompella, and L. Liu, 'A survey on large language model-based game agents,' arXiv preprint arXiv:2404.02039 , 2024. - · arXiv
  11. [12][12] X. Xu, Y. Wang, C. Xu, Z. Ding, J. Jiang, Z. Ding, and B. F. Karlsson, 'A survey on game playing agents and large models: Methods, applications, and challenges,' arXiv preprint arXiv:2403.10249 , 2024. - · arXiv
  12. [13][13] M. Xu, H. Du, D. Niyato, J. Kang, Z. Xiong, S. Mao, Z. Han, A. Jamalipour, D. I. Kim, X. Shen et al. , 'Unleashing the power of edge-cloud generative ai in mobile networks: A survey of aigc services,' IEEE Communications Surveys & Tutorials , vol. 26, no. 2, pp. 1127-1170, 2024. -
  13. [14][14] G. Qu, Q. Chen, W. Wei, Z. Lin, X. Chen, and K. Huang, 'Mobile edge intelligence for large language models: A contemporary survey,' IEEE Communications Surveys & Tutorials , 2025. -
  14. [15][15] Z. Durante, Q. Huang, N. Wake, R. Gong, J. S. Park, B. Sarkar, R. Taori, Y. Noda, D. Terzopoulos, Y. Choi et al. , 'Agent ai: Surveying the horizons of multimodal interaction,' arXiv preprint arXiv:2401.03568 , 2024. - · arXiv
  15. [16][16] Y. Wang, Y. Pan, Q. Zhao, Y. Deng, Z. Su, L. Du, and T. H. Luan, 'Large model agents: State-of-the-art, cooperation paradigms, security and privacy, and future trends,' arXiv preprint arXiv:2409.14457 , 2024. - · arXiv
  16. [17][17] L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin et al. , 'A survey on large language model based autonomous agents,' Frontiers of Computer Science , vol. 18, no. 6, p. 186345, 2024. -
  17. [18][18] X. Li, S. Wang, S. Zeng, Y. Wu, and Y. Yang, 'A survey on llm-based multi-agent systems: workflow, infrastructure, and challenges,' Vicinagearth , vol. 1, no. 1, p. 9, 2024. -
  18. [19][19] X. Li, 'A review of prominent paradigms for llm-based agents: Tool use (including rag), planning, and feedback learning,' arXiv preprint arXiv:2406.05804 , 2024. - · arXiv
  19. [20][20] W. Jin, H. Du, B. Zhao, X. Tian, B. Shi, and G. Yang, 'A comprehensive survey on multi-agent cooperative decision-making: Scenarios, approaches, challenges and perspectives,' arXiv preprint arXiv:2503.13415 , 2025. - · arXiv
  20. [21][21] Y. Ma, Z. Song, Y. Zhuang, J. Hao, and I. King, 'A survey on vision-language-action models for embodied ai,' arXiv preprint arXiv:2405.14093 , 2024. - · arXiv
  21. [22][22] T. Guo, X. Chen, Y. Wang, R. Chang, S. Pei, N. V. Chawla, O. Wiest, and X. Zhang, 'Large language model based multiagents: A survey of progress and challenges,' arXiv preprint arXiv:2402.01680 , 2024. - · arXiv
  22. [24][24] Y. Cheng, C. Zhang, Z. Zhang, X. Meng, S. Hong, W. Li, Z. Wang, Z. Wang, F. Yin, J. Zhao et al. , 'Exploring large language model based intelligent agents: Definitions, methods, and prospects,' arXiv preprint arXiv:2401.03428 , 2024. - · arXiv
  23. [25][25] G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem, 'Camel: Communicative agents for 'mind' exploration of large language model society,' in NeurIPS , 2023. -
  24. [27][27] S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou et al. , 'Metagpt: Meta programming for a multi-agent collaborative framework,' in ICLR , 2024. -
  25. [28][28] C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong et al. , 'Chatdev: Communicative agents for software development,' in ACL , 2024, pp. 15 174-15 186. -
  26. [29][29] J. Zhang, J. Xiang, Z. Yu, F. Teng, X.-H. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, B. Liu, Y. Luo, and C. Wu, 'AFlow: Automating agentic workflow generation,' in ICLR , 2025. -
  27. [30][30] J. S. Park, J. O'Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, 'Generative agents: Interactive simulacra of human behavior,' in UIST , 2023, pp. 1-22. -
  28. [31][31] L. Wang, J. Zhang, H. Yang, Z.-Y. Chen, J. Tang, Z. Zhang, X. Chen, Y. Lin, H. Sun, R. Song et al. , 'User behavior simulation with large language model-based agents,' ACM Transactions on Information Systems , vol. 43, no. 2, pp. 1-37, 2025. -
  29. [32][32] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts, 'Dspy: Compiling declarative language model calls into self-improving pipelines,' in ICLR , 2024. -
  30. [33][33] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, 'React: Synergizing reasoning and acting in language models,' in ICLR , 2023. -
  31. [34][34] M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, M. Podstawski, L. Gianinazzi, J. Gajda, T. Lehmann, H. Niewiadomski, P. Nyczyk et al. , 'Graph of thoughts: Solving elaborate problems with large language models,' in AAAI , vol. 38, no. 16, 2024, pp. 17 682-17 690. -
  32. [35][35] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, 'Voyager: An open-ended embodied agent with large language models,' TMLR , 2023. -
  33. [36][36] X. Zhu, Y. Chen, H. Tian, C. Tao, W. Su, C. Yang, G. Huang, B. Li, L. Lu, X. Wang et al. , 'Ghost in the minecraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory,' arXiv preprint arXiv:2305.17144 , 2023. - · arXiv
  34. [37][37] A. Zhao, D. Huang, Q. Xu, M. Lin, Y.-J. Liu, and G. Huang, 'Expel: Llm agents are experiential learners,' in AAAI , 2024, pp. 19 63219 642. -
  35. [38][38] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, 'Reflexion: Language agents with verbal reinforcement learning,' NeurIPS , vol. 36, pp. 8634-8652, 2023. -
  36. [39][39] J. Ruan, Y. Chen, B. Zhang, Z. Xu, T. Bao, H. Mao, Z. Li, X. Zeng, R. Zhao et al. , 'Tptu: Task planning and tool usage of large language model-based ai agents,' in NeurIPS , 2023. -
  37. [40][40] T. Xie, F. Zhou, Z. Cheng, P. Shi, L. Weng, Y. Liu, T. J. Hua, J. Zhao, Q. Liu, C. Liu et al. , 'Openagents: An open platform for language agents in the wild,' arXiv preprint arXiv:2310.10634 , 2023. - · arXiv
  38. [41][41] H. Wang, H. Xin, C. Zheng, Z. Liu, Q. Cao, Y. Huang, J. Xiong, H. Shi, E. Xie, J. Yin et al. , 'Lego-prover: Neural theorem proving with growing libraries,' in ICLR , 2024. -
  39. [42][42] C. Packer, V. Fang, S. G. Patil, K. Lin, S. Wooders, and J. E. Gonzalez, 'Memgpt: Towards llms as operating systems,' CoRR , 2023. -
  40. [43][43] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. K¨ uttler, M. Lewis, W.-t. Yih, T. Rockt¨ aschel et al. , 'Retrievalaugmented generation for knowledge-intensive nlp tasks,' NeurIPS , vol. 33, pp. 9459-9474, 2020. -
  41. [44][44] D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson, 'From local to global: A graph rag approach to query-focused summarization,' arXiv preprint arXiv:2404.16130 , 2024. - · arXiv
  42. [45][45] Y. Zhang, R. Sun, Y. Chen, T. Pfister, R. Zhang, and S. Arik, 'Chain of agents: Large language models collaborating on long-context tasks,' Advances in Neural Information Processing Systems , vol. 37, pp. 132 208-132 237, 2024. -
  43. [46][46] H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal, 'Interleaving retrieval with chain-of-thought reasoning for knowledgeintensive multi-step questions,' arXiv preprint arXiv:2212.10509 , 2022. - · arXiv
  44. [47][47] X. Li, C. Zhu, L. Li, Z. Yin, T. Sun, and X. Qiu, 'Llatrieval: Llmverified retrieval for verifiable generation,' in NAACL , 2024, pp. 5453-5471. -
  45. [49][49] X. Guan, J. Zeng, F. Meng, C. Xin, Y. Lu, H. Lin, X. Han, L. Sun, and J. Zhou, 'Deeprag: Thinking to retrieval step by step for large language models,' arXiv preprint arXiv:2502.01142 , 2025. - · arXiv
  46. [50][50] L. Wang, W. Xu, Y. Lan, Z. Hu, Y. Lan, R. K.-W. Lee, and E.P. Lim, 'Plan-and-solve prompting: Improving zero-shot chainof-thought reasoning by large language models,' arXiv preprint arXiv:2305.04091 , 2023. - · arXiv
  47. [51][51] E. H. Durfee, 'Distributed problem solving and planning,' in ECCAI Advanced Course on Artificial Intelligence . Springer, 2001, pp. 118-149. -
  48. [52][52] M. Tao, D. Zhao, and Y. Feng, 'Chain-of-discussion: A multimodel framework for complex evidence-based question answering,' arXiv preprint arXiv:2402.16313 , 2024. - · arXiv
  49. [53][53] M. Hu, Y. Mu, X. Yu, M. Ding, S. Wu, W. Shao, Q. Chen, B. Wang, Y. Qiao, and P. Luo, 'Tree-planner: Efficient closeloop task planning with large language models,' arXiv preprint arXiv:2310.08582 , 2023. - · arXiv
  50. [55][55] J. Long, 'Large language model guided tree-of-thought,' arXiv preprint arXiv:2305.08291 , 2023. - · arXiv
  51. [56][56] D. Zhang, S. Zhoubian, Z. Hu, Y. Yue, Y. Dong, and J. Tang, 'Restmcts*: Llm self-training via process reward guided tree search,' NeurIPS , vol. 37, pp. 64 735-64 772, 2024. -
  52. [57][57] A. Lykov, M. Dronova, N. Naglov, M. Litvinov, S. Satsevich, A. Bazhenov, V. Berman, A. Shcherbak, and D. Tsetserukou, 'Llmmars: Large language model for behavior tree generation and nlpenhanced dialogue in multi-agent robot systems,' arXiv preprint arXiv:2312.09348 , 2023. - · arXiv
  53. [58][58] J. Ao, F. Wu, Y. Wu, A. Swikir, and S. Haddadin, 'Llm as btplanner: Leveraging llms for behavior tree generation in robot task planning,' arXiv preprint arXiv:2409.10444 , 2024. - · arXiv
  54. [59][59] C. Rivera, G. Byrd, W. Paul, T. Feldman, M. Booker, E. Holmes, D. Handelman, B. Kemp, A. Badger, A. Schmidt et al. , 'Conceptagent: Llm-driven precondition grounding and tree search for robust task planning and execution,' arXiv preprint arXiv:2410.06108 , 2024. - · arXiv
  55. [60][60] V. Bhat, A. U. Kaypak, P. Krishnamurthy, R. Karri, and F. Khorrami, 'Grounding llms for robot task planning using closed-loop state feedback,' arXiv preprint arXiv:2402.08546 , 2024. - · arXiv
  56. [61][61] H. Li, H. Jiang, T. Zhang, Z. Yu, A. Yin, H. Cheng, S. Fu, Y. Zhang, and W. He, 'Traineragent: Customizable and efficient model training through llm-powered multi-agent system,' arXiv preprint arXiv:2311.06622 , 2023. - · arXiv
  57. [62][62] G. Wan, Y. Wu, J. Chen, and S. Li, 'Dynamic self-consistency: Leveraging reasoning paths for efficient llm sampling,' arXiv preprint arXiv:2408.17017 , 2024. - · arXiv
  58. [63][63] S. Seo, J. Lee, S. Noh, and H. Kang, 'Llm-based cooperative agents using information relevance and plan validation,' arXiv preprint arXiv:2405.16751 , 2024. - · arXiv
  59. [64][64] H. Sun, Y. Zhuang, L. Kong, B. Dai, and C. Zhang, 'Adaplanner: Adaptive planning from feedback with language models,' NeurIPS , vol. 36, pp. 58 202-58 245, 2023. -
  60. [65][65] M. Jafaripour, S. Golestan, S. Miwa, Y. Mitsuka, and O. Zaiane, 'Adaptive iterative feedback prompting for obstacle-aware path planning via llms,' in AAAI Workshop , 2025. -
  61. [66][66] S. Qiao, H. Gui, C. Lv, Q. Jia, H. Chen, and N. Zhang, 'Making language models better tool learners with execution feedback,' arXiv preprint arXiv:2305.13068 , 2023. - · arXiv
  62. [67][67] R. Yang, L. Song, Y. Li, S. Zhao, Y. Ge, X. Li, and Y. Shan, 'Gpt4tools: Teaching large language model to use tools via selfinstruction,' NeurIPS , vol. 36, pp. 71 995-72 007, 2023. -
  63. [68][68] S. Yuan, K. Song, J. Chen, X. Tan, Y. Shen, R. Kan, D. Li, and D. Yang, 'Easytool: Enhancing llm-based agents with concise tool instruction,' arXiv preprint arXiv:2401.06201 , 2024. - · arXiv
  64. [69][69] S. Wu, S. Zhao, Q. Huang, K. Huang, M. Yasunaga, K. Cao, V. Ioannidis, K. Subbian, J. Leskovec, and J. Y. Zou, 'Avatar: Optimizing llm agents for tool usage via contrastive reasoning,' NeurIPS , vol. 37, pp. 25 981-26 010, 2025. -
  65. [70][70] Y. Huang, J. Sansom, Z. Ma, F. Gervits, and J. Chai, 'Drivlme: Enhancing llm-based autonomous driving agents with embodied and social experiences,' in IROS . IEEE, 2024, pp. 3153-3160. -
  66. [71][71] Y. Zhang, S. Yang, C. Bai, F. Wu, X. Li, Z. Wang, and X. Li, 'Towards efficient llm grounding for embodied multi-agent collaboration,' arXiv preprint arXiv:2405.14314 , 2024. - · arXiv
  67. [73][73] D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes, 'Autonomous chemical research with large language models,' Nature , vol. 624, no. 7992, pp. 570-578, 2023. -
  68. [75][75] S. Qiao, N. Zhang, R. Fang, Y. Luo, W. Zhou, Y. E. Jiang, C. Lv, and H. Chen, 'Autoact: Automatic agent learning from scratch for qa via self-planning,' arXiv preprint arXiv:2401.05268 , 2024. - · arXiv
  69. [76][76] M. Suzgun and A. T. Kalai, 'Meta-prompting: Enhancing language models with task-agnostic scaffolding,' arXiv preprint arXiv:2401.12954 , 2024. - · arXiv
  70. [77][77] A. Khan, J. Hughes, D. Valentine, L. Ruis, K. Sachan, A. Radhakrishnan, E. Grefenstette, S. R. Bowman, T. Rockt¨ aschel, and E. Perez, 'Debating with more persuasive llms leads to more truthful answers,' arXiv preprint arXiv:2402.06782 , 2024. - · arXiv
  71. [78][78] X. Tang, A. Zou, Z. Zhang, Z. Li, Y. Zhao, X. Zhang, A. Cohan, and M. Gerstein, 'Medagents: Large language models as collaborators for zero-shot medical reasoning,' arXiv preprint arXiv:2311.10537 , 2023. - · arXiv
  72. [79][79] J. C.-Y. Chen, S. Saha, and M. Bansal, 'Reconcile: Round-table conference improves reasoning via consensus among diverse llms,' arXiv preprint arXiv:2309.13007 , 2023. - · arXiv
  73. [80][80] T. Liang, Z. He, W. Jiao, X. Wang, Y. Wang, R. Wang, Y. Yang, S. Shi, and Z. Tu, 'Encouraging divergent thinking in large language models through multi-agent debate,' arXiv preprint arXiv:2305.19118 , 2023. - · arXiv
  74. [81][81] K. Kim, S. Lee, K.-H. Huang, H. P. Chan, M. Li, and H. Ji, 'Can llms produce faithful explanations for fact-checking? towards faithful explainable fact-checking via multi-agent debate,' arXiv preprint arXiv:2402.07401 , 2024. - · arXiv
  75. [82][82] Y. Du, S. Li, A. Torralba, J. B. Tenenbaum, and I. Mordatch, 'Improving factuality and reasoning in language models through multiagent debate,' in ICML , 2023. -
  76. [83][83] Y. Zhu, S. Qiao, Y. Ou, S. Deng, N. Zhang, S. Lyu, Y. Shen, L. Liang, J. Gu, and H. Chen, 'Knowagent: Knowledge-augmented planning for llm-based agents,' arXiv preprint arXiv:2403.03101 , 2024. - · arXiv
  77. [84][84] S. Qiao, R. Fang, N. Zhang, Y. Zhu, X. Chen, S. Deng, Y. Jiang, P. Xie, F. Huang, and H. Chen, 'Agent planning with world knowledge model,' NeurIPS , vol. 37, pp. 114 843-114 871, 2024. -
  78. [85][85] R. Fang, S. Qiao, and Z. Xi, 'Refining guideline knowledge for agent planning using textgrad,' in ICKG . IEEE, 2024, pp. 102-103. -
  79. [86][86] Q. Zhong, L. Ding, J. Liu, B. Du, and D. Tao, 'Self-evolution learning for discriminative language model pretraining,' in ACL Findings , 2023, pp. 4130-4145. -
  80. [87][87] T. Akiba, M. Shing, Y. Tang, Q. Sun, and D. Ha, 'Evolutionary optimization of model merging recipes,' Nature Machine Intelligence , pp. 1-10, 2025. -
  81. [88][88] S. Wu, K. Lu, B. Xu, J. Lin, Q. Su, and C. Zhou, 'Self-evolved diverse data sampling for efficient instruction tuning,' arXiv preprint arXiv:2311.08182 , 2023. - · arXiv
  82. [89][89] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang et al. , 'Self-refine: Iterative refinement with self-feedback,' NeurIPS , vol. 36, pp. 46 534-46 594, 2023. -
  83. [90][90] E. Zelikman, Y. Wu, J. Mu, and N. D. Goodman, 'Star: Self-taught reasoner bootstrapping reasoning with reasoning,' in NeurIPS , vol. 1126, 2024. -
  84. [92][92] Y. Weng, M. Zhu, F. Xia, B. Li, S. He, S. Liu, B. Sun, K. Liu, and J. Zhao, 'Large language models are better reasoners with self-verification,' in EMNLP Findings , 2023, pp. 2550-2575. -
  85. [94][94] K. Yang, D. Klein, A. Celikyilmaz, N. Peng, and Y. Tian, 'Rlcd: Reinforcement learning from contrastive distillation for lm alignment,' in ICLR , 2024. -
  86. [95][95] J.-C. Pang, P. Wang, K. Li, X.-H. Chen, J. Xu, Z. Zhang, and Y. Yu, 'Language model self-improvement by reinforcement learning contemplation,' in ICLR , 2024. -
  87. [96][96] C. Zhang, K. Yang, S. Hu, Z. Wang, G. Li, Y. Sun, C. Zhang, Z. Zhang, A. Liu, S.-C. Zhu et al. , 'Proagent: building proactive cooperative agents with large language models,' in AAAI , vol. 38, no. 16, 2024, pp. 17 591-17 599. -
  88. [97][97] H. Ma, T. Hu, Z. Pu, L. Boyin, X. Ai, Y. Liang, and M. Chen, 'Coevolving with the other you: Fine-tuning llm with sequential cooperative multi-agent reinforcement learning,' NeurIPS , vol. 37, pp. 15 497-15 525, 2024. -
  89. [98][98] C. Ma, Z. Yang, H. Ci, J. Gao, M. Gao, X. Pan, and Y. Yang, 'Evolving diverse red-team language models in multi-round multiagent games,' arXiv preprint arXiv:2310.00322 , 2023. - · arXiv
  90. [99][99] T. Liang, Z. He, W. Jiao, X. Wang, Y. Wang, R. Wang, Y. Yang, S. Shi, and Z. Tu, 'Encouraging divergent thinking in large language models through multi-agent debate,' in EMNLP , 2024, pp. 17 88917 904. -
  91. [100][100] Z. Gou, Z. Shao, Y. Gong, Y. Yang, N. Duan, W. Chen et al. , 'Critic: Large language models can self-correct with tool-interactive critiquing,' in ICLR , 2024. -
  92. [101][101] Y. Song, D. Yin, X. Yue, J. Huang, S. Li, and B. Y. Lin, 'Trial and error: Exploration-based trajectory optimization of llm agents,' in ACL , 2024, pp. 7584-7600. -
  93. [102][102] S. Jiang, Y. Wang, and Y. Wang, 'Selfevolve: A code evolution framework via large language models,' arXiv preprint arXiv:2306.02907 , 2023. - · arXiv
  94. [103][103] X. Huang, W. Liu, X. Chen, X. Wang, H. Wang, D. Lian, Y. Wang, R. Tang, and E. Chen, 'Understanding the planning of llm agents: A survey,' arXiv preprint arXiv:2402.02716 , 2024. - · arXiv
  95. [104][104] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, 'Large language models are zero-shot reasoners,' NeurIPS , vol. 35, pp. 22 199-22 213, 2022. -
  96. [105][105] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al. , 'Chain-of-thought prompting elicits reasoning in large language models,' NeurIPS , vol. 35, pp. 24 824-24 837, 2022. -
  97. [106][106] X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou, 'Self-consistency improves chain of thought reasoning in language models,' arXiv preprint arXiv:2203.11171 , 2022. - · arXiv
  98. [107][107] W. Li and W. Pan, 'Enhancing chain-of-thought reasoning in large language models through text style diversity and prompt fusion,' in EIBDCT , vol. 13181. SPIE, 2024, pp. 226-232. -
  99. [108][108] J. Jiang, Z. Chen, Y. Min, J. Chen, X. Cheng, J. Wang, Y. Tang, H. Sun, J. Deng, W. X. Zhao et al. , 'Technical report: Enhancing llm reasoning with reward-guided tree search,' arXiv preprint arXiv:2411.11694 , 2024. - · arXiv
  100. [109][109] C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton, 'A survey of monte carlo tree search methods,' IEEE Transactions on Computational Intelligence and AI in games , vol. 4, no. 1, pp. 1-43, 2012. -
  101. [110][110] H. Guo, Z. Liu, Y. Zhang, and Z. Wang, 'Can large language models play games? a case study of a self-play approach,' arXiv preprint arXiv:2403.05632 , 2024. - · arXiv
  102. [111][111] Y. Liu, P. Sun, and H. Li, 'Large language models as agents in two-player games,' arXiv preprint arXiv:2402.08078 , 2024. - · arXiv
  103. [112][112] A. R. Laleh and M. N. Ahmadabadi, 'A survey on enhancing reinforcement learning in complex environments: Insights from human and llm feedback,' arXiv preprint arXiv:2411.13410 , 2024. - · arXiv
  104. [113][113] Z. Shen, 'Llm with tools: A survey,' arXiv preprint arXiv:2409.18807 , 2024. - · arXiv
  105. [114][114] C. Y. Kim, C. P. Lee, and B. Mutlu, 'Understanding large-language model (llm)-powered human-robot interaction,' in HRI , 2024, pp. 371-380. -
  106. [115][115] B. Li, Y. Wang, J. Gu, K.-W. Chang, and N. Peng, 'Metal: A multiagent framework for chart generation with test-time scaling,' arXiv preprint arXiv:2502.17651 , 2025. - · arXiv
  107. [116][116] S. Guo, C. Deng, Y. Wen, H. Chen, Y. Chang, and J. Wang, 'Ds-agent: Automated data science by empowering large language models with case-based reasoning,' arXiv preprint arXiv:2402.17453 , 2024. - · arXiv
  108. [117][117] Z. Yin, Q. Sun, C. Chang, Q. Guo, J. Dai, X. Huang, and X. Qiu, 'Exchange-of-thought: Enhancing large language model capabilities through cross-model communication,' arXiv preprint arXiv:2312.01823 , 2023. - · arXiv
  109. [118][118] Y. Li, S. Ren, P. Wu, S. Chen, C. Feng, and W. Zhang, 'Learning distilled collaboration graph for multi-agent perception,' NeurIPS , vol. 34, pp. 29 541-29 552, 2021. -
  110. [120][120] Y. Kim, C. Park, H. Jeong, Y. S. Chan, X. Xu, D. McDuff, H. Lee, M. Ghassemi, C. Breazeal, H. Park et al. , 'Mdagents: An adaptive collaboration of llms for medical decision-making,' NeurIPS , vol. 37, pp. 79 410-79 452, 2024. -
  111. [121][121] L. Ying, T. Zhi-Xuan, V. Mansinghka, and J. B. Tenenbaum, 'Inferring the goals of communicating agents from actions and instructions,' in Proceedings of the AAAI Symposium Series , vol. 2, no. 1, 2023, pp. 26-33. -
  112. [122][122] J. Vyas and M. Mercang¨ oz, 'Autonomous industrial control using an agentic framework with large language models,' arXiv preprint arXiv:2411.05904 , 2024. - · arXiv
  113. [123][123] D. Dell'Anna, N. Alechina, F. Dalpiaz, M. Dastani, and B. Logan, 'Data-driven revision of conditional norms in multi-agent systems,' Journal of Artificial Intelligence Research , vol. 75, pp. 1549-1593, 2022. -
  114. [124][124] X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang et al. , 'Agentbench: Evaluating llms as agents,' arXiv preprint arXiv:2308.03688 , 2023. - · arXiv
  115. [125][125] X. Deng, Y. Gu, B. Zheng, S. Chen, S. Stevens, B. Wang, H. Sun, and Y. Su, 'Mind2web: Towards a generalist agent for the web,' NeurIPS , vol. 36, pp. 28 091-28 114, 2023. -
  116. [126][126] G. Yin, H. Bai, S. Ma, F. Nan, Y. Sun, Z. Xu, S. Ma, J. Lu, X. Kong, A. Zhang et al. , 'Mmau: A holistic benchmark of agent capabilities across diverse domains,' arXiv preprint arXiv:2407.18961 , 2024. - · arXiv
  117. [127][127] K. Gu, R. Shang, R. Jiang, K. Kuang, R.-J. Lin, D. Lyu, Y. Mao, Y. Pan, T. Wu, J. Yu et al. , 'Blade: Benchmarking language model agents for data-driven science,' arXiv preprint arXiv:2408.09667 , 2024. - · arXiv
  118. [128][128] X. Liu, T. Zhang, Y. Gu, I. L. Iong, Y. Xu, X. Song, S. Zhang, H. Lai, X. Liu, H. Zhao et al. , 'Visualagentbench: Towards large multimodal models as visual foundation agents,' arXiv preprint arXiv:2408.06327 , 2024. - · arXiv
  119. [129][129] M. Li, S. Zhao, Q. Wang, K. Wang, Y. Zhou, S. Srivastava, C. Gokmen, T. Lee, E. L. Li, R. Zhang et al. , 'Embodied agent interface: Benchmarking llms for embodied decision making,' NeurIPS , vol. 37, pp. 100 428-100 534, 2025. -
  120. [130][130] T. Xu, L. Chen, D.-J. Wu, Y. Chen, Z. Zhang, X. Yao, Z. Xie, Y. Chen, S. Liu, B. Qian et al. , 'Crab: Cross-platfrom agent benchmark for multi-modal embodied language model agents,' in NeurIPS Workshop , 2024. -
  121. [131][131] N. Butt, V. Chandrasekaran, N. Joshi, B. Nushi, and V. Balachandran, 'Benchagents: Automated benchmark creation with agent interaction,' arXiv preprint arXiv:2410.22584 , 2024. - · arXiv
  122. [132][132] S. Wang, Z. Long, Z. Fan, Z. Wei, and X. Huang, 'Benchmark selfevolving: A multi-agent framework for dynamic llm evaluation,' arXiv preprint arXiv:2402.11443 , 2024. - · arXiv
  123. [133][133] W. Wang, Z. Ma, P. Liu, and M. Chen, 'Revisiting benchmark and assessment: An agent-based exploratory dynamic evaluation framework for llms,' arXiv preprint arXiv:2410.11507 , 2024. - · arXiv
  124. [134][134] M. Wu, T. Zhu, H. Han, C. Tan, X. Zhang, and W. Chen, 'Seal-tools: Self-instruct tool learning dataset for agent tuning and detailed benchmark,' in NLPCC . Springer, 2024, pp. 372-384. -
  125. [135][135] Z. Guo, Y. Huang, and D. Xiong, 'Ctooleval: a chinese benchmark for llm-powered agent evaluation in real-world api interactions,' in ACL Findings , 2024, pp. 15 711-15 724. -
  126. [136][136] Y. Jiang, K. C. Black, G. Geng, D. Park, A. Y. Ng, and J. H. Chen, 'Medagentbench: Dataset for benchmarking llms as agents in medical applications,' arXiv preprint arXiv:2501.14654 , 2025. - · arXiv
  127. [137][137] Z. Fan, J. Tang, W. Chen, S. Wang, Z. Wei, J. Xi, F. Huang, and J. Zhou, 'Ai hospital: Benchmarking large language models in a multi-agent medical interaction simulator,' arXiv preprint arXiv:2402.09742 , 2024. - · arXiv
  128. [138][138] Y. Ma, C. Cui, X. Cao, W. Ye, P. Liu, J. Lu, A. Abdelraouf, R. Gupta, K. Han, A. Bera et al. , 'Lampilot: An open benchmark dataset for autonomous driving with language model programs,' in CVPR , 2024, pp. 15 141-15 151. -
  129. [139][139] Y. Zhang, Q. Jiang, X. Han, N. Chen, Y. Yang, and K. Ren, 'Benchmarking data science agents,' arXiv preprint arXiv:2402.17168 , 2024. - · arXiv
  130. [140][140] Y. Huang, J. Luo, Y. Yu, Y. Zhang, F. Lei, Y. Wei, S. He, L. Huang, X. Liu, J. Zhao et al. , 'Da-code: Agent data science code generation benchmark for large language models,' arXiv preprint arXiv:2410.07331 , 2024. - · arXiv
  131. [141][141] B. Huang, Y. Yu, J. Huang, X. Zhang, and J. Ma, 'Dcabench: A benchmark for dataset curation agents,' arXiv preprint arXiv:2406.07275 , 2024. - · arXiv
  132. [142][142] J. Xie, K. Zhang, J. Chen, T. Zhu, R. Lou, Y. Tian, Y. Xiao, and Y. Su, 'Travelplanner: A benchmark for real-world planning with language agents,' arXiv preprint arXiv:2402.01622 , 2024. - · arXiv
  133. [143][143] Q. Huang, J. Vora, P. Liang, and J. Leskovec, 'Benchmarking large language models as ai research agents,' in NeurIPS Workshop , 2023. -
  134. [144][144] J. S. Chan, N. Chowdhury, O. Jaffe, J. Aung, D. Sherburn, E. Mays, G. Starace, K. Liu, L. Maksin, T. Patwardhan et al. , 'Mlebench: Evaluating machine learning agents on machine learning engineering,' arXiv preprint arXiv:2410.07095 , 2024. - · arXiv
  135. [145][145] M. Andriushchenko, A. Souly, M. Dziemian, D. Duenas, M. Lin, J. Wang, D. Hendrycks, A. Zou, J. Z. Kolter, M. Fredrikson et al. , 'Agentharm: Benchmarking robustness of llm agents on harmful tasks,' in ICLR , 2024. -
  136. [146][146] T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, J. H. Toh, Z. Cheng, D. Shin, F. Lei et al. , 'Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments,' NeurIPS , vol. 37, pp. 52 040-52 094, 2025. -
  137. [147][147] K. Xu, Y. Kordi, T. Nayak, A. Asija, Y. Wang, K. Sanders, A. Byerly, J. Zhang, B. Van Durme, and D. Khashabi, 'Tur [k] ingbench: A challenge benchmark for web agents,' arXiv preprint arXiv:2403.11905 , 2024. - · arXiv
  138. [148][148] R. Kapoor, Y. P. Butala, M. Russak, J. Y. Koh, K. Kamble, W. AlShikh, and R. Salakhutdinov, 'Omniact: A dataset and benchmark for enabling multimodal generalist autonomous agents for desktop and web,' in ECCV . Springer, 2024, pp. 161-178. -
  139. [149][149] J. Yang, S. Liu, H. Guo, Y. Dong, X. Zhang, S. Zhang, P. Wang, Z. Zhou, B. Xie, Z. Wang et al. , 'Egolife: Towards egocentric life assistant,' arXiv preprint arXiv:2503.03803 , 2025. - · arXiv
  140. [150][150] J. Wang, M. Zerun, Y. Li, S. Zhang, C. Chen, K. Chen, and X. Le, 'Gta: a benchmark for general tool agents,' in NeurIPS , 2024. -
  141. [151][151] F. F. Xu, Y. Song, B. Li, Y. Tang, K. Jain, M. Bao, Z. Z. Wang, X. Zhou, Z. Guo, M. Cao et al. , 'Theagentcompany: benchmarking llm agents on consequential real world tasks,' arXiv preprint arXiv:2412.14161 , 2024. - · arXiv
  142. [152][152] R. Barbarroxa, L. Gomes, and Z. Vale, 'Benchmarking large language models for multi-agent systems: A comparative analysis of autogen, crewai, and taskweaver,' in International Conference on Practical Applications of Agents and Multi-Agent Systems . Springer, 2024, pp. 39-48. -
  143. [153][153] Z. Li, X. Wu, H. Du, H. Nghiem, and G. Shi, 'Benchmark evaluations, applications, and challenges of large vision language models: A survey,' arXiv preprint arXiv:2501.02189 , 2025. - · arXiv
  144. [154][154] M. Kenney, 'Ml research benchmark,' arXiv preprint arXiv:2410.22553 , 2024. - · arXiv
  145. [155][155] R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, X. Jiang, K. Cobbe, T. Eloundou, G. Krueger, K. Button, M. Knight, B. Chess, and J. Schulman, 'Webgpt: Browser-assisted question-answering with human feedback,' 2022. -
  146. [156][156] Y. Qin, Z. Cai, D. Jin, L. Yan, S. Liang, K. Zhu, Y. Lin, X. Han, N. Ding, H. Wang, R. Xie, F. Qi, Z. Liu, M. Sun, and J. Zhou, 'WebCPM: Interactive web search for Chinese long-form question answering,' in ACL , A. Rogers, J. Boyd-Graber, and N. Okazaki, Eds. Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 8968-8988. -
  147. [158][158] S. Robertson, H. Zaragoza et al. , 'The probabilistic relevance framework: Bm25 and beyond,' Foundations and Trends® in Information Retrieval , vol. 3, no. 4, pp. 333-389, 2009. -
  148. [161][161] X. Wang, Y. Chen, L. Yuan, Y. Zhang, Y. Li, H. Peng, and H. Ji, 'Executable code actions elicit better llm agents,' ArXiv , vol. abs/2402.01030, 2024. - · arXiv
  149. [162][162] T. Schick, J. Dwivedi-Yu, R. Dess` ı, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom, 'Toolformer: Language models can teach themselves to use tools,' Advances in Neural Information Processing Systems , vol. 36, pp. 68 539-68 551, 2023. -
  150. [163][163] B. Paranjape, S. Lundberg, S. Singh, H. Hajishirzi, L. Zettlemoyer, and M. T. Ribeiro, 'Art: Automatic multi-step reasoning and tooluse for large language models,' arXiv preprint arXiv:2303.09014 , 2023. - · arXiv
  151. [165][165] A. Saha, L. Mandal, B. Ganesan, S. Ghosh, R. Sindhgatta, C. Eberhardt, D. Debrunner, and S. Mehta, 'Sequential API function calling using GraphQL schema,' in EMNLP , Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds., Miami, Florida, USA, Nov. 2024, pp. 19 452-19 458. -
  152. [166][166] L. Yuan, Y. Chen, X. Wang, Y. R. Fung, H. Peng, and H. Ji, 'Craft: Customizing llms by creating and retrieving from specialized toolsets,' arXiv preprint arXiv:2309.17428 , 2023. - · arXiv
  153. [167][167] C. Qian, C. Xiong, Z. Liu, and Z. Liu, 'Toolink: Linking toolkit creation and using through chain-of-solving on open-source model,' in NAACL , 2024, pp. 831-854. -
  154. [168][168] C. Qian, C. Han, Y. Fung, Y. Qin, Z. Liu, and H. Ji, 'CREATOR: Tool creation for disentangling abstract and concrete reasoning of large language models,' in EMNLP Findings , H. Bouamor, J. Pino, and K. Bali, Eds., Singapore, Dec. 2023, pp. 6922-6939. -
  155. [175][175] A. Li, Y. Zhou, V. C. Raghuram, T. Goldstein, and M. Goldblum, 'Commercial llm agents are already vulnerable to simple yet dangerous attacks,' arXiv preprint arXiv:2502.08586 , 2025. - · arXiv
  156. [176][176] W. Zhang, K. Tang, H. Wu, M. Wang, Y. Shen, G. Hou, Z. Tan, P. Li, Y. Zhuang, and W. Lu, 'Agent-pro: Learning to evolve via policy-level reflection and optimization,' in ACL , 2024, pp. 5348-5375. -
  157. [177][177] L. Mo, Z. Liao, B. Zheng, Y. Su, C. Xiao, and H. Sun, 'A trembling house of cards? mapping adversarial attacks against language agents,' arXiv preprint arXiv:2402.10196 , 2024. - · arXiv
  158. [178][178] E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tramer, 'Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents,' in NeurIPS , vol. 37, 2024, pp. 82 895-82 920. -
  159. [179][179] C. H. Wu, J. Y. Koh, R. Salakhutdinov, D. Fried, and A. Raghunathan, 'Adversarial attacks on multimodal agents,' arXiv preprint arXiv:2406.12814 , 2024. - · arXiv
  160. [180][180] L.-b. Ning, S. Wang, W. Fan, Q. Li, X. Xu, H. Chen, and F. Huang, 'Cheatagent: Attacking llm-empowered recommender systems via llm agent,' in KDD , 2024, pp. 2284-2295. -
  161. [181][181] W. Yu, K. Hu, T. Pang, C. Du, M. Lin, and M. Fredrikson, 'Infecting llm agents via generalizable adversarial attack,' in NeurIPS Workshop , 2024. -
  162. [182][182] G. Lin and Q. Zhao, 'Large language model sentinel: Llm agent for adversarial purification,' arXiv preprint arXiv:2405.20770 , 2024. - · arXiv
  163. [183][183] S. Chern, Z. Fan, and A. Liu, 'Combating adversarial attacks with multi-agent debate,' arXiv preprint arXiv:2401.05998 , 2024. - · arXiv
  164. [184][184] X. Wang, J. Peng, K. Xu, H. Yao, and T. Chen, 'Reinforcement learning-driven llm agent for automated attacks on llms,' in ACL Findings , 2024, pp. 170-177. -
  165. [185][185] Y. Dong, Z. Li, X. Meng, N. Yu, and S. Guo, 'Jailbreaking text-to-image models with llm-based agents,' arXiv preprint arXiv:2408.00523 , 2024. - · arXiv
  166. [186][186] X. Chen, Y. Nie, W. Guo, and X. Zhang, 'When llm meets drl: Advancing jailbreaking efficiency via drl-guided search,' in NeurIPS , 2024. -
  167. [187][187] Z. Lin, W. Ma, M. Zhou, Y. Zhao, H. Wang, Y. Liu, J. Wang, and L. Li, 'Pathseeker: Exploring llm security vulnerabilities with a reinforcement learning-based jailbreak approach,' arXiv preprint arXiv:2409.14177 , 2024. - · arXiv
  168. [188][188] Y. Zeng, Y. Wu, X. Zhang, H. Wang, and Q. Wu, 'Autodefense: Multi-agent llm defense against jailbreak attacks,' arXiv preprint arXiv:2403.04783 , 2024. - · arXiv
  169. [189][189] S. Barua, M. Rahman, M. J. Sadek, R. Islam, S. Khaled, and A. Kabir, 'Guardians of the agentic system: Preventing many shots jailbreak with agentic system,' arXiv preprint arXiv:2502.16750 , 2025. - · arXiv
  170. [190][190] Z. Ni, H. Wang, and H. Wang, 'Shieldlearner: A new paradigm for jailbreak attack defense in llms,' arXiv preprint arXiv:2502.13162 , 2025. - · arXiv
  171. [191][191] P. Zhu, Z. Zhou, Y. Zhang, S. Yan, K. Wang, and S. Su, 'Demonagent: Dynamically encrypted multi-backdoor implantation attack on llm-based agent,' arXiv preprint arXiv:2502.12575 , 2025. - · arXiv
  172. [192][192] W. Yang, X. Bi, Y. Lin, S. Chen, J. Zhou, and X. Sun, 'Watch out for your agents! investigating backdoor threats to llm-based agents,' NeurIPS , vol. 37, pp. 100 938-100 964, 2025. -
  173. [193][193] Y. Wang, D. Xue, S. Zhang, and S. Qian, 'Badagent: Inserting and activating backdoor attacks in llm agents,' in ACL , 2024, pp. 9811-9827. -
  174. [194][194] T. Tong, F. Wang, Z. Zhao, and M. Chen, 'Badjudge: Backdoor vulnerabilities of llm-as-a-judge,' in ICLR , 2025. -
  175. [195][195] Z. Guo and R. Tourani, 'Darkmind: Latent chain-of-thought backdoor in customized llms,' arXiv preprint arXiv:2501.18617 , 2025. - · arXiv
  176. [196][196] Z. Zhou, Z. Li, J. Zhang, Y. Zhang, K. Wang, Y. Liu, and Q. Guo, 'Corba: Contagious recursive blocking attacks on multiagent systems based on large language models,' arXiv preprint arXiv:2502.14529 , 2025. - · arXiv
  177. [197][197] P. He, Y. Lin, S. Dong, H. Xu, Y. Xing, and H. Liu, 'Red-teaming llm multi-agent systems via communication attacks,' arXiv preprint arXiv:2502.14847 , 2025. - · arXiv
  178. [198][198] M. Yu, S. Wang, G. Zhang, J. Mao, C. Yin, Q. Liu, Q. Wen, K. Wang, and Y. Wang, 'Netsafe: Exploring the topological safety of multiagent networks,' arXiv preprint arXiv:2410.15686 , 2024. - · arXiv
  179. [199][199] S. Wang, G. Zhang, M. Yu, G. Wan, F. Meng, C. Guo, K. Wang, and Y. Wang, 'G-safeguard: A topology-guided security lens and treatment on llm-based multi-agent systems,' arXiv preprint arXiv:2502.11127 , 2025. - · arXiv
  180. [200][200] W. Hua, X. Yang, M. Jin, Z. Li, W. Cheng, R. Tang, and Y. Zhang, 'Trustagent: Towards safe and trustworthy llm-based agents through agent constitution,' in EMNLP Findings , 2024. -
  181. [201][201] Z. Zhang, Y. Zhang, L. Li, H. Gao, L. Wang, H. Lu, F. Zhao, Y. Qiao, and J. Shao, 'Psysafe: A comprehensive framework for psychological-based attack, defense, and evaluation of multi-agent system safety,' arXiv preprint arXiv:2401.11880 , 2024. - · arXiv
  182. [202][202] Z. Deng, Y. Guo, C. Han, W. Ma, J. Xiong, S. Wen, and Y. Xiang, 'Ai agents under threat: A survey of key security challenges and future pathways,' ACM Computing Surveys , 2024. -
  183. [203][203] E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tram` er, 'Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents,' NeurIPS , vol. 37, pp. 82 895-82 920, 2025. -
  184. [204][204] X. Li, Z. Li, Y. Kosuga, Y. Yoshida, and V. Bian, 'Targeting the core: A simple and effective method to attack rag-based agents via direct llm manipulation,' arXiv preprint arXiv:2412.04415 , 2024. - · arXiv
  185. [205][205] Q. Zhan, Z. Liang, Z. Ying, and D. Kang, 'Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents,' arXiv preprint arXiv:2403.02691 , 2024. - · arXiv
  186. [206][206] D. Pasquini, E. M. Kornaropoulos, and G. Ateniese, 'Hacking back the ai-hacker: Prompt injection as a defense against llm-driven cyberattacks,' arXiv preprint arXiv:2410.20911 , 2024. - · arXiv
  187. [207][207] S. Abdelnabi, A. Gomaa, E. Bagdasarian, P. O. Kristensson, and R. Shokri, 'Firewalls to secure dynamic llm agentic networks,' arXiv preprint arXiv:2502.01822 , 2025. - · arXiv
  188. [208][208] P. Y. Zhong, S. Chen, R. Wang, M. McCall, B. L. Titzer, and H. Miller, 'Rtbas: Defending llm agents against prompt injection and privacy leakage,' arXiv preprint arXiv:2502.08966 , 2025. - · arXiv
  189. [209][209] F. Jia, T. Wu, X. Qin, and A. Squicciarini, 'The task shield: Enforcing task alignment to defend against indirect prompt injection in llm agents,' arXiv preprint arXiv:2412.16682 , 2024. - · arXiv
  190. [210][210] Y. Tian, X. Yang, J. Zhang, Y. Dong, and H. Su, 'Evil geniuses: Delving into the safety of llm-based agents,' arXiv preprint arXiv:2311.11855 , 2023. - · arXiv
  191. [211][211] C. Wang, Q. Long, X. Meng, X. Cai, C. Wu, Z. Meng, X. Wang, and Y. Zhou, 'Biorag: A rag-llm framework for biological question reasoning,' arXiv preprint arXiv:2408.01107 , 2024. - · arXiv
  192. [213][213] Z. Xiang, Y. Zeng, M. Kang, C. Xu, J. Zhang, Z. Yuan, Z. Chen, C. Xie, F. Jiang, M. Pan et al. , 'Clas 2024: The competition for llm and agent safety,' in NeurIPS Workshop , 2024. -
  193. [214][214] F. Wu, S. Wu, Y. Cao, and C. Xiao, 'Wipi: A new web threat for llm-driven web agents,' arXiv preprint arXiv:2402.16965 , 2024. - · arXiv
  194. [215][215] I. Nakash, G. Kour, G. Uziel, and A. Anaby-Tavor, 'Breaking react agents: Foot-in-the-door attack will get you in,' arXiv preprint arXiv:2410.16950 , 2024. - · arXiv
  195. [216][216] Z. Chen, Z. Xiang, C. Xiao, D. Song, and B. Li, 'Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases,' NeurIPS , vol. 37, pp. 130 185-130 213, 2025. -
  196. [217][217] B. Wang, W. He, P. He, S. Zeng, Z. Xiang, Y. Xing, and J. Tang, 'Unveiling privacy risks in llm agent memory,' arXiv preprint arXiv:2502.13172 , 2025. - · arXiv
  197. [219][219] Y. Li, H. Wen, W. Wang, X. Li, Y. Yuan, G. Liu, J. Liu, W. Xu, X. Wang, Y. Sun et al. , 'Personal llm agents: Insights and survey about the capability, efficiency and security,' arXiv preprint arXiv:2401.05459 , 2024. - · arXiv
  198. [220][220] X. Gu, X. Zheng, T. Pang, C. Du, Q. Liu, Y. Wang, J. Jiang, and M. Lin, 'Agent smith: A single image can jailbreak one million multimodal llm agents exponentially fast,' arXiv preprint arXiv:2402.08567 , 2024. - · arXiv
  199. [221][221] D. Lee and M. Tiwari, 'Prompt infection: Llm-to-llm prompt injection within multi-agent systems,' arXiv preprint arXiv:2410.07283 , 2024. - · arXiv
  200. [222][222] B. Chen, G. Li, X. Lin, Z. Wang, and J. Li, 'Blockagents: Towards byzantine-robust llm-based multi-agent coordination via blockchain,' in ACM Turing Award Celebration Conference , 2024, pp. 187-192. -

200 of 302 shown

Cited by

Sorted by citations

Related

Authors
Luo, J., Zhang, W., Yuan, Y., Zhao, Y., Yang, J., Gu, Y., Wu, B., Chen, B., Qiao, Z., Long, Q., Tu, R., Luo, X., Ju, W., Xiao, Z., Wang, Y., Xiao, M., Liu, C., Yuan, J., Zhang, S., Jin, Y., Zhang, F., Wu, X., Zhao, H., Tao, D., Yu, P. S., & Zhang, M.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2503.21460
arXiv
2503.21460
OpenAlex
W4415062702
Cited by
5
References
0
Citations by year
2026: 3 · 2025: 2
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
LLM agents, agent architecture, agent cooperation, agent evolution, agent evaluation, tool use, goal-directed behavior, artificial general intelligence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review