PreprintCited by 36Evolutionary Algorithms and ApplicationsMulti-Agent Systems and NegotiationAI-based Problem Solving and Planning

EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms

Open access
Yuan, Siyu, Song, Kaitao, Chen, Jiangjie, Tan, Xu, Li, Dongsheng, Yang, Deqing·2024·arXiv

Abstract

The rise of powerful large language models (LLMs) has spurred a new trend in building LLM-based autonomous agents for solving complex tasks, especially multi-agent systems. Despite the remarkable progress, we notice that existing works are heavily dependent on human-designed frameworks, which greatly limits the functional scope and scalability of agent systems. How to automatically extend the specialized agent to multi-agent systems to improve task-solving capability still remains a significant challenge. In this paper, we introduce EvoAgent, a generic method to automatically extend specialized agents to multi-agent systems via the evolutionary algorithm, thereby improving the effectiveness of LLM-based agents in solving tasks. Specifically, we consider the existing agent frameworks as the initial individual and then apply a series of evolutionary operators (e.g., mutation, crossover, selection, etc.) to generate multiple agents with diverse settings. Experimental results across various tasks show that EvoAgent can significantly enhance the task-solving capability of LLM-based agents, and can be generalized to any LLM-based agent framework to extend them into multi-agent systems. Resources are available at https://evo-agent.github.io/.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2406.14228

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior agentic automation pipelines have generally excluded M from their search space, often preselecting a single LLM M to instantiate all agents, which constrains the development of more diverse and capability-rich agentic systems.

    “prior agentic automation pipelines have generally excluded M from their search space (Zhang et al., 2024c; Yuan et al., 2024; Zhuge et al., 2024), often preselecting a single LLM M to instantiate all agents, which constrains the development of more diverse and capability-rich agentic systems.”

    From EvoFlow: Evolving Diverse Agentic Workflows On The Fly (2025) · cites this paper as Yuan et al., 2024

    The Ablation Studies section says EVOAGENT uses an LLM with the same backbone as its initial agent, and the NLP and Multi-Modal Tasks section describes selecting backbone models for experiments. The paper also criticizes prior frameworks for relying on handcrafted agent settings (Agent Generation), but it does not establish that prior pipelines generally exclude LLM choice from their search space or that this specifically limits model diversity and capability. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • EvoAgent and EvoPrompt employ simple genetic algorithms to optimize agent profiles and prompts, with a highly constrained level of automation, focusing solely on single-agent prompt optimization and failing to evolve at the workflow level.

    “EvoAgent (Yuan et al., 2024) and EvoPrompt (Guo et al., 2023) employ simple genetic algorithms to optimize agent profiles and prompts, whose, however, level of automation is highly constrained, focusing solely on single-agent prompt optimization and failing to evolve at the workflow level”

    From EvoFlow: Evolving Diverse Agentic Workflows On The Fly (2025) · cites this paper as Yuan et al., 2024

    The Introduction and §3.2 describe evolutionary operators that update agent settings, including roles, skills, and prompts, so the evolutionary-optimization part is supported. But they describe automatically generating multiple specialized agents that collaborate, not focusing solely on single-agent prompt optimization. The paper does not establish the claim's characterization of EvoAgent's automation as highly constrained; it says generation requires no additional human design. The cited passages do not show workflow-level evolution. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Another line of research focuses on optimizing the configuration of multi-agent systems.

    “Another line of research focuses on optimizing the configuration of multi-agent systems (Chen et al., 2023;Yuan et al., 2024;Li et al., 2023;Zhuge et al., 2024;Wang et al., 2023b).”

    From AgentSquare: Automatic LLM Agent Search in Modular Design Space (2024) · cites this paper as Yuan et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 40

  1. Thomas Bäck and Hans-Paul Schwefel. 1993. An overview of evolutionary algorithms for parameter optimization. Evol. Comput., 1(1):1-23.
  2. Thomas Bartz-Beielstein, Jürgen Branke, Jörn Mehnen, and Olaf Mersmann. 2014. Evolutionary algorithms. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 4(3):178-195.
  3. Angelica Chen, David Dohan, and David R. So. 2023a. Evoprompting: Language models for code-level neu- ral architecture search. In Advances in Neural In- formation Processing Systems 36: Annual Confer- ence on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 -16, 2023.
  4. Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F Karlsson, Jie Fu, and Yemin Shi. 2023b. Autoagents: A framework for automatic agent generation. arXiv preprint arXiv:2309.17288. · arXiv
  5. Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, et al. 2024a. From persona to person- alization: A survey on role-playing language agents. arXiv preprint arXiv:2404.18231. · arXiv
  6. Jiangjie Chen, Siyu Yuan, Rong Ye, Bodhisattwa Prasad Majumder, and Kyle Richardson. 2023c. Put your money where your mouth is: Evaluating strategic planning and execution of llm agents in an auction arena. arXiv preprint arXiv:2310.05746. · arXiv
  7. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2024b. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In The Twelfth International Conference on Learning Representations.
  8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. · arXiv
  9. Agoston E Eiben, James E Smith, AE Eiben, and JE Smith. 2015. What is an evolutionary algorithm? Introduction to evolutionary computing, pages 25- 48.
  10. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yu- jiu Yang. 2023. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. CoRR, abs/2309.08532. · arXiv
  11. John H Holland. 1992. Adaptation in natural and artifi- cial systems: an introductory analysis with applica- tions to biology, control, and artificial intelligence. MIT press.
  12. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2024. MetaGPT: Meta pro- gramming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations.
  13. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023a. CAMEL: Communicative agents for "mind" exploration of large language model society. In Thirty-seventh Conference on Neural Information Processing Systems.
  14. Yuan Li, Yixuan Zhang, and Lichao Sun. 2023b. Metaa- gents: Simulating interactions of human behav- iors for llm-based task-oriented coordination via collaborative generative agents. arXiv preprint arXiv:2310.06500. · arXiv
  15. Bill Yuchen Lin, Yicheng Fu, Karina Yang, Faeze Brah- man, Shiyu Huang, Chandra Bhagavatula, Prithviraj Ammanabrolu, Yejin Choi, and Xiang Ren. 2023. Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. In Thirty- seventh Conference on Neural Information Process- ing Systems.
  16. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdan- bakhsh, and Peter Clark. 2023. Self-refine: Itera- tive refinement with self-feedback. In Thirty-seventh Conference on Neural Information Processing Sys- tems.
  17. Seyedali Mirjalili, Jin Song Dong, Ali Safa Sadiq, and Hossam Faris. 2020. Genetic algorithm: Theory, lit- erature review, and application in image reconstruc- tion. Nature-inspired optimizers: Theories, literature reviews and applications, pages 69-85.
  18. Melanie Mitchell. 1998. An introduction to genetic algorithms. MIT press.
  19. OpenAI. 2022. Chatgpt. OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774. · arXiv
  20. Joon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023b. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST 2023, San Francisco, CA, USA, 29 October 2023-1 November 2023, pages 2:1-2:22. ACM.
  21. Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. 2023. Communicative agents for software de- velopment. arXiv preprint arXiv:2307.07924. · arXiv
  22. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. CoRR, abs/2302.04761. · arXiv
  23. Lothar M Schmitt. 2001. Theory of genetic algorithms. Theoretical Computer Science, 259(1-2):1-61.
  24. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023a. Hugging- gpt: Solving AI tasks with chatgpt and its friends in huggingface. CoRR, abs/2303.17580. · arXiv
  25. Yongliang Shen, Kaitao Song, Xu Tan, Wenqi Zhang, Kan Ren, Siyu Yuan, Weiming Lu, Dongsheng Li, and Yueting Zhuang. 2023b. Taskbench: Benchmark- ing large language models for task automation. arXiv preprint arXiv:2311.18760. · arXiv
  26. Noah Shinn, Beck Labash, and Ashwin Gopinath. 2023. Reflexion: an autonomous agent with dy- namic memory and self-reflection. arXiv preprint arXiv:2303.11366. · arXiv
  27. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, An- thony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Di- ana Liskovich, Yinghai Lu, Yuning Mao, Xavier Mar- tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly- bog, Yixin Nie, Andrew Poulton, Jeremy Reizen- stein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subrama- nian, Xiaoqing Ellen Tan, Binh Tang, Ross Tay- lor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Ro- driguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine- tuned chat models. CoRR, abs/2307.09288. · arXiv
  28. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. 2023. Unleash- ing cognitive synergy in large language models: A task-solving agent through multi-persona self- collaboration. arXiv preprint arXiv:2307.05300. · arXiv
  29. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022a. Chain of thought prompt- ing elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
  30. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022b. Chain-of-thought prompt- ing elicits reasoning in large language models. In NeurIPS.
  31. Jason Weston, Sumit Chopra, and Antoine Bordes. 2015. Memory networks. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceed- ings.
  32. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. Auto- gen: Enabling next-gen llm applications via multi- agent conversation framework. arXiv preprint arXiv:2308.08155. · arXiv
  33. Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024. Travelplanner: A benchmark for real- world planning with language agents. arXiv preprint arXiv:2402.01622. · arXiv
  34. Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. 2023. Expertprompting: Instructing large language models to be distinguished experts. arXiv preprint arXiv:2305.14688. · arXiv
  35. Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023. The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421, 9(1):1. · arXiv
  36. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations.
  37. Siyu Yuan, Jiangjie Chen, Ziquan Fu, Xuyang Ge, So- ham Shah, Charles Robert Jankowski, Yanghua Xiao, and Deqing Yang. 2023. Distilling script knowledge from large language models for constrained language planning. arXiv preprint arXiv:2305.05252. · arXiv
  38. Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Ren Kan, Dongsheng Li, and De- qing Yang. 2024. Easytool: Enhancing llm-based agents with concise tool instruction. arXiv preprint arXiv:2401.06201. · arXiv
  39. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. 2023. Mmmu: A massive multi-discipline multimodal understand- ing and reasoning benchmark for expert agi. arXiv preprint arXiv:2311.16502. · arXiv
  40. Yikai Zhang, Siyu Yuan, Caiyu Hu, Kyle Richard- son, Yanghua Xiao, and Jiangjie Chen. 2024. Timearena: Shaping efficient multitasking language agents in a time-aware simulation. arXiv preprint arXiv:2402.05733. · arXiv

Cited by

Sorted by citations

Authors
Yuan, S., Song, K., Chen, J., Tan, X., Li, D., & Yang, D.
Affiliations
  • Fudan University (CN)
  • Microsoft Research Asia (China) (CN)
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
OpenAlex
W4399911620
Cited by
0
References
0
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Funders
Major Research Plan
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
multi-agent systems, LLM agents, evolutionary algorithms, autonomous agents, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review