SEMAG: Self-Evolutionary Multi-Agent Code Generation
Open accessAbstract
Large Language Models (LLMs) have made significant progress in handling complex programming tasks. However, current methods rely on manual model selection and fixed workflows, which limit their ability to adapt to changing task complexities. To address this, we propose SEMAG, a Self-Evolutionary Multi-Agent code Generation framework that mimics human coding practices. It decomposes programming tasks into stages, including planning, coding, debugging, and discussion, while adapting workflows to task difficulty. Its self-evolutionary agents can access the latest models in real time and automatically upgrade the backbone model. SEMAG sets new state-of-the-art Pass@1 accuracy across benchmarks. Using identical backbone models, SEMAG outperforms prior methods by 3.3% on CodeContests. When augmented with self-evolutionary model selection that automatically identifies optimal backbones, SEMAG reaches 52.6%, showcasing both framework effectiveness and adaptability to evolving LLM capabilities.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Other recent multi-agent coding systems [Pan et al., 2026, Islam et al., 2025, Peng et al., 2026] share the LLM-team-coding goal but use sequential or staged-role paradigms.
“Other recent multi-agent coding systems [Pan et al., 2026, Islam et al., 2025, Peng et al., 2026] share the LLM-team-coding goal but use sequential or staged-role paradigms.”
From AgentRoom: Concurrent Multi-Agent Coding in a CRDT-Backed Shared Workspace (2026) · cites this paper as Peng et al., 2026
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Self-collaboration code generation via ChatGPT
Dong, Y., Jiang, X., Jin, Z., & Li, G. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672459 · Cited by 256 · cited 1× · see citing works
“Experimental results show that SEMAG achieves new state-of-the-art performance, including 98.8% Pass@1(Chen et al., 2021; Dong et al., 2024) on HumanEval, 87.6% on MBPP, and 65.0% on LiveCode.”
Influential
Flagged first, then by citations
Research works printed but not linked: 37
- Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 techni- cal report. Preprint, arXiv:2303.08774. · arXiv
- Antonis Antoniades, Albert Örwall, Kexun Zhang, Yuxi Xie, Anirudh Goyal, and William Wang. 2025. Swe- search: Enhancing software agents with monte carlo tree search and iterative refinement. In Interna- tional Conference on Representation Learning, vol- ume 2025, pages 64485-64515.
- Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. Program synthesis with large language models. Preprint, arXiv:2108.07732. · arXiv
- R BALZER. 1985. A 15 year perspective on automatic programming. IEEE transactions on software engi- neering, 11(11):1257-1268.
- Devichand Budagam, Ashutosh Kumar, Mahsa Khosh- noodi, Sankalp KJ, Vinija Jain, and Aman Chadha. 2025. Hierarchical prompting taxonomy: A univer- sal evaluation framework for large language models aligned with human cognitive principles. In First In- ternational KDD Workshop on Prompt Optimization, 2025.
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Ka- plan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. Preprint, arXiv:2107.03374. · arXiv
- Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023. Teaching large language models to self-debug. Preprint, arXiv:2304.05128. · arXiv
- Yihong Dong, Jiazheng Ding, Xue Jiang, Ge Li, Zhuo Li, and Zhi Jin. 2025. Codescore: Evaluating code generation by learning code execution. ACM Trans- actions on Software Engineering and Methodology, 34(3):1-22.
- Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific lan- guage model pretraining for biomedical natural lan- guage processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1-23.
- Dan Hendrycks, Steven Basart, Saurav Kadavath, Man- tas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring coding challenge com- petence with apps. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1.
- Yaojie Hu, Qiang Zhou, Qihong Chen, Xiaopeng Li, Linbo Liu, Dejiao Zhang, Amit Kachroo, Talha Oz, and Omer Tripp. 2025. Qualityflow: An agentic workflow for program synthesis controlled by llm quality checks. Preprint, arXiv:2501.17167. · arXiv
- Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2. 5-coder technical report. Preprint, arXiv:2409.12186. · arXiv
- Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. Preprint, arXiv:2410.21276. · arXiv
- Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. 2024. Mapcoder: Multi-agent code generation for competitive problem solving. In Proceedings of the 62nd Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 4912-4944. · DOI · Paper page
- Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. 2025. Live- codebench: Holistic and contamination free evalua- tion of large language models for code. In Interna- tional Conference on Representation Learning, pages 58791-58831.
- Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, and 1 others. 2024a. Mixtral of experts. Preprint, arXiv:2401.04088. · arXiv
- Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. 2024b. Self-planning code generation with large language models. ACM Transactions on Software Engineering and Methodology, 33(7):1-30.
- Chao Lei, Yanchuan Chang, Nir Lipovetzky, and Krista A Ehinger. 2024. Planning-driven program- ming: A large language model programming work- flow. Preprint, arXiv:2411.14503. · arXiv
- Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E Gonzalez, and Ion Stoica. 2025. S*: Test time scal- ing for code generation. Preprint, arXiv:2502.14382. · arXiv
- Kefan Li and Yuan Yuan. 2024. Large language models as test case generators: Performance evaluation and enhancement. Preprint, arXiv:2404.13340. · arXiv
- Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, and 1 others. 2022. Competition-level code generation with alphacode. Science, 378(6624):1092-1097.
- Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. · arXiv
- Zohar Manna and Richard J Waldinger. 1971. Toward automatic program synthesis. Communications of the ACM, 14(3):151-165.
- Noble Saji Mathews and Meiyappan Nagappan. 2024. Test-driven development for code generation. Preprint, arXiv:2402.13521. · arXiv
- John McCarthy. 1978. History of lisp. SIGPLAN Not., 13(8):217-223.
- Marjan Mernik, Jan Heering, and Anthony M Sloane. 2005. When and how to develop domain-specific lan- guages. ACM computing surveys (CSUR), 37(4):316- 344.
- Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, and 1 others. 2023. Code llama: Open foundation models for code. Preprint, arXiv:2308.12950. · arXiv
- Elliot Soloway. 1986. Learning to program = learning to construct mechanisms and explanations. Commu- nications of the ACM, 29(9):850-858.
- Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. Gemini: a family of highly capable multimodal models. Preprint, arXiv:2312.11805. · arXiv
- Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, and 1 others. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. Preprint, arXiv:2403.05530. · arXiv
- Richard J Waldinger and Richard CT Lee. 1969. Prow: A step toward automatic program writing. In Pro- ceedings of the 1st international joint conference on Artificial intelligence, pages 241-252.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elic- its reasoning in large language models. Advances in neural information processing systems, 35:24824- 24837.
- Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023a. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809-11822.
- Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023b. React: Synergizing reasoning and acting in language models. In International Conference on Representa- tion Learning.
- Yifan Zhang, Jingqin Yang, Yang Yuan, and Andrew Chi-Chih Yao. 2023. Cumulative reasoning with large language models. Preprint, arXiv:2308.04371. · arXiv
- Li Zhong, Zilong Wang, and Jingbo Shang. 2024. De- bug like a human: A large language model debugger via verifying runtime execution step by step. In Find- ings of the Association for Computational Linguistics: ACL 2024, pages 851-870. · DOI
- Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and 1 oth- ers. 2022. Least-to-most prompting enables com- plex reasoning in large language models. Preprint, arXiv:2205.10625. · arXiv
Cited by
Sorted by citations
AgentRoom: Concurrent Multi-Agent Coding in a CRDT-Backed Shared Workspace
Cho, S., & Lee, D. (2026) · arXiv · doi:10.48550/arxiv.2608.23740
- Authors
- Peng, Y., Hou, H., Zhu, X., He, Y. T., & Yu, F. R.
- Affiliations
- Shenzhen University (CN)
- Carleton University (CA)
- ORCID
- Hou, Haowen
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2603.15707
- OpenAlex
- W7139087799
- Cited by
- 0
- References
- 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Keywords
- large language models, dynamic model selection, debugging, adaptive workflows, model upgrade
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
