CoChat
Sign Up
PreprintCited by 81Software Engineering ResearchSoftware Engineering Techniques and PracticesAdvanced Software Engineering Methodologies

ChatDev: Communicative Agents for Software Development

Open access
Qian, Chen, Liu, Wei, Liu, Hongzhang, Chen, Nuo, Dang, Yufan, Li, Jiahao, Yang, Cheng, Chen, Weize +6 more·2023·arXiv
DOI 10.48550/arxiv.2307.07924arXiv 2307.07924
Open this paper in CoChat — free accountOpen PDF

Abstract

Software development is a complex task that necessitates cooperation among multiple members with diverse skills. Numerous studies used deep learning to improve specific phases in a waterfall model, such as design, coding, and testing. However, the deep learning model in each phase requires unique designs, leading to technical inconsistencies across various phases, which results in a fragmented and ineffective development process. In this paper, we introduce ChatDev, a chat-powered software development framework in which specialized agents driven by large language models (LLMs) are guided in what to communicate (via chat chain) and how to communicate (via communicative dehallucination). These agents actively contribute to the design, coding, and testing phases through unified language-based communication, with solutions derived from their multi-turn dialogues. We found their utilization of natural language is advantageous for system design, and communicating in programming language proves helpful in debugging. This paradigm demonstrates how linguistic communication facilitates multi-agent collaboration, establishing language as a unifying bridge for autonomous task-solving among LLM agents. The code and data are available at https://github.com/OpenBMB/ChatDev.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2307.07924

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Autonomous agents typically perform tasks by assuming specific roles, such as coders, teachers and domain experts.

    “Autonomous agents typically perform tasks by assuming specific roles, such as coders, teachers and domain experts [18,19].”

    From A survey on large language model based autonomous agents (2024) · cites this paper as [18]

    The Introduction describes autonomous agents with specialized roles cooperating on software-development tasks, and the Ablation Study discusses programmer and reviewer roles. The paper supports the role-based approach and the coder example, but the cited passages do not establish teachers or domain experts as agent roles, or that agents typically use roles across tasks generally. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • LLMs can usually generate high-quality conversations, enabling agents to behave more like humans.

    “LLMs can usually generate high-quality conversations.This capability enables the agent to behave more like humans.”

    From A survey on large language model based autonomous agents (2024) · cites this paper as [18]

    The Introduction describes LLMs’ strong role-playing capacity and agents that collaborate through multi-turn dialogue. Section 3.1 (Chat Chain) describes how ChatDev structures those exchanges, but also says that simply exchanging responses does not ensure effective communication. The paper does not establish that LLMs usually generate high-quality conversations or that conversation makes agents behave more like humans. Page numbers are unavailable in our copy. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

  • In ChatDev, a virtual CEO meta-agent distributes subtasks to departmental agents and integrates their outputs into a unified strategic response.

    “In ChatDev [155], for example, a virtual CEO meta-agent distributes subtasks to departmental agents and integrates their outputs into a unified strategic response.”

    From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [155]

    Section 3 and §3.1 Chat Chain (page not available) support that ChatDev divides software development into subtasks handled by agents with specialized roles, and carries solutions forward between subtasks. They do not describe a virtual CEO meta-agent distributing work to departmental agents or integrating their outputs into a strategic response; §3.1 instead describes a sequential workflow with two agents per subtask. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Common efficiency metrics for LLM-based agents include length of planning, cost associated with development, speed of inference, and number of clarification dialogues.

    “Commonly considered metrics encompass the length of planning [57], the cost associated with development [18], the speed of inference [16,38], and number of clarification dialogues [122].”

    From A survey on large language model based autonomous agents (2024) · cites this paper as [18]

  • Platforms like AutoGen and ChatDev exemplify agentic coordination through task decomposition, role assignment, and recursive feedback loops.

    “For example, platforms like AutoGen [100] and ChatDev [155] exemplify agentic coordination through task decomposition, role assignment, and recursive feedback loops.”

    From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [155]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 66

  1. Silvia T Acuna, Natalia Juristo, and Ana M Moreno. 2006. Emphasizing Human Capabilities in Software Development. In IEEE Software, volume 23, pages 94-101.
  2. Mansi Agnihotri and Anuradha Chug. 2020. A Sys- tematic Literature Survey of Software Metrics, Code Smells and Refactoring Techniques. In booktitle of Information Processing Systems, volume 16, pages 915-934.
  3. Rajiv D Banker, Gordon B Davis, and Sandra A Slaugh- ter. 1998. Software Development Practices, Software Complexity, and Software Maintenance Performance: A Field Study. In Management science, volume 44, pages 433-450. · DOI
  4. Victor R Basili. 1989. Software Development: A Paradigm for The Future. In Proceedings of the An- nual International Computer Software and Applica- tions Conference, pages 471-485. IEEE.
  5. Thorsten Brants, Ashok C Popat, Peng Xu, Franz J Och, and Jeffrey Dean. 2007. Large Language Models in Machine Translation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 858- 867.
  6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Ad- vances in Neural Information Processing Systems (NeurIPS), volume 33, pages 1877-1901.
  7. Sébastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Pe- ter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of artificial general in- telligence: Early experiments with gpt-4. In arXiv preprint arXiv:2303.12712. · arXiv
  8. Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. 2023. Large Language Models as Tool Makers. In arXiv preprint arXiv:2305.17126. · arXiv
  9. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate. In arXiv preprint arXiv:2308.07201. · arXiv
  10. Dake Chen, Hanbin Wang, Yunhao Huo, Yuzhao Li, and Haoyang Zhang. 2023a. GameGPT: Multi-agent Collaborative Framework for Game Development. In arXiv preprint arXiv:2310.08067. · arXiv
  11. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating Large Lan- guage Models Trained on Code. In arXiv preprint arXiv:2107.03374. · arXiv
  12. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. 2023b. AgentVerse: Facilitating Multi-agent Collaboration and Explor- ing Emergent Behaviors in Agents. In International Conference on Learning Representations (ICLR).
  13. Roi Cohen, May Hamri, Mor Geva, and Amir Glober- son. 2023. LM vs LM: Detecting Factual Er- rors via Cross Examination. In ArXiv, volume abs/2305.13281. · arXiv
  14. Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2023. Chain-of-Verification Reduces Hallucination in Large Language Models. In arXiv preprint arXiv:2309.11495. · arXiv
  15. Shiying Ding, Xinyi Chen, Yan Fang, Wenrui Liu, Yiwu Qiu, and Chunlei Chai. 2023. DesignGPT: Multi- Agent Collaboration in Design. In arXiv preprint arXiv:2311.11591. · arXiv
  16. Michael D. Ernst. 2017. Natural Language is a Pro- gramming Language: Applying Natural Language Processing to Software Development. In Advances in Programming Languages (SNAPL), volume 71, pages 4:1-4:14. · DOI
  17. Saad Ezzini, Sallam Abualhaija, Chetan Arora, and Mehrdad Sabetzadeh. 2022. Automated Handling of Anaphoric Ambiguity in Requirements: A Multi- solution Study. In International Conference on Soft- ware Engineering (ICSE), pages 187-199. · DOI
  18. Robert France and Bernhard Rumpe. 2007. Model- driven Development of Complex Software: A Re- search Roadmap. In Future of Software Engineering (FOSE), pages 37-54.
  19. Peter Freeman, Donald J. Bagert, Hossein Saiedian, Mary Shaw, Robert Dupuis, and J. Barrie Thompson. 2001. Software Engineering Body of Knowledge (SWEBOK). In Proceedings of the International Conference on Software Engineering (ICSE), pages 693-696.
  20. Sa Gao, Chunyang Chen, Zhenchang Xing, Yukun Ma, Wen Song, and Shang-Wei Lin. 2019. A Neural Model for Method Name Generation from Functional Description. In 26th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 411-421. · DOI
  21. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2023. MetaGPT: Meta Pro- gramming for A Multi-Agent Collaborative Frame- work. In International Conference on Learning Rep- resentations (ICLR).
  22. Wenyue Hua, Lizhou Fan, Lingyao Li, Kai Mei, Jianchao Ji, Yingqiang Ge, Libby Hemphill, and Yongfeng Zhang. 2023. War and Peace (WarA- gent): Large Language Model-based Multi-Agent Simulation of World Wars. In arXiv preprint arXiv:2311.17227. · arXiv
  23. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hal- lucination in Natural Language Generation. In ACM Computing Surveys, volume 55, pages 1-38. · DOI · Paper page
  24. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling Laws for Neural Language Models. In arXiv preprint arXiv:2001.08361. · arXiv
  25. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023a. CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS).
  26. Yuan Li, Yixuan Zhang, and Lichao Sun. 2023b. MetaAgents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordina- tion via Collaborative Generative Agents. In arXiv preprint arXiv:2310.06500. · arXiv
  27. Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinecke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpit, Ran Xu, Phil Mui, Huan Wang, Caiming Xiong, and Silvio Savarese. 2023. BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents. In arXiv preprint arXiv:2308.05960. · arXiv
  28. Kaixin Ma, Hongming Zhang, Hongwei Wang, Xiao- man Pan, and Dong Yu. 2023. LASER: LLM Agent with State-Space Exploration for Web Navigation. In arXiv preprint arXiv:2309.08172. · arXiv
  29. Cuauhtémoc López Martín and Alain Abran. 2015. Neu- ral networks for predicting the duration of new soft- ware projects. In J. Syst. Softw., volume 101, pages 127-135. · DOI
  30. Nadia Nahar, Shurui Zhou, Grace A. Lewis, and Christian Kästner. 2022. Collaboration Challenges in Building ML-Enabled Systems: Communica- tion, Documentation, Engineering, and Process. In IEEE/ACM International Conference on Software Engineering (ICSE), pages 413-425. · DOI
  31. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthe- sis. In The International Conference on Learning Representations (ICLR).
  32. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Pe- ter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training Language Models to Follow Instructions with Human Feedback. In arXiv preprint arXiv:2203.02155. · arXiv
  33. Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Mered- ith Ringel Morris, Percy Liang, and Michael S. Bern- stein. 2023. Generative Agents: Interactive Simu- lacra of Human Behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST). · DOI · Paper page
  34. Florian Pudlitz, Florian Brokhausen, and Andreas Vo- gelsang. 2019. Extraction of System States from Natural Language Requirements. In IEEE Interna- tional Requirements Engineering Conference (RE), pages 211-222. · DOI
  35. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023a. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-World APIs. In arXiv preprint arXiv:2307.16789. · arXiv
  36. Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Don- ald Metzler, Xuanhui Wang, and Michael Bendersky. 2023b. Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting. In arXiv preprint arXiv:2306.17563. · arXiv
  37. Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng, and Rui Zhao. 2023. TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage. In arXiv preprint arXiv:2308.03427. · arXiv
  38. Steve Sawyer and Patricia J. Guinan. 1998. Software development: Processes and Performance. In IBM Systems Journal, volume 37, pages 552-569.
  39. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. ToolFormer: Language Models Can Teach Themselves to Use Tools. In arXiv preprint arXiv:2302.04761. · arXiv
  40. Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role Play with Large Language Models. In Nature, volume 623, pages 493-498. · DOI · Paper page
  41. Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. 2023. Cognitive Archi- tectures for Language Agents. In arXiv preprint arXiv:2309.02427. · arXiv
  42. Hannes Thaller, Lukas Linsbauer, and Alexander Egyed. 2019. Feature Maps: A Comprehensible Software Representation for Design Pattern Detection. In IEEE International Conference on Software Anal- ysis, Evolution and Reengineering (SANER), pages 207-217. · DOI
  43. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. LLaMA: Open and Efficient Foundation Language Models. In arXiv preprint arXiv:2302.13971. · arXiv
  44. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Advances in Neural Information Pro- cessing Systems (NeurIPS), volume 30.
  45. Chengcheng Wan, Shicheng Liu, Sophie Xie, Yifan Liu, Henry Hoffmann, Michael Maire, and Shan Lu. 2022. Automated Testing of Software that Uses Ma- chine Learning APIs. In IEEE/ACM International Conference on Software Engineering (ICSE), pages 212-224. · DOI
  46. Yao Wan, Zhou Zhao, Min Yang, Guandong Xu, Haochao Ying, Jian Wu, and Philip S. Yu. 2018. Im- proving Automatic Source Code Summarization via Deep Reinforcement Learning. In Proceedings of the ACM/IEEE International Conference on Automated Software Engineering (ASE), pages 397-407. · DOI
  47. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Man- dlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and An- ima Anandkumar. 2023a. Voyager: An Open-ended Embodied Agent with Large Language Models. In arXiv preprint arXiv:2305.16291. · arXiv
  48. Lei Wang, Jingsen Zhang, Hao Yang, Zhiyuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Rui- hua Song, Wayne Xin Zhao, Jun Xu, Zhicheng Dou, Jun Wang, and Ji-Rong Wen. 2023b. When Large Language Model based Agent Meets User Behavior Analysis: A Novel User Simulation Paradigm. In arXiv preprint arXiv:2306.02552. · arXiv
  49. Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023c. Large Language Models are not Fair Evaluators. In arXiv preprint arXiv:2305.17926. · arXiv
  50. Song Wang, Taiyue Liu, and Lin Tan. 2016. Automati- cally Learning Semantic Features for Defect Predic- tion. In Proceedings of the International Conference on Software Engineering (ICSE), pages 297-308. · DOI
  51. Song Wang, Nishtha Shrestha, Abarna Kucheri Subbu- raman, Junjie Wang, Moshi Wei, and Nachiappan Nagappan. 2021. Automatic Unit Test Generation for Machine Learning Libraries: How Far Are We? In IEEE/ACM International Conference on Software Engineering (ICSE), pages 1548-1560. · DOI
  52. Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Hao- tian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P. Xing, and Zhiting Hu. 2023d. PromptAgent: Strategic Planning with Language Models Enables Expert- level Prompt Optimization. In arXiv preprint arXiv:2310.16427. · arXiv
  53. Zhilin Wang, Yu Ying Chiu, and Yu Cheung Chiu. 2023e. Humanoid Agents: Platform for Simulating Human-like Generative Agents. In arXiv preprint arXiv:2310.05418. · arXiv
  54. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022a. Emergent Abilities of Large Language Mod- els. In arXiv preprint arXiv:2206.07682. · arXiv
  55. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022b. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 24824-24837.
  56. Jonas Winkler, Jannis Grönberg, and Andreas Vogel- sang. 2020. Predicting How to Test Requirements: An Automated Approach. In Software Engineering, volume P-300 of LNI, pages 141-142. · DOI
  57. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2023a. Large Language Models as Optimizers. In arXiv preprint arXiv:2309.03409. · arXiv
  58. Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. 2023b. GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction. In Advances in Neural Information Processing Systems (NeurIPS).
  59. Murat Yilmaz, Rory V O'Connor, and Paul Clarke. 2012. A Systematic Approach to the Comparison of Roles in the Software Development Processes. In Inter- national Conference on Software Process Improve- ment and Capability Determination, pages 198-209.
  60. Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. 2023. Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communi- cation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 15135-15153. · DOI · Paper page
  61. An Zhang, Leheng Sheng, Yuxin Chen, Hao Li, Yang Deng, Xiang Wang, and Tat-Seng Chua. 2023a. On Generative Agents in Recommendation. In arXiv preprint arXiv:2310.10108. · arXiv
  62. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023b. Siren's Song in the AI Ocean: A Survey on Hallucination in Large Lan- guage Models. In arXiv preprint arXiv:2309.01219. · arXiv
  63. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2023. ExpeL: LLM Agents Are Experiential Learners. In AAAI Conference on Artificial Intelligence (AAAI).
  64. Tianming Zhao, Chunyang Chen, Yuanning Liu, and Xiaodong Zhu. 2021. GUIGAN: Learning to Gener- ate GUI Designs Using Generative Adversarial Net- works. In IEEE/ACM International Conference on Software Engineering (ICSE), pages 748-760. · DOI
  65. Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, et al. 2023a. We- bArena: A Realistic Web Environment for Build- ing Autonomous Agents. In arXiv preprint arXiv:2307.13854. · arXiv
  66. Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. 2023b. Agents: An Open-source Framework for Autonomous Language Agents. In arXiv preprint arXiv:2309.07870. · arXiv

Cited by

Sorted by citations

Authors
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., Xu, J., Li, D., Liu, Z., & Sun, M.
Affiliations
The University of Sydney (AU)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2307.07924
arXiv
2307.07924
OpenAlex
W4384812233
Cited by
81
References
1
Citations by year
2026: 7 · 2025: 29 · 2024: 38 · 2023: 7
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Tencent
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Software Engineering Methodologies
Keywords
large language models, multi-agent systems, software development, multi-turn dialogue, software design, debugging, natural language programming, agent cooperation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review