CoChat

Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View

Zhang, Jintian, Xu, Xin, Zhang, Ningyu, Liu, Ruibo, Hooi, Bryan, Deng, Shumin · 2024 · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Cited by 53

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

As Natural Language Processing (NLP) systems are increasingly employed in intricate social environments, a pressing query emerges: Can these NLP systems mirror human-esque collaborative intelligence, in a multi-agent society consisting of multiple large language models (LLMs)? This paper probes the collaboration mechanisms among contemporary NLP systems by melding practical experiments with theoretical insights. We fabricate four unique `societies' comprised of LLM agents, where each agent is characterized by a specific `trait' (easy-going or overconfident) and engages in collaboration with a distinct `thinking pattern' (debate or reflection). Through evaluating these multi-agent societies on three benchmark datasets, we discern that certain collaborative strategies not only outshine previous top-tier approaches, but also optimize efficiency (using fewer API tokens). Moreover, our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity and consensus reaching, mirroring foundational social psychology theories. In conclusion, we integrate insights from social psychology to contextualize the collaboration of LLM agents, inspiring further investigations into the collaboration mechanism for LLMs. We commit to sharing our code and datasets\footnote{\url{https://github.com/zjunlp/MachineSoM}.}, hoping to catalyze further research in this promising avenue.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2024.acl-long.782

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Research works printed but not linked: 82

  1. Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, and Eric Schulz. 2023. Playing repeated games with large language models. CoRR, abs/2305.16867. · DOI · Paper page
  2. Lize Alberts, Geoff Keeling, and Amanda McCroskery. 2024. What makes for a 'good' social actor? using re- spect as a lens to evaluate interactions with language agents. CoRR, abs/2401.09082. · arXiv
  3. Vernon L. Allen and John M. Levine. 1969. Consen- sus and conformity. Journal of Experimental Social Psychology, 5(4):389-399. · DOI
  4. Leila Amgoud and Henri Prade. 2009. Using arguments for making and explaining decisions. Artif. Intell., 173(3-4):413-436. · DOI · Paper page
  5. Thilo Hagendorff. 2023. Machine psychology: Investi- gating emergent capabilities and behavior in large lan- guage models using psychological methods. CoRR, abs/2303.13988. · DOI · Paper page
  6. Shanshan Han, Qifan Zhang, Yuhang Yao, Weizhao Jin, Zhaozhuo Xu, and Chaoyang He. 2024. Llm multi-agent systems: Challenges and open problems. CoRR, abs/2402.03578. · arXiv
  7. Rui Hao, Linmei Hu, Weijian Qi, Qingliu Wu, Yirui Zhang, and Liqiang Nie. 2023. Chatllm net- work: More brains, more intelligence. CoRR, abs/2304.12998. · DOI
  8. Zhitao He, Pengfei Cao, Yubo Chen, Kang Liu, Ruopeng Li, Mengshu Sun, and Jun Zhao. 2023. LEGO: A multi-agent collaborative framework with role-playing and iterative feedback for causality ex- planation generation. In EMNLP (Findings), pages 9142-9163. Association for Computational Linguis- tics.
  9. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021a. Measuring massive multitask language under- standing. In ICLR. OpenReview.net.
  10. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. Measuring mathematical problem solving with the MATH dataset. In NeurIPS Datasets and Benchmarks.
  11. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2024. Metagpt: Meta pro- gramming for multi-agent collaborative framework. In ICLR. OpenReview.net.
  12. Yaqing Hou, Mingyang Sun, Yifeng Zeng, Yew-Soon Ong, Yaochu Jin, Hongwei Ge, and Qiang Zhang. 2023. A multi-agent cooperative learning system with evolution of social roles. IEEE Transactions on Evolutionary Computation, pages 1-1. · DOI
  13. Karen A Jehn. 1995. A multimethod examination of the benefits and detriments of intragroup conflict. Administrative science quarterly, pages 256-282. · DOI · Paper page
  14. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput. Surv., 55(12):248:1-248:38. · DOI · Paper page
  15. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Re- nard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timo- thée Lacroix, and William El Sayed. 2023. Mistral 7b. CoRR, abs/2310.06825. · DOI · Paper page
  16. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gi- anna Lengyel, Guillaume Bour, Guillaume Lam- ple, Lélio Renard Lavaud, Lucile Saulnier, Marie- Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024. Mix- tral of experts. CoRR, abs/2401.04088. · arXiv
  17. David W Johnson and Roger T Johnson. 2009. An edu- cational psychology success story: Social interdepen- dence theory and cooperative learning. Educational researcher, 38(5):365-379.
  18. Luoma Ke, Song Tong, Peng Chen, and Kaiping Peng. 2024. Exploring the frontiers of llms in psycholog- ical applications: A comprehensive review. CoRR, abs/2401.01519. · arXiv
  19. Chuyi Kong, Yaxin Fan, Xiang Wan, Feng Jiang, and Benyou Wang. 2023. Large language model as a user simulator. CoRR, abs/2308.11534. · arXiv · doi:10.48550/arxiv.2308.11534
  20. Tatsuki Kuribayashi, Yohei Oseki, and Timothy Bald- win. 2024. Psychometric predictive power of large language models. In NAACL (Findings). Association for Computational Linguistics. · arXiv · doi:10.48550/arxiv.2311.07484
  21. Marc Lanctot, Kate Larson, Yoram Bachrach, Luke Marris, Zun Li, Avishkar Bhoopchand, Thomas W. Anthony, Brian Tanner, and Anna Koop. 2023. Eval- uating agents using social choice theory. CoRR, abs/2312.03121. · DOI
  22. Alan M Leslie, Ori Friedman, and Tim P German. 2004. Core mechanisms in 'theory of mind'. Trends in cognitive sciences, 8(12):528-533. · DOI · Paper page
  23. Yuan Li, Yixuan Zhang, and Lichao Sun. 2023b. Metaa- gents: Simulating interactions of human behaviors for llm-based task-oriented coordination via collabo- rative generative agents. CoRR, abs/2310.06500. · DOI · Paper page
  24. Yujia Li, David H. Choi, Junyoung Chung, Nate Kush- man, Julian Schrittwieser, Rémi Leblond, Tom Ec- cles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Mas- son d'Autume, Igor Babuschkin, Xinyun Chen, Po- Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. 2022. Competition-level code generation with alpha- code. Science, 378(6624):1092-1097. · DOI · Paper page
  25. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023. Encouraging divergent thinking in large language models through multi-agent debate. CoRR, abs/2305.19118. · DOI · Paper page
  26. Yuanzhi Liang, Linchao Zhu, and Yi Yang. 2024. An- teval: Quantitatively evaluating informativeness and expressiveness of agent social interactions. CoRR, abs/2401.06509. · arXiv
  27. Ruibo Liu, Ruixin Yang, Chenyan Jia, Ge Zhang, Denny Zhou, Andrew M. Dai, Diyi Yang, and Soroush Vosoughi. 2024. Training socially aligned language models in simulated human society. In ICLR. Open- Review.net.
  28. Ruibo Liu, Ge Zhang, Xinyu Feng, and Soroush Vosoughi. 2022. Aligning generative language mod- els with human values. In NAACL-HLT (Findings), pages 241-252. Association for Computational Lin- guistics. · DOI
  29. Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. 2023. Dynamic llm-agent network: An llm- agent collaboration framework with agent team opti- mization. CoRR, abs/2310.02170. · DOI
  30. Jan Lorenz, Martin Neumann, and Tobias Schröder. 2021. Individual attitude change and societal dynam- ics: Computational experiments with psychological theories. Psychological Review, 128(4):623. · DOI · Paper page
  31. Junliang Luo, Tianyu Li, Di Wu, Michael Jenkin, Steve Liu, and Gregory Dudek. 2024. Hallucination detec- tion and hallucination mitigation: An investigation. CoRR, abs/2401.08358. · arXiv
  32. Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. 2024a. Agentboard: An an- alytical evaluation board of multi-turn llm agents. CoRR, abs/2401.13178. · arXiv
  33. Qun Ma, Xiao Xue, Deyu Zhou, Xiangning Yu, Donghua Liu, Xuwen Zhang, Zihan Zhao, Yifan Shen, Peilin Ji, Juanjuan Li, Gang Wang, and Wan- peng Ma. 2024b. Computational experiments meet large language model based agents: A survey and perspective. CoRR, abs/2402.00262. · arXiv
  34. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdan- bakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. In NeurIPS.
  35. Qiaozhu Mei, Yutong Xie, Walter Yuan, and Matthew O. Jackson. 2024. A turing test of whether ai chat- bots are behaviorally similar to humans. Pro- ceedings of the National Academy of Sciences, 121(9):e2313925121. · DOI · Paper page
  36. Jack Mezirow. 2018. Transformative learning theory. In Contemporary theories of learning, pages 114-128. Routledge.
  37. Don A Moore and Paul J Healy. 2008. The trouble with overconfidence. Psychological review, 115(2):502.
  38. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welin- der, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instruc- tions with human feedback. In NeurIPS.
  39. Joon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive sim- ulacra of human behavior. In UIST, pages 2:1-2:22. ACM. · DOI · Paper page
  40. Chaim Perelman. 1971. The new rhetoric. Springer. · DOI · Paper page
  41. Pouya Pezeshkpour, Eser Kandogan, Nikita Bhutani, Sajjadur Rahman, Tom Mitchell, and Estevam Hr- uschka. 2024. Reasoning capacity in multi-agent sys- tems: Limitations, challenges and human-centered solutions. CoRR, abs/2402.01108. · arXiv
  42. Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Chengfei Lv, and Huajun Chen. 2024. Autoact: Automatic agent learning from scratch via self-planning. In ACL. Association for Computational Linguistics.
  43. Vipula Rawte, Amit P. Sheth, and Amitava Das. 2023. A survey of hallucination in large foundation models. CoRR, abs/2309.05922. · DOI · Paper page
  44. Thomas J Scheff. 1967. Toward a sociological model of consensus. American Sociological Review, pages 32-46. · DOI · Paper page
  45. David Wyatt Seal, Laura M Bogart, and Anke A Ehrhardt. 1998. Small group dynamics: The util- ity of focus group discussions as a research method. Group Dynamics: Theory, Research, and Practice, 2(4):253. · DOI
  46. Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models. Nat., 623(7987):493-498. · DOI · Paper page
  47. Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2024. Clever hans or neural theory of mind? stress testing social reasoning in large language models. In EACL (1), pages 2257- 2273. Association for Computational Linguistics.
  48. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. 2024. Towards understanding syco- phancy in language models. In ICLR. OpenRe- view.net.
  49. Noah Shinn, Beck Labash, and Ashwin Gopinath. 2023. Reflexion: an autonomous agent with dy- namic memory and self-reflection. arXiv preprint, abs/2303.11366. · DOI · Paper page
  50. Michael Siegal and Rosemary Varley. 2002. Neural sys- tems involved in 'theory of mind'. Nature Reviews Neuroscience, 3(6):463-471. · DOI · Paper page
  51. Push Singh. 2003. Examining the society of mind. Com- put. Artif. Intell., 22(6):521-543.
  52. Paul Smaldino. 2023. Modeling social behavior: Math- ematical and agent-based models of social dynamics and cultural evolution. Princeton University Press.
  53. Nikita Soni, Niranjan Balasubramanian, H. Andrew Schwartz, and Dirk Hovy. 2024. Comparing human-centered language modeling: Is it better to model groups, individual traits, or both? CoRR, abs/2401.12492. · arXiv
  54. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Par- rish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Santilli, Andreas Stuhlmüller, Andrew M. Dai, Andrew La, Andrew K. Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Anto- nio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Her- rick, Avia Efrat, Aykut Erdem, Ayla Karakas, and et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint, abs/2206.04615. · DOI · Paper page
  55. Peter Stone and Manuela M. Veloso. 2000. Multiagent systems: A survey from a machine learning perspec- tive. Auton. Robots, 8(3):345-383. · DOI · Paper page
  56. Cass R Sunstein. 2005. Why societies need dissent. Harvard University Press. · doi:10.4159/9780674267657 · Paper page
  57. Henri Tajfel. 1982. Social psychology of intergroup relations. Annual review of psychology, 33(1):1-39. · DOI · Paper page
  58. Henri Tajfel and John C Turner. 2004. The social iden- tity theory of intergroup behavior. In Political psy- chology, pages 276-293. Psychology Press.
  59. Yashar Talebirad and Amirhossein Nadiri. 2023. Multi- agent collaboration: Harnessing the power of intelli- gent LLM agents. CoRR, abs/2306.03314. · DOI · Paper page
  60. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. CoRR, abs/2302.13971. · DOI · Paper page
  61. Cheng Wang, Chuwen Wang, Yu Zhao, Shirong Zeng, Wang Zhang, and Ronghui Ning. 2024a. Behavioral simulation: Exploring a possible next paradigm for science. CoRR, abs/2401.09851. · arXiv
  62. Haonan Wang, James Zou, Michael Mozer, Anirudh Goyal, Alex Lamb, Linjun Zhang, Weijie J Su, Zhun Deng, Michael Qizhe Xie, Hannah Brown, and Kenji Kawaguchi. 2024b. Can ai be as creative as humans? CoRR, abs/2401.01623. · arXiv
  63. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. 2024c. A survey on large language model based autonomous agents. Front. Comput. Sci., 18. · DOI · Paper page
  64. Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, Qingqing Zhu, Zhenzhu Yang, Adam Nik, Qi Liu, Chenghua Lin, Shi Wang, Ruibo Liu, Wenhu Chen, Ke Xu, Dayiheng Liu, Yike Guo, and Jie Fu. 2023. Interactive natural language processing. CoRR, abs/2305.13246. · DOI
  65. Gerhard Weiß. 1995. Adaptation and learning in multi- agent systems: Some remarks and a bibliography. In Adaption and Learning in Multi-Agent Systems, volume 1042 of Lecture Notes in Computer Science, pages 1-21. Springer. · DOI
  66. Michael J. Wooldridge. 2009. An Introduction to Multi- Agent Systems, Second Edition. Wiley.
  67. Anita Williams Woolley, Christopher F. Chabris, Alex Pentland, Nada Hashmi, and Thomas W. Malone. 2010. Evidence for a collective intelligence fac- tor in the performance of human groups. Science, 330(6004):686-688. · DOI · Paper page
  68. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yi- wen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiao- ran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xi- angyang Liu, Zhangyue Yin, Shihan Dou, Rongxi- ang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huan, and Tao Gui. 2023. The rise and potential of large lan- guage model based agents: A survey. arxiv preprint, abs/2309.07864. · DOI · Paper page
  69. Yang Xiao, Yi Cheng, Jinlan Fu, Jiashuo Wang, Wenjie Li, and Pengfei Liu. 2023. How far are we from believable ai agents? a framework for evaluating the believability of human behavior simulation. CoRR, abs/2312.17115. · arXiv
  70. Chengxing Xie, Canyu Chen, Feiran Jia, Ziyu Ye, Kai Shu, Adel Bibi, Ziniu Hu, Philip H. S. Torr, Bernard Ghanem, and Guohao Li. 2024. Can large language model agents simulate human trust behaviors? CoRR, abs/2402.04559. · DOI
  71. Tianbao Xie, Fan Zhou, Zhoujun Cheng, Peng Shi, Lu- oxuan Weng, Yitao Liu, Toh Jing Hua, Junning Zhao, Qian Liu, Che Liu, Leo Z. Liu, Yiheng Xu, Hongjin Su, Dongchan Shin, Caiming Xiong, and Tao Yu. 2023. Openagents: An open platform for language agents in the wild. CoRR, abs/2310.10634. · DOI · Paper page
  72. Lin Xu, Zhiyuan Hu, Daquan Zhou, Hongyu Ren, Zhen Dong, Kurt Keutzer, See-Kiong Ng, and Jiashi Feng. 2023. Magic: Investigation of large language model powered multi-agent in cognition, adaptability, ratio- nality and collaboration. CoRR, abs/2311.08562. · DOI
  73. Ruoxi Xu, Yingfei Sun, Mengjie Ren, Shiguang Guo, Ruotong Pan, Hongyu Lin, Le Sun, and Xianpei Han. 2024. Ai for social science and social science of ai: A survey. Information Processing & Management, 61(3):103665. · DOI
  74. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. A survey on multimodal large language models. arXiv preprint, abs/2306.13549. · DOI
  75. Ceyao Zhang, Kaijie Yang, Siyi Hu, Zihao Wang, Guanghe Li, Yihang Sun, Cheng Zhang, Zhaowei Zhang, Anji Liu, Song-Chun Zhu, Xiaojun Chang, Junge Zhang, Feng Yin, Yitao Liang, and Yaodong Yang. 2024. Proagent: Building proactive cooper- ative agents with large language models. In AAAI, pages 17591-17599. AAAI Press. · DOI · Paper page
  76. Xijia Zhang, Yue Guo, Simon Stepputtis, Katia P. Sycara, and Joseph Campbell. 2023a. Explaining agent behavior with large language models. CoRR, abs/2309.10346. · DOI
  77. Xijia Zhang, Yue Guo, Simon Stepputtis, Katia P. Sycara, and Joseph Campbell. 2023b. Understanding your agent: Leveraging large language models for behavior explanation. CoRR, abs/2311.18062. · DOI
  78. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023c. Siren's song in the AI ocean: A survey on hallucination in large language models. CoRR, abs/2309.01219. · DOI · Paper page
  79. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models. arXiv preprint, abs/2303.18223. · DOI · Paper page
  80. Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. 2023. Agents: An open-source framework for autonomous language agents. CoRR, abs/2309.07870. · DOI · Paper page
  81. Yuqi Zhu, Xiaohan Wang, Jing Chen, Shuofei Qiao, Yixin Ou, Yunzhi Yao, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities. CoRR, abs/2305.13168. · DOI
  82. Mingchen Zhuge, Haozhe Liu, Francesco Faccio, Dy- lan R. Ashley, Róbert Csordás, Anand Gopalakrish- nan, Abdullah Hamdi, Hasan Abed Al Kader Ham- moud, Vincent Herrmann, Kazuki Irie, Louis Kirsch, Bing Li, Guohao Li, Shuming Liu, Jinjie Mai, Pi- otr Piekos, Aditya Ramesh, Imanol Schlag, Weimin Shi, Aleksandar Stanic, Wenyi Wang, Yuhui Wang, Mengmeng Xu, Deng-Ping Fan, Bernard Ghanem, and Jürgen Schmidhuber. 2023. Mindstorms in natural language-based societies of mind. CoRR, abs/2305.17066. · DOI · Paper page

Cited by

See all 53

Metadata

Authors
Zhang, J., Xu, X., Zhang, N., Liu, R., Hooi, B., & Deng, S.
Venue
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Published
2024
Pages
14544-14607
Cited by
53
References
3

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View. https://app.cochat.ai/papers/10.18653/v1/2024.acl-long.782
The paper
Zhang, J., Xu, X., Zhang, N., Liu, R., Hooi, B., & Deng, S. (2024). Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v1/2024.acl-long.782

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.