CoChat
Sign Up
PreprintCited by 7Multi-Agent Systems and NegotiationTopic ModelingNatural Language Processing Techniques

Creativity in LLM-based Multi-Agent Systems: A Survey

Open access
Lin, Yi-Cheng, Chen, Kang-Chieh, Li, Zhe-Yan, Wu, Tzu-Heng, Wu, Tzu-Hsuan, Chen, Kuan-Yu, Lee, Hung-yi, Chen, Yun-Nung·2025·arXiv
DOI 10.48550/arxiv.2505.21116arXiv 2505.21116

Preprint — published as 10.18653/v1/2025.emnlp-main.1403

Open this paper in CoChat — free accountOpen PDF

Abstract

Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts. While existing surveys provide comprehensive overviews of MAS infrastructures, they largely overlook the dimension of \emph{creativity}, including how novel outputs are generated and evaluated, how creativity informs agent personas, and how creative workflows are coordinated. This is the first survey dedicated to creativity in MAS. We focus on text and image generation tasks, and present: (1) a taxonomy of agent proactivity and persona design; (2) an overview of generation techniques, including divergent exploration, iterative refinement, and collaborative synthesis, as well as relevant datasets and evaluation metrics; and (3) a discussion of key challenges, such as inconsistent evaluation standards, insufficient bias mitigation, coordination conflicts, and the lack of unified benchmarks. This survey offers a structured framework and roadmap for advancing the development, evaluation, and standardization of creative MAS.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2505.21116

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The key insight is that when multiple agents collaborate through workflows, the system's overall performance can exceed the sum of the individual capabilities of all agents within the system.

    “The key insight is that when multiple agents collaborate through such workflows, the system's overall performance can exceed the sum of the individual capabilities of all agents within the system (Lin et al., 2025; Luo et al., 2025a).”

    From A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems (2025) · cites this paper as Lin et al., 2025

    Section 3.3, “Collaborative Synthesis” (page unavailable), says agents integrate diverse perspectives through structured workflows and that this benefits creative tasks. It does not compare system performance with the sum of the agents’ individual capabilities, so the claimed superadditive effect is unsupported by this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 133

  1. Safinah Ali, Ayat Abodayeh, Zahra Dhuliawala, Cyn- thia Breazeal, and Hae Won Park. 2025. Towards Inclusive Co-creative Child-robot Interaction: Can Social Robots Support Neurodivergent Children's Creativity? In Proceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interac- tion, HRI '25. · DOI
  2. Alfonso Amayuelas, Jingbo Yang, Saaket Agashe, Ashwin Nagarajan, Antonis Antoniades, Xin Eric Wang, and William Wang. 2025. Self-resource al- location in multi-agent LLM systems. Preprint, arXiv:2504.02051. · arXiv
  3. Zhangir Azerbayev, Bartosz Piotrowski, Hailey Schoelkopf, Edward W. Ayers, Dragomir Radev, and Jeremy Avigad. 2023. ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathe- matics. Preprint, arXiv:2302.12433. · arXiv
  4. Stephanie P. Besemer and Donald J. Treffinger. 1981. Analysis of creative products: Review and synthesis. Journal of Creative Behavior, 15(3):158-178. · DOI
  5. Margaret A Boden. 2004. The creative mind: Myths and mechanisms. Psychology Press.
  6. Angana Borah and Rada Mihalcea. 2024. Towards im- plicit bias detection and mitigation in multi-agent LLM interactions. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9306-9326. Association for Computational Linguis- tics. · DOI · Paper page
  7. Tom B. Brown et al. 2020. Language Models are Few- Shot Learners. Preprint, arXiv:2005.14165. · arXiv
  8. Erika K Carlson. 2020. Artificial intelligence can invent but not patent-for now. Engineering, 6(11):1212- 1213.
  9. Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025. Why Do Multi-Agent LLM Systems Fail? Preprint, arXiv:2503.13657. · arXiv
  10. Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brah- man, and Smaranda Muresan. 2024. Creativity Sup- port in the Age of Large Language Models: An Em- pirical Study Involving Emerging Writers. Preprint, arXiv:2309.12570. · arXiv
  11. Jing Chen, Xinyu Zhu, Cheng Yang, Chufan Shi, Yadong Xi, Yuxiang Zhang, Junjie Wang, Jiashu Pu, Rongsheng Zhang, Yujiu Yang, and Tian Feng. 2024. HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing. CoRR.
  12. Erin Cherry and Celine Latulipe. 2014. Quantifying the Creativity Support of Digital Tools through the Creativity Support Index. ACM Trans. Comput.-Hum. Interact., 21(4). · DOI · Paper page
  13. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anasta- sios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. Chatbot Arena: An Open Platform for Evaluating LLMs by Human Pref- erence. Preprint, arXiv:2403.04132. · arXiv
  14. Niall Creech, Natalia Criado Pacheco, and Simon Miles. 2021. Resource allocation in dynamic multiagent systems. Preprint, arXiv:2102.08317. · arXiv
  15. Mike D'Arcy, Tom Hope, Larry Birnbaum, and Doug Downey. 2024. Marg: Multi-agent review generation for scientific papers. Preprint, arXiv:2401.04259. · arXiv
  16. Allegra De Filippo, Michela Milano, et al. 2024. Large language models for human-AI co-creation of robotic dance performances. In IJCAI, pages 7627-7635.
  17. DeepSeek-AI. 2024. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. · arXiv
  18. John M Digman. 1990. Personality structure: Emer- gence of the five-factor model. Annual review of psychology. · DOI
  19. Shiying Ding, Xinyi Chen, Yan Fang, Wenrui Liu, Yiwu Qiu, and Chunlei Chai. 2023. DesignGPT: Multi- Agent Collaboration in Design. In 2023 16th Inter- national Symposium on Computational Intelligence and Design (ISCID), pages 204-208. · DOI
  20. Kangkang Duan and Zhengbo Zou. 2025. Enhancing construction robot collaboration via multiagent rein- forcement learning. Journal of Intelligent Construc- tion, 3(2):1-16.
  21. Yifan Duan, Yihong Tang, Xuefeng Bai, Kehai Chen, Juntao Li, and Min Zhang. 2025. The Power of Personality: A Human Simulation Perspective to In- vestigate Large Language Model Agents. Preprint, arXiv:2502.20859. · arXiv
  22. Xianzhe Fan, Zihan Wu, Chun Yu, Fenggui Rao, Weinan Shi, and Teng Tu. 2024. ContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation. In Proceedings of the 2024 CHI Con- ference on Human Factors in Computing Systems, CHI '24. Association for Computing Machinery. · DOI · Paper page
  23. Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. 2024. Large language models empowered agent-based mod- eling and simulation: A survey and perspectives. Humanities and Social Sciences Communications, 11(1):1-24.
  24. Shijun Ge, Yuanbo Sun, Yin Cui, and Dapeng Wei. 2025. An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Inte- grate LLM-Based Agents With Concept Generation Methods. IEEE Access, 13:10499-10512. · DOI
  25. Juraj Gottweis et al. 2025. Towards an AI co-scientist. Preprint, arXiv:2502.18864. · arXiv
  26. Aaron Grattafiori et al. 2024. The llama 3 herd of mod- els. Preprint, arXiv:2407.21783. · arXiv
  27. J. P. Guilford. 1950. Creativity. American Psychologist, 5(9):444-454. · DOI · Paper page
  28. J. P. Guilford. 1967. The Nature of Human Intelligence, chapter 8. McGraw-Hill, New York.
  29. Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xi- angliang Zhang. 2024a. Large Language Model Based Multi-agents: A Survey of Progress and Chal- lenges. In Proceedings of the Thirty-Third Inter- national Joint Conference on Artificial Intelligence, IJCAI-24, pages 8048-8057. International Joint Con- ferences on Artificial Intelligence Organization. Sur- vey Track. · DOI · Paper page
  30. Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xi- angliang Zhang. 2024b. Large language model based multi-agents: A survey of progress and challenges. Preprint, arXiv:2402.01680. · arXiv
  31. Shashank Gupta, Vaishnavi Shrivastava, Ameet Desh- pande, Ashwin Kalyan, Peter Clark, Ashish Sabhar- wal, and Tushar Khot. 2024. Bias runs deep: Implicit reasoning biases in persona-assigned LLMs. In The Twelfth International Conference on Learning Repre- sentations.
  32. Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Tak Yeon Lee, So-Yeon Ahn, and Alice Oh. 2024a. RECIPE4U: Student-ChatGPT interaction dataset in EFL writing education. In Proceedings of the 2024 Joint International Conference on Computa- tional Linguistics, Language Resources and Evalu- ation (LREC-COLING 2024), pages 13666-13676, Torino, Italia. ELRA and ICCL.
  33. Shanshan Han, Qifan Zhang, Yuhang Yao, Weizhao Jin, Zhaozhuo Xu, and Chaoyang He. 2024b. LLM multi-agent systems: Challenges and open problems. Preprint, arXiv:2402.03578. · arXiv
  34. Jessica He, Stephanie Houde, Gabriel E. Gonzalez, Darío Andrés Silva Moran, Steven I. Ross, Michael Muller, and Justin D. Weisz. 2024. Ai and the future of collaborative work: Group ideation with an llm in a virtual canvas. In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer In- teraction for Work, CHIWORK '24, New York, NY, USA. Association for Computing Machinery. · DOI
  35. Jie He, Tao Wang, Deyi Xiong, and Qun Liu. 2020. The box is in the pen: Evaluating commonsense rea- soning in neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3662-3672, Online. Association for Computational Linguistics. · DOI
  36. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2018. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Preprint, arXiv:1706.08500. · arXiv
  37. Yihan Hou, Manling Yang, Hao Cui, Lei Wang, Jie Xu, and Wei Zeng. 2024. C2ideas: Supporting creative interior color design ideation with a large language model. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI '24. · DOI
  38. Stephanie Houde, Kristina Brimijoin, Michael Muller, Steven I. Ross, Dario Andres Silva Moran, Gabriel Enrique Gonzalez, Siya Kunde, Morgan A. Foreman, and Justin D. Weisz. 2025. Controlling ai agent participation in group conversations: A human- centered approach. In Proceedings of the 30th Inter- national Conference on Intelligent User Interfaces, IUI '25, page 390-408, New York, NY, USA. Asso- ciation for Computing Machinery. · DOI
  39. Tiancheng Hu and Nigel Collier. 2024. Quantifying the persona effect in llm simulations. Preprint, arXiv:2402.10811. · arXiv
  40. Kaiyi Huang, Yukun Huang, Xuefei Ning, Zinan Lin, Yu Wang, and Xihui Liu. 2024. Genmac: Composi- tional text-to-video generation with multi-agent col- laboration. Preprint, arXiv:2412.04440. · arXiv
  41. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Day- iheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. · arXiv
  42. Fantine Huot, Reinald Kim Amplayo, Jennimaria Palo- maki, Alice Shoshana Jakobovits, Elizabeth Clark, and Mirella Lapata. 2025. Agents' room: Narra- tive generation through multi-step collaboration. In The Thirteenth International Conference on Learning Representations.
  43. Thorsten Händler. 2023. Balancing autonomy and alignment: A multi-dimensional taxonomy for au- tonomous LLM-powered multi-agent architectures. Preprint, arXiv:2310.03659. · arXiv
  44. Francisco Ibarrola, Tomas Lawton, and Kazjon Grace. 2024. A Collaborative, Interactive and Context- Aware Drawing Agent for Co-Creative Design. IEEE Transactions on Visualization and Computer Graph- ics. · DOI
  45. Naomi Imasato, Kazuki Miyazawa, Takayuki Nagai, and Takato Horii. 2024. Creative agents: Simulat- ing the systems model of creativity with generative agents. Preprint, arXiv:2411.17065. · arXiv
  46. Masaki Ishizaka, Akihito Taya, and Yoshito Tobe. 2024. Sparkit: A mind map-based mas for idea generation support. In Engineering Multi-Agent Systems, pages 1-22, Cham. Springer Nature Switzerland.
  47. Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2024. Per- sonaLLM: Investigating the Ability of Large Lan- guage Models to Express Personality Traits. Preprint, arXiv:2305.02547. · arXiv
  48. Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kai- jie Zhu, Yijia Xiao, and Jindong Wang. 2024. Agen- tReview: Exploring peer review dynamics with LLM agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1208-1226, Miami, Florida, USA. Association for Computational Linguistics. · DOI · Paper page
  49. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A Large Scale Dis- tantly Supervised Challenge Dataset for Reading Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1601-1611. Association for Computational Linguistics.
  50. Huang Kai, Zhang Zhonghua, Qin Zheng, and Liu Bing. 2010. Conflict resolution in multi-agent systems based on negotiation and arbitrage. In 2010 2nd IEEE International Conference on Information Man- agement and Engineering, pages 304-307. · DOI
  51. Junseok Kim, Nakyeong Yang, and Kyomin Jung. 2024. Persona is a double-edged sword: Mitigating the negative impact of role-playing prompts in zero-shot reasoning tasks. Preprint, arXiv:2408.08631. · arXiv
  52. Chun-Yi Kuan, Chih-Kai Yang, Wei-Ping Huang, Ke- Han Lu, and Hung yi Lee. 2024. Speech-copilot: Leveraging large language models for speech pro- cessing via task decomposition, modularization, and program generation. Preprint, arXiv:2407.09886. · arXiv
  53. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic Uncertainty: Linguistic Invariances for Un- certainty Estimation in Natural Language Generation. Preprint, arXiv:2302.09664. · arXiv
  54. Harsh Kumar, Jonathan Vincentius, Ewan Jordan, and Ashton Anderson. 2025. Human Creativity in the Age of LLMs: Randomized Experiments on Diver- gent and Convergent Thinking. In International Con- ference on Human Factors in Computing Systems. · DOI · Paper page
  55. Mohammad Lataifeh, Xavier A Carrasco, Ashraf M Elnagar, Naveed Ahmed, and Imran Junejo. 2024. Human-machine co-creation: a complementary cog- nitive approach to creative character design pro- cess using GANs. The Journal of Supercomputing, 80(11):16574-16610. · DOI
  56. Tomas Lawton, Francisco J Ibarrola, Dan Ventura, and Kazjon Grace. 2023. Drawing with Reframer: Emer- gence and Control in Co-Creative AI. In Proceedings of the 28th International Conference on Intelligent User Interfaces, IUI '23. · DOI
  57. John D. Lee and Katrina A. See. 2004. Trust in automa- tion: Designing for appropriate reliance. Human Factors, 46(1):50-80. · DOI · Paper page
  58. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting ob- jective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 110-119, San Diego, California. Association for Computational Linguistics. · DOI
  59. Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent sys- tems: workflow, infrastructure, and challenges. Vici- nagearth, 1(1):9. · DOI · Paper page
  60. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.
  61. Gionnieve Lim and Simon T. Perrault. 2024. Rapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language Models. arXiv preprint arXiv:2403.12928. · arXiv
  62. Yi-Cheng Lin, Tzu-Quan Lin, Chih-Kai Yang, Ke-Han Lu, Wei-Chih Chen, Chun-Yi Kuan, and Hung-Yi Lee. 2024a. Listen and speak fairly: a study on semantic gender bias in speech integrated large lan- guage models. In 2024 IEEE Spoken Language Tech- nology Workshop (SLT), pages 439-446. · DOI
  63. Zheng Lin, Zhenxing Niu, Zhibin Wang, and Yinghui Xu. 2024b. Interpreting and mitigating hallucina- tion in mllms through multi-agent debate. Preprint, arXiv:2407.20505. · arXiv
  64. Zhiyu Lin, Rohan Agarwal, and Mark Riedl. 2022. Cre- ative wand: a system to study effects of communi- cations in co-creative settings. In Proceedings of the Eighteenth AAAI Conference on Artificial Intel- ligence and Interactive Digital Entertainment, AI- IDE'22. AAAI Press. · DOI
  65. Andy Liu, Mona Diab, and Daniel Fried. 2024a. Evalu- ating large language model biases in persona-steered generation. In Findings of the Association for Com- putational Linguistics: ACL 2024, pages 9832-9850, Bangkok, Thailand. Association for Computational Linguistics. · DOI
  66. Yiren Liu, Si Chen, and Haocong et al. Cheng. 2024b. How AI Processing Delays Foster Creativity: Explor- ing Research Question Co-Creation with an LLM- based Agent. In Proc. CHI '24. · DOI · Paper page
  67. Yiren Liu, Pranav Sharma, Mehul Jitendra Oswal, Hai- jun Xia, and Yun Huang. 2024c. PersonaFlow: Boosting Research Ideation with LLM-Simulated Ex- pert Personas. Preprint, arXiv:2409.12538. · arXiv
  68. Tao Long, Katy Ilonka Gero, and Lydia B Chilton. 2024. Not just novelty: A longitudinal study on utility and customization of an ai workflow. In Proceedings of the 2024 ACM Designing Interactive Systems Confer- ence, DIS '24, page 782-803, New York, NY, USA. Association for Computing Machinery. · DOI
  69. Li-Chun Lu, Shou-Jen Chen, Tsung-Min Pai, Chan- Hung Yu, Hung yi Lee, and Shao-Hua Sun. 2024. Llm discussion: Enhancing the creativity of large language models via discussion framework and role- play. Preprint, arXiv:2405.06373. · arXiv
  70. Alaeddine Mellouli et al. 2024. Storeys: A neurosym- bolic approach to human-ai co-creation of novel action-oriented narratives in known story worlds. In Proceedings of the 15th International Conference on Computational Creativity, ICCC 2024, Jönköping, Sweden, June 17-21, 2024.
  71. Chunjiang Mu, Hao Guo, Yang Chen, Chen Shen, Die Hu, Shuyue Hu, and Zhen Wang. 2024. Multi-agent, human-agent and beyond: A survey on cooperation in social dilemmas. Neurocomputing, 610:128514. · DOI
  72. Anirban Mukherjee and Hannah Hanwen Chang. 2025. Stochastic, dynamic, fluid autonomy in agentic ai: Implications for authorship, inventorship, and liabil- ity. Preprint, arXiv:2504.04058. · arXiv
  73. Mohammad Naiseh, Nan Jiang, Jianbing Ma, and Raian Ali. 2020. Personalising explainable recommenda- tions: Literature and conceptualisation. In Trends and Innovations in Information Systems and Tech- nologies, pages 518-533, Cham. Springer Interna- tional Publishing.
  74. Harshit Nigam, Manasi Patwardhan, Lovekesh Vig, and Gautam Shroff. 2024. An interactive co-pilot for ac- celerated research ideation. In Proceedings of the Third Workshop on Bridging Human-Computer In- teraction and Natural Language Processing, pages 60-73, Mexico City, Mexico. Association for Com- putational Linguistics. · DOI
  75. Bo Pan, Jiaying Lu, Ke Wang, Li Zheng, Zhen Wen, Yingchaojie Feng, Minfeng Zhu, and Wei Chen. 2024. Agentcoord: Visually exploring coordination strategy for llm-based multi-agent collaboration. Preprint, arXiv:2404.11943. · arXiv
  76. Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Mered- ith Ringel Morris, Percy Liang, and Michael S. Bern- stein. 2023. Generative Agents: Interactive Simu- lacra of Human Behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST '23, New York, NY, USA. Association for Computing Machinery. · DOI · Paper page
  77. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ChatDev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15174-15186, Bangkok, Thailand. Association for Computational Linguistics. · DOI · Paper page
  78. Hua Xuan Qin, Shan Jin, Ze Gao, Mingming Fan, and Pan Hui. 2024. CharacterMeet: Supporting Creative Writers' Entire Story Character Construction Pro- cesses Through Conversation with LLM-Powered Chatbot Avatars. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI '24. Association for Computing Machinery.
  79. Yansheng Qiu, Haoquan Zhang, Zhaopan Xu, Ming Li, Diping Song, Zheng Wang, and Kaipeng Zhang. 2025. AI Idea Bench 2025: AI Research Idea Gener- ation Benchmark. Preprint, arXiv:2504.14191. · arXiv
  80. Marissa Radensky. 2024. Mixed-Initiative Methods for Co-Creation in Scientific Research. In Proceedings of the 16th Conference on Creativity & Cognition, C&C '24, page 1-7. Association for Computing Ma- chinery. · DOI
  81. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. Preprint, arXiv:1908.10084. · arXiv
  82. David Rein, Betty Li Hou, Asa Cooper Stickland, Jack- son Petty, Richard Yuanzhe Pang, Julien Dirani, Ju- lian Michael, and Samuel R. Bowman. 2024. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  83. Kai Ruan, Xuan Wang, Jixiang Hong, Peng Wang, Yang Liu, and Hao Sun. 2024. Liveideabench: Evaluating llms' scientific creativity and idea generation with minimal context. arXiv preprint arXiv:2412.17596. · arXiv
  84. Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpurohit, Ameet Deshpande, Karthik Narasimhan, and Vishvak Murahari. 2024. Personagym: Evaluating persona agents and llms. Preprint, arXiv:2407.18416. · arXiv
  85. Greg Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2025. Personality Traits in Large Language Mod- els. Preprint, arXiv:2307.00184. · arXiv
  86. Orit Shaer, Angelora Cooper, Osnat Mokryn, Andrew L Kun, and Hagit Ben Shoshan. 2024. Ai-augmented brainwriting: Investigating the use of llms in group ideation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI '24, New York, NY, USA. Association for Computing Machinery. · DOI · Paper page
  87. C. E. Shannon. 1948. A mathematical theory of com- munication. The Bell System Technical Journal, 27(3):379-423. · DOI · Paper page
  88. Li Shi, Houjiang Liu, Yian Wong, Utkarsh Mujumdar, Dan Zhang, Jacek Gwizdka, and Matthew Lease. 2024. Argumentative Experience: Reducing Con- firmation Bias on Controversial Issues through LLM- Generated Multi-Persona Debates. arXiv preprint arXiv:2412.04629. · arXiv
  89. Peiyang Song, Kaiyu Yang, and Anima Anandkumar. 2025. Lean copilot: Large language models as copilots for theorem proving in lean. Preprint, arXiv:2404.12534. · arXiv
  90. Aarohi Srivastava et al. 2023. Beyond the imitation game: Quantifying and extrapolating the capabili- ties of language models. Transactions on Machine Learning Research.
  91. Haoyang Su, Renqi Chen, Shixiang Tang, Zhenfei Yin, Xinzhe Zheng, Jinzhe Li, Biqing Qi, Qi Wu, Hui Li, Wanli Ouyang, Philip Torr, Bowen Zhou, and Nanqing Dong. 2025. Many Heads Are Bet- ter Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System. Preprint, arXiv:2410.09403. · arXiv
  92. Jingyun Sun, Chengxiao Dai, Zhongze Luo, Yangbo Chang, and Yang Li. 2024. LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultation. Preprint, arXiv:2407.16252. · arXiv
  93. Yuqian Sun and Stefano Gualeni. 2025. Between puppet and actor: Reframing authorship in this age of ai agents. Preprint, arXiv:2501.15346. · arXiv
  94. Kamil Szczepanik and Jarosław Chudziak. 2025. TRIZ Agents: A Multi-Agent LLM Approach for TRIZ- Based Innovation. In 17th International Conference on Agents and Artificial Intelligence. SciTePress.
  95. Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. ArnetMiner: Extraction and Mining of Academic Social Networks. In Proceed- ings of the 14th ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining, pages 990-998. ACM. · DOI
  96. Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ro- nan Le Bras, Raja Marjieh, Nanyun Peng, Yejin Choi, Thomas L. Griffiths, and Faeze Brahman. 2024. Mac- Gyver: Are Large Language Models Creative Prob- lem Solvers? In Proceedings of the 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL).
  97. E. Paul Torrance. 1966. Torrance Tests of Creative Thinking. Personnel Press, Princeton, NJ.
  98. Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Pham, Barry O'Sullivan, and Hoang D. Nguyen. 2025. Multi-Agent Collaboration Mechanisms: A Survey of LLMs. Preprint, arXiv:2501.06322. · arXiv
  99. Ishwara Vasista, Imran Mirza, Cole Huang, Rohan Ra- jasekhara Patil, Aslihan Akalin, Kevin Zhu, and Sean O'Brien. 2025. MALIBU Benchmark: Multi-Agent LLM Implicit Bias Uncovered. In ICLR 2025 Work- shop on Building Trust in Language Models and Ap- plications.
  100. Tony Veale and F Amílcar Cardoso. 2019. Computa- tional Creativity: The Philosophy and Engineering of Autonomously Creative Systems. Springer.
  101. P Venkadesh, SV Divya, and K Subash Kumar. 2024. Unlocking AI Creativity: A Multi-Agent Approach with CrewAI. Journal of Trends in Computer Science Smart Technology.
  102. Kavana Venkatesh, Connor Dunlop, and Pinar Yanardag. 2025. CREA: A Collaborative Multi-Agent Frame- work for Creative Content Generation with Diffusion Models. Preprint, arXiv:2504.05306. · arXiv
  103. Saranya Venkatraman, Nafis Irtiza Tripto, and Dongwon Lee. 2024. Collabstory: Multi-llm collaborative story generation and authorship analysis. In Proc. NAACL '25.
  104. Qian Wan, Siying Hu, Yu Zhang, Piaohong Wang, Bo Wen, and Zhicong Lu. 2024. "It Felt Like Hav- ing a Second Mind": Investigating Human-AI Co- creativity in Prewriting with Large Language Models. Proc. ACM Hum.-Comput. Interact., 8. · DOI · Paper page
  105. Yun Wan and Yoram M Kalman. 2025. Using Gener- ative AI Personas Increases Collective Diversity in Human Ideation. Preprint, arXiv:2504.13868. · arXiv
  106. Boshi Wang, Xiang Deng, and Huan Sun. 2022. Itera- tively prompt pre-trained language models for chain of thought. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2714-2730. Association for Computational Linguistics.
  107. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024a. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345.
  108. Yi Wang, Qian Zhou, and David Ledo. 2024b. Story- verse: Towards co-authoring dynamic plot with llm- based character simulation via narrative planning. In Proceedings of the 19th International Conference on the Foundations of Digital Games, FDG '24, New York, NY, USA. Association for Computing Machin- ery. · DOI
  109. Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. 2024c. Unleashing the emergent cognitive synergy in large language mod- els: A task-solving agent through multi-persona self- collaboration. Preprint, arXiv:2307.05300. · arXiv
  110. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompt- ing elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824-24837.
  111. Geraint A Wiggins. 2006. A preliminary framework for description, analysis and comparison of creative systems. Knowledge-based systems, 19(7):449-458.
  112. Haibin Wu, Xuanjun Chen, Yi-Cheng Lin, Kai wei Chang, Ho-Lam Chung, Alexander H. Liu, and Hung yi Lee. 2024. Towards audio language modeling -an overview. Preprint, arXiv:2402.13236. · arXiv
  113. Zhikun Wu, Thomas Weber, and Florian Müller. 2025. One Does Not Simply Meme Alone: Evaluating Co- Creativity Between LLMs and Humans in the Gener- ation of Humor. In Proceedings of the 30th Interna- tional Conference on Intelligent User Interfaces, IUI '25, page 1082-1092. ACM. · DOI
  114. Chengxing Xie and Difan Zou. 2024. A human-like reasoning framework for multi-phases planning task with large language models. In ICML 2024 Workshop on LLMs and Cognition.
  115. Frank F. Xu, Yufan Song, Boxuan Li, Yuxuan Tang, Kritanjali Jain, Mengxue Bao, Zora Z. Wang, Xuhui Zhou, Zhitong Guo, Murong Cao, Mingyang Yang, Hao Yang Lu, Amaad Martin, Zhe Su, Leander Maben, Raj Mehta, Wayne Chi, Lawrence Jang, Yiqing Xie, Shuyan Zhou, and Graham Neubig. 2024. TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. Preprint, arXiv:2412.14161. · arXiv
  116. Bingyu Yan, Xiaoming Zhang, Litian Zhang, Lian Zhang, Ziyi Zhou, Dezhuang Miao, and Chaozhuo Li. 2025. Beyond self-talk: A communication-centric survey of llm-based multi-agent systems. Preprint, arXiv:2502.14321. · arXiv
  117. Kaixun Yang, Yixin Cheng, Linxuan Zhao, Mladen Raković, Zachari Swiecki, Dragan Gašević, and Guanliang Chen. 2024. Ink and algorithm: Exploring temporal dynamics in human-ai collaborative writing. Preprint, arXiv:2406.14885. · arXiv
  118. Lyumanshan Ye, Jiandong Jiang, Yuhan Liu, Yihan Ran, Pengfei Liu, and Danni Chang. 2024. Colin: A Mul- timodal Human-AI Co-Creation Storytelling System To Support Children's Multi-Level Narrative Skills. Preprint, arXiv:2405.06495. Version 4, last revised 17 Mar 2025. · DOI
  119. Junwei Yu, Yepeng Ding, and Hiroyuki Sato. 2025. DynTaskMAS: A dynamic task graph-driven frame- work for asynchronous and parallel llm-based multi- agent systems. Preprint, arXiv:2503.07675. · arXiv
  120. Mingyue Yuan, Jieshan Chen, and Aaron Quigley. 2024. Maxprototyper: A multi-agent generation system for interactive user interface prototyping. Preprint, arXiv:2405.07131. · arXiv
  121. J.D. Zamfirescu-Pereira, Eunice Jun, Michael Terry, Qian Yang, and Bjoern Hartmann. 2025. Beyond code generation: Llm-supported exploration of the program design space. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI '25, page 1-17. ACM. · DOI
  122. Lidong Zhai, Zhijie Qiu, Xizhong Guo, and Jiaqi Li. 2025. The Athenian Academy: A Seven-Layer Ar- chitecture Model for Multi-Agent Systems. arXiv preprint arXiv:2504.12735. · arXiv
  123. An Zhang, Yuxin Chen, Leheng Sheng, Xiang Wang, and Tat-Seng Chua. 2024a. On generative agents in recommendation. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '24, page 1807-1817, New York, NY, USA. Association for Computing Machinery. · DOI
  124. Chao Zhang, Xuechen Liu, Katherine Ziska, Soobin Jeon, Chi-Lin Yu, and Ying Xu. 2024b. Mathemyths: Leveraging large language models to teach mathe- matical language through child-ai co-creative story- telling. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI '24, New York, NY, USA. Association for Computing Machinery. · DOI · Paper page
  125. Chao Zhang, Cheng Yao, Jiayi Wu, Weijia Lin, Lijuan Liu, Ge Yan, and Fangtian Ying. 2022. StoryDrawer: A Child-AI Collaborative Drawing System to Sup- port Children's Creative Visual Storytelling. In Pro- ceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI '22. Association for Computing Machinery. · DOI · Paper page
  126. Guibin Zhang, Luyang Niu, Junfeng Fang, Kun Wang, Lei Bai, and Xiang Wang. 2025. Multi-agent ar- chitecture search via agentic supernet. Preprint, arXiv:2502.04180. · arXiv
  127. Jingyue Zhang and Ian Arawjo. 2025. ChainBuddy: An AI-assisted Agent System for Generating LLM Pipelines. In Proceedings of the 2025 CHI Confer- ence on Human Factors in Computing Systems, CHI '25, page 1-21. ACM. · DOI
  128. Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. 2024c. Exploring collaboration mechanisms for LLM agents: A social psychology view. In Proceedings of the 62nd An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14544- 14607, Bangkok, Thailand. Association for Compu- tational Linguistics. · DOI
  129. Zixiao Zhao, Jing Sun, Zhe Hou, Zhiyuan Wei, Cheng-Hao Cai, Miao Qiao, and Jin Song Dong. 2025. Mactg: Multi-agent collaborative thought graph for automatic programming. arXiv preprint arXiv:2410.19245. · arXiv
  130. Lianmin Zheng et al. 2023a. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems. · DOI
  131. Qingxiao Zheng, Zhongwei Xu, Abhinav Choudhry, Yuting Chen, Yongming Li, and Yun Huang. 2023b. Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based Agents. Preprint, arXiv:2310.15065. · arXiv
  132. Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Zhe Wang, Zhenhailong Wang, Cheng Qian, Xiangru Tang, Heng Ji, and Jiaxuan You. 2025. Multiagentbench: Evaluating the col- laboration and competition of llm agents. Preprint, arXiv:2503.01935. · arXiv
  133. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models. Preprint, arXiv:1802.01886. · arXiv

Cited by

Sorted by citations

Related

Authors
Lin, Y.-C., Chen, K.-C., Li, Z.-Y., Wu, T.-H., Wu, T.-H., Chen, K.-Y., Lee, H.-Y., & Chen, Y.-N.
Affiliations
National Taiwan University (TW)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
Pages
27572-27595
DOI
10.48550/arxiv.2505.21116
arXiv
2505.21116
OpenAlex
W4416035654
Cited by
6
References
0
Citations by year
2026: 4 · 2025: 2
FWCI
5.69
Type
conference paper
Language
en
Open access
Licence
Locations
  • Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • Ministry of Education, India
  • National Taiwan University
  • National Science and Technology Council
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
creativity, large language models, multi-agent systems, computational creativity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review