Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 313 of the 313 references it cites were found in a registry or an index.

A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges

Open access
Li, Xinyi, Wang, Sai, Zeng, Siqi, Wu, Yu, Yang, Yi · 2024 · Vicinagearth

Abstract

The pursuit of more intelligent and credible autonomous systems, akin to human society, has been a long-standing endeavor for humans. Leveraging the exceptional reasoning and planning capabilities of large language models (LLMs), LLM-based agents have been proposed and have achieved remarkable success across a wide array of tasks. Notably, LLM-based multi-agent systems (MAS) are considered a promising pathway towards realizing general artificial intelligence that is equivalent to or surpasses human-level intelligence. In this paper, we present a comprehensive survey of these studies, offering a systematic review of LLM-based MAS. Adhering to the workflow of LLM-based multi-agent systems, we synthesize a general structure encompassing five key components: profile, perception, self-action, mutual interaction, and evolution. This unified framework encapsulates much of the previous work in the field. Furthermore, we illuminate the extensive applications of LLM-based MAS in two principal areas: problem-solving and world simulation. Finally, we discuss in detail several contemporary challenges and provide insights into potential future directions in this domain.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s44336-024-00009-2

Cites

313 of 313 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 252 references

Research works printed but not linked: 86

  1. A. Brohan, Y. Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian et al., Do as i can, not as i say: Grounding language in robotic affordances. in Conference on robot learning (PMLR, Atlanta, GA, USA, 2023), pp. 287–318
  2. K. Li, A.K. Hopkins, D. Bau, F. Viégas, H. Pfister, M. Wattenberg, Emergent world representations: Exploring a sequence model trained on a synthetic task. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
  3. N. Renella, M. Eger, Towards automated video game commentary using generative ai. in Proceedings of the Experimental Artificial Intelligence in Games Workshop co-located with the 19th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2023) (AAAI Press, Washington, DC, USA, 2023)
  4. Z. Ma, Y. Mei, Z. Su, Understanding the benefits and challenges of using large language model-based conversational agents for mental well-being support. in AMIA Annual Symposium Proceedings (American Medical Informatics Association, New Orleans, LA, 2023). pp. 1105
  5. G. Weiss, Multiagent systems: a modern approach to distributed artificial intelligence (MIT Press, Cambridge, MA, United States, 1999), pp. 547
  6. H.V.D. Parunak, Applications of distributed artificial intelligence in industry. Found. Distrib. Artif. Intell. 2(1), 18 (1996)
  7. I. Dasgupta, C. Kaeser-Chen, K. Marino, A. Ahuja, S. Babayan, F. Hill, R. Fergus, Collaborating with language models for embodied reasoning. in NeurIPS 2022 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States of America, 2022)
  8. S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S.K.S. Yau, Z. Lin, L. Zhou et al., Metagpt: Meta programming for multi-agent collaborative framework. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
  9. H. Zhang, W. Du, J. Shan, Q. Zhou, Y. Du, J.B. Tenenbaum, T. Shu, C. Gan, Building cooperative embodied agents modularly with large language models. in NeurIPS 2023 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States, 2023)
  10. Y. Du, S. Li, A. Torralba, J.B. Tenenbaum, I. Mordatch, Improving factuality and reasoning in language models through multiagent debate. in Proceedings of the 41st International Conference on Machine Learning. Proceedings of Machine Learning Research (Vienna, Austria, 2024). 235, pp. 11733–11763. PMLR.
  11. C.M. Chan, W. Chen, Y. Su, J. Yu, W. Xue, S. Zhang, J. Fu, Z. Liu, Chateval: Towards better llm-based evaluators through multi-agent debate. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
  12. J. Light, M. Cai, S. Shen, Z. Hu, Avalonbench: Evaluating llms playing the game of avalon. in NeurIPS 2023 Foundation Models for Decision Making Workshop (2023) (New Orleans, United States, 2023)
  13. G. Mukobi, H. Erlebach, N. Lauffer, L. Hammond, A. Chan, J. Clifton, Welfare diplomacy: Benchmarking language model cooperation. in NeurIPS 2023 Socially Responsible Language Modelling Research workshop (New Orleans, United States, 2023)
  14. G.V. Aher, R.I. Arriaga, A.T. Kalai, Using large language models to simulate multiple humans and replicate human subject studies. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 337–371
  15. A. Zhang, L. Sheng, Y. Chen, H. Li, Y. Deng, X. Wang, T.S. Chua, On generative agents in recommendation. in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Association for Computing Machinery, New York, NY, USA, 2024). SIGIR ’24, pp. 1807–1817. https://doi.org/10.1145/3626772.3657844 · DOI
  16. L. Wang, J. Zhang, X. Chen, Y. Lin, R. Song, W.X. Zhao, J.R. Wen, Recagent: A novel simulation paradigm for recommender systems. arXiv preprint (2023) arXiv:2306.0255
  17. M. Shridhar, X. Yuan, M.A. Côté, Y. Bisk, A. Trischler, M. Hausknecht, Alfworld: Aligning text and embodied environments for interactive learning. International Conference on Learning Representation (2021)
  18. D. Driess, F. Xia, M.S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu et al., Palm-e: An embodied multimodal language model. in Proceedings of the 40th International Conference on Machine Learning (Honolulu, Hawaii, USA, 2023). ICML’23. JMLR.org 340, 20.
  19. W. Tan, Z. Ding, W. Zhang, B. Li, B. Zhou, J. Yue, H. Xia, J. Jiang, L. Zheng, X. Xu et al., Towards general computer control: A multimodal agent for red dead redemption ii as a case study. in ICLR 2024 Workshop on Large Language Model (LLM) Agents (Vienna, Austria, 2024).
  20. S. Reed, K. Zolna, E. Parisotto, S.G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, Y. Sulsky, J. Kay, J.T. Springenberg et al., A generalist agent. Transactions on Machine Learning Research (2022).
  21. S. Zheng, Y. Feng, Z. Lu et al., Steve-eye: Equipping llm-based embodied agents with visual perception in open worlds. in The Twelfth International Conference on Learning Representations (Kigali, Rwanda, 2023)
  22. P.F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, D. Amodei, Deep reinforcement learning from human preferences. Adv. Neural Inf. Process. Syst. 30 (2017). pp. 4302-4310
  23. J. Wei, M. Bosma, V.Y. Zhao, K. Guu, A.W. Yu, B. Lester, N. Du, A.M. Dai, Q.V. Le, Finetuned language models are zero-shot learners. in International Conference on Learning Representations (2022) (Virtual Event).
  24. W. Li, L. Zhu, L. Wen, Y. Yang, Decap: Decoding clip latents for zero-shot captioning via text-only training. in International Conference on Learning Representations (2023) (Kigali, Rwanda, 2023).
  25. S. Zhao, X. Wang, L. Zhu, Y. Yang, Test-time adaptation with clip reward for zero-shot generalization in vision-language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
  26. J. Chen, H. Guo, K. Yi, B. Li, M. Elhoseiny, Visualgpt: Data-efficient image captioning by balancing visual input and linguistic knowledge from pretraining. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE, New Orleans, LA, USA, 2022), pp. 18030–18040
  27. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., An image is worth 16x16 words: Transformers for image recognition at scale. in International Conference on Learning Representations (2021) (Vienna, Austria).
  28. A. Van Den Oord, O. Vinyals et al., Neural discrete representation learning. Adv. Neural Inf. Process. Syst. 30 (2017). pp. 6309-6318
  29. S. Mehta, M. Rastegari, Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. in International Conference on Learning Representations (2022) (Virtual Event).
  30. J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. in International conference on machine learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 19730–19742
  31. D. Zhu, J. Chen, X. Shen, X. Li, M. Elhoseiny, Minigpt-4: Enhancing vision-language understanding with advanced large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
  32. Y. Su, T. Lan, H. Li, J. Xu, Y. Wang, D. Cai, Pandagpt: One model to instruction-follow them all. in Proceedings of the 1st Workshop on Taming Large Language Models: Controllability in the era of Interactive Assistants! (Association for Computational Linguistics, Prague, Czech Republic, 2023), pp. 11-23.
  33. Z. Peng, W. Wang, L. Dong, Y. Hao, S. Huang, S. Ma, F. Wei, Kosmos-2: Grounding multimodal large language models to the world. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
  34. L. Qian, J. Li, Y. Wu, Y. Ye, H. Fei, T.S. Chua, Y. Zhuang, S. Tang, Momentor: Advancing video large language model with fine-grained temporal reasoning. in Forty-first International Conference on Machine Learning (Vienna, Austria, 2024).
  35. Z. Yang, G. Chen, X. Li, W. Wang, Y. Yang, Doraemongpt: Toward understanding dynamic scenes with large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
  36. L.H. Marshall, H.W. Magoun, Discoveries in the human brain: neuroscience prehistory, brain structure, and function (Springer Science & Business Media, Humana Totowa, NJ, 2013)
  37. L. Zheng, R. Wang, X. Wang, B. An, Synapse: Trajectory-as-exemplar prompting with memory for computer control. in The Twelfth International Conference on Learning Representations (Messe Wien Exhibition and Congress Center, Vienna, Austria, 2023)
  38. J. Kang, R. Laroche, X. Yuan, A. Trischler, X. Liu, J. Fu, Think before you act: Decision transformers with internal working memory. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
  39. A. Modarressi, A. Imani, M. Fayyaz, H. Schütze, Ret-llm: Towards a general read-write memory for large language models. in ICLR 2024 Workshop: How Far Are We From AGI (Vienna, Austria, 2024)
  40. T. Silver, V. Hariprasad, R.S. Shuttleworth, N. Kumar, T. Lozano-Pérez, L.P. Kaelbling, Pddl planning with pretrained large language models. in NeurIPS 2022 foundation models for decision making workshop (New Orleans, Louisiana, USA, 2022)
  41. Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, H. Soh, Translating natural language to planning goals with large-language models. in The International Journal of Robotics Research (2020). 2019, pp. 1
  42. M. Tsimpoukelli, J.L. Menick, S. Cabi, S. Eslami, O. Vinyals, F. Hill, Multimodal few-shot learning with frozen language models. Adv. Neural Inf. Process. Syst. 34, 200–212 (2021)
  43. H. Liu, C. Li, Q. Wu, Y.J. Lee, Visual instruction tuning. Adv. Neural Inf. Process. Syst. 36(2024). pp. 34892-34916
  44. L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, G. Neubig, Pal: Program-aided language models. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 10764–10799
  45. X. Cheng, D. Luo, X. Chen, L. Liu, D. Zhao, R. Yan, Lift yourself up: Retrieval-augmented text generation with self-memory. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 43780-43799
  46. W. Yu, D. Iter, S. Wang, Y. Xu, M. Ju, S. Sanyal, C. Zhu, M. Zeng, M. Jiang, Generate rather than retrieve: Large language models are strong context generators. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
  47. A. Asai, Z. Wu, Y. Wang, A. Sil, H. Hajishirzi, Self-rag: Learning to retrieve, generate, and critique through self-reflection. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
  48. F. Xu, W. Shi, E. Choi, Recomp: Improving retrieval-augmented lms with compression and selective augmentation. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
  49. E. Mitchell, C. Lin, A. Bosselut, C.D. Manning, C. Finn, Memory-based model editing at scale. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 15817–15831
  50. J. Maynez, S. Narayan, B. Bohnet, R. McDonald, On faithfulness and factuality in abstractive summarization. in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (Association for Computational Linguistics, Online, 2022), pp. 1906-1919
  51. J. Thorne, A. Vlachos, Automated fact checking: Task formulations, methods and future directions. in Proceedings of the 27th International Conference on Computational Linguistics (Association for Computational Linguistics, Santa Fe, New Mexico, USA, 2018), pp. 3346-3359
  52. C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu et al., Lima: Less is more for alignment. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 55006-55021
  53. Z. Wu, Y. Hu, W. Shi, N. Dziri, A. Suhr, P. Ammanabrolu, N.A. Smith, M. Ostendorf, H. Hajishirzi, Fine-grained human feedback gives better rewards for language model training. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 59008-59033
  54. Y.S. Chuang, Y. Xie, H. Luo, Y. Kim, J. Glass, P. He, Dola: Decoding by contrasting layers improves factuality in large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
  55. P.C. Wason, P.N. Johnson-Laird, Psychology of reasoning: Structure and content. Harvard University Press, Cambridge, MA, USA, 1972. 86, pp. 246-252
  56. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le et al., Least-to-most prompting enables complex reasoning in large language models. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
  57. M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman et al., Do as i can, not as i say: Grounding language in robotic affordances. in Conference on Robot Learning (Atlanta, GA, United States, 2023), pp. 287–318. PMLR
  58. S.S. Raman, V. Cohen, E. Rosen, I. Idrees, D. Paulius, S. Tellex, Planning with large language models via corrective re-prompting. in NeurIPS 2022 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, USA, 2022)
  59. T. Kojima, S.S. Gu, M. Reid, Y. Matsuo, Y. Iwasawa, Large language models are zero-shot reasoners. Adv. Neural Inf. Process. Syst. 35, 22199–22213 (2022)
  60. B.Y. Lin, Y. Fu, K. Yang, F. Brahman, S. Huang, C. Bhagavatula, P. Ammanabrolu, Y. Choi, X. Ren, Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 23813-23825
  61. S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, K. Narasimhan, Tree of thoughts: Deliberate problem solving with large language models. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 11809-11822
  62. W. Huang, P. Abbeel, D. Pathak, I. Mordatch, Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 9118–9147
  63. K. Nottingham, P. Ammanabrolu, A. Suhr, Y. Choi, H. Hajishirzi, S. Singh, R. Fox, Do embodied agents dream of pixelated sheep: Embodied decision making using language guided world modelling. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 26311–26325
  64. J. Ruan, Y. Chen, B. Zhang, Z. Xu, T. Bao, G. Du, S. Shi, H. Mao, X. Zeng, R. Zhao, Tptu: Task planning and tool usage of large language model-based ai agents. in NeurIPS 2023 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States of America, 2023)
  65. Z. Zhao, E. Wallace, S. Feng, D. Klein, S. Singh, Calibrate before use: Improving few-shot performance of language models. in International conference on machine learning (PMLR, Virtual Event, 2021), pp. 12697–12706
  66. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
  67. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, T. Scialom, Toolformer: Language models can teach themselves to use tools. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 68539-68551
  68. M. Carroll, R. Shah, M.K. Ho, T. Griffiths, S. Seshia, P. Abbeel, A. Dragan, On the utility of learning about humans for human-ai coordination. Adv. Neural Inf. Process. Syst. 32(2019), pp. 5174-5185
  69. X. Ning, Z. Lin, Z. Zhou, Z. Wang, H. Yang, Y. Wang, Skeleton-of-thought: Prompting llms for efficient parallel generation. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
  70. S. Srivastava, C. Li, M. Lingelbach, R. Martín-Martín, F. Xia, K.E. Vainio, Z. Lian, C. Gokmen, S. Buch, K. Liu et al., Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments. in Conference on robot learning (PMLR, London, UK, 2021), pp. 477–490
  71. C. Colas, L. Teodorescu, P.Y. Oudeyer, X. Yuan, M.A. Côté, Augmenting autotelic agents with large language models. in Conference on Lifelong Learning Agents (PMLR, McGill University, Montréal, Québec, Canada, 2023), pp. 205–226
  72. L. Zhu, L. Hu, J. Lin, S. Han, Lift: Efficient layer-wise fine-tuning for large model models. in Proceedings of the 31st ACM International Conference on -Multimedia (Association for Computing Machinery, New York, NY, United States, 2023), pp. 4678-4687
  73. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, S. Gelly, Parameter-efficient transfer learning for nlp. in International conference on machine learning (PMLR, Long Beach, California, USA, 2019), pp. 2790–2799
  74. T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettlemoyer, Qlora: Efficient finetuning of quantized llms. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 10088-10115
  75. A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, S. Bowman, Superglue: A stickier benchmark for general-purpose language understanding systems. Adv. Neural Inf. Process. Syst. 32 (2019). pp. 3266-3280
  76. A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang et al., Self-refine: Iterative refinement with self-feedback. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 46534-46594
  77. N. Shinn, B. Labash, A. Gopinath, Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint (2023) arXiv: 2303.11366 · arXiv
  78. E. Brooks, L. Walls, R.L. Lewis, S. Singh, Large language models can implement policy iteration. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 30349-30366
  79. T. Carta, C. Romac, T. Wolf, S. Lamprier, O. Sigaud, P.Y. Oudeyer, Grounding large language models in interactive environments with online reinforcement learning. in International Conference on Machine Learing (PMLR, Honolulu, Hawaii, USA, 2023), pp. 3676–3713
  80. S. Sudhakaran, M. González-Duque, M. Freiberger, C. Glanois, E. Najarro, S. Risi, Mariogpt: Open-ended text2level generation through large language models. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 54213-54227
  81. F. Cardoso Durier da Silva, R. Vieira, A.C. Garcia, Can machines learn to detect fake news? a survey focused on social media, in 2021 IEEE 4th International Conference on Computing, Power and Communication Technologies (Kuala Lumpur, Malaysia, 2021), pp.1-7
  82. B. Buchanan, A. Lohn, M. Musser, K. Sedova, Truth, lies, and automation. Cent. Secur. Emerg. Technol. 1(1), 2 (2021)
  83. N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., Extracting training data from large language models. in 30th USENIX Security Symposium (USENIX Security 21) (USENIX, Berkeley, CA, 2021), pp. 2633–2650
  84. Z. Xi, T. Du, C. Li, R. Pang, S. Ji, J. Chen, F. Ma, T. Wang, Defending pre-trained language models as few-shot learners against backdoor attacks. Adv. Neural Inf. Process. Syst. 36(2024), pp. 32748-32764
  85. E. Ostrom, Tragedy of the commons. New Palgrave Dictionary Econ. 2, 1–4 (2008)
  86. T. Sun, Y. Shao, H. Qian, X. Huang, X. Qiu, Black-box tuning for language-model-as-a-service. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 20841–20855

Cited by

See all 412

Metadata

Authors
Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y.
Affiliations
  • Wuhan University (CN)
  • Zhejiang University (CN)
Corresponding author
  • Li, Xinyi
  • Wang, Sai
  • Zeng, Siqi
  • Wu, Yu
  • Yang, Yi
Venue
Vicinagearth
Published
2024-10-08
Volume
1
Issue
1
ISSN
3005-060X (ISSN-L)
OpenAlex
W4403203873
Cited by
404
References
252
Citations by year
2026: 254 · 2025: 147 · 2024: 2
FWCI
59.35
Type
journal article
Language
en
Open access
Licence
Locations
Vicinagearth. (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
multi-agent LLM systems, large language models, multi-agent systems, artificial general intelligence, agent architecture, agent perception, agent interaction, agent evolution, problem solving, world simulation, autonomous systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges. https://app.cochat.ai/papers/10.1007/s44336-024-00009-2
The paper
Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y. (2024). A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth. https://doi.org/10.1007/s44336-024-00009-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.