A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges
Open accessThe DOI matches its Crossref record; 322 of the 322 references it cites were found in a registry or an index.
Abstract
The pursuit of more intelligent and credible autonomous systems, akin to human society, has been a long-standing endeavor for humans. Leveraging the exceptional reasoning and planning capabilities of large language models (LLMs), LLM-based agents have been proposed and have achieved remarkable success across a wide array of tasks. Notably, LLM-based multi-agent systems (MAS) are considered a promising pathway towards realizing general artificial intelligence that is equivalent to or surpasses human-level intelligence. In this paper, we present a comprehensive survey of these studies, offering a systematic review of LLM-based MAS. Adhering to the workflow of LLM-based multi-agent systems, we synthesize a general structure encompassing five key components: profile, perception, self-action, mutual interaction, and evolution. This unified framework encapsulates much of the previous work in the field. Furthermore, we illuminate the extensive applications of LLM-based MAS in two principal areas: problem-solving and world simulation. Finally, we discuss in detail several contemporary challenges and provide insights into potential future directions in this domain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“MAS, in which multiple autonomous agents collaborate as a team, are emerging as a powerful paradigm for biomedical and clinical data analysis [1,2].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, current RAG models face challenges related to frequent knowledge updates [2,11,21,22].”
From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [11]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Evaluating multi-agent collaboration in LLM-based systems requires different methodologies compared to traditional reinforcement learning-driven coordination [7, 48, 89].”
From Evaluation and Benchmarking of LLM Agents: A Survey (2025) · cites this paper as [48]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–323 of 323
TidyBot: Personalized Robot Assistance with Large Language Models
Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S., & Funkhouser, T. (2023) · 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341577 · Cited by 77
Memory Augmented Large Language Models are Computationally UniversalPreprint
Schuurmans, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.04589 · Cited by 76
RecMind: Large Language Model Powered Agent For Recommendation
Wang, Y., Jiang, Z., Chen, Z., Yang, F., Zhou, Y., Cho, E., Fan, X., Lu, Y., Huang, X., & Yang, Y. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.271 · Cited by 74 · cited 1×
“In RecMind [232], a self-inspiring mechanism is designed where discarded historical information in the planning process is leveraged to derive new reasoning steps.”
Self-citationWang, Z., Mao, S., Wu, W., Ge, T., Wei, F., & Ji, H. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.15 · Cited by 74
Decoding the Threat Landscape : ChatGPT, FraudGPT, and WormGPT in Social Engineering Attacks
Falade, P. V. (2023) · International Journal of Scientific Research in Computer Science, Engineering and Information Technology · doi:10.32628/cseit2390533 · Cited by 72 · cited 1×
“Despite the powerful capabilities of MAS and LLMs, they can be maliciously exploited for large-scale disinformation generation [340–342], cyber-attacks [343–346], and other inappropriate behaviors [347, 348].”
Emergent autonomous scientific research capabilities of large language models
Boiko, D. A., MacKnight, R., & Gomes, G. (2023) · arXiv · doi:10.48550/arxiv.2304.05332 · Cited by 72
CGMI: Configurable General Multi-Agent Interaction FrameworkPreprint
Jinxin, S., Zhao, J., Yilei, W., Wu, X., Li, J., & He, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.12503 · Cited by 72
Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation using Large Language Models
Yu, B., Kasaei, H., & Cao, M. (2023) · arXiv.org · doi:10.48550/arxiv.2310.07937 · Cited by 71
Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration
Zhang, Y., Yang, S., Bai, C., Wu, F., Li, X., Wang, Z., & Li, X.-L. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2405.14314 · Cited by 70
Self-citationPerturbation augmentation for fairer NLP
Qian, R., Ross, C., Fernandes, J., Smith, E. M., Kiela, D., & Williams, A. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.646 · Cited by 69
Explore Spurious Correlations at the Concept Level in Language Models for Text Classification
Zhou, Y., Xu, P., Liu, X., An, B., Ai, W., & Huang, F. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.28 · Cited by 69
Retrieval-Augmented Generative Question Answering for Event Argument Extraction
Du, X., & Ji, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.307 · Cited by 68
Queer People are People First: Deconstructing Sexual Identity Stereotypes in Large Language ModelsPreprint
Dhingra, H., Jayashanker, P., Moghe, S., & Strubell, E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.00101 · Cited by 68
Ghaffarzadegan, N., Majumdar, A., Williams, R., & Hosseinichimeh, N. (2023) · arXiv · doi:10.48550/arxiv.2309.11456 · Cited by 68
Learning from Richer Human Guidance: Augmenting Comparison-Based Learning with Feature Queries
Basu, C., Singhal, M., & Dragan, A. D. (2018) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3171221.3171284 · Cited by 67
Large Language Model-Empowered Agents for Simulating Macroeconomic ActivitiesPreprint
Li, N., Gao, C., Li, Y., & Liao, Q.-M. (2023) · Social Science Research Network · doi:10.48550/arxiv.2310.10436 · Cited by 67
Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
Xiong, K., Ding, X., Cao, Y., Liu, T., & Qin, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.508 · Cited by 66
Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents
Talebirad, Y., & Nadiri, A. (2023) · arXiv · doi:10.48550/arxiv.2306.03314 · Cited by 65 · cited 1×
“To address these difficulties, researchers have adopted various strategies: Firstly, static adjustment and dynamic scaling methods are widely applied [356].”
MemGPT: Towards LLMs as Operating Systems
Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., & Gonzalez, J. E. (2023) · arXiv · doi:10.48550/arxiv.2310.08560 · Cited by 64
Inferring Rewards from Language in Context
Lin, J., Fried, D., Klein, D., & Dragan, A. D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.585 · Cited by 63
Memory-assisted prompt editing to improve GPT-3 after deployment
Madaan, A., Tandon, N., Clark, P., & Yang, Y. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.183 · Cited by 60
Self-citationText Style Transfer for Bias Mitigation using Masked Language Modeling
Tokpo, E. K., & Calders, T. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW) · doi:10.18653/v1/2022.naacl-srw.21 · Cited by 60
Is ChatGPT a Good Recommender? A Preliminary Study
Liu, J., Liu, C., Zhou, P., Lv, R., Zhou, K., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2304.10149 · Cited by 60
ChatLLM Network: More brains, More intelligencePreprint
Hao, R., Hu, L., Qi, W., et al. (2023) · AI Open · doi:10.48550/arxiv.2304.12998 · Cited by 58
(2007) · doi:10.1515/9783110198843 · Cited by 57
Combining Fine-Tuning and LLM-Based Agents for Intuitive Smart Contract Auditing with Justifications
Ma, W., Wu, D., Sun, Y., Wang, T., Liu, S., Zhang, J., Xue, Y., & Liu, Y. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) · doi:10.1109/icse55347.2025.00027 · Cited by 57
Wang, Z., Cai, S., Chen, G., Liu, A., Ma, X., & Liang, Y. (2023) · arXiv · doi:10.48550/arxiv.2302.01560 · Cited by 55
Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View
Zhang, J., Xu, X., Zhang, N., Liu, R., Hooi, B., & Deng, S. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.782 · Cited by 53
Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data
Li, X., Liu, Z., Xiong, C., Shi, Y., Gu, Y., Liu, Z., & Yu, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.734 · Cited by 52
Self-citationLLM As DBAPreprint
Zhou, X., Li, G., & Liu, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.05481 · Cited by 51
MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning LibraryPreprint
Hu, S., Zhong, Y., Gao, M., Wang, W., Dong, H., Liang, X., Li, Z., Chang, X., & Yang, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.13708 · Cited by 50
Self-citationKarpas, E., Abend, O., Belinkov, Y., Lenz, B., Lieber, O., Ratner, N., Shoham, Y., Bata, H., Levine, Y., Leyton-Brown, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.00445 · Cited by 49
Mehta, N., Teruel, M., Deng, X., Sanz, S. F., Awadallah, A. H., & Kiseleva, J. (2024) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2024.findings-eacl.87 · Cited by 49
Large Language Model-Empowered Agents for Simulating Macroeconomic ActivitiesPreprint
Li, N., Gao, C., Li, Y., & Liao, Q. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4606937 · Cited by 48
WavJourney: Compositional Audio Creation with Large Language ModelsPreprint
Liu, X.-B., Zhu, Z., Liu, H., et al. (2023) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.48550/arxiv.2307.14335 · Cited by 48
Attention-Guided Contrastive Role Representations for Multi-Agent Reinforcement Learning
Hu, Z., Zhang, Z., Li, H., Chen, C., Ding, H., & Wang, Z. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2312.04819 · Cited by 48
ScienceWorld: Is your Agent Smarter than a 5th Grader?
Wang, R., Jansen, P., Côté, M.-A., & Ammanabrolu, P. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.775 · Cited by 45
Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
Xiong, K., Ding, X., Cao, Y., Liu, T., & Qin, B. (2023) · arXiv · doi:10.48550/arxiv.2305.11595 · Cited by 45 · cited 1×
“Static adjustment methods [37, 255] design systems by pre-determining the number and roles of agents, which is effective for fixed tasks or goals but lacks flexibility in response to task changes.”
ScienceWorld: Is your Agent Smarter than a 5th Grader?
Wang, R., Jansen, P., Côté, M.-A., & Ammanabrolu, P. (2022) · arXiv · doi:10.48550/arxiv.2203.07540 · Cited by 45
PMET: Precise Model Editing in a Transformer
Li, X., Li, S., Song, S., Yang, J., Ma, J., & Yu, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29818 · Cited by 44
Self-citationCan Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis
Fan, C., Chen, J., Jin, Y., & He, H. (2023) · arXiv · doi:10.48550/arxiv.2312.05488 · Cited by 44
Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness
Zayed, A., Parthasarathi, P., Mordido, G., Palangi, H., Shabanian, S., & Chandar, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26706 · Cited by 43
Detect and Perturb: Neutral Rewriting of Biased and Sensitive Text via Gradient-based Decoding
He, Z., Majumder, B. P., & McAuley, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.352 · Cited by 42
Is Prompt All You Need? No. A Comprehensive and Broader View of Instruction LearningPreprint
Lou, R., Zhang, K., & Yin, W. (2023) · arXiv.org · doi:10.48550/arxiv.2303.10475 · Cited by 42
AgentTuning: Enabling Generalized Agent Abilities for LLMs
Zeng, A., Liu, M., Lu, R., Wang, B., Liu, X., Dong, Y., & Tang, J. (2023) · arXiv · doi:10.48550/arxiv.2310.12823 · Cited by 40
AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
Chen, W., Su, Y., Zuo, J., Yang, C., Yuan, C., Chan, C.-M., Yu, H., Lu, Y., Hung, Y.-H., Qian, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.10848 · Cited by 39
Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis
Fan, C., Chen, J., Jin, Y., & He, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29751 · Cited by 38
Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration
Lyu, C., Wu, M., Wang, L., Huang, X., Liu, B., Du, Z., Shi, S., & Tu, Z. (2023) · arXiv · doi:10.48550/arxiv.2306.09093 · Cited by 38
PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
Deng, G., Liu, Y., Mayoral-Vilches, V., Liu, P., Li, Y., Xu, Y., Zhang, T., Liu, Y., Pinzger, M., & Rass, S. (2023) · arXiv · doi:10.48550/arxiv.2308.06782 · Cited by 37 · cited 2×
“Through ongoing interactions, agents continuously assess and enhance their knowledge base, [32], ChatEDA [303], LIBRO [304], PENTESTGPT [305] Industrial Engineering Mehta et al. [2], Xia et al. [3], Li et al. [4] Embodied Agents SayCan [8], Inner Monologue [9], TidyBot [10], RoCo [34], CoELA [35]…”
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
Robey, A., Wong, E., Hassani, H., & Pappas, G. J. (2023) · arXiv · doi:10.48550/arxiv.2310.03684 · Cited by 37
CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents
Zhao, Q., Wang, J., Zhang, Y., Jin, Y., Zhu, K., Chen, H., & Xie, X. (2023) · arXiv · doi:10.48550/arxiv.2310.17512 · Cited by 37 · cited 3×
“The task execution scenario focuses on how agents execute specific actions based on predefined task allocations, which may include role-playing games [12, 256, 257], distributed task assignments [258–260], and more.”
Retrieval-Augmented Generative Question Answering for Event Argument Extraction
Du, X., & Ji, H. (2022) · arXiv · doi:10.48550/arxiv.2211.07067 · Cited by 37
TransHuman: A Transformer-based Human Representation for Generalizable Neural Human Rendering
Pan, X., Yang, Z., Ma, J., Zhou, C., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00328 · Cited by 36
Self-citationQuantifying the Impact of Large Language Models on Collective Opinion DynamicsPreprint
Li, C., Su, X., Han, H.-Y., Xue, C., Zheng, C.-M., & Fan, C. (2023) · arXiv.org · doi:10.48550/arxiv.2308.03313 · Cited by 36
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking
Zhuang, S., Liu, B., Koopman, B., & Zuccon, G. (2023) · arXiv · doi:10.48550/arxiv.2310.13243 · Cited by 36 · cited 1×
“In the generation stage, the model improves the quality of responses from LLMs by re-ranking document segments to highlight the most relevant results [186] or by selecting or compressing contexts to reduce redundant information and manage overly long inputs [187, 188].”
Black-box Prompt Tuning for Vision-Language Model as a Service
Yu, L., Chen, Q., Lin, J., & He, L. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/187 · Cited by 35 · cited 1×
“The introduction of LLM multi-agent systems as a service (AaaS) heralds a significant shift in the service model within the AI domain [365, 366].”
TALM: Tool Augmented Language Models
Parisi, A., Zhao, Y., & Fiedel, N. (2022) · arXiv · doi:10.48550/arxiv.2205.12255 · Cited by 35
Memory Sandbox: Transparent and Interactive Memory Management for Conversational Agents
Huang, Z., Gutierrez, S., Kamana, H., & Macneil, S. (2023) · Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586182.3615796 · Cited by 34 · cited 2×
“In [145], it involves translating sentences into triplet configurations, while others perceive memory as a unique data object, facilitating diverse interactions [144].”
CapHuman: Capture Your Moments in Parallel Universes
Liang, C., Ma, F., Zhu, L., Deng, Y., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00612 · Cited by 33
Self-citationLLM Multi-Agent Systems: Challenges and Open Problems
Han, S., Zhang, Q., Jin, W., & Xu, Z. (2024) · arXiv · doi:10.48550/arxiv.2402.03578 · Cited by 33 · cited 1×
“A prevalent approach is to represent audio information as embeddings [118, 119], which usually requires integration with data from other modalities.”
Zhu, X., Chen, Y., Tian, H., Tao, C., Su, W., Yang, C., Huang, G., Li, B., Lu, L., Wang, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.17144 · Cited by 33
MindAgent: Emergent Gaming Interaction
Gong, R., Huang, Q., Ma, X., Noda, Y., Durante, Z., Zheng, Z., Terzopoulos, D., Fei-Fei, L., Gao, J., & Vo, H. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.200 · Cited by 32
Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf
Xu, Y., Wang, S., Li, P., Luo, F., Wang, X., Liu, W., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.04658 · Cited by 32
Self-citationLarge Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning
Tang, R., Kong, D., Huang, L., & Xue, H. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.284 · Cited by 31 · cited 1×
“For example, in sentiment analysis tasks, a model might skew towards predicting positive sentiment due to an overabundance of positive samples, even when the text actually conveys negative sentiment [214].”
CoLT5: Faster Long-Range Transformers with Conditional Computation
Ainslie, J., Lei, T., de Jong, M., Ontanon, S., Brahma, S., Zemlyanskiy, Y., Uthus, D., Guo, M., Lee-Thorp, J., Tay, Y., et al. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.309 · Cited by 31
SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)
Sun, R., Arik, S. Ö., Muzio, A., Miculicich, L., Gundabathula, S., Yin, P., Dai, H., Nakhost, H., Sinha, R., Wang, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.00739 · Cited by 31
Switchable Novel Object Captioner
Wu, Y., Jiang, L., & Yang, Y. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3144984 · Cited by 30 · cited 1×
“Regarding the extraction of visual features, a straightforward approach involves generating corresponding textual descriptions for visual messages through visual caption models [94–96] or user definitions [83, 84, 97].”
Self-citationA psychometric framework for evaluating and shaping personality traits in large language models
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Romero, P., Abdulhai, M., Faust, A., & Matarić, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01115-6 · Cited by 30
Semantically Aligned Task Decomposition in Multi-Agent Reinforcement LearningPreprint
Li, W., Qiao, D., Wang, B., Wang, X., Jin, B., & Zha, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.10865 · Cited by 30
SpeechAgents: Human-Communication Simulation with Multi-Modal Multi-Agent SystemsPreprint
Zhang, D., Li, Z., Wang, P., Zhang, X., Zhou, Y., & Qiu, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.03945 · Cited by 30
Advances in Multi-turn Dialogue Comprehension: A SurveyPreprint
Zhang, Z., & Zhao, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04984 · Cited by 29
Certifying LLM Safety against Adversarial Prompting
Kumar, A., Agarwal, C., Srinivas, S., Li, A. J., Feizi, S., & Lakkaraju, H. (2023) · arXiv · doi:10.48550/arxiv.2309.02705 · Cited by 28
Charan, P. V. S., Chunduri, H., Anand, P. M., & Shukla, S. K. (2023) · arXiv · doi:10.48550/arxiv.2305.15336 · Cited by 25
Yang, H., Li, Z., Zhang, Y., Wang, J., Cheng, N., Li, M., & Xiao, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.326 · Cited by 23
Faithful Chain-of-Thought Reasoning
Lyu, Q., Havaldar, S., Stein, A., Zhang, L., Rao, D., Wong, E., Apidianaki, M., & Callison-Burch, C. (2023) · arXiv · doi:10.48550/arxiv.2301.13379 · Cited by 23
Nie, Y., Huang, H., Wei, W., & Mao, X.-L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.336 · Cited by 22
Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations
Huang, S., Mamidanna, S., Jangam, S., Zhou, Y., & Gilpin, L. H. (2023) · arXiv · doi:10.48550/arxiv.2310.11207 · Cited by 22
Text Adversarial Purification as Defense against Adversarial Attacks
Li, L., Song, D., & Qiu, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.20 · Cited by 20 · cited 1×
“For instance, some studies employ methods such as instruction processing and malicious detection to eliminate potential adversarial contexts or malicious intents [349–351].”
A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration
Liu, Z., Zhang, Y., Li, P., Liu, Y., & Yang, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.02170 · Cited by 20
Epidemic Modeling with Generative Agents
Williams, R., Hosseinichimeh, N., Majumdar, A., & Ghaffarzadegan, N. (2023) · arXiv · doi:10.48550/arxiv.2307.04986 · Cited by 19
Yang, H., Li, Z., Zhang, Y., Wang, J., Cheng, N., Li, M., & Xiao, J. (2023) · arXiv · doi:10.48550/arxiv.2310.18347 · Cited by 18 · cited 1×
“In the generation stage, the model improves the quality of responses from LLMs by re-ranking document segments to highlight the most relevant results [186] or by selecting or compressing contexts to reduce redundant information and manage overly long inputs [187, 188].”
Active Retrieval Augmented Generation
Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · arXiv · doi:10.48550/arxiv.2305.06983 · Cited by 18
Self-citationReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
Xu, B., Peng, Z., Lei, B., Mukherjee, S., Liu, Y., & Xu, D. (2023) · arXiv · doi:10.48550/arxiv.2305.18323 · Cited by 17 · cited 1×
“Similarly, ReWOO [222] introduces a paradigm where plans and external observations are generated independently by the agents.”
First the Worst: Finding Better Gender Translations During Beam Search
Saunders, D., Sallis, R., & Byrne, B. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.301 · Cited by 16
Poisoning scientific knowledge using large language modelsPreprint
Yang, J., Xu, H., Mirzoyan, S., Chen, T., Liu, Z., Ju, W., Liu, L., Zhang, M., & Wang, S. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.06.565928 · Cited by 15 · cited 1×
“Despite the powerful capabilities of MAS and LLMs, they can be maliciously exploited for large-scale disinformation generation [340–342], cyber-attacks [343–346], and other inappropriate behaviors [347, 348].”
Self-citationEnabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach
Hu, B., Zhao, C., Zhang, P., Zhou, Z., Yang, Y., Xu, Z., & Liu, B. (2023) · arXiv · doi:10.48550/arxiv.2306.03604 · Cited by 15
Self-citationKnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases
Wang, X., Yang, Q., Qiu, Y., Liang, J., He, Q., Gu, Z., Xiao, Y., & Wang, W. (2023) · arXiv · doi:10.48550/arxiv.2308.11761 · Cited by 15
Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models
Sel, B., Al-Tawaha, A., Khattar, V., Jia, R., & Jin, M. (2023) · arXiv · doi:10.48550/arxiv.2308.10379 · Cited by 15 · cited 1×
“Algorithm of Thought [231] introduces a novel method to enhance LLM reasoning by incorporating algorithmic examples into the prompts, remarkably requiring only one or a few queries to the LLM.”
AgentSims: An Open-Source Sandbox for Large Language Model Evaluation
Lin, J., Zhao, H., Zhang, A., Wu, Y., Ping, H., & Chen, Q. (2023) · arXiv · doi:10.48550/arxiv.2308.04026 · Cited by 14 · cited 3×
“Another notable approach considers the retrieved information as a learnable representation, such as embeddings and vectors [30, 129, 135], which serve as soft guidance for fine-tuning the model to accommodate various tasks.”
Self-citationDong, Y., Zhu, X., Pan, Z., Zhu, L., & Yang, Y. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.964 · Cited by 13
Self-citationChatDB: Augmenting LLMs with Databases as Their Symbolic Memory
Hu, C., Fu, J., Du, C., Luo, S., Zhao, J., & Zhao, H. (2023) · arXiv · doi:10.48550/arxiv.2306.03901 · Cited by 13
Sparse is Enough in Fine-tuning Pre-trained Large Language ModelsPreprint
Song, W., Li, Z., Zhang, L., Zhao, H. Y., & Du, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.11875 · Cited by 13
Li, Y., Yu, Y., Li, H., Chen, Z., & Khashanah, K. (2023) · arXiv · doi:10.48550/arxiv.2309.03736 · Cited by 12
(2022) · doi:10.48550/arxiv.2210.08726 · Cited by 12
Quantifying the Impact of Large Language Models on Collective Opinion DynamicsPreprint
Li, C., Su, X., Han, H., Xue, C., Zheng, C., & Fan, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4688547 · Cited by 11
AppAgent: Multimodal Agents as Smartphone Users
Zhang, C., Yang, Z., Liu, J., Li, Y., Han, Y., Chen, X., Huang, Z., Fu, B., & Yu, G. (2023) · arXiv · doi:10.48550/arxiv.2312.13771 · Cited by 11
Wang, Z., Mao, S., Wu, W., Ge, T., Wei, F., & Ji, H. (2023) · arXiv · doi:10.48550/arxiv.2307.05300 · Cited by 11
Saying the Unseen: Video Descriptions via Dialog Agents
Zhu, Y., Wu, Y., Yang, Y., & Yan, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3093360 · Cited by 10 · cited 1×
“Consequently, agents can leverage their image perception capabilities to understand and interpret video content, which necessitates additional attention to the transitions and changes across the sequence of frames [109–111].”
Self-citationSparsity and Sentence Structure in Encoder-Decoder Attention of Summarization Systems
Manakul, P., & Gales, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.739 · Cited by 10 · cited 1×
“Memory storage is typically realized through the use of natural language formed text [147– 149], although it also encompasses multi-modal information such as visual and audio data [58].”
AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System
Liu, Z., Yao, W., Zhang, J., Yang, L., Liu, Z., Tan, J., Choubey, P. K., Lan, T., Wu, J., Wang, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.15538 · Cited by 10 · cited 1×
“AgentLite [265] builds on this by providing a lightweight platform that allows developers to easily implement and extend complex interactions and collaborative tasks among LLM agents based on this hierarchical concept.”
Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
Yao, W., Heinecke, S., Niebles, J. C., Liu, Z., Feng, Y., Xue, L., Murthy, R., Chen, Z., Zhang, J., Arpit, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.02151 · Cited by 10
A Unified Framework for Emotion Identification and Generation in Dialogues
Madasu, A., Firdaus, M., & Ekbal, A. (2023) · Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (EACL SRW) · doi:10.18653/v1/2023.eacl-srw.7 · Cited by 9
FireAct: Toward Language Agent Fine-tuning
Chen, B., Shu, C., Shareghi, E., Collier, N., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2310.05915 · Cited by 9
LLM4Laser: Large Language Models Automate the Design of LasersPreprint
Li, R., Zhang, C., Mao, S., Zhou, X., Yin, F., Theodoridis, S., & Zhang, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.12145 · Cited by 9
Plan, Eliminate, and Track - Language Models are Good Teachers for Embodied Agents
Wu, Y., Min, S. Y., Bisk, Y., Salakhutdinov, R., Azaria, A., Li, Y., Mitchell, T., & Prabhumoye, S. (2023) · arXiv · doi:10.48550/arxiv.2305.02412 · Cited by 7 · cited 1×
“Additionally, some methods emphasize hierarchical planning, while others underscore a strategy in which final plans are derived from reasoning steps structured in a tree-like format [226, 227].”
Self-citationTowards Open-World Recommendation with Knowledge Augmentation from Large Language ModelsPreprint
Xi, Y., Liu, W., Lin, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.10933 · Cited by 7
Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?Preprint
Chen, Y., Arkin, J., Zhang, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.15943 · Cited by 6
Embodied LLM Agents Learn to Cooperate in Organized TeamsPreprint
Guo, X., Huang, K., Liu, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.12482 · Cited by 6
Affective social anthropomorphic intelligent system
Mamun, M. A., Abdullah, H., Alam, M. G. R., Hassan, M. M., & Uddin, M. Z. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14597-6 · Cited by 5 · cited 1×
“Additionally, it considers emotionally nuanced information such as ambiance and atmosphere [73, 74].”
INMS: Memory Sharing for Large Language Model based Agents
Gao, H., & Zhang, Y. (2024) · arXiv · doi:10.48550/arxiv.2404.09982 · Cited by 2 · cited 1×
“Similarly, prompt engineering and fine-tuning techniques have been employed for more accurate outputs [87, 88].”
RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM AgentsPreprint
Shu, Y., Zhang, H., Gu, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.09904 · Cited by 2
Reflective Linguistic Programming (RLP): A Stepping Stone in Socially-Aware AGI (SocialAGI)
Fischer, K. A. (2023) · arXiv · doi:10.48550/arxiv.2305.12647 · Cited by 2
Lyfe Agents: Generative agents for low-cost real-time social interactions
Kaiya, Z., Naim, M., Kondic, J., Cortes, M., Ge, J., Luo, S., Yang, G. R., & Ahn, A. (2023) · arXiv · doi:10.48550/arxiv.2310.02172 · Cited by 2
Semantically Grounded QFormer for Efficient Vision Language UnderstandingPreprint
Choraria, M., Wu, X., Basu, S., Sekhar, N., Wu, Y., Zhang, X., Singhal, P., & Varshney, L. R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.07449 · Cited by 2
Self-citationMuliana, R., Sukmawati, R. A., & Suryaningsih, Y. (2023) · Computing and Education Technology Journal · doi:10.20527/cetj.v3i1.8099 · Cited by 1
Randomized Positional Encodings Boost Length Generalization of Transformers
Ruoss, A., Delétang, G., Genewein, T., Grau-Moya, J., Csordás, R., Bennani, M., Legg, S., & Veness, J. (2023) · arXiv · doi:10.48550/arxiv.2305.16843 · Cited by 1
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
RestGPT: Connecting Large Language Models with Real-World RESTful APIs
Song, Y., Xiong, W., Zhu, D., Wu, W., Qian, H., Song, M., Huang, H., Li, C., Wang, K., Yao, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.06624
Large Language Models and Games: A Survey and Roadmap
Gallotta, R., Todd, G., Zammit, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.18659
Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
Cheng, Y., Zhang, C., Zhang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.03428
Empowering LLMs with Pseudo-Untrimmed Videos for Audio-Visual Temporal Understanding
Tang, Y. Y., Shimada, D., Bi, J., Feng, M., Hua, H., & Xu, C. (2024) · arXiv · doi:10.48550/arxiv.2403.16276
LaGR-SEQ: Language-Guided Reinforcement Learning with Sample-Efficient Querying
Karimpanal, T. G., Semage, L. B., Rana, S., Le, H., Tran, T., Gupta, S., & Venkatesh, S. (2023) · arXiv · doi:10.48550/arxiv.2308.13542
de Wynter, A. (2024) · arXiv · doi:10.48550/arxiv.2403.05468
Research works printed but not linked: 83
- A. Brohan, Y. Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian et al., Do as i can, not as i say: Grounding language in robotic affordances. in Conference on robot learning (PMLR, Atlanta, GA, USA, 2023), pp. 287–318
- K. Li, A.K. Hopkins, D. Bau, F. Viégas, H. Pfister, M. Wattenberg, Emergent world representations: Exploring a sequence model trained on a synthetic task. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
- N. Renella, M. Eger, Towards automated video game commentary using generative ai. in Proceedings of the Experimental Artificial Intelligence in Games Workshop co-located with the 19th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2023) (AAAI Press, Washington, DC, USA, 2023)
- Z. Ma, Y. Mei, Z. Su, Understanding the benefits and challenges of using large language model-based conversational agents for mental well-being support. in AMIA Annual Symposium Proceedings (American Medical Informatics Association, New Orleans, LA, 2023). pp. 1105
- G. Weiss, Multiagent systems: a modern approach to distributed artificial intelligence (MIT Press, Cambridge, MA, United States, 1999), pp. 547
- H.V.D. Parunak, Applications of distributed artificial intelligence in industry. Found. Distrib. Artif. Intell. 2(1), 18 (1996)
- I. Dasgupta, C. Kaeser-Chen, K. Marino, A. Ahuja, S. Babayan, F. Hill, R. Fergus, Collaborating with language models for embodied reasoning. in NeurIPS 2022 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States of America, 2022)
- S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S.K.S. Yau, Z. Lin, L. Zhou et al., Metagpt: Meta programming for multi-agent collaborative framework. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- H. Zhang, W. Du, J. Shan, Q. Zhou, Y. Du, J.B. Tenenbaum, T. Shu, C. Gan, Building cooperative embodied agents modularly with large language models. in NeurIPS 2023 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States, 2023)
- Y. Du, S. Li, A. Torralba, J.B. Tenenbaum, I. Mordatch, Improving factuality and reasoning in language models through multiagent debate. in Proceedings of the 41st International Conference on Machine Learning. Proceedings of Machine Learning Research (Vienna, Austria, 2024). 235, pp. 11733–11763. PMLR.
- C.M. Chan, W. Chen, Y. Su, J. Yu, W. Xue, S. Zhang, J. Fu, Z. Liu, Chateval: Towards better llm-based evaluators through multi-agent debate. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- J. Light, M. Cai, S. Shen, Z. Hu, Avalonbench: Evaluating llms playing the game of avalon. in NeurIPS 2023 Foundation Models for Decision Making Workshop (2023) (New Orleans, United States, 2023)
- G. Mukobi, H. Erlebach, N. Lauffer, L. Hammond, A. Chan, J. Clifton, Welfare diplomacy: Benchmarking language model cooperation. in NeurIPS 2023 Socially Responsible Language Modelling Research workshop (New Orleans, United States, 2023)
- G.V. Aher, R.I. Arriaga, A.T. Kalai, Using large language models to simulate multiple humans and replicate human subject studies. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 337–371
- A. Zhang, Y. Chen, L. Sheng, X. Wang, T.S. Chua, On generative agents in recommendation. in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (2024) (Association for Computing Machinery, New York, NY, United State), pp. 1807–1817 · DOI · Paper page
- L. Wang, J. Zhang, X. Chen, Y. Lin, R. Song, W.X. Zhao, J.R. Wen, Recagent: A novel simulation paradigm for recommender systems. arXiv preprint (2023) arXiv:2306.0255
- M. Shridhar, X. Yuan, M.A. Côté, Y. Bisk, A. Trischler, M. Hausknecht, Alfworld: Aligning text and embodied environments for interactive learning. International Conference on Learning Representation (2021)
- D. Driess, F. Xia, M.S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu et al., Palm-e: An embodied multimodal language model. in Proceedings of the 40th International Conference on Machine Learning (Honolulu, Hawaii, USA, 2023). ICML’23. JMLR.org 340, 20.
- W. Tan, Z. Ding, W. Zhang, B. Li, B. Zhou, J. Yue, H. Xia, J. Jiang, L. Zheng, X. Xu et al., Towards general computer control: A multimodal agent for red dead redemption ii as a case study. in ICLR 2024 Workshop on Large Language Model (LLM) Agents (Vienna, Austria, 2024).
- S. Reed, K. Zolna, E. Parisotto, S.G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, Y. Sulsky, J. Kay, J.T. Springenberg et al., A generalist agent. Transactions on Machine Learning Research (2022).
- S. Zheng, Y. Feng, Z. Lu et al., Steve-eye: Equipping llm-based embodied agents with visual perception in open worlds. in The Twelfth International Conference on Learning Representations (Kigali, Rwanda, 2023)
- P.F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, D. Amodei, Deep reinforcement learning from human preferences. Adv. Neural Inf. Process. Syst. 30 (2017). pp. 4302-4310
- J. Wei, M. Bosma, V.Y. Zhao, K. Guu, A.W. Yu, B. Lester, N. Du, A.M. Dai, Q.V. Le, Finetuned language models are zero-shot learners. in International Conference on Learning Representations (2022) (Virtual Event).
- W. Li, L. Zhu, L. Wen, Y. Yang, Decap: Decoding clip latents for zero-shot captioning via text-only training. in International Conference on Learning Representations (2023) (Kigali, Rwanda, 2023).
- S. Zhao, X. Wang, L. Zhu, Y. Yang, Test-time adaptation with clip reward for zero-shot generalization in vision-language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- J. Chen, H. Guo, K. Yi, B. Li, M. Elhoseiny, Visualgpt: Data-efficient image captioning by balancing visual input and linguistic knowledge from pretraining. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE, New Orleans, LA, USA, 2022), pp. 18030–18040
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., An image is worth 16x16 words: Transformers for image recognition at scale. in International Conference on Learning Representations (2021) (Vienna, Austria).
- A. Van Den Oord, O. Vinyals et al., Neural discrete representation learning. Adv. Neural Inf. Process. Syst. 30 (2017). pp. 6309-6318
- S. Mehta, M. Rastegari, Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. in International Conference on Learning Representations (2022) (Virtual Event).
- J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. in International conference on machine learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 19730–19742
- D. Zhu, J. Chen, X. Shen, X. Li, M. Elhoseiny, Minigpt-4: Enhancing vision-language understanding with advanced large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
- Y. Su, T. Lan, H. Li, J. Xu, Y. Wang, D. Cai, Pandagpt: One model to instruction-follow them all. in Proceedings of the 1st Workshop on Taming Large Language Models: Controllability in the era of Interactive Assistants! (Association for Computational Linguistics, Prague, Czech Republic, 2023), pp. 11-23.
- Z. Peng, W. Wang, L. Dong, Y. Hao, S. Huang, S. Ma, F. Wei, Kosmos-2: Grounding multimodal large language models to the world. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
- L. Qian, J. Li, Y. Wu, Y. Ye, H. Fei, T.S. Chua, Y. Zhuang, S. Tang, Momentor: Advancing video large language model with fine-grained temporal reasoning. in Forty-first International Conference on Machine Learning (Vienna, Austria, 2024).
- Z. Yang, G. Chen, X. Li, W. Wang, Y. Yang, Doraemongpt: Toward understanding dynamic scenes with large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
- L.H. Marshall, H.W. Magoun, Discoveries in the human brain: neuroscience prehistory, brain structure, and function (Springer Science & Business Media, Humana Totowa, NJ, 2013)
- L. Zheng, R. Wang, X. Wang, B. An, Synapse: Trajectory-as-exemplar prompting with memory for computer control. in The Twelfth International Conference on Learning Representations (Messe Wien Exhibition and Congress Center, Vienna, Austria, 2023)
- J. Kang, R. Laroche, X. Yuan, A. Trischler, X. Liu, J. Fu, Think before you act: Decision transformers with internal working memory. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
- A. Modarressi, A. Imani, M. Fayyaz, H. Schütze, Ret-llm: Towards a general read-write memory for large language models. in ICLR 2024 Workshop: How Far Are We From AGI (Vienna, Austria, 2024)
- T. Silver, V. Hariprasad, R.S. Shuttleworth, N. Kumar, T. Lozano-Pérez, L.P. Kaelbling, Pddl planning with pretrained large language models. in NeurIPS 2022 foundation models for decision making workshop (New Orleans, Louisiana, USA, 2022)
- Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, H. Soh, Translating natural language to planning goals with large-language models. in The International Journal of Robotics Research (2020). 2019, pp. 1
- H. Liu, C. Li, Q. Wu, Y.J. Lee, Visual instruction tuning. Adv. Neural Inf. Process. Syst. 36(2024). pp. 34892-34916
- L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, G. Neubig, Pal: Program-aided language models. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 10764–10799
- X. Cheng, D. Luo, X. Chen, L. Liu, D. Zhao, R. Yan, Lift yourself up: Retrieval-augmented text generation with self-memory. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 43780-43799
- W. Yu, D. Iter, S. Wang, Y. Xu, M. Ju, S. Sanyal, C. Zhu, M. Zeng, M. Jiang, Generate rather than retrieve: Large language models are strong context generators. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
- A. Asai, Z. Wu, Y. Wang, A. Sil, H. Hajishirzi, Self-rag: Learning to retrieve, generate, and critique through self-reflection. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- F. Xu, W. Shi, E. Choi, Recomp: Improving retrieval-augmented lms with compression and selective augmentation. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- E. Mitchell, C. Lin, A. Bosselut, C.D. Manning, C. Finn, Memory-based model editing at scale. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 15817–15831
- J. Maynez, S. Narayan, B. Bohnet, R. McDonald, On faithfulness and factuality in abstractive summarization. in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (Association for Computational Linguistics, Online, 2022), pp. 1906-1919
- J. Thorne, A. Vlachos, Automated fact checking: Task formulations, methods and future directions. in Proceedings of the 27th International Conference on Computational Linguistics (Association for Computational Linguistics, Santa Fe, New Mexico, USA, 2018), pp. 3346-3359
- C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu et al., Lima: Less is more for alignment. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 55006-55021
- Z. Wu, Y. Hu, W. Shi, N. Dziri, A. Suhr, P. Ammanabrolu, N.A. Smith, M. Ostendorf, H. Hajishirzi, Fine-grained human feedback gives better rewards for language model training. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 59008-59033
- Y.S. Chuang, Y. Xie, H. Luo, Y. Kim, J. Glass, P. He, Dola: Decoding by contrasting layers improves factuality in large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- P.C. Wason, P.N. Johnson-Laird, Psychology of reasoning: Structure and content. Harvard University Press, Cambridge, MA, USA, 1972. 86, pp. 246-252
- D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le et al., Least-to-most prompting enables complex reasoning in large language models. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
- M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman et al., Do as i can, not as i say: Grounding language in robotic affordances. in Conference on Robot Learning (Atlanta, GA, United States, 2023), pp. 287–318. PMLR
- S.S. Raman, V. Cohen, E. Rosen, I. Idrees, D. Paulius, S. Tellex, Planning with large language models via corrective re-prompting. in NeurIPS 2022 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, USA, 2022)
- B.Y. Lin, Y. Fu, K. Yang, F. Brahman, S. Huang, C. Bhagavatula, P. Ammanabrolu, Y. Choi, X. Ren, Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 23813-23825
- S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, K. Narasimhan, Tree of thoughts: Deliberate problem solving with large language models. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 11809-11822
- W. Huang, P. Abbeel, D. Pathak, I. Mordatch, Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 9118–9147
- K. Nottingham, P. Ammanabrolu, A. Suhr, Y. Choi, H. Hajishirzi, S. Singh, R. Fox, Do embodied agents dream of pixelated sheep: Embodied decision making using language guided world modelling. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 26311–26325
- J. Ruan, Y. Chen, B. Zhang, Z. Xu, T. Bao, G. Du, S. Shi, H. Mao, X. Zeng, R. Zhao, Tptu: Task planning and tool usage of large language model-based ai agents. in NeurIPS 2023 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States of America, 2023)
- Z. Zhao, E. Wallace, S. Feng, D. Klein, S. Singh, Calibrate before use: Improving few-shot performance of language models. in International conference on machine learning (PMLR, Virtual Event, 2021), pp. 12697–12706
- A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
- T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, T. Scialom, Toolformer: Language models can teach themselves to use tools. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 68539-68551
- M. Carroll, R. Shah, M.K. Ho, T. Griffiths, S. Seshia, P. Abbeel, A. Dragan, On the utility of learning about humans for human-ai coordination. Adv. Neural Inf. Process. Syst. 32(2019), pp. 5174-5185
- X. Ning, Z. Lin, Z. Zhou, Z. Wang, H. Yang, Y. Wang, Skeleton-of-thought: Prompting llms for efficient parallel generation. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- S. Srivastava, C. Li, M. Lingelbach, R. Martín-Martín, F. Xia, K.E. Vainio, Z. Lian, C. Gokmen, S. Buch, K. Liu et al., Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments. in Conference on robot learning (PMLR, London, UK, 2021), pp. 477–490
- C. Colas, L. Teodorescu, P.Y. Oudeyer, X. Yuan, M.A. Côté, Augmenting autotelic agents with large language models. in Conference on Lifelong Learning Agents (PMLR, McGill University, Montréal, Québec, Canada, 2023), pp. 205–226
- L. Zhu, L. Hu, J. Lin, S. Han, Lift: Efficient layer-wise fine-tuning for large model models. in Proceedings of the 31st ACM International Conference on -Multimedia (Association for Computing Machinery, New York, NY, United States, 2023), pp. 4678-4687
- T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettlemoyer, Qlora: Efficient finetuning of quantized llms. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 10088-10115
- A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, S. Bowman, Superglue: A stickier benchmark for general-purpose language understanding systems. Adv. Neural Inf. Process. Syst. 32 (2019). pp. 3266-3280
- A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang et al., Self-refine: Iterative refinement with self-feedback. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 46534-46594
- N. Shinn, B. Labash, A. Gopinath, Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint (2023) arXiv: 2303.11366 · arXiv
- E. Brooks, L. Walls, R.L. Lewis, S. Singh, Large language models can implement policy iteration. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 30349-30366
- T. Carta, C. Romac, T. Wolf, S. Lamprier, O. Sigaud, P.Y. Oudeyer, Grounding large language models in interactive environments with online reinforcement learning. in International Conference on Machine Learing (PMLR, Honolulu, Hawaii, USA, 2023), pp. 3676–3713
- S. Sudhakaran, M. González-Duque, M. Freiberger, C. Glanois, E. Najarro, S. Risi, Mariogpt: Open-ended text2level generation through large language models. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 54213-54227
- F. Cardoso Durier da Silva, R. Vieira, A.C. Garcia, Can machines learn to detect fake news? a survey focused on social media, in 2021 IEEE 4th International Conference on Computing, Power and Communication Technologies (Kuala Lumpur, Malaysia, 2021), pp.1-7
- B. Buchanan, A. Lohn, M. Musser, K. Sedova, Truth, lies, and automation. Cent. Secur. Emerg. Technol. 1(1), 2 (2021)
- N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., Extracting training data from large language models. in 30th USENIX Security Symposium (USENIX Security 21) (USENIX, Berkeley, CA, 2021), pp. 2633–2650
- Z. Xi, T. Du, C. Li, R. Pang, S. Ji, J. Chen, F. Ma, T. Wang, Defending pre-trained language models as few-shot learners against backdoor attacks. Adv. Neural Inf. Process. Syst. 36(2024), pp. 32748-32764
- E. Ostrom, Tragedy of the commons. New Palgrave Dictionary Econ. 2, 1–4 (2008)
- T. Sun, Y. Shao, H. Qian, X. Huang, X. Qiu, Black-box tuning for language-model-as-a-service. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 20841–20855
Cited by
Liu, S., McCoy, A. B., & Wright, A. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf008 · Cited by 200
“57 For example, clinicians and patients have diverse information access needs, some needing to analyze text from a knowledge base, others needing to incorporate structured data from an EHR.”
Bandi, A., Kongari, B., Naguru, R., Pasnoor, S., & Vilipala, S. V. (2025) · Future Internet · doi:10.3390/fi17090404 · Cited by 207
A Survey on the Memory Mechanism of Large Language Model based Agents
Zhang, Z., Bo, X., Ma, C., Li, R., Chen, X., Dai, Q., Zhu, J., Dong, Z., & Wen, J.-R. (2024) · arXiv · doi:10.48550/arxiv.2404.13501 · Cited by 149
Yigit, Y., Ferrag, M. A., Ghanem, M. C., Sarker, I. H., Maglaras, L. A., Chrysoulas, C., Moradpoor, N., Tihanyi, N., & Janicke, H. (2025) · Sensors · doi:10.3390/s25061666 · Cited by 69
“In that case, specialized AI agents can instantly coordinate across multiple teams: public utilities, government agencies, and private contractors, ensuring everyone has the latest threat intelligence and situational updates [118].”
Evaluation and Benchmarking of LLM Agents: A Survey
Mohammadi, M., Li, Y., Lo, J., & Yip, W. (2025) · Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · doi:10.1145/3711896.3736570 · Cited by 74
Metadata
- Authors
- Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y.
- Affiliations
- Wuhan University (CN)
- Zhejiang University (CN)
- Corresponding author
- Li, Xinyi
- Wang, Sai
- Zeng, Siqi
- Wu, Yu
- Yang, Yi
- Venue
- Vicinagearth
- Published
- 2024-10-08
- Volume
- 1
- Issue
- 1
- ISSN
- 3005-060X (ISSN-L)
- OpenAlex
- W4403203873
- Semantic Scholar
- fc8ce12d6186ddaa797e2b36d5e8eb7921425308
- Cited by
- 404
- References
- 252
- Citations by year
- 2026: 254 · 2025: 147 · 2024: 2
- FWCI
- 59.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Vicinagearth. (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- multi-agent LLM systems, large language models, multi-agent systems, artificial general intelligence, agent architecture, agent perception, agent interaction, agent evolution, problem solving, world simulation, autonomous systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.