A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges
Open accessThe DOI matches its Crossref record; 322 of the 322 references it cites were found in a registry or an index.
Abstract
The pursuit of more intelligent and credible autonomous systems, akin to human society, has been a long-standing endeavor for humans. Leveraging the exceptional reasoning and planning capabilities of large language models (LLMs), LLM-based agents have been proposed and have achieved remarkable success across a wide array of tasks. Notably, LLM-based multi-agent systems (MAS) are considered a promising pathway towards realizing general artificial intelligence that is equivalent to or surpasses human-level intelligence. In this paper, we present a comprehensive survey of these studies, offering a systematic review of LLM-based MAS. Adhering to the workflow of LLM-based multi-agent systems, we synthesize a general structure encompassing five key components: profile, perception, self-action, mutual interaction, and evolution. This unified framework encapsulates much of the previous work in the field. Furthermore, we illuminate the extensive applications of LLM-based MAS in two principal areas: problem-solving and world simulation. Finally, we discuss in detail several contemporary challenges and provide insights into potential future directions in this domain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“MAS, in which multiple autonomous agents collaborate as a team, are emerging as a powerful paradigm for biomedical and clinical data analysis [1,2].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, current RAG models face challenges related to frequent knowledge updates [2,11,21,22].”
From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [11]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Evaluating multi-agent collaboration in LLM-based systems requires different methodologies compared to traditional reinforcement learning-driven coordination [7, 48, 89].”
From Evaluation and Benchmarking of LLM Agents: A Survey (2025) · cites this paper as [48]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 3×
“Feedback information can take on multiple forms, such as prompt contexts [240], embeddings [287], tokens [288, 291–293].”
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680 · cited 1×
“This information is then converted into intermediate representations through perception modules, which then determine the agent’s autonomous decision-making outcomes and behavioral responses [62, 63].”
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
Self-citationOvercoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 2×
“(2) Hallucination : LLMs can produce information that sounds plausible but is factually incorrect or nonsensical, known as hallucination [199].”
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · cited 6×
“…external knowledge bases to enhance information accuracy [143, 238], increasing model transparency to foster understanding of decision processes [86, 202, 203], developing fact-checking systems to verify outputs [200, 201], and designing hallucination detection tasks [325, 326] to evaluate and…”
InfluentialTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 3×
“Instruct-GPT [205] learns by collecting evaluations from human annotators on the model’s output, which include preference rankings for the text generated by the model, serving as a feedback signal.”
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 2×
“Beyond text, visual, and audio data, LLMs often need to utilize specialized knowledge from specific domains such as scientific research [160, 161], medical information [162–164], or technical specifications [165–167].”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 2×
“Retrieval-augmented generation methods [175, 183, 208] enhances the accuracy of language generation by introducing additional source material and providing mechanisms to check for inconsistencies between the generated response and the source material.”
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · cited 2×
“Feedback information can take on multiple forms, such as prompt contexts [240], embeddings [287], tokens [288, 291–293].”
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092 · cited 2×
“Feedback information can take on multiple forms, such as prompt contexts [240], embeddings [287], tokens [288, 291–293].”
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 2×
“Database and knowledge base queries involve accessing structured data from repositories like Google Knowledge Graph, PubMed [168], and other domain-specific databases.”
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224 · cited 1×
“For instance, a language model might incorrectly predict due to the frequent co-occurrence of function words with specific labels in the training set [211].”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 6×
“For example, Generative Agent [28], immersed in the context of a software company, utilizes authored natural language descriptions to define each agent’s identity, encompassing their occupation and inter-agent relationships, serving as seed memories.”
InfluentialGenerative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
MLP-Mixer: An all-MLP Architecture for Vision
Tolstikhin, I., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.01601 · Cited by 2,087 · cited 1×
“Another representative approach, exemplified by works like ViT and VQVAE, encodes visual information into visual tokens typically based on transformers, similar to how LLMs process textual information by converting text into discrete tokens [99–102].”
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · cited 4×
“This mechanism is vividly demonstrated in complex task planning and robotic simulations within dynamic environmental scenarios [17, 132, 268].”
InfluentialA Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834 · cited 1×
“For example, P-Tuning [288] stabilizes the training process by combining continuous prompt embeddings with discrete prompts and has achieved significant performance improvements in natural language understanding tasks such as LAMA [289] and SuperGLUE [290].”
Self-citationLarge Language Model based Multi-Agents: A Survey of Progress and Challenges
Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., & Zhang, X. (2024) · arXiv · doi:10.48550/arxiv.2402.01680 · Cited by 1,713
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · cited 1×
“For instance, previous research efforts like Flamingo [112] and VideoAgent [113] extract video frames at certain frequencies and rigorously follow the chronological order to perform visual understanding on each frame.”
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
Wason, P. C. (1968) · Quarterly Journal of Experimental Psychology · doi:10.1080/14640746808400161 · Cited by 1,476 · cited 1×
“These abilities are fundamental human capabilities that underpin problem-solving, decision-making, and critical analysis, forming the bedrock of human perception and social interaction [216–218].”
Expel: LLM agents are experiential learners
Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., & Huang, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29936 · Cited by 1,451
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
Dai, W., Li, J., Li, D., Tiong, A. M. H., Zhao, J., Wang, W., Li, B., Fung, P., & Hoi, S. C. H. (2023) · neural information processing systems · doi:10.52202/075280-2142 · Cited by 1,359
Logothetis, N. K., & Sheinberg, D. L. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.19.030196.003045 · Cited by 1,351 · cited 1×
“This information is then converted into intermediate representations through perception modules, which then determine the agent’s autonomous decision-making outcomes and behavioral responses [62, 63].”
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
Zhang, K., Yang, Z., & Başar, T. (2021) · Studies in Systems, Decision and Control · doi:10.1007/978-3-030-60990-0_12 · Cited by 1,264
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264 · cited 1×
“Leveraging the robust text-based reasoning and planning capacity inherent in large language models (LLMs) [65, 66], it becomes essential for LLM-based agents to effectively perceive complex environments, execute intricate tasks, and engage in human-like interactions.”
Meshed-Memory Transformer for Image Captioning
Cornia, M., Stefanini, M., Baraldi, L., & Cucchiara, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01059 · Cited by 1,259
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245 · cited 1×
“(3 ) Bias : Bias in LLMs manifests as the propagation and amplification of discriminatory tendencies present in training data, such as racial and gender biases, leading to unfair or harmful outputs [327–330].”
Multiagent systems: A survey from a machine learning perspective
Stone, P., & Veloso, M. (2000) · Autonomous Robots · doi:10.1023/a:1008942012299 · Cited by 1,218 · cited 1×
“Similarly, In [26], four different MAS prototypes were summarized based on the dimensions of agent heterogeneity and communication level.”
AST: Audio Spectrogram Transformer
Gong, Y., Chung, Y.-A., & Glass, J. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-698 · Cited by 1,159
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · cited 2×
“This mechanism is vividly demonstrated in complex task planning and robotic simulations within dynamic environmental scenarios [17, 132, 268].”
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · cited 3×
“Existing multi-agent cooperation models are mainly divided into unordered cooperation and ordered cooperation [20, 262].”
Flamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966
Liu, X., Zheng, Y., Du, Z., Ding, M., Qian, Y., Yang, Z., & Tang, J. (2021) · arXiv · doi:10.48550/arxiv.2103.10385 · Cited by 908
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · cited 1×
“For example, the TruthfulQA [207] task aims to detect whether the model mimics human false statements.”
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2021) · arXiv · doi:10.48550/arxiv.2109.07958 · Cited by 899
A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME
Salih, A. M., Raisi‐Estabragh, Z., Galazzo, I. B., Radeva, P., Petersen, S. E., Lekadir, K., & Menegaz, G. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202400304 · Cited by 897
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 798 · cited 1×
“…a deeper understanding of textual nuances (Ye et al.: Tool-eyes: Fine-grained evaluation for tool learning capabilities of large language models in real-world scenarios, under review) [92], which helps deduce the speaker’s preferences and leads to more personalized and accurate agent responses.”
Augmenting large language models with chemistry tools
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00832-8 · Cited by 794
Out of One, Many: Using Language Models to Simulate Human Samples
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023) · Political Analysis · doi:10.1017/pan.2023.2 · Cited by 783 · cited 1×
“Similarly, In [61], it is focused on assigning roles to GPT-3 based on the demographic backgrounds-such as race and ethnicity.”
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
Ben Zaken, E., Goldberg, Y., & Ravfogel, S. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.1 · Cited by 765
AST: Audio Spectrogram TransformerPreprint
Gong, Y., Chung, Y.-A., & Glass, J. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.01778 · Cited by 738
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2338 · Cited by 686
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.45 · Cited by 682 · cited 1×
“Each interaction message usually signifies an independent information exchange between two specific agents, characterized by individual specificity and temporal relevance, given the multi-round nature of inter-agent communication [76, 77].”
AudioPaLM: A Large Language Model That Can Speak and ListenPreprint
Rubenstein, P. K., Asawaroengchai, C., Nguyen, D. D., Bapna, A., Borsos, Z., Quitry, F. D. C., Chen, P., Badawy, D. E., Han, W., Kharitonov, E., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.12925 · Cited by 674
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2021) · arXiv · doi:10.48550/arxiv.2104.08786 · Cited by 615 · cited 1×
“Additionally, models might develop preferences based on the order of training samples, such as the position of answers in question-answering tasks potentially influencing model judgments [212].”
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.556 · Cited by 610 · cited 1×
“Additionally, models might develop preferences based on the order of training samples, such as the position of answers in question-answering tasks potentially influencing model judgments [212].”
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
Zhang, H., Li, X., & Bing, L. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2023.emnlp-demo.49 · Cited by 586
Self-citationTraining Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575 · cited 3×
“Beyond text, visual, and audio data, LLMs often need to utilize specialized knowledge from specific domains such as scientific research [160, 161], medical information [162–164], or technical specifications [165–167].”
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29720 · Cited by 556 · cited 1×
“The Graph of Thought [233] expands the tree-like reasoning structure in ToT to graph structures, resulting in more robust prompting strategies.”
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.09687 · Cited by 556
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Xu, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.01219 · Cited by 544 · cited 1×
“Hallucination refers to the phenomenon where LLM agents generate text that deviates from reality [196–198].”
A systematic evaluation of large language models of code
Xu, F. F., Alon, U., Neubig, G., & Hellendoorn, V. J. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534862 · Cited by 540
Artificial Intelligence (AI) Ethics: Ethics of AI and Ethical AI
Siau, K., & Wang, W. (2020) · Journal of Database Management · doi:10.4018/jdm.2020040105 · Cited by 533 · cited 1×
“Additionally, establishing AI ethics and policies guides the development and deployment of agent systems, ensuring they operate within ethical and legal frameworks, thereby reducing the risk of misuse [354, 355].”
A Survey on Automated Fact-Checking
Guo, Z., Schlichtkrull, M., & Vlachos, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00454 · Cited by 500 · cited 2×
“…to enhance information accuracy [143, 238], increasing model transparency to foster understanding of decision processes [86, 202, 203], developing fact-checking systems to verify outputs [200, 201], and designing hallucination detection tasks [325, 326] to evaluate and improve model performance.”
Active Retrieval Augmented Generation
Jiang, Z., Xu, F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.495 · Cited by 497
Self-citationChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.810 · Cited by 490 · cited 7×
“For instance, agents in ChatDev [30] collaborate through communication to collectively accomplish software development tasks.”
InfluentialTowards Generalist Biomedical AI
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang, P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I., et al. (2024) · NEJM AI · doi:10.1056/aioa2300138 · Cited by 483
Towards Generalist Biomedical AI
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang, P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.14334 · Cited by 483 · cited 1×
“Beyond text, visual, and audio data, LLMs often need to utilize specialized knowledge from specific domains such as scientific research [160, 161], medical information [162–164], or technical specifications [165–167].”
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
Shen, Y., Song, K., Tan, X., Li, D., Lu, W., & Zhuang, Y. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1657 · Cited by 475
Bao, K., Zhang, J., Zhang, Y., Wang, W., Feng, F., & He, X. (2023) · Proceedings of the 17th ACM Conference on Recommender Systems · doi:10.1145/3604915.3608857 · Cited by 469 · cited 2×
“…(Financial trading) Horton et al. [307], Akata et al. [308], Guo et al. [256], CompeteAI [257] Recommender Systems Zhang et al. [309], TALLRec [310], Hou et al. [311], Liu et al. [312], Chat-Rec [313], Dai et al. [314], KAR [315], GENRE [316], LLMRec [317], RecAgent [60], RecSim [318],…”
VideoAgent: Long-form Video Understanding with Large Language Model as Agent
Wang, X.-H., Zhang, Y., Zohar, O., et al. (2024) · European Conference on Computer Vision · doi:10.48550/arxiv.2403.10517 · Cited by 442
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer
Pfeiffer, J., Vulić, I., Gurevych, I., & Ruder, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.617 · Cited by 437
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Song, C. H., Sadler, B. M., Wu, J., Chao, W.-L., Washington, C., & Su, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00280 · Cited by 435 · cited 1×
“Occasionally, it serves as supplementary background information [35, 75], influencing both the agents’ self-interactions and their interactions with one another.”
Audiolm: A language modeling approach to audio generation
Borsos, Z., Marinier, R., Vincent, D., Kharitonov, E., Pietquin, O., Sharifi, M., Roblek, D., Teboul, O., Grangier, D., Tagliasacchi, M., et al. (2023) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3288409 · Cited by 432 · cited 1×
“Previous work, such as WavJourney [114] and AudioLM [115], has often employed audio encoders to convert audio into discrete tokens, which are then integrated into the LLM’s knowledge base.”
Towards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.67 · Cited by 428
Towards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2022) · arXiv · doi:10.48550/arxiv.2212.10403 · Cited by 428
Gender bias and stereotypes in Large Language Models
Kotek, H., Dockum, R., & Sun, D. (2023) · Proceedings of The ACM Collective Intelligence Conference · doi:10.1145/3582269.3615599 · Cited by 381
AutoAgents: A Framework for Automatic Agent Generation
Chen, G., Dong, S., Shu, Y., Zhang, G., Sesay, J., Karlsson, B. F., Fu, J., & Shi, Y. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/3 · Cited by 379
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
Tang, X., Zou, A., Zhang, Z., Li, Z., Zhao, Y., Zhang, X., Cohan, A., & Gerstein, M. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.33 · Cited by 376
Multimodal Few-Shot Learning with Frozen Language Models
Tsimpoukelli, M., Menick, J., Cabi, S., Eslami, S. M. A., Vinyals, O., & Hill, F. (2021) · arXiv · doi:10.48550/arxiv.2106.13884 · Cited by 355
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.557 · Cited by 349
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022) · arXiv · doi:10.48550/arxiv.2212.10509 · Cited by 349 · cited 1×
“Beyond the commonly used single-step retrieval, RAG incorporates three types of retrieval enhancement processes: iterative retrieval [181], recursive retrieval [182], and adaptive retrieval [183, 184], which are designed to improve efficiency and accuracy in solving complex queries [185].”
Towards Expert-Level Medical Question Answering with Large Language Models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.09617 · Cited by 338
Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?
Horton, J., Filippas, A., & Manning, B. (2023) · National Bureau of Economic Research · doi:10.3386/w31122 · Cited by 329
Large Language Models are Zero-Shot Rankers for Recommender Systems
Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J., & Zhao, W. X. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-56060-6_24 · Cited by 327
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., & Kaplan, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08073 · Cited by 326 · cited 1×
“Feedback information can take on multiple forms, such as prompt contexts [240], embeddings [287], tokens [288, 291–293].”
Approaches to studying formal and everyday reasoning.
Galotti, K. M. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.3.331 · Cited by 322 · cited 1×
“These abilities are fundamental human capabilities that underpin problem-solving, decision-making, and critical analysis, forming the bedrock of human perception and social interaction [216–218].”
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., & Tu, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.19118 · Cited by 320
Self-citationCamel: Communicative agents for" mind" exploration of large language model society
Li, G., Hammoud, H., Itani, H., Khizbullin, D., & Ghanem, B. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2264 · Cited by 314
Yang, Y., Zhuang, Y., & Pan, Y. (2021) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2100463 · Cited by 309 · cited 1×
“After taking into account various sources of information messages, we also emphasize the importance of enabling agents to receive and comprehend multiple modalities of information through perception modules [82].”
Self-citationA Survey on Fairness in Large Language ModelsPreprint
Li, Y., Du, M., Song, R., Wang, X., & Ying, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.10149 · Cited by 299
A Survey of Adversarial Defenses and Robustness in NLP
Goyal, S., Doddapaneni, S., Khapra, M. M., & Ravindran, B. (2023) · ACM Computing Surveys · doi:10.1145/3593042 · Cited by 289 · cited 1×
“Adversarial training and prompting [352, 353] enhance the robustness of agents, enabling them to withstand malicious inputs and attacks.”
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · arXiv · doi:10.48550/arxiv.2101.00190 · Cited by 289
Self-citationInner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287 · cited 3×
“…LIBRO [304], PENTESTGPT [305] Industrial Engineering Mehta et al. [2], Xia et al. [3], Li et al. [4] Embodied Agents SayCan [8], Inner Monologue [9], TidyBot [10], RoCo [34], CoELA [35] Science Experiments Ghafarollahi et al. [5], Boiko et al. [6], ChemCrow [7] Science Debate Du et al. [36],…”
InfluentialAcademic integrity and artificial intelligence: is ChatGPT hype, hero or heresy?
Currie, G. M. (2023) · Seminars in Nuclear Medicine · doi:10.1053/j.semnuclmed.2023.04.008 · Cited by 283 · cited 1×
“Despite the powerful capabilities of MAS and LLMs, they can be maliciously exploited for large-scale disinformation generation [340–342], cyber-attacks [343–346], and other inappropriate behaviors [347, 348].”
Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.212 · Cited by 282
Self-citationWebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272 · cited 1×
“Several studies have integrated LLMs with specific tools like web search [171], compiler [172], and calculator [161].”
LLMRec: Large Language Models with Graph Augmentation for Recommendation
Wei, W., Ren, X., Tang, J., Wang, Q., Su, L., Cheng, S., Wang, J., Yin, D., & Huang, C. (2024) · Proceedings of the 17th ACM International Conference on Web Search and Data Mining · doi:10.1145/3616855.3635853 · Cited by 268
Towards a Unified View of Parameter-Efficient Transfer Learning
He, J., Zhou, C., Ma, X., Berg-Kirkpatrick, T., & Neubig, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04366 · Cited by 268
Self-collaboration code generation via ChatGPT
Dong, Y., Jiang, X., Jin, Z., & Li, G. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672459 · Cited by 256 · cited 4×
“Through ongoing interactions, agents continuously assess and enhance their knowledge base, [32], ChatEDA [303], LIBRO [304], PENTESTGPT [305] Industrial Engineering Mehta et al. [2], Xia et al. [3], Li et al. [4] Embodied Agents SayCan [8], Inner Monologue [9], TidyBot [10], RoCo [34], CoELA [35]…”
InfluentialSelf-collaboration Code Generation via ChatGPT
Dong, Y., Jiang, X., Jin, Z., & Li, G. (2023) · arXiv · doi:10.48550/arxiv.2304.07590 · Cited by 256
MemoryBank: Enhancing Large Language Models with Long-Term Memory
Zhong, W., Guo, L., Gao, Q., Ye, H., & Wang, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.10250 · Cited by 253 · cited 3×
“For example, several methods adopt embedding vectors to represent memory sections and history dialogues [129, 134, 135].”
Gender Bias in Neural Natural Language Processing
Lu, K., Mardziel, P., Wu, F., Amancharla, P., & Datta, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-62077-6_14 · Cited by 250
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Liu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.00234 · Cited by 248
LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead
He, J., Treude, C., & Lo, D. (2024) · arXiv · doi:10.48550/arxiv.2404.04834 · Cited by 246 · cited 2×
“They can also interact with other agents by simulating real-world collaboration patterns such as cooperation, competition, and hierarchical organization [19, 23], thereby enhancing overall collaborative efficiency.”
RecSim: A Configurable Simulation Platform for Recommender SystemsPreprint
Ie, E., Hsu, C., Mladenov, M., Jain, V., Narvekar, S., Wang, J., Wu, R., & Boutilier, C. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04847 · Cited by 244
From ethical AI frameworks to tools: a review of approaches
Prem, E. (2023) · AI and Ethics · doi:10.1007/s43681-023-00258-9 · Cited by 242 · cited 1×
“Additionally, establishing AI ethics and policies guides the development and deployment of agent systems, ensuring they operate within ethical and legal frameworks, thereby reducing the risk of misuse [354, 355].”
Human-level play in the game of Diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models
Hu, Z., Wang, L., Lan, Y., Xu, W., Lim, E.-P., Bing, L., Xu, X., Poria, S., & Lee, R. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.319 · Cited by 238
LongT5: Efficient Text-To-Text Transformer for Long Sequences
Guo, M., Ainslie, J., Uthus, D., Ontanon, S., Ni, J., Sung, Y.-H., & Yang, Y. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.55 · Cited by 230
Self-citationLongT5: Efficient Text-To-Text Transformer for Long Sequences
Guo, M., Ainslie, J., Uthus, D., Ontanon, S., Ni, J., Sung, Y.-H., & Yang, Y. (2021) · arXiv · doi:10.48550/arxiv.2112.07916 · Cited by 230
Self-citationGorilla: Large Language Model Connected with Massive APIs
Patil, S. G., Zhang, T., Wang, X., & Gonzalez, J. E. (2023) · arXiv · doi:10.48550/arxiv.2305.15334 · Cited by 226 · cited 2×
“These external tools encompass domain-specific skills such as APIs, expert models, and techniques involving external databases [174, 237, 238], renowned for their proficiency and high accuracy in specific domains.”
MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis
Zhou, D., Li, Y., Ma, F., Zhang, X., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00651 · Cited by 224 · cited 1×
“LLMs are transforming problem-solving across various domains by leveraging their advanced capabilities in natural language understanding and other application [320–323].”
Self-citationVoyager: An Open-Ended Embodied Agent with Large Language Models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023) · arXiv · doi:10.48550/arxiv.2305.16291 · Cited by 222
Dong, X., Yu, S.-I., Weng, X., Wei, S.-E., Yang, Y., & Sheikh, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00045 · Cited by 212
Self-citationSocial Simulacra: Creating Populated Prototypes for Social Computing Systems
Park, J. S., Popowski, L., Cai, C., Morris, M. R., Liang, P., & Bernstein, M. S. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545616 · Cited by 212 · cited 2×
“Moreover, the profile may summarize social relationships and environmental contexts relevant to the agents’ interactions [40, 60].”
TidyBot: personalized robot assistance with large language models
Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S., & Funkhouser, T. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10139-z · Cited by 212 · cited 4×
“…PENTESTGPT [305] Industrial Engineering Mehta et al. [2], Xia et al. [3], Li et al. [4] Embodied Agents SayCan [8], Inner Monologue [9], TidyBot [10], RoCo [34], CoELA [35] Science Experiments Ghafarollahi et al. [5], Boiko et al. [6], ChemCrow [7] Science Debate Du et al. [36], Liang et al.…”
InfluentialMitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-Based Retrofitting
Guan, X., Liu, Y., Lin, H., Lu, Y., He, B., Han, X., & Sun, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29770 · Cited by 212
On the Risk of Misinformation Pollution with Large Language Models
Pan, Y., Pan, L., Chen, W., Nakov, P., Kan, M.-Y., & Wang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.97 · Cited by 210
Distributed problem-solving techniques: A survey
Decker, K. S. (1987) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1987.6499280 · Cited by 209 · cited 1×
“The four-dimensional framework proposed by [27] emphasizes the diversity of communication protocols, distinguishing between blackboard and message-based systems, and setting different gradients from low to high-level content.”
Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction
Kang, S., Yoon, J., & Yoo, S. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00194 · Cited by 204 · cited 2×
“Through ongoing interactions, agents continuously assess and enhance their knowledge base, [32], ChatEDA [303], LIBRO [304], PENTESTGPT [305] Industrial Engineering Mehta et al. [2], Xia et al. [3], Li et al. [4] Embodied Agents SayCan [8], Inner Monologue [9], TidyBot [10], RoCo [34], CoELA [35]…”
Faithful Chain-of-Thought Reasoning
Lyu, Q., Havaldar, S., Stein, A., Zhang, L., Rao, D., Wong, E., Apidianaki, M., & Callison-Burch, C. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.20 · Cited by 201
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
Shao, Z., Gong, Y., Shen, Y., Huang, M., Duan, N., & Chen, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.620 · Cited by 200
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
Shao, Z., Gong, Y., Shen, Y., Huang, M., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.15294 · Cited by 200 · cited 1×
“Beyond the commonly used single-step retrieval, RAG incorporates three types of retrieval enhancement processes: iterative retrieval [181], recursive retrieval [182], and adaptive retrieval [183, 184], which are designed to improve efficiency and accuracy in solving complex queries [185].”
Unlimiformer: Long-Range Transformers with Unlimited Length Input
Bertsch, A., Alon, U., Neubig, G., & Gormley, M. R. (2023) · neural information processing systems · doi:10.52202/075280-1543 · Cited by 194
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.08155 · Cited by 193
Self-citationHu, S., Chen, X., & Torkelson, J. M. (2019) · ACS Sustainable Chemistry & Engineering · doi:10.1021/acssuschemeng.9b01239 · Cited by 189
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
Zhang, D., Li, S., Zhang, X., Zhan, J., Wang, P., Zhou, Y., & Qiu, X. (2023) · arXiv · doi:10.48550/arxiv.2305.11000 · Cited by 189 · cited 1×
“For instance, in SpeechGPT [155], speech tokens generated by the HuBERT [121] encoder are embedded into the LLaMA [156] vocabulary, enabling the LLM to process speech input.”
Uncovering ChatGPT’s Capabilities in Recommender Systems
Dai, S., Shao, N., Zhao, H., Yu, W., Si, Z., Xu, C., Sun, Z., Zhang, X., & Xu, J. (2023) · Proceedings of the 17th ACM Conference on Recommender Systems · doi:10.1145/3604915.3610646 · Cited by 187 · cited 1×
“…et al. [256], CompeteAI [257] Recommender Systems Zhang et al. [309], TALLRec [310], Hou et al. [311], Liu et al. [312], Chat-Rec [313], Dai et al. [314], KAR [315], GENRE [316], LLMRec [317], RecAgent [60], RecSim [318], Agent4Rec [59], AgentCF [49] Disease Propagation Simulation Williams et al.…”
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
Zhang, D., Li, S., Zhang, X., Zhan, J., Wang, P., Zhou, Y., & Qiu, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.1055 · Cited by 187
Braess' paradox: Some new insights
Pas, E. I., & Principio, S. L. (1997) · Transportation Research Part B: Methodological · doi:10.1016/s0191-2615(96)00024-0 · Cited by 183
RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
Mandi, Z., Jain, S., & Song, S. (2023) · arXiv · doi:10.48550/arxiv.2307.04738 · Cited by 180
Chen, F., Han, M., Zhao, H., Zhang, Q., Shi, J., Xu, S. B., & Xu, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.04160 · Cited by 180
ExpeL: LLM Agents Are Experiential Learners
Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., & Huang, G. (2023) · arXiv · doi:10.48550/arxiv.2308.10144 · Cited by 179
FreezeOut: Accelerate Training by Progressively Freezing LayersPreprint
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04983 · Cited by 177
"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters
Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.-W., & Peng, N. (2023) · arXiv · doi:10.48550/arxiv.2310.09219 · Cited by 175
Reasoning with Language Model is Planning with World Model
Hao, S., Gu, Y., Ma, H., Hong, J., Wang, Z., Wang, D., & Hu, Z. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.507 · Cited by 168
RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
Mandi, Z., Jain, S., & Song, S. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610855 · Cited by 167
Reasoning with Language Model is Planning with World Model
Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z., & Hu, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.14992 · Cited by 166
Connecting Speech Encoder and Large Language Model for ASR
Yu, W., Tang, C., Sun, G., Chen, X., Tan, T., Li, W., Lu, L., Ma, Z., & Zhang, C. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445874 · Cited by 160 · cited 1×
“…approach involves aligning speech encoders with the LLM using connectors such as fully connected layers, multi-head cross-attention [157], or Q-Former [103], which preserves more speech information and achieves efficient compression, thereby supporting the processing of long speech segments [158].”
A Decentralized Cluster Formation Containment Framework for Multirobot Systems
Hu, J., Bhowmick, P., Jang, I., Arvin, F., & Lanzon, A. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3071615 · Cited by 159 · cited 1×
“MAS is a complex system composed of multiple interacting intelligent agents [21], capable of simulating social interactions and teamwork in the real world, enhancing overall adaptability and efficiency through decentralized decision-making processes and information sharing.”
Peng, B., Galley, M., He, P., Cheng, H., Xie, Y., Hu, Y., Huang, Q., Liden, L., Yu, Z., Chen, W., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.12813 · Cited by 156 · cited 1×
“Current approaches include leveraging external knowledge sources to supplement the knowledge base of LLMs [170, 192].”
Randomized Positional Encodings Boost Length Generalization of Transformers
Ruoss, A., Delétang, G., Genewein, T., Grau-Moya, J., Csordás, R., Bennani, M., Legg, S., & Veness, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.161 · Cited by 153
ChatEDA: A Large Language Model Powered Autonomous Agent for EDA
Wu, H., He, Z., Zhang, X., Yao, X., Zheng, S., Zheng, H., & Yu, B. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3383347 · Cited by 152 · cited 2×
“Through ongoing interactions, agents continuously assess and enhance their knowledge base, [32], ChatEDA [303], LIBRO [304], PENTESTGPT [305] Industrial Engineering Mehta et al. [2], Xia et al. [3], Li et al. [4] Embodied Agents SayCan [8], Inner Monologue [9], TidyBot [10], RoCo [34], CoELA [35]…”
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning
Zhang, Z.-R., Tan, C., Xu, H., Wang, C., Huang, J. G., & Huang, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.107 · Cited by 147
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach
Zhang, J., Xie, R., Hou, Y., Zhao, W. X., Lin, L., & Wen, J.-R. (2023) · arXiv · doi:10.48550/arxiv.2305.07001 · Cited by 146
Getting pwn’d by AI: Penetration Testing with Large Language Models
Happe, A., & Cito, J. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3613083 · Cited by 144
The Curious Case of Hallucinations in Neural Machine Translation
Raunak, V., Menezes, A., & Junczys-Dowmunt, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.92 · Cited by 142
Avalon's Game of Thoughts: Battle Against Deception through Recursive ContemplationPreprint
Wang, S., Liu, C., Zheng, Z., Qi, S., Chen, S., Yang, Q., Zhao, A., Wang, C., Song, S., & Huang, G. M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.01320 · Cited by 142
Self-citationThe Curious Case of Hallucinations in Neural Machine Translation
Raunak, V., Menezes, A., & Junczys-Dowmunt, M. (2021) · arXiv · doi:10.48550/arxiv.2104.06683 · Cited by 142
RARR: Researching and revising what language models say, using language models
Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V., Lao, N., Lee, H., Juan, D.-C., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.910 · Cited by 141 · cited 1×
“…to enhance information accuracy [143, 238], increasing model transparency to foster understanding of decision processes [86, 202, 203], developing fact-checking systems to verify outputs [200, 201], and designing hallucination detection tasks [325, 326] to evaluate and improve model performance.”
TableGPT: Towards Unifying Tables, Nature Language and Commands into One GPTPreprint
Zha, L., Zhou, J., Li, L., Wang, R., Huang, Q., Yang, S., Yuan, J., Su, C., Li, X., Su, A., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.08674 · Cited by 138
Self-citationLarge Language Models and Games: A Survey and Roadmap
Gallotta, R., Todd, G., Zammit, M., Earle, S., Liapis, A., Togelius, J., & Yannakakis, G. N. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3461510 · Cited by 137 · cited 2×
“The application of LLM agents in the gaming domain encompasses various roles, ranging from acting as players participating in games, simulating non-player character (NPC) dialogues and behaviors, to providing player assistance and game design support [11].”
Personality Traits in Large Language Models
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Romero, P., Abdulhai, M., Faust, A., & Matarić, M. (2023) · arXiv · Cited by 137
“Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters
Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.-W., & Peng, N. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.243 · Cited by 136
Personality Traits in Large Language Models
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Abdulhai, M., Faust, A., & Matarić, M. (2023) · Research Square · doi:10.21203/rs.3.rs-3296728/v1 · Cited by 134
Improving alignment of dialogue agents via targeted human judgements
Glaese, A., McAleese, N., Trębacz, M., Aslanides, J., Firoiu, V., Ewalds, T., Rauh, M., Weidinger, L., Chadwick, M., Thacker, P., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.14375 · Cited by 134 · cited 1×
“For instance, through instruction tuning [359, 360] and alignment tuning [205, 361], LLMs can better adapt to specific tasks and human values.”
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
Huang, R., Li, M., Yang, D., Shi, J., Chang, X., Ye, Z., Wu, Y., Hong, Z., Huang, J., Liu, J., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30570 · Cited by 131
Self-citationAudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
Huang, R., Li, M., Yang, D., Shi, J., Chang, X., Ye, Z., Wu, Y., Hong, Z., Huang, J., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.12995 · Cited by 131
Self-citationLarge Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient ReasoningPreprint
Yue, M., Zhao, J., Zhang, M., Du, L., & Yao, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.03094 · Cited by 126
Pallagani, V., Muppasani, B. C., Roy, K., Fabiano, F., Loreggia, A., Murugesan, K., Srivastava, B., Rossi, F., Horesh, L., & Sheth, A. (2024) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v34i1.31503 · Cited by 125 · cited 1×
“Beyond text, visual, and audio data, LLMs often need to utilize specialized knowledge from specific domains such as scientific research [160, 161], medical information [162–164], or technical specifications [165–167].”
Devising and Detecting Phishing Emails Using Large Language Models
Heiding, F., Schneier, B., Vishwanath, A., Bernstein, J., & Park, P. S. (2024) · IEEE Access · doi:10.1109/access.2024.3375882 · Cited by 123
SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction
Zhang, Z., Yang, Z., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00948 · Cited by 122 · cited 1×
“These modalities encompass 3D point cloud maps, GPS location data, human pose information, among others [125–127].”
Self-citationG-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering
He, X., Tian, Y., Sun, Y., Chawla, N. V., Laurent, T., LeCun, Y., Bresson, X., & Hooi, B. (2024) · arXiv · doi:10.48550/arxiv.2402.07630 · Cited by 122
ChemCrow: Augmenting large-language models with chemistry tools
Bran, A. M., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2023) · arXiv · doi:10.48550/arxiv.2304.05376 · Cited by 122
CATR: Combinatorial-Dependence Audio-Queried Transformer for Audio-Visual Video Segmentation
Li, K., Yang, Z., Chen, L., Yang, Y., & Xiao, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611724 · Cited by 121 · cited 1×
“An interesting alternative method involves converting the perception of audio information into the encoding of visual information [120–122], as audio spectrograms can be visualized as flat images.”
Self-citationGlobal-to-Local Modeling for Video-Based 3D Human Pose and Shape Estimation
Shen, X., Yang, Z., Wang, X., Ma, J., Zhou, C., & Yang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00858 · Cited by 120 · cited 1×
“These modalities encompass 3D point cloud maps, GPS location data, human pose information, among others [125–127].”
Self-citation(2023) · doi:10.18653/v1/2023.emnlp-main.632 · Cited by 115
Editing Large Language Models: Problems, Methods, and Opportunities
Yao, Y., Wang, P., Tian, B., Cheng, S., Li, Z., Deng, S., Chen, H., & Zhang, N. (2023) · arXiv · doi:10.48550/arxiv.2305.13172 · Cited by 115
Language Models of Code are Few-Shot Commonsense Learners
Madaan, A., Zhou, S., Alon, U., Yang, Y., & Neubig, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.90 · Cited by 114
Self-citationS3: Social-network Simulation System with Large Language Model-Empowered Agents
Gao, C., Lan, X., Lu, Z., Mao, J., Piao, J., Wang, H., Jin, D., & Li, Y. (2023) · doi:10.2139/ssrn.4607026 · Cited by 113
On Generative Agents in Recommendation
Zhang, A., Chen, Y., Sheng, L., Wang, X., & Chua, T.-S. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657844 · Cited by 113 · cited 3×
“Additionally, the profile may include psychological attributes like current emotions, personality traits, and life goals, thereby reflecting the distinct personalities of the agents [58, 59].”
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking
Zhuang, S., Liu, B., Koopman, B., & Zuccon, G. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.590 · Cited by 111
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
Yang, J., Dong, Y., Liu, S., Li, B., Wang, Z.-Y., Jiang, C., Tan, H., Kang, J.-M., Zhang, Y.-H., Zhou, K.-Y., et al. (2023) · European Conference on Computer Vision · doi:10.48550/arxiv.2310.08588 · Cited by 109
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs
Liang, Y., Wu, C., Song, T., Wu, W., Xia, Y., Liu, Y., Ou, Y., Lu, S., Ji, L., Mao, S., et al. (2024) · Intelligent Computing · doi:10.34133/icomputing.0063 · Cited by 108
Ghafarollahi, A., & Buehler, M. J. (2024) · Digital Discovery · doi:10.1039/d4dd00013g · Cited by 106
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
Yang, J., Dong, Y., Liu, S., Li, B., Wang, Z., Tan, H., Jiang, C., Kang, J., Zhang, Y., Zhou, K., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_2 · Cited by 106
Large Language Models Are Semi-Parametric Reinforcement Learning Agents
Zhang, D., Chen, L., Zhang, S., Xu, H., Zhao, Z., & Yu, K. (2023) · neural information processing systems · doi:10.52202/075280-3419 · Cited by 106
LLM+P: Empowering Large Language Models with Optimal Planning Proficiency
Liu, B., Jiang, Y., Zhang, X., Liu, Q., Zhang, S., Biswas, J., & Stone, P. (2023) · arXiv · doi:10.48550/arxiv.2304.11477 · Cited by 104
AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuningPreprint
Liu, Y., Agarwal, S., & Venkataraman, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01386 · Cited by 104
Xia, Y., Shenoy, M., Jazdi, N., & Weyrich, M. (2023) · 2023 IEEE 28th International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa54631.2023.10275362 · Cited by 103 · cited 2×
“…assess and enhance their knowledge base, [32], ChatEDA [303], LIBRO [304], PENTESTGPT [305] Industrial Engineering Mehta et al. [2], Xia et al. [3], Li et al. [4] Embodied Agents SayCan [8], Inner Monologue [9], TidyBot [10], RoCo [34], CoELA [35] Science Experiments Ghafarollahi et al. [5],…”
Playing repeated games with Large Language Models
Akata, E., Schulz, L., Coda-Forno, J., Oh, S. J., Bethge, M., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2305.16867 · Cited by 99
Putting Humans in the Natural Language Processing Loop: A SurveyPreprint
Wang, Z. J., Choi, D., Xu, S., & Yang, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.04044 · Cited by 98
Logothetis, N. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.19.1.577 · Cited by 97
Shortcut Learning of Large Language Models in Natural Language Understanding
Du, M., He, F., Zou, N., Tao, D., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3596490 · Cited by 94 · cited 1×
“Shortcut learning can be mitigated through methods such as data debias-ing, adversarial training, interpretive regularization, and confidence regularization [213].”
AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems
Zhang, J., Hou, Y., Xie, R., Sun, W., McAuley, J., Zhao, W. X., Lin, L., & Wen, J.-R. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645537 · Cited by 94 · cited 3×
“…al. [311], Liu et al. [312], Chat-Rec [313], Dai et al. [314], KAR [315], GENRE [316], LLMRec [317], RecAgent [60], RecSim [318], Agent4Rec [59], AgentCF [49] Disease Propagation Simulation Williams et al. [52], Ghaffarzadegan et al. [319] generating daily insights into evolving personality traits.”
Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?
Chen, Y., Arkin, J., Zhang, Y., Roy, N., & Fan, C. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610676 · Cited by 93 · cited 1×
“For example, in the decentralized multi-agent communication frame - work (DMAS) [253], each robot’s LLM agent engages in task planning through turn-taking dialogues.”
VideoChat: Chat-Centric Video UnderstandingPreprint
Li, K., He, Y., Wang, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06355 · Cited by 89
ProAgent: Building Proactive Cooperative Agents with Large Language Models
Zhang, C., Yang, K., Hu, S., Wang, Z., Li, G., Sun, Y., Zhang, C., Zhang, Z., Liu, A., Zhu, S.-C., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29710 · Cited by 88
Self-citationIntegrating action knowledge and LLMs for task planning and situation handling in open worlds
Ding, Y. F., Zhang, X., Amiri, S., Cao, N., Yang, H., Kaminski, A., Esselink, C., & Zhang, S. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10133-5 · Cited by 87 · cited 2×
“Beyond text, visual, and audio data, LLMs often need to utilize specialized knowledge from specific domains such as scientific research [160, 161], medical information [162–164], or technical specifications [165–167].”
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding
Shi, W., Han, X., Lewis, M., Tsvetkov, Y., Zettlemoyer, L., & Yih, W.-T. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers) · doi:10.18653/v1/2024.naacl-short.69 · Cited by 83
Cheng, Y., Li, L., Xu, Y., Li, X., Yang, Z., Wang, W., & Yang, Y. P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06558 · Cited by 79
Self-citationChat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System
Gao, Y., Sheng, T., Xiang, Y., Xiong, Y., Wang, H., & Zhang, J. (2023) · arXiv · doi:10.48550/arxiv.2303.14524 · Cited by 79
A First Look at LLM-Powered Generative News Recommendation
Liu, Q., Chen, N., Sakai, T., & Wu, X.-M. (2023) · arXiv.org · doi:10.48550/arxiv.2305.06566 · Cited by 79
Does Gender Matter? Towards Fairness in Dialogue Systems
Liu, H., Dacon, J., Fan, W., Liu, H., Liu, Z., & Tang, J. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.390 · Cited by 78
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action
Yang, Z., Li, L., Wang, J., Lin, K., Azarnasab, E., Ahmed, F., Liu, Z., Liu, C., Zeng, M., & Wang, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.11381 · Cited by 78
TidyBot: Personalized Robot Assistance with Large Language Models
Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S., & Funkhouser, T. (2023) · 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341577 · Cited by 77
Memory Augmented Large Language Models are Computationally UniversalPreprint
Schuurmans, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.04589 · Cited by 76
Research works printed but not linked: 83
- A. Brohan, Y. Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian et al., Do as i can, not as i say: Grounding language in robotic affordances. in Conference on robot learning (PMLR, Atlanta, GA, USA, 2023), pp. 287–318
- K. Li, A.K. Hopkins, D. Bau, F. Viégas, H. Pfister, M. Wattenberg, Emergent world representations: Exploring a sequence model trained on a synthetic task. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
- N. Renella, M. Eger, Towards automated video game commentary using generative ai. in Proceedings of the Experimental Artificial Intelligence in Games Workshop co-located with the 19th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE 2023) (AAAI Press, Washington, DC, USA, 2023)
- Z. Ma, Y. Mei, Z. Su, Understanding the benefits and challenges of using large language model-based conversational agents for mental well-being support. in AMIA Annual Symposium Proceedings (American Medical Informatics Association, New Orleans, LA, 2023). pp. 1105
- G. Weiss, Multiagent systems: a modern approach to distributed artificial intelligence (MIT Press, Cambridge, MA, United States, 1999), pp. 547
- H.V.D. Parunak, Applications of distributed artificial intelligence in industry. Found. Distrib. Artif. Intell. 2(1), 18 (1996)
- I. Dasgupta, C. Kaeser-Chen, K. Marino, A. Ahuja, S. Babayan, F. Hill, R. Fergus, Collaborating with language models for embodied reasoning. in NeurIPS 2022 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States of America, 2022)
- S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S.K.S. Yau, Z. Lin, L. Zhou et al., Metagpt: Meta programming for multi-agent collaborative framework. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- H. Zhang, W. Du, J. Shan, Q. Zhou, Y. Du, J.B. Tenenbaum, T. Shu, C. Gan, Building cooperative embodied agents modularly with large language models. in NeurIPS 2023 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States, 2023)
- Y. Du, S. Li, A. Torralba, J.B. Tenenbaum, I. Mordatch, Improving factuality and reasoning in language models through multiagent debate. in Proceedings of the 41st International Conference on Machine Learning. Proceedings of Machine Learning Research (Vienna, Austria, 2024). 235, pp. 11733–11763. PMLR.
- C.M. Chan, W. Chen, Y. Su, J. Yu, W. Xue, S. Zhang, J. Fu, Z. Liu, Chateval: Towards better llm-based evaluators through multi-agent debate. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- J. Light, M. Cai, S. Shen, Z. Hu, Avalonbench: Evaluating llms playing the game of avalon. in NeurIPS 2023 Foundation Models for Decision Making Workshop (2023) (New Orleans, United States, 2023)
- G. Mukobi, H. Erlebach, N. Lauffer, L. Hammond, A. Chan, J. Clifton, Welfare diplomacy: Benchmarking language model cooperation. in NeurIPS 2023 Socially Responsible Language Modelling Research workshop (New Orleans, United States, 2023)
- G.V. Aher, R.I. Arriaga, A.T. Kalai, Using large language models to simulate multiple humans and replicate human subject studies. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 337–371
- A. Zhang, L. Sheng, Y. Chen, H. Li, Y. Deng, X. Wang, T.S. Chua, On generative agents in recommendation. in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Association for Computing Machinery, New York, NY, USA, 2024). SIGIR ’24, pp. 1807–1817. https://doi.org/10.1145/3626772.3657844 · DOI · Paper page
- L. Wang, J. Zhang, X. Chen, Y. Lin, R. Song, W.X. Zhao, J.R. Wen, Recagent: A novel simulation paradigm for recommender systems. arXiv preprint (2023) arXiv:2306.0255
- M. Shridhar, X. Yuan, M.A. Côté, Y. Bisk, A. Trischler, M. Hausknecht, Alfworld: Aligning text and embodied environments for interactive learning. International Conference on Learning Representation (2021)
- D. Driess, F. Xia, M.S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu et al., Palm-e: An embodied multimodal language model. in Proceedings of the 40th International Conference on Machine Learning (Honolulu, Hawaii, USA, 2023). ICML’23. JMLR.org 340, 20.
- W. Tan, Z. Ding, W. Zhang, B. Li, B. Zhou, J. Yue, H. Xia, J. Jiang, L. Zheng, X. Xu et al., Towards general computer control: A multimodal agent for red dead redemption ii as a case study. in ICLR 2024 Workshop on Large Language Model (LLM) Agents (Vienna, Austria, 2024).
- S. Reed, K. Zolna, E. Parisotto, S.G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, Y. Sulsky, J. Kay, J.T. Springenberg et al., A generalist agent. Transactions on Machine Learning Research (2022).
- S. Zheng, Y. Feng, Z. Lu et al., Steve-eye: Equipping llm-based embodied agents with visual perception in open worlds. in The Twelfth International Conference on Learning Representations (Kigali, Rwanda, 2023)
- P.F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, D. Amodei, Deep reinforcement learning from human preferences. Adv. Neural Inf. Process. Syst. 30 (2017). pp. 4302-4310
- J. Wei, M. Bosma, V.Y. Zhao, K. Guu, A.W. Yu, B. Lester, N. Du, A.M. Dai, Q.V. Le, Finetuned language models are zero-shot learners. in International Conference on Learning Representations (2022) (Virtual Event).
- W. Li, L. Zhu, L. Wen, Y. Yang, Decap: Decoding clip latents for zero-shot captioning via text-only training. in International Conference on Learning Representations (2023) (Kigali, Rwanda, 2023).
- S. Zhao, X. Wang, L. Zhu, Y. Yang, Test-time adaptation with clip reward for zero-shot generalization in vision-language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- J. Chen, H. Guo, K. Yi, B. Li, M. Elhoseiny, Visualgpt: Data-efficient image captioning by balancing visual input and linguistic knowledge from pretraining. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE, New Orleans, LA, USA, 2022), pp. 18030–18040
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., An image is worth 16x16 words: Transformers for image recognition at scale. in International Conference on Learning Representations (2021) (Vienna, Austria).
- A. Van Den Oord, O. Vinyals et al., Neural discrete representation learning. Adv. Neural Inf. Process. Syst. 30 (2017). pp. 6309-6318
- S. Mehta, M. Rastegari, Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. in International Conference on Learning Representations (2022) (Virtual Event).
- J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. in International conference on machine learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 19730–19742
- D. Zhu, J. Chen, X. Shen, X. Li, M. Elhoseiny, Minigpt-4: Enhancing vision-language understanding with advanced large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
- Y. Su, T. Lan, H. Li, J. Xu, Y. Wang, D. Cai, Pandagpt: One model to instruction-follow them all. in Proceedings of the 1st Workshop on Taming Large Language Models: Controllability in the era of Interactive Assistants! (Association for Computational Linguistics, Prague, Czech Republic, 2023), pp. 11-23.
- Z. Peng, W. Wang, L. Dong, Y. Hao, S. Huang, S. Ma, F. Wei, Kosmos-2: Grounding multimodal large language models to the world. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
- L. Qian, J. Li, Y. Wu, Y. Ye, H. Fei, T.S. Chua, Y. Zhuang, S. Tang, Momentor: Advancing video large language model with fine-grained temporal reasoning. in Forty-first International Conference on Machine Learning (Vienna, Austria, 2024).
- Z. Yang, G. Chen, X. Li, W. Wang, Y. Yang, Doraemongpt: Toward understanding dynamic scenes with large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
- L.H. Marshall, H.W. Magoun, Discoveries in the human brain: neuroscience prehistory, brain structure, and function (Springer Science & Business Media, Humana Totowa, NJ, 2013)
- L. Zheng, R. Wang, X. Wang, B. An, Synapse: Trajectory-as-exemplar prompting with memory for computer control. in The Twelfth International Conference on Learning Representations (Messe Wien Exhibition and Congress Center, Vienna, Austria, 2023)
- J. Kang, R. Laroche, X. Yuan, A. Trischler, X. Liu, J. Fu, Think before you act: Decision transformers with internal working memory. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024).
- A. Modarressi, A. Imani, M. Fayyaz, H. Schütze, Ret-llm: Towards a general read-write memory for large language models. in ICLR 2024 Workshop: How Far Are We From AGI (Vienna, Austria, 2024)
- T. Silver, V. Hariprasad, R.S. Shuttleworth, N. Kumar, T. Lozano-Pérez, L.P. Kaelbling, Pddl planning with pretrained large language models. in NeurIPS 2022 foundation models for decision making workshop (New Orleans, Louisiana, USA, 2022)
- Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, H. Soh, Translating natural language to planning goals with large-language models. in The International Journal of Robotics Research (2020). 2019, pp. 1
- H. Liu, C. Li, Q. Wu, Y.J. Lee, Visual instruction tuning. Adv. Neural Inf. Process. Syst. 36(2024). pp. 34892-34916
- L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, G. Neubig, Pal: Program-aided language models. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 10764–10799
- X. Cheng, D. Luo, X. Chen, L. Liu, D. Zhao, R. Yan, Lift yourself up: Retrieval-augmented text generation with self-memory. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 43780-43799
- W. Yu, D. Iter, S. Wang, Y. Xu, M. Ju, S. Sanyal, C. Zhu, M. Zeng, M. Jiang, Generate rather than retrieve: Large language models are strong context generators. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
- A. Asai, Z. Wu, Y. Wang, A. Sil, H. Hajishirzi, Self-rag: Learning to retrieve, generate, and critique through self-reflection. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- F. Xu, W. Shi, E. Choi, Recomp: Improving retrieval-augmented lms with compression and selective augmentation. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- E. Mitchell, C. Lin, A. Bosselut, C.D. Manning, C. Finn, Memory-based model editing at scale. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 15817–15831
- J. Maynez, S. Narayan, B. Bohnet, R. McDonald, On faithfulness and factuality in abstractive summarization. in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (Association for Computational Linguistics, Online, 2022), pp. 1906-1919
- J. Thorne, A. Vlachos, Automated fact checking: Task formulations, methods and future directions. in Proceedings of the 27th International Conference on Computational Linguistics (Association for Computational Linguistics, Santa Fe, New Mexico, USA, 2018), pp. 3346-3359
- C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu et al., Lima: Less is more for alignment. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 55006-55021
- Z. Wu, Y. Hu, W. Shi, N. Dziri, A. Suhr, P. Ammanabrolu, N.A. Smith, M. Ostendorf, H. Hajishirzi, Fine-grained human feedback gives better rewards for language model training. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 59008-59033
- Y.S. Chuang, Y. Xie, H. Luo, Y. Kim, J. Glass, P. He, Dola: Decoding by contrasting layers improves factuality in large language models. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- P.C. Wason, P.N. Johnson-Laird, Psychology of reasoning: Structure and content. Harvard University Press, Cambridge, MA, USA, 1972. 86, pp. 246-252
- D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le et al., Least-to-most prompting enables complex reasoning in large language models. in The Eleventh International Conference on Learning Representations (Kigali, Rwanda, 2023)
- M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman et al., Do as i can, not as i say: Grounding language in robotic affordances. in Conference on Robot Learning (Atlanta, GA, United States, 2023), pp. 287–318. PMLR
- S.S. Raman, V. Cohen, E. Rosen, I. Idrees, D. Paulius, S. Tellex, Planning with large language models via corrective re-prompting. in NeurIPS 2022 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, USA, 2022)
- B.Y. Lin, Y. Fu, K. Yang, F. Brahman, S. Huang, C. Bhagavatula, P. Ammanabrolu, Y. Choi, X. Ren, Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 23813-23825
- S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, K. Narasimhan, Tree of thoughts: Deliberate problem solving with large language models. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 11809-11822
- W. Huang, P. Abbeel, D. Pathak, I. Mordatch, Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 9118–9147
- K. Nottingham, P. Ammanabrolu, A. Suhr, Y. Choi, H. Hajishirzi, S. Singh, R. Fox, Do embodied agents dream of pixelated sheep: Embodied decision making using language guided world modelling. in International Conference on Machine Learning (PMLR, Honolulu, Hawaii, USA, 2023), pp. 26311–26325
- J. Ruan, Y. Chen, B. Zhang, Z. Xu, T. Bao, G. Du, S. Shi, H. Mao, X. Zeng, R. Zhao, Tptu: Task planning and tool usage of large language model-based ai agents. in NeurIPS 2023 Foundation Models for Decision Making Workshop (New Orleans, Louisiana, United States of America, 2023)
- Z. Zhao, E. Wallace, S. Feng, D. Klein, S. Singh, Calibrate before use: Improving few-shot performance of language models. in International conference on machine learning (PMLR, Virtual Event, 2021), pp. 12697–12706
- A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
- T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, T. Scialom, Toolformer: Language models can teach themselves to use tools. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 68539-68551
- M. Carroll, R. Shah, M.K. Ho, T. Griffiths, S. Seshia, P. Abbeel, A. Dragan, On the utility of learning about humans for human-ai coordination. Adv. Neural Inf. Process. Syst. 32(2019), pp. 5174-5185
- X. Ning, Z. Lin, Z. Zhou, Z. Wang, H. Yang, Y. Wang, Skeleton-of-thought: Prompting llms for efficient parallel generation. in The Twelfth International Conference on Learning Representations (Vienna, Austria, 2024)
- S. Srivastava, C. Li, M. Lingelbach, R. Martín-Martín, F. Xia, K.E. Vainio, Z. Lian, C. Gokmen, S. Buch, K. Liu et al., Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments. in Conference on robot learning (PMLR, London, UK, 2021), pp. 477–490
- C. Colas, L. Teodorescu, P.Y. Oudeyer, X. Yuan, M.A. Côté, Augmenting autotelic agents with large language models. in Conference on Lifelong Learning Agents (PMLR, McGill University, Montréal, Québec, Canada, 2023), pp. 205–226
- L. Zhu, L. Hu, J. Lin, S. Han, Lift: Efficient layer-wise fine-tuning for large model models. in Proceedings of the 31st ACM International Conference on -Multimedia (Association for Computing Machinery, New York, NY, United States, 2023), pp. 4678-4687
- T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettlemoyer, Qlora: Efficient finetuning of quantized llms. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 10088-10115
- A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, S. Bowman, Superglue: A stickier benchmark for general-purpose language understanding systems. Adv. Neural Inf. Process. Syst. 32 (2019). pp. 3266-3280
- A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang et al., Self-refine: Iterative refinement with self-feedback. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 46534-46594
- N. Shinn, B. Labash, A. Gopinath, Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint (2023) arXiv: 2303.11366 · arXiv
- E. Brooks, L. Walls, R.L. Lewis, S. Singh, Large language models can implement policy iteration. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 30349-30366
- T. Carta, C. Romac, T. Wolf, S. Lamprier, O. Sigaud, P.Y. Oudeyer, Grounding large language models in interactive environments with online reinforcement learning. in International Conference on Machine Learing (PMLR, Honolulu, Hawaii, USA, 2023), pp. 3676–3713
- S. Sudhakaran, M. González-Duque, M. Freiberger, C. Glanois, E. Najarro, S. Risi, Mariogpt: Open-ended text2level generation through large language models. Adv. Neural Inf. Process. Syst. 36 (2024). pp. 54213-54227
- F. Cardoso Durier da Silva, R. Vieira, A.C. Garcia, Can machines learn to detect fake news? a survey focused on social media, in 2021 IEEE 4th International Conference on Computing, Power and Communication Technologies (Kuala Lumpur, Malaysia, 2021), pp.1-7
- B. Buchanan, A. Lohn, M. Musser, K. Sedova, Truth, lies, and automation. Cent. Secur. Emerg. Technol. 1(1), 2 (2021)
- N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., Extracting training data from large language models. in 30th USENIX Security Symposium (USENIX Security 21) (USENIX, Berkeley, CA, 2021), pp. 2633–2650
- Z. Xi, T. Du, C. Li, R. Pang, S. Ji, J. Chen, F. Ma, T. Wang, Defending pre-trained language models as few-shot learners against backdoor attacks. Adv. Neural Inf. Process. Syst. 36(2024), pp. 32748-32764
- E. Ostrom, Tragedy of the commons. New Palgrave Dictionary Econ. 2, 1–4 (2008)
- T. Sun, Y. Shao, H. Qian, X. Huang, X. Qiu, Black-box tuning for language-model-as-a-service. in International Conference on Machine Learning (PMLR, Baltimore, Maryland, USA, 2022), pp. 20841–20855
Cited by
Liu, S., McCoy, A. B., & Wright, A. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf008 · Cited by 200
“57 For example, clinicians and patients have diverse information access needs, some needing to analyze text from a knowledge base, others needing to incorporate structured data from an EHR.”
Bandi, A., Kongari, B., Naguru, R., Pasnoor, S., & Vilipala, S. V. (2025) · Future Internet · doi:10.3390/fi17090404 · Cited by 207
A Survey on the Memory Mechanism of Large Language Model based Agents
Zhang, Z., Bo, X., Ma, C., Li, R., Chen, X., Dai, Q., Zhu, J., Dong, Z., & Wen, J.-R. (2024) · arXiv · doi:10.48550/arxiv.2404.13501 · Cited by 149
Yigit, Y., Ferrag, M. A., Ghanem, M. C., Sarker, I. H., Maglaras, L. A., Chrysoulas, C., Moradpoor, N., Tihanyi, N., & Janicke, H. (2025) · Sensors · doi:10.3390/s25061666 · Cited by 69
“In that case, specialized AI agents can instantly coordinate across multiple teams: public utilities, government agencies, and private contractors, ensuring everyone has the latest threat intelligence and situational updates [118].”
Evaluation and Benchmarking of LLM Agents: A Survey
Mohammadi, M., Li, Y., Lo, J., & Yip, W. (2025) · Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · doi:10.1145/3711896.3736570 · Cited by 74
Metadata
- Authors
- Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y.
- Affiliations
- Wuhan University (CN)
- Zhejiang University (CN)
- Corresponding author
- Li, Xinyi
- Wang, Sai
- Zeng, Siqi
- Wu, Yu
- Yang, Yi
- Venue
- Vicinagearth
- Published
- 2024-10-08
- Volume
- 1
- Issue
- 1
- ISSN
- 3005-060X (ISSN-L)
- OpenAlex
- W4403203873
- Semantic Scholar
- fc8ce12d6186ddaa797e2b36d5e8eb7921425308
- Cited by
- 404
- References
- 252
- Citations by year
- 2026: 254 · 2025: 147 · 2024: 2
- FWCI
- 59.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Vicinagearth. (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- multi-agent LLM systems, large language models, multi-agent systems, artificial general intelligence, agent architecture, agent perception, agent interaction, agent evolution, problem solving, world simulation, autonomous systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.