CoChat
Sign Up
PreprintCited by 972Artificial Intelligence ApplicationsTopic ModelingAI-based Problem Solving and Planning

The Rise and Potential of Large Language Model Based Agents: A Survey

Open access
Xi, Zhiheng, Chen, Wenxiang, Guo, Xin, He, Wei, Ding, Yiwen, Hong, Boyang, Zhang, Ming, Wang, Junzhe +21 more·2023·arXiv
DOI 10.48550/arxiv.2309.07864arXiv 2309.07864
Open this paper in CoChat — free accountOpen PDF

Abstract

For a long time, humanity has pursued artificial intelligence (AI) equivalent to or surpassing the human level, with AI agents considered a promising vehicle for this pursuit. AI agents are artificial entities that sense their environment, make decisions, and take actions. Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks. Actually, what the community lacks is a general and powerful model to serve as a starting point for designing AI agents that can adapt to diverse scenarios. Due to the versatile capabilities they demonstrate, large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI), offering hope for building general AI agents. Many researchers have leveraged LLMs as the foundation to build AI agents and have achieved significant progress. In this paper, we perform a comprehensive survey on LLM-based agents. We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. Building upon this, we present a general framework for LLM-based agents, comprising three main components: brain, perception, and action, and the framework can be tailored for different applications. Subsequently, we explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation. Following this, we delve into agent societies, exploring the behavior and personality of LLM-based agents, the social phenomena that emerge from an agent society, and the insights they offer for human society. Finally, we discuss several key topics and open problems within the field. A repository for the related papers at https://github.com/WooooDyy/LLM-Agent-Paper-List.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2309.07864

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The rise of generative AI tools and their capacity to generate even more human-like text poses a significant threat to academic integrity.

    “Thus, the rise of such generative AI tools and their capacity to generate even more human-like text poses a significant threat to academic integrity [14].”

    From Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity (2024) · cites this paper as [14]

    Section 3.3.1 (page not provided) says LLM-based agents generate high-quality text, and the discussion of multi-turn conversation describes steady improvements in coherence and grammatical accuracy. The passages on risks do not identify academic integrity as a threatened area; searches for academic integrity, plagiarism, and academic dishonesty found no supporting passage for that part of the claim. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • LLM-based agents are characterized by a self-evolving capability that is the basis for solving real-world problems requiring long-term and complex agent-environment interactions.

    “Compared with original LLMs, LLMbased agents are featured in their self-evolving capability, which is the basis for solving real-world problems that need long-term and complex agent-environment interactions.”

    From A Survey on the Memory Mechanism of Large Language Model-based Agents (2025) · cites this paper as [4]

    The discussion of the ability to evolve continually describes agents adapting over time, and §4.1.3 links continuous exploration and skill development to a long-term life cycle. But the survey presents this as a challenging goal, not an established feature distinguishing LLM-based agents from original LLMs or a demonstrated basis for solving complex real-world problems. Page numbers are not available in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Recent years have seen many studies on LLM-based agents, where the key is to equip LLMs with additional modules to enhance their self-evolving capability in real-world environments.

    “To further push the boundary of LLMs towards AGI, recent years have witnessed a large number of studies on LLM-based agents [3, 4], where the key is to equip LLMs with additional modules to enhance their self-evolving capability in real-world environments.”

    From A Survey on the Memory Mechanism of Large Language Model-based Agents (2025) · cites this paper as [4]

    Section 6.1 says research on LLM-based agents has rapidly progressed, and the Agent Action section describes agents built around brain, perception, and action modules. The Utility section discusses continual evolution, but its example is a virtual world; the paper does not establish that adding modules is the key to self-evolution in real-world environments. No page numbers were provided for these passages. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • There is growing interest in developing AI agents that take actions affecting the world.

    “There is simultaneously a growing interest in developing AI agents , models that are trained and equipped to take actions that affect the world [4], [5].”

    From Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated (2025) · cites this paper as [5]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 485 · Cited by 972

Research works printed but not linked: 378

  1. Russell, S. J. Artificial intelligence a modern approach. Pearson Education, Inc., 2010.
  2. Turing, A. M. Computing machinery and intelligence. Springer, 2009.
  3. Wooldridge, M. J., N. R. Jennings. Intelligent agents: theory and practice. Knowl. Eng. Rev., 10(2):115-152, 1995.
  4. Agha, G. A. Actors: a Model of Concurrent Computation in Distributed Systems (Parallel Processing, Semantics, Open, Programming Languages, Artificial Intelligence). Ph.D. thesis, University of Michigan, USA, 1985.
  5. Green, S., L. Hurst, B. Nangle, et al. Software agents: A review. Department of Computer Science, Trinity College Dublin, Tech. Rep. TCS-CS-1997-06, 1997.
  6. Genesereth, M. R., S. P. Ketchpel. Software agents. Commun. ACM, 37(7):48-53, 1994.
  7. Goodwin, R. Formalizing properties of agents. J. Log. Comput., 5(6):763-781, 1995.
  8. Padgham, L., M. Winikoff. Developing intelligent agent systems: A practical guide. John Wiley & Sons, 2005.
  9. Shoham, Y. Agent oriented programming. In M. Masuch, L. Pólos, eds., Knowledge Repre- sentation and Reasoning Under Uncertainty, Logic at Work [International Conference Logic at Work, Amsterdam, The Netherlands, December 17-19, 1992], vol. 808 of Lecture Notes in Computer Science, pages 123-129. Springer, 1992.
  10. Hutter, M. Universal artificial intelligence: Sequential decisions based on algorithmic probability. Springer Science & Business Media, 2004.
  11. Fikes, R., N. J. Nilsson. STRIPS: A new approach to the application of theorem proving to problem solving. In D. C. Cooper, ed., Proceedings of the 2nd International Joint Confer- ence on Artificial Intelligence. London, UK, September 1-3, 1971, pages 608-620.
  12. Sacerdoti, E. D. Planning in a hierarchy of abstraction spaces. In N. J. Nilsson, ed., Proceedings of the 3rd International Joint Conference on Artificial Intelligence. Standford, CA, USA, August 20-23, 1973, pages 412-422.
  13. Brooks, R. A. Intelligence without representation. Artificial intelligence, 47(1-3):139-159, 1991.
  14. Maes, P. Designing autonomous agents: Theory and practice from biology to engineering and back. MIT press, 1990.
  15. Ribeiro, C. Reinforcement learning agents. Artificial intelligence review, 17:223-250, 2002.
  16. Kaelbling, L. P., M. L. Littman, A. W. Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237-285, 1996.
  17. Guha, R. V., D. B. Lenat. Enabling agents to work together. Communications of the ACM, 37(7):126-142, 1994.
  18. Kaelbling, L. P., et al. An architecture for intelligent reactive systems. Reasoning about actions and plans, pages 395-410, 1987.
  19. Sutton, R. S., A. G. Barto. Reinforcement learning: An introduction. MIT press, 2018.
  20. Ouyang, L., J. Wu, X. Jiang, et al. Training language models to follow instructions with human feedback. In NeurIPS. 2022.
  21. Wei, J., Y. Tay, R. Bommasani, et al. Emergent abilities of large language models. Trans. Mach. Learn. Res., 2022, 2022.
  22. Bisk, Y., A. Holtzman, J. Thomason, et al. Experience grounds language. In B. Webber, T. Cohn, Y. He, Y. Liu, eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 8718- 8735. Association for Computational Linguistics, 2020.
  23. Anscombe, G. E. M. Intention. Harvard University Press, 2000.
  24. Davidson, D. Actions, reasons, and causes. The Journal of Philosophy, 60(23):685-700, 1963.
  25. -. I. agency. In A. Marras, R. N. Bronaugh, R. W. Binkley, eds., Agent, Action, and Reason, pages 1-37. University of Toronto Press, 1971.
  26. Dennett, D. C. Précis of the intentional stance. Behavioral and brain sciences, 11(3):495-505, 1988.
  27. Barandiaran, X. E., E. Di Paolo, M. Rohde. Defining agency: Individuality, normativity, asymmetry, and spatio-temporality in action. Adaptive Behavior, 17(5):367-386, 2009.
  28. Rosenschein, S. J., L. P. Kaelbling. The synthesis of digital machines with provable epistemic properties. In Theoretical aspects of reasoning about knowledge, pages 83-98. Elsevier, 1986.
  29. Radford, A., J. Wu, R. Child, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  30. Brown, T. B., B. Mann, N. Ryder, et al. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin, eds., Advances in Neural In- formation Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. 2020.
  31. Lin, C., A. Jaech, X. Li, et al. Limitations of autoregressive models and their alternatives. In K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tür, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, Y. Zhou, eds., Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 5147-5173. Association for Computational Linguistics, 2021.
  32. Tomasello, M. Constructing a language: A usage-based theory of language acquisition. Harvard university press, 2005.
  33. Bloom, P. How children learn the meanings of words. MIT press, 2002.
  34. Andreas, J. Language models as agent models. In Y. Goldberg, Z. Kozareva, Y. Zhang, eds., Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 5769-5779. Association for Computational Linguistics, 2022.
  35. Li, B. Z., M. I. Nye, J. Andreas. Implicit representations of meaning in neural language models. In C. Zong, F. Xia, W. Li, R. Navigli, eds., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 1813-1827. Association for Computational Linguistics, 2021.
  36. Mukhopadhyay, U., L. M. Stephens, M. N. Huhns, et al. An intelligent system for document retrieval in distributed office environments. J. Am. Soc. Inf. Sci., 37(3):123-135, 1986.
  37. Maes, P. Situated agents can have goals. Robotics Auton. Syst., 6(1-2):49-70, 1990.
  38. Müller, J. P., M. Pischel. Modelling interacting agents in dynamic environments. In Proceed- ings of the 11th European Conference on Artificial Intelligence, pages 709-713. 1994.
  39. Brooks, R. A robust layered control system for a mobile robot. IEEE journal on robotics and automation, 2(1):14-23, 1986.
  40. Brooks, R. A. Intelligence without reason. In The artificial life route to artificial intelligence, pages 25-81. Routledge, 2018.
  41. Newell, A., H. A. Simon. Computer science as empirical inquiry: Symbols and search. Commun. ACM, 19(3):113-126, 1976.
  42. Ginsberg, M. L. Essentials of Artificial Intelligence. Morgan Kaufmann, 1993.
  43. Wilkins, D. E. Practical planning -extending the classical AI planning paradigm. Morgan Kaufmann series in representation and reasoning. Morgan Kaufmann, 1988.
  44. Shardlow, N. Action and agency in cognitive science. Ph.D. thesis, Master's thesis, Department of Psychlogy, University of Manchester, Oxford . . . , 1990.
  45. Sacerdoti, E. D. The nonlinear nature of plans. In Advance Papers of the Fourth International Joint Conference on Artificial Intelligence, Tbilisi, Georgia, USSR, September 3-8, 1975, pages 206-214. 1975.
  46. Russell, S. J., E. Wefald. Do the right thing: studies in limited rationality. MIT press, 1991.
  47. Schoppers, M. Universal plans for reactive robots in unpredictable environments. In J. P. Mc- Dermott, ed., Proceedings of the 10th International Joint Conference on Artificial Intelligence.
  48. Morgan Kaufmann, 1987.
  49. Brooks, R. A. A robust layered control system for a mobile robot. IEEE J. Robotics Autom., 2(1):14-23, 1986.
  50. Minsky, M. Steps toward artificial intelligence. Proceedings of the IRE, 49(1):8-30, 1961.
  51. Isbell, C., C. R. Shelton, M. Kearns, et al. A social reinforcement learning agent. In Proceedings of the fifth international conference on Autonomous agents, pages 377-384. 2001.
  52. Rummery, G. A., M. Niranjan. On-line Q-learning using connectionist systems, vol. 37. University of Cambridge, Department of Engineering Cambridge, UK, 1994.
  53. Tesauro, G., et al. Temporal difference learning and td-gammon. Communications of the ACM, 38(3):58-68, 1995.
  54. Silver, D., A. Huang, C. J. Maddison, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484-489, 2016.
  55. Dulac-Arnold, G., N. Levine, D. J. Mankowitz, et al. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Mach. Learn., 110(9):2419-2468, 2021.
  56. Ghosh, D., J. Rahme, A. Kumar, et al. Why generalization in RL is difficult: Epistemic pomdps and implicit partial observability. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, J. W. Vaughan, eds., Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 25502-25515. 2021.
  57. Brys, T., A. Harutyunyan, M. E. Taylor, et al. Policy transfer using reward shaping. In G. Weiss, P. Yolum, R. H. Bordini, E. Elkind, eds., Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2015, Istanbul, Turkey, May 4-8, 2015, pages 181-188. ACM, 2015.
  58. Finn, C., P. Abbeel, S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In D. Precup, Y. W. Teh, eds., Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, vol. 70 of Proceedings of Machine Learning Research, pages 1126-1135. PMLR, 2017.
  59. Gupta, A., R. Mendonca, Y. Liu, et al. Meta-reinforcement learning of structured exploration strategies. In S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Gar- nett, eds., Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 5307-5316. 2018.
  60. Rakelly, K., A. Zhou, C. Finn, et al. Efficient off-policy meta-reinforcement learning via probabilistic context variables. In K. Chaudhuri, R. Salakhutdinov, eds., Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, vol. 97 of Proceedings of Machine Learning Research, pages 5331-5340. PMLR, 2019.
  61. Tirinzoni, A., A. Sessa, M. Pirotta, et al. Importance weighted transfer of samples in reinforce- ment learning. In J. G. Dy, A. Krause, eds., Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, vol. 80 of Proceedings of Machine Learning Research, pages 4943-4952. PMLR, 2018.
  62. Beck, J., R. Vuorio, E. Z. Liu, et al. A survey of meta-reinforcement learning. CoRR, abs/2301.08028, 2023. · arXiv
  63. Yao, S., J. Zhao, D. Yu, et al. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
  64. Wei, J., X. Wang, D. Schuurmans, et al. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS. 2022.
  65. Kojima, T., S. S. Gu, M. Reid, et al. Large language models are zero-shot reasoners. In NeurIPS. 2022.
  66. Wang, X., J. Wei, D. Schuurmans, et al. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
  67. Zhou, D., N. Schärli, L. Hou, et al. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
  68. Akyürek, A. F., E. Akyürek, A. Kalyan, et al. RL4F: generating natural language feedback with reinforcement learning for repairing model outputs. In A. Rogers, J. L. Boyd-Graber, N. Okazaki, eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 7716-7733. Association for Computational Linguistics, 2023.
  69. Wei, J., M. Bosma, V. Y. Zhao, et al. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
  70. Sanh, V., A. Webson, C. Raffel, et al. Multitask prompted training enables zero-shot task generalization. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
  71. Castelfranchi, C. Guarantees for autonomy in cognitive agent architecture. In M. J. Wooldridge, N. R. Jennings, eds., Intelligent Agents, ECAI-94 Workshop on Agent Theories, Architectures, and Languages, Amsterdam, The Netherlands, August 8-9, 1994, Proceedings, vol. 890 of Lecture Notes in Computer Science, pages 56-70. Springer, 1994.
  72. Yuan, A., A. Coenen, E. Reif, et al. Wordcraft: Story writing with large language models. In G. Jacucci, S. Kaski, C. Conati, S. Stumpf, T. Ruotsalo, K. Gajos, eds., IUI 2022: 27th International Conference on Intelligent User Interfaces, Helsinki, Finland, March 22 -25, 2022, pages 841-852. ACM, 2022.
  73. Franceschelli, G., M. Musolesi. On the creativity of large language models. CoRR, abs/2304.00008, 2023. · arXiv
  74. Yin, S., C. Fu, S. Zhao, et al. A survey on multimodal large language models. CoRR, abs/2306.13549, 2023. · arXiv
  75. Driess, D., F. Xia, M. S. M. Sajjadi, et al. Palm-e: An embodied multimodal language model. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett, eds., International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, vol. 202 of Proceedings of Machine Learning Research, pages 8469-8488. PMLR, 2023.
  76. Brown, J. W. Beyond conflict monitoring: Cognitive control and the neural basis of thinking before you act. Current Directions in Psychological Science, 22(3):179-185, 2013.
  77. Liu, H., C. Sferrazza, P. Abbeel. Chain of hindsight aligns language models with feedback. CoRR, abs/2302.02676, 2023. · arXiv
  78. Lin, J., D. Fried, D. Klein, et al. Inferring rewards from language in context. In S. Muresan, P. Nakov, A. Villavicencio, eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 8546-8560. Association for Computational Linguistics, 2022.
  79. Lu, A., H. Zhang, Y. Zhang, et al. Bounding the capabilities of large language models in open text generation with prompt constraints. In A. Vlachos, I. Augenstein, eds., Findings of the Association for Computational Linguistics: EACL 2023, Dubrovnik, Croatia, May 2-6, 2023, pages 1937-1963. Association for Computational Linguistics, 2023.
  80. Hill, F., K. Cho, A. Korhonen. Learning distributed representations of sentences from un- labelled data. In K. Knight, A. Nenkova, O. Rambow, eds., NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pages 1367-1377. The Association for Computational Linguistics, 2016.
  81. Roberts, A., C. Raffel, N. Shazeer. How much knowledge can you pack into the parameters of a language model? In B. Webber, T. Cohn, Y. He, Y. Liu, eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 5418-5426. Association for Computational Linguistics, 2020.
  82. Tandon, N., A. S. Varde, G. de Melo. Commonsense knowledge in machine intelligence. SIGMOD Rec., 46(4):49-52, 2017.
  83. Vulic, I., E. M. Ponti, R. Litschko, et al. Probing pretrained language models for lexical semantics. In B. Webber, T. Cohn, Y. He, Y. Liu, eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 7222-7240. Association for Computational Linguistics, 2020.
  84. Hewitt, J., C. D. Manning. A structural probe for finding syntax in word representations. In J. Burstein, C. Doran, T. Solorio, eds., Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4129-4138. Association for Computational Linguistics, 2019.
  85. Rau, L. F., P. S. Jacobs, U. Zernik. Information extraction and text summarization using linguistic knowledge acquisition. Inf. Process. Manag., 25(4):419-428, 1989.
  86. Yang, K., Z. Chen, Y. Cai, et al. Improved automatic keyword extraction given more semantic knowledge. In H. Gao, J. Kim, Y. Sakurai, eds., Database Systems for Advanced Applications -DASFAA 2016 International Workshops: BDMS, BDQM, MoI, and SeCoP, Dallas, TX, USA, April 16-19, 2016, Proceedings, vol. 9645 of Lecture Notes in Computer Science, pages 112-125. Springer, 2016.
  87. Beloucif, M., C. Biemann. Probing pre-trained language models for semantic attributes and their values. In M. Moens, X. Huang, L. Specia, S. W. Yih, eds., Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pages 2554-2559. Association for Computational Linguistics, 2021.
  88. Safavi, T., D. Koutra. Relational world knowledge representation in contextual language models: A review. In M. Moens, X. Huang, L. Specia, S. W. Yih, eds., Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 1053-1067. Association for Computational Linguistics, 2021.
  89. Jiang, Z., F. F. Xu, J. Araki, et al. How can we know what language models know. Trans. Assoc. Comput. Linguistics, 8:423-438, 2020.
  90. Madaan, A., S. Zhou, U. Alon, et al. Language models of code are few-shot commonsense learners. In Y. Goldberg, Z. Kozareva, Y. Zhang, eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 1384-1403. Association for Computational Linguistics, 2022.
  91. Xu, F. F., U. Alon, G. Neubig, et al. A systematic evaluation of large language models of code. In S. Chaudhuri, C. Sutton, eds., MAPS@PLDI 2022: 6th ACM SIGPLAN International Symposium on Machine Programming, San Diego, CA, USA, 13 June 2022, pages 1-10. ACM, 2022.
  92. Thirunavukarasu, A. J., D. S. J. Ting, K. Elangovan, et al. Large language models in medicine. Nature medicine, pages 1-11, 2023.
  93. Lai, Y., C. Li, Y. Wang, et al. DS-1000: A natural and reliable benchmark for data science code generation. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett, eds., International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, vol. 202 of Proceedings of Machine Learning Research, pages 18319-18345. PMLR, 2023.
  94. Kemker, R., M. McClure, A. Abitino, et al. Measuring catastrophic forgetting in neural networks. In S. A. McIlraith, K. Q. Weinberger, eds., Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 3390-3398. AAAI Press, 2018.
  95. Cao, N. D., W. Aziz, I. Titov. Editing factual knowledge in language models. In M. Moens, X. Huang, L. Specia, S. W. Yih, eds., Proceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 6491-6506. Association for Computational Linguistics, 2021.
  96. Mitchell, E., C. Lin, A. Bosselut, et al. Memory-based model editing at scale. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvári, G. Niu, S. Sabato, eds., International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, vol. 162 of Proceedings of Machine Learning Research, pages 15817-15831. PMLR, 2022.
  97. Lewis, M., Y. Liu, N. Goyal, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In D. Jurafsky, J. Chai, N. Schluter, J. R. Tetreault, eds., Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7871-7880. Association for Computational Linguistics, 2020.
  98. Park, H. H., Y. Vyas, K. Shah. Efficient classification of long documents using transformers. In S. Muresan, P. Nakov, A. Villavicencio, eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 702-709. Association for Computational Linguistics, 2022.
  99. Guo, M., J. Ainslie, D. C. Uthus, et al. Longt5: Efficient text-to-text transformer for long sequences. In M. Carpuat, M. de Marneffe, I. V. M. Ruíz, eds., Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 724-736. Association for Computational Linguistics, 2022.
  100. Ruoss, A., G. Delétang, T. Genewein, et al. Randomized positional encodings boost length generalization of transformers. In A. Rogers, J. L. Boyd-Graber, N. Okazaki, eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 1889-1903. Association for Computational Linguistics, 2023.
  101. Shinn, N., B. Labash, A. Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. CoRR, abs/2303.11366, 2023. · arXiv
  102. Creswell, A., M. Shanahan, I. Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
  103. Ichter, B., A. Brohan, Y. Chebotar, et al. Do as I can, not as I say: Grounding language in robotic affordances. In K. Liu, D. Kulic, J. Ichnowski, eds., Conference on Robot Learning, CoRL 2022, 14-18 December 2022, Auckland, New Zealand, vol. 205 of Proceedings of Machine Learning Research, pages 287-318. PMLR, 2022.
  104. Huang, W., F. Xia, T. Xiao, et al. Inner monologue: Embodied reasoning through planning with language models. In K. Liu, D. Kulic, J. Ichnowski, eds., Conference on Robot Learning, CoRL 2022, 14-18 December 2022, Auckland, New Zealand, vol. 205 of Proceedings of Machine Learning Research, pages 1769-1782. PMLR, 2022.
  105. Wu, T., M. Terry, C. J. Cai. AI chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In S. D. J. Barbosa, C. Lampe, C. Appert, D. A. Shamma, S. M. Drucker, J. R. Williamson, K. Yatani, eds., CHI '22: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022 -5 May 2022, pages 385:1-385:22. ACM, 2022.
  106. Wang, X., W. Wang, Y. Cao, et al. Images speak in images: A generalist painter for in-context visual learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 6830-6839. IEEE, 2023.
  107. Razdaibiedina, A., Y. Mao, R. Hou, et al. Progressive prompts: Continual learning for language models. In The Eleventh International Conference on Learning Representations. 2023.
  108. Marshall, L. H., H. W. Magoun. Discoveries in the human brain: neuroscience prehistory, brain structure, and function. Springer Science & Business Media, 2013.
  109. Searle, J. R. What is language: some preliminary remarks. Explorations in Pragmatics. Linguistic, cognitive and intercultural aspects, pages 7-37, 2007.
  110. Zhuge, M., H. Liu, F. Faccio, et al. Mindstorms in natural language-based societies of mind. CoRR, abs/2305.17066, 2023. · arXiv
  111. Roller, S., E. Dinan, N. Goyal, et al. Recipes for building an open-domain chatbot. In P. Merlo, J. Tiedemann, R. Tsarfaty, eds., Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19 -23, 2021, pages 300-325. Association for Computational Linguistics, 2021.
  112. Raffel, C., N. Shazeer, A. Roberts, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485-5551, 2020.
  113. Rajpurkar, P., J. Zhang, K. Lopyrev, et al. Squad: 100, 000+ questions for machine com- prehension of text. In J. Su, X. Carreras, K. Duh, eds., Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 2383-2392. The Association for Computational Linguistics, 2016.
  114. See, A., A. Pappu, R. Saxena, et al. Do massively pretrained language models make better storytellers? In M. Bansal, A. Villavicencio, eds., Proceedings of the 23rd Conference on Computational Natural Language Learning, CoNLL 2019, Hong Kong, China, November 3-4, 2019, pages 843-861. Association for Computational Linguistics, 2019.
  115. Radford, A., J. Wu, D. Amodei, et al. Better language models and their implications. OpenAI blog, 1(2), 2019.
  116. Tellex, S., T. Kollar, S. Dickerson, et al. Understanding natural language commands for robotic navigation and mobile manipulation. In W. Burgard, D. Roth, eds., Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2011, San Francisco, California, USA, August 7-11, 2011, pages 1507-1514. AAAI Press, 2011.
  117. Christiano, P. F., J. Leike, T. B. Brown, et al. Deep reinforcement learning from human preferences. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, R. Garnett, eds., Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4299-4307. 2017.
  118. Basu, C., M. Singhal, A. D. Dragan. Learning from richer human guidance: Augmenting comparison-based learning with feature queries. In T. Kanda, S. Sabanovic, G. Hoffman, A. Tapus, eds., Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, HRI 2018, Chicago, IL, USA, March 05-08, 2018, pages 132-140. ACM, 2018.
  119. Sumers, T. R., M. K. Ho, R. X. D. Hawkins, et al. Learning rewards from linguistic feedback. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 6002-6010. AAAI Press, 2021.
  120. Jeon, H. J., S. Milli, A. D. Dragan. Reward-rational (implicit) choice: A unifying formalism for reward learning. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin, eds., Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. 2020.
  121. McShane, M. Reference resolution challenges for intelligent agents: The need for knowledge. IEEE Intell. Syst., 24(4):47-58, 2009.
  122. Gururangan, S., A. Marasovic, S. Swayamdipta, et al. Don't stop pretraining: Adapt language models to domains and tasks. In D. Jurafsky, J. Chai, N. Schluter, J. R. Tetreault, eds., Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 8342-8360. Association for Computational Linguistics, 2020.
  123. Shi, F., X. Chen, K. Misra, et al. Large language models can be easily distracted by irrelevant context. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett, eds., International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, vol. 202 of Proceedings of Machine Learning Research, pages 31210-31227. PMLR, 2023.
  124. Nuxoll, A. M., J. E. Laird. Extending cognitive architecture with episodic memory. In AAAI, pages 1560-1564. 2007.
  125. Squire, L. R. Mechanisms of memory. Science, 232(4758):1612-1619, 1986.
  126. Schwabe, L., K. Nader, J. C. Pruessner. Reconsolidation of human memory: brain mechanisms and clinical relevance. Biological psychiatry, 76(4):274-280, 2014.
  127. Zhang, X., F. Wei, M. Zhou. HIBERT: document level pre-training of hierarchical bidirectional transformers for document summarization. In A. Korhonen, D. R. Traum, L. Màrquez, eds., Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019, Volume 1: Long Papers, pages 5059-5069. Association for Computational Linguistics, 2019.
  128. Nie, Y., H. Huang, W. Wei, et al. Capturing global structural information in long document question answering with compressive graph selector network. In Y. Goldberg, Z. Kozareva, Y. Zhang, eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 5036-5047. Association for Computational Linguistics, 2022.
  129. Manakul, P., M. J. F. Gales. Sparsity and sentence structure in encoder-decoder attention of summarization systems. In M. Moens, X. Huang, L. Specia, S. W. Yih, eds., Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 9359-9368. Association for Computational Linguistics, 2021.
  130. Zaheer, M., G. Guruganesh, K. A. Dubey, et al. Big bird: Transformers for longer sequences. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin, eds., Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. 2020.
  131. Wason, P. C. Reasoning about a rule. Quarterly journal of experimental psychology, 20(3):273- 281, 1968.
  132. Wason, P. C., P. N. Johnson-Laird. Psychology of reasoning: Structure and content, vol. 86. Harvard University Press, 1972.
  133. Galotti, K. M. Approaches to studying formal and everyday reasoning. Psychological bulletin, 105(3):331, 1989.
  134. Huang, J., K. C. Chang. Towards reasoning in large language models: A survey. In A. Rogers, J. L. Boyd-Graber, N. Okazaki, eds., Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 1049-1065. Association for Computational Linguistics, 2023.
  135. Grafman, J., L. Spector, M. J. Rattermann. Planning and the brain. In The cognitive psychology of planning, pages 191-208. Psychology Press, 2004.
  136. Unterrainer, J. M., A. M. Owen. Planning and problem solving: from neuropsychology to functional neuroimaging. Journal of Physiology-Paris, 99(4-6):308-317, 2006.
  137. Zula, K. J., T. J. Chermack. Integrative literature review: Human capital planning: A review of literature and implications for human resource development. Human Resource Development Review, 6(3):245-262, 2007.
  138. Bratman, M. E., D. J. Israel, M. E. Pollack. Plans and resource-bounded practical reasoning. Computational intelligence, 4(3):349-355, 1988.
  139. Russell, S., P. Norvig. Artificial intelligence -a modern approach, 2nd Edition. Prentice Hall series in artificial intelligence. Prentice Hall, 2003.
  140. Fainstein, S. S., J. DeFilippis. Readings in planning theory. John Wiley & Sons, 2015.
  141. Sebastia, L., E. Onaindia, E. Marzal. Decomposition of planning problems. Ai Communica- tions, 19(1):49-81, 2006.
  142. Crosby, M., M. Rovatsos, R. Petrick. Automated agent decomposition for classical planning. In Proceedings of the International Conference on Automated Planning and Scheduling, vol. 23, pages 46-54. 2013.
  143. Raman, S. S., V. Cohen, E. Rosen, et al. Planning with large language models via corrective re-prompting. CoRR, abs/2211.09935, 2022. · arXiv
  144. Huang, W., P. Abbeel, D. Pathak, et al. Language models as zero-shot planners: Extracting ac- tionable knowledge for embodied agents. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvári, G. Niu, S. Sabato, eds., International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, vol. 162 of Proceedings of Machine Learning Research, pages 9118-9147. PMLR, 2022.
  145. Peters, M. E., M. Neumann, M. Iyyer, et al. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237. Association for Computational Linguistics, New Orleans, Louisiana, 2018.
  146. Devlin, J., M. Chang, K. Lee, et al. BERT: pre-training of deep bidirectional transformers for language understanding. In J. Burstein, C. Doran, T. Solorio, eds., Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4171-4186. Association for Computational Linguistics, 2019.
  147. Solaiman, I., C. Dennison. Process for adapting language models to society (palms) with values-targeted datasets. Advances in Neural Information Processing Systems, 34:5861-5873, 2021.
  148. Bach, S. H., V. Sanh, Z. X. Yong, et al. Promptsource: An integrated development environment and repository for natural language prompts. In V. Basile, Z. Kozareva, S. Stajner, eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 -System Demonstrations, Dublin, Ireland, May 22-27, 2022, pages 93-104. Association for Computational Linguistics, 2022.
  149. Winston, P. H. Learning and reasoning by analogy. Commun. ACM, 23(12):689-703, 1980.
  150. Lu, Y., M. Bartolo, A. Moore, et al. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In S. Muresan, P. Nakov, A. Villavicencio, eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 8086-8098. Association for Computational Linguistics, 2022.
  151. Tsimpoukelli, M., J. Menick, S. Cabi, et al. Multimodal few-shot learning with frozen language models. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, J. W. Vaughan, eds., Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 200-212. 2021.
  152. Bar, A., Y. Gandelsman, T. Darrell, et al. Visual prompting via image inpainting. In NeurIPS. 2022.
  153. Zhang, J., J. Zhang, K. Pertsch, et al. Bootstrap your own skills: Learning to solve new tasks with large language model guidance. In 7th Annual Conference on Robot Learning. 2023.
  154. Kirkpatrick, J., R. Pascanu, N. Rabinowitz, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526, 2017.
  155. Li, Z., D. Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2947, 2017.
  156. Farajtabar, M., N. Azizan, A. Mott, et al. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics, pages 3762-3773. PMLR, 2020.
  157. Lopez-Paz, D., M. Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
  158. de Masson D'Autume, C., S. Ruder, L. Kong, et al. Episodic memory in lifelong language learning. Advances in Neural Information Processing Systems, 32, 2019.
  159. Rolnick, D., A. Ahuja, J. Schwarz, et al. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019.
  160. Serrà, J., D. Surís, M. Miron, et al. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning. 2018.
  161. Dosovitskiy, A., L. Beyer, A. Kolesnikov, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  162. van den Oord, A., O. Vinyals, K. Kavukcuoglu. Neural discrete representation learning. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, R. Garnett, eds., Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6306-6315. 2017.
  163. Mehta, S., M. Rastegari. Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
  164. Tolstikhin, I. O., N. Houlsby, A. Kolesnikov, et al. Mlp-mixer: An all-mlp architecture for vision. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, J. W. Vaughan, eds., Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 24261-24272. 2021.
  165. Li, J., D. Li, S. Savarese, et al. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett, eds., International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, vol. 202 of Proceedings of Machine Learning Research, pages 19730-19742. PMLR, 2023.
  166. Alayrac, J., J. Donahue, P. Luc, et al. Flamingo: a visual language model for few-shot learning. In NeurIPS. 2022.
  167. Gong, Y., Y. Chung, J. R. Glass. AST: audio spectrogram transformer. In H. Hermansky, H. Cernocký, L. Burget, L. Lamel, O. Scharenborg, P. Motlícek, eds., Interspeech 2021, 22nd Annual Conference of the International Speech Communication Association, Brno, Czechia, 30 August -3 September 2021, pages 571-575. ISCA, 2021.
  168. Hsu, W., B. Bolte, Y. H. Tsai, et al. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE ACM Trans. Audio Speech Lang. Process., 29:3451-3460, 2021.
  169. Hubel, D. H., T. N. Wiesel. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of physiology, 160(1):106, 1962.
  170. Logothetis, N. K., D. L. Sheinberg. Visual object recognition. Annual review of neuroscience, 19(1):577-621, 1996.
  171. Lu, J., X. Ren, Y. Ren, et al. Improving contextual language models for response retrieval in multi-turn conversation. In J. X. Huang, Y. Chang, X. Cheng, J. Kamps, V. Murdock, J. Wen, Y. Liu, eds., Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, pages 1805-1808. ACM, 2020.
  172. Huang, L., W. Wang, J. Chen, et al. Attention on attention for image captioning. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 -November 2, 2019, pages 4633-4642. IEEE, 2019.
  173. Pan, Y., T. Yao, Y. Li, et al. X-linear attention networks for image captioning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 10968-10977. Computer Vision Foundation / IEEE, 2020.
  174. Cornia, M., M. Stefanini, L. Baraldi, et al. M 2 : Meshed-memory transformer for image captioning. CoRR, abs/1912.08226, 2019. · arXiv
  175. Chen, J., H. Guo, K. Yi, et al. Visualgpt: Data-efficient image captioning by balancing visual input and linguistic knowledge from pretraining. CoRR, abs/2102.10407, 2021. · arXiv
  176. Vaswani, A., N. Shazeer, N. Parmar, et al. Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, R. Garnett, eds., Advances in Neural Information Processing Systems 30: Annual Conference on Neural In- formation Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008. 2017.
  177. Touvron, H., M. Cord, M. Douze, et al. Training data-efficient image transformers & distil- lation through attention. In M. Meila, T. Zhang, eds., Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, vol. 139 of Proceedings of Machine Learning Research, pages 10347-10357. PMLR, 2021.
  178. Lu, J., C. Clark, R. Zellers, et al. UNIFIED-IO: A unified model for vision, language, and multi-modal tasks. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
  179. Radford, A., J. W. Kim, T. Xu, et al. Robust speech recognition via large-scale weak su- pervision. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett, eds., International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, vol. 202 of Proceedings of Machine Learning Research, pages 28492-28518. PMLR, 2023.
  180. Ren, Y., Y. Ruan, X. Tan, et al. Fastspeech: Fast, robust and controllable text to speech. In H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. B. Fox, R. Garnett, eds., Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 3165-3174. 2019.
  181. In L. D. Raedt, ed., Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, pages 4468-4474. ijcai.org, 2022.
  182. Kim, J., J. Kong, J. Son. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In M. Meila, T. Zhang, eds., Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, vol. 139 of Proceedings of Machine Learning Research, pages 5530-5540. PMLR, 2021.
  183. Wang, Z., S. Cornell, S. Choi, et al. Tf-gridnet: Integrating full-and sub-band modeling for speech separation. IEEE ACM Trans. Audio Speech Lang. Process., 31:3221-3236, 2023.
  184. Liu, J., C. Li, Y. Ren, et al. Diffsinger: Singing voice synthesis via shallow diffusion mechanism. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty- Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 -March 1, 2022, pages 11020-11028. AAAI Press, 2022.
  185. Inaguma, H., S. Dalmia, B. Yan, et al. Fast-md: Fast multi-decoder end-to-end speech transla- tion with non-autoregressive hidden intermediates. In IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2021, Cartagena, Colombia, December 13-17, 2021, pages 922-929. IEEE, 2021.
  186. Flanagan, J. L. Speech analysis synthesis and perception, vol. 3. Springer Science & Business Media, 2013.
  187. Schwarz, B. Mapping the world in 3d. Nature Photonics, 4(7):429-430, 2010.
  188. Parkinson, B. W., J. J. Spilker. Progress in astronautics and aeronautics: Global positioning system: Theory and applications, vol. 164. Aiaa, 1996.
  189. Clarebout, G., J. Elen, N. A. J. Collazo, et al. Metacognition and the Use of Tools, pages 187-195. Springer New York, New York, NY, 2013.
  190. Qian, C., C. Han, Y. R. Fung, et al. CREATOR: disentangling abstract and concrete reasonings of large language models through tool creation. CoRR, abs/2305.14318, 2023. · arXiv
  191. Shah, D., B. Osinski, B. Ichter, et al. Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action. In K. Liu, D. Kulic, J. Ichnowski, eds., Conference on Robot Learning, CoRL 2022, 14-18 December 2022, Auckland, New Zealand, vol. 205 of Proceedings of Machine Learning Research, pages 492-504. PMLR, 2022.
  192. Fan, L., G. Wang, Y. Jiang, et al. Minedojo: Building open-ended embodied agents with internet-scale knowledge. In NeurIPS. 2022.
  193. Nottingham, K., P. Ammanabrolu, A. Suhr, et al. Do embodied agents dream of pixelated sheep: Embodied decision making using language guided world modelling. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett, eds., International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, vol. 202 of Proceedings of Machine Learning Research, pages 26311-26325. PMLR, 2023.
  194. Sumers, T., K. Marino, A. Ahuja, et al. Distilling internet-scale vision-language models into embodied agents. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett, eds., International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, vol. 202 of Proceedings of Machine Learning Research, pages 32797-32818. PMLR, 2023.
  195. Savelka, J., K. D. Ashley, M. A. Gray, et al. Can GPT-4 support analysis of textual data in tasks requiring highly specialized domain expertise? In F. Lagioia, J. Mumford, D. Odekerken, H. Westermann, eds., Proceedings of the 6th Workshop on Automated Semantic Analysis of Information in Legal Text co-located with the 19th International Conference on Artificial Intelligence and Law (ICAIL 2023), Braga, Portugal, 23rd September, 2023, vol. 3441 of CEUR Workshop Proceedings, pages 1-12. CEUR-WS.org, 2023.
  196. Linardatos, P., V. Papastefanopoulos, S. Kotsiantis. Explainable AI: A review of machine learning interpretability methods. Entropy, 23(1):18, 2021.
  197. Hussein, A., M. M. Gaber, E. Elyan, et al. Imitation learning: A survey of learning methods. ACM Comput. Surv., 50(2):21:1-21:35, 2017.
  198. Liu, Y., A. Gupta, P. Abbeel, et al. Imitation from observation: Learning to imitate behaviors from raw video via context translation. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, pages 1118-1125. IEEE, 2018.
  199. Baker, B., I. Akkaya, P. Zhokov, et al. Video pretraining (VPT): learning to act by watching unlabeled online videos. In NeurIPS. 2022.
  200. Levine, S., P. Pastor, A. Krizhevsky, et al. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int. J. Robotics Res., 37(4-5):421-436, 2018.

200 of 378 shown

Authors
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., Zheng, R., Fan, X., Wang, X., Xiong, L., Zhou, Y., Wang, W., Jiang, C., Zou, Y., Liu, X., Yin, Z., Dou, S., Weng, R., Cheng, W., Zhang, Q., Qin, W., Zheng, Y., Qiu, X., Huang, X., & Gui, T.
Affiliations
  • Fudan University (CN)
  • Meituan (China) (CN)
Corresponding author
  • Zhang, Qi
  • Gui, Tao
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Volume
68
Issue
2
ISSN
1869-1919 (ISSN-L)
DOI
10.48550/arxiv.2309.07864
arXiv
2309.07864
OpenAlex
W4406800520
Cited by
972
References
485
Citations by year
2026: 464 · 2025: 338 · 2024: 159 · 2023: 11
FWCI
320.54
Type
journal article
Language
en
Open access
Licence
Locations
  • Science China Information Sciences (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
large language models, LLM agents, AI agents, artificial general intelligence, agent applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review