CoChat
Sign Up
PreprintCited by 7Domain Adaptation and Few-Shot LearningPsychiatry, Mental Health, NeuroscienceAI-based Problem Solving and Planning

The DOI matches its DataCite record; the references it cites are not checked yet.

Long Term Memory: The Foundation of AI Self-Evolution

Open access
Jiang, Xun, Li, Feng, Zhao, Han, Qiu, Jiahao, Wang, Jiaying, Shao, Jun, Xu, Shihao, Zhang, Shu, Chen, Weiling, Tang, Xavier, Chen, Yize, Wu, Mengyue, Ma, Weizhi, Wang, Mengdi, Chen, Tianqiao·2024·arXiv
DOI 10.48550/arxiv.2410.15665arXiv 2410.15665
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) like GPTs, trained on vast datasets, have demonstrated impressive capabilities in language understanding, reasoning, and planning, achieving human-level performance in various tasks. Most studies focus on enhancing these models by training on ever-larger datasets to build more powerful foundation models. While training stronger models is important, enabling models to evolve during inference is equally crucial, a process we refer to as AI self-evolution. Unlike large-scale training, self-evolution may rely on limited data or interactions. Inspired by the columnar organization of the human cerebral cortex, we hypothesize that AI models could develop cognitive abilities and build internal representations through iterative interactions with their environment. To achieve this, models need long-term memory (LTM) to store and manage processed interaction data. LTM supports self-evolution by representing diverse experiences across environments and agents. In this report, we explore AI self-evolution and its potential to enhance models during inference. We examine LTM's role in lifelong learning, allowing models to evolve based on accumulated interactions. We outline the structure of LTM and the systems needed for effective data retention and representation. We also classify approaches for building personalized models with LTM data and show how these models achieve self-evolution through interaction. Using LTM, our multi-agent framework OMNE achieved first place on the GAIA benchmark, demonstrating LTM's potential for AI self-evolution. Finally, we present a roadmap for future research, emphasizing the importance of LTM for advancing AI technology and its practical applications.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2410.15665

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Memory-augmented agents treating long-term storage as a first-class primitive persistently accumulate introspective state histories across interactions, establishing a form of temporal self-continuity.

    “memory-augmented agents treating long-term storage as a first-class primitive [71] persistently accumulate introspective state histories across interactions, establishing a form of temporal self-continuity.”

    From Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement (2026) · cites this paper as [71]

    Section 5.4 (page not provided) describes long-term memory as a component of multi-agent systems, with stored information updated across interactions and periodically reviewed through self-reflection. The paper does not say that agents accumulate introspective state histories or establish temporal self-continuity; those parts go beyond its account. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Most existing work on memory mechanisms focuses on analyzing and explaining memory from the temporal (time) dimension, specifically short-term and long-term memory.

    “Although previous studies and reviews have provided detailed explanations of memory mechanisms, most of the existing work focuses on analyzing and explaining memory from the temporal (time) dimension, specifically in terms of short-term and long-term memory [8, 7, 17].”

    From From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs (2025) · cites this paper as [8]

    Section 3.3, “Inspiration from Human LTM” (page not provided), says current research typically divides human memory into working, short-term, and long-term memory. That supports the claim that short- and long-term memory are common categories, but the paper does not say most existing work analyzes memory mechanisms primarily through a temporal dimension. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

The citation lists haven’t been fetched yet

Cited by 7

Part of this paper’s reference list could not be split into single entries.

Research works printed but not linked: 124

  1. Min Xu, Jeanne M. David, and Suk Hi Kim. The fourth industrial revolution: Opportunities and challenges. International Journal of Financial Research, 9(2):90-95, April 2018.
  2. I. Lehr Brisbin, Charles T. Collins, Gary C. White, and D. Alasdair McCallum. A new paradigm for the analysis and interpretation of growth data: The shape of things to come. The Auk, 104(3):552-554, 1987.
  3. Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm-pruner: On the structural pruning of large language models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 21702-21720.
  4. Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Frozen pretrained transformers as universal computation engines. Proceedings of the AAAI Conference on Artificial Intelligence, 36(7):7628-7636, 2022.
  5. Fabricio Ballarini, Diego Moncada, Maria Cecilia Martinez, Nadia Alen, and Haydée Viola. Behavioral tagging is a general mechanism of long-term memory formation. Proceedings of the National Academy of Sciences, 106(34):14599-14604, 2009.
  6. Brian R Johnson and Timothy A Linksvayer. Deconstructing the superorganism: social physiology, groundplans, and sociogenomics. The Quarterly Review of Biology, 85(1):57-79, 2010.
  7. Huao Li, Yu Chong, Simon Stepputtis, Joseph Campbell, Dana Hughes, Charles Lewis, and Katia Sycara. Theory of mind for multi-agent collaboration via large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2023.
  8. Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abilities of large language models a mirage? In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 55565-55581.
  9. Christopher J. Hewer and Robert Roberts. History, culture and cognition: Towards a dynamic model of social memory. Culture & Psychology, 18(2):167-183, 2012.
  10. Ali Oroojlooy and Darius Adam Hajinezhad. A review of cooperative multi-agent deep reinforcement learning. Applied Intelligence, 53:13677-13722, 2023.
  11. Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, and Xia Hu. Data-centric ai: Perspectives and challenges. In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM), pages 945-948. Society for Industrial and Applied Mathematics, 2023.
  12. Ajay Bandi, Pydi Venkata Satya Ramesh Adapa, and Yudu Eswar Vinay Pratap Kumar Kuchi. The power of generative ai: A review of requirements, models, input-output formats, evaluation metrics, and challenges. Future Internet, 15:260, 2023.
  13. E. Talvitie. Self-correcting models for model-based reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31. AAAI Press, 2017.
  14. J. C. Schön. Structure prediction in low dimensions: concepts, issues and examples. Philosoph- ical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 381(2250), Jul 2023.
  15. Marcel Nonnenmacher, Srinivas C Turaga, and Jakob H Macke. Extracting low-dimensional dynamics from multiple large-scale neural population recordings by learning to predict corre- lations. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30.
  16. Leonard A Smith. Identification and prediction of low dimensional dynamics. Physica D: Nonlinear Phenomena, 58(1):50-76, 1992.
  17. S Hochreiter. Long short-term memory. Neural Computation MIT-Press, 1997.
  18. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, volume 159, pages 1877-1901, 2020.
  19. James L. McGaugh. Memory-a century of consolidation. Science, 287(5451):248-251, 2000.
  20. Larry R. Squire. Memory systems of the brain: A brief history and current perspective. Neurobiology of Learning and Memory, 82(3):171-177, 2004.
  21. Yadin Dudai. The neurobiology of consolidations, or, how stable is the engram? Annual Review of Psychology, 55:51-86, 2004.
  22. Larry R. Squire and John T. Wixted. The cognitive neuroscience of human memory since h.m. Annual Review of Neuroscience, 34:259-288, 2011.
  23. Angelina R. Sutin, Justin Brown, Martina Luchetti, Damaris Aschwanden, Yannick Stephan, and Antonio Terracciano. Five-factor model personality traits and the trajectories of episodic memory: A meta-analysis of individual participant data from 120,640 participants and 471,821 memory assessments. The Journals of Gerontology: Series B, 78(3):421-433, March 2023.
  24. Y. N. Kenett, S. Humphries, and A. Chatterjee. A thirst for knowledge: Grounding curiosity, creativity, and aesthetics in memory and reward neural systems. Creativity Research Journal, 35(3):412-426, 2023.
  25. D. Z. Hambrick and R. W. Engle. The role of working memory in problem solving. In J. E. Davidson and R. J. Sternberg, editors, The Psychology of Problem Solving, pages 176-206. Cambridge University Press, 2003.
  26. M. S. Weldon and K. D. Bellinger. Collective memory: Collaborative and individual processes in remembering. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(5):1160-1175, 1997.
  27. Wanjun Zhong, Lin Guo, Qian Gao, Hao Ye, and Yijun Wang. Memorybank: Enhancing large language models with long-term memory. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):19724-19731, 2024.
  28. Yiming Du, Hongru Wang, Zhengyi Zhao, Bin Liang, Baojun Wang, Wanjun Zhong, Zezhong Wang, and Kam-Fai Wong. Perltqa: A personal long-term memory dataset for memory classification, retrieval, and fusion in question answering. In Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10), pages 152-164. Association for Computational Linguistics, 2024.
  29. Y. Wang, N. L. Zhang, and T. Chen. Latent tree models and approximate inference in bayesian networks. arXiv preprint arXiv:1401.3429, 2014. · arXiv
  30. W. Wei and L. Liu. Trustworthy distributed ai systems: Robustness, privacy, and governance. arXiv preprint arXiv:2402.01096, 2024. · arXiv
  31. Longbing Cao. In-depth behavior understanding and use: The behavior informatics approach. CoRR, abs/2007.15516, 2020. · arXiv
  32. Benediktus Rolando and Herry Mulyono. Unlocking the power of data: Effective data-driven marketing strategies to engage millennial consumers. Transekonomika: Akuntansi, Bisnis Dan Keuangan, 4(3):303-321, 2024.
  33. Luka Abb and Jana-Rebecca Rehse. A reference data model for process-related user interaction logs. Springer, pages 57-74, 2022.
  34. Muhammad Umar Khan, Zainoor Ahmad Choudry, Sumair Aziz, Syed Zohaib Hassan Naqvi, Afeefa Aymin, and Muhammad Atif Imtiaz. Biometric authentication based on emg signals of speech. IEEE, pages 1-5, 2020.
  35. Sonia Sabir, Inayat Ali, and Eraj Khan. H-lps: a hybrid approach for user's location privacy in location-based services. arXiv preprint arXiv:2212.08241, 2022. · arXiv
  36. Bing Yao, Changying Shi, Lei Zou, et al. D4: a chinese dialogue dataset for depression- diagnosis-oriented chat. arXiv preprint arXiv:2205.11764, 2022. · arXiv
  37. Alexey Kurakin, Natalia Ponomareva, Umar Syed, Liam MacDermed, and Andreas Terzis. Harnessing large-language models to generate private synthetic text. arXiv preprint arXiv:2306.01684, 2023. · arXiv
  38. Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, et al. Be- yond human data: Scaling self-training for problem-solving with language models. CoRR, abs/2312.06585, 2023. · arXiv
  39. Fabrizio Gilardi, Meysam Alizadeh, and Ma.l Kubli. Chatgpt outperforms crowd-workers for text-annotation tasks. CoRR, abs/2303.15056, 2023. · arXiv
  40. R. Wang, W. Zhou, and M. Sachan. Let's synthesize step by step: Iterative dataset synthe- sis with large language models by extrapolating errors from small models. arXiv preprint arXiv:2310.13671, 2023. · arXiv
  41. C. Cui, X. Peng, and M. Riedl. Thespian: Multi-character text role-playing game agents. arXiv preprint arXiv:2308.01872, 2023. · arXiv
  42. Abhay Zala, Jaemin Cho, Han Lin, Jaehong Yoon, and Mohit Bansal. Envgen: Generating and adapting environments via llms for training embodied agents. arXiv preprint arXiv:2403.12014, 2024. · arXiv
  43. Y. Yu, Q. Zhang, J. Li, Q. Fu, and D. Ye. Affordable generative agents. arXiv preprint arXiv:2402.02053, 2024. · arXiv
  44. Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xin- lei Chen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, and Carlos Guestrin. Learning to (learn at test time): Rnns with expressive hidden states. arXiv preprint arXiv:2407.04620, 2024. · arXiv
  45. Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William W. Cohen. Murag: Multimodal retrieval-augmented generator for open question answering over images and text. In arXiv preprint arXiv:2210.02928, 2022. · arXiv
  46. Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Ra-cm3: Retrieval-augmented causal masked multimodal model. In arXiv preprint arXiv:2211.12561, 2022. · arXiv
  47. Daixuan Cheng, Shaohan Huang, Junyu Bi, Yuefeng Zhan, Jianfeng Liu, Yujing Wang, Hao Sun, Furu Wei, Denvy Deng, and Qi Zhang. Uprise: Universal prompt retrieval for improving zero-shot evaluation. In arXiv preprint arXiv:2303.08518, 2023. · arXiv
  48. Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. Raptor: Recursive abstractive processing for tree-organized retrieval. In arXiv preprint arXiv:2401.18059, 2024. · arXiv
  49. Lucrezia Bottegoni. Memwalker: Memory tree approach for long document question answer- ing. In arXiv preprint arXiv:2310.0502, 2023.
  50. H. Yang, Z. Lin, W. Wang, H. Wu, Z. Li, B. Tang, W. Wei, J. Wang, Z. Tang, S. Song, C. Xi, Y. Yu, K. Chen, F. Xiong, L. Tang, and W. E. Memory³: Language modeling with explicit memory. arXiv preprint arXiv:2407.01178, 2024. · arXiv
  51. Kunal Sawarkar, Abhilasha Mangal, and Shivam Raj Solanki. Blended rag: Improving rag accuracy with semantic search and hybrid query-based retrievers. In arXiv preprint arXiv:2404.07220, 2024. · arXiv
  52. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511, 2023. · arXiv
  53. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023. · arXiv
  54. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Canton Ferrer, Cristian, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernan- des, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Singh Koura, Punit, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Michael Smith, Eric, Ranjan Sub- ramanian, Ellen Tan, Xiaoqing, Binh Tang, Ross Taylor, Adina Williams, Xiang Kuan, Jian, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. · arXiv
  55. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Le Scao, Teven, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and El Sayed, William. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. · arXiv
  56. Jupinder Parmar, Sanjev Satheesh, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catan- zaro. Reuse, don't retrain: A recipe for continued pretraining of language models. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
  57. Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Elle: Efficient lifelong pre-training for emerging data. Findings of the Association for Computational Linguistics, 2022.
  58. Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Stanley Choi Jungkyu, and Minjoon Seo. Towards continual knowledge learning of language models. arXiv preprint arXiv:2110.03215, 2022. · arXiv
  59. Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf, Dominic Culver, Rui Melo, Caio Corro, Andre F. T. Martins, Fabrizio Esposito, Vera Lucia Raposo, Sofia Morgado, and Desa Michael. Saullm-7b: A pioneering large language model for law. arXiv preprint arXiv:2403.03883, 2024. · arXiv
  60. Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. Continual pre-training of language models. arXiv preprint arXiv:2302.03241, 2023. · arXiv
  61. Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don't stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2302.03241, 2023. · arXiv
  62. Kshitij Gupta, Benjamin Therien, Adam Ibrahim, Mats L. Richter, Quentin Anthony, Belilovsky Eugene, Irina Rish, and Timothee Lesort. Continual pre-training of large lan- guage models: How to (re)warm your model? arXiv preprint arXiv:2308.04014, 2023. · arXiv
  63. Genta Indra Winata, Lingjue Xie, Karthik Radhakrishnan, Shijie Wu, Xisen Jin, Pengxiang Cheng, Mayank Kulkarni, and Daniel Preotiuc-Pietro. Overcoming catastrophic forgetting in massively multilingual continual learning. arXiv preprint arXiv:2305.16252, 2023. · arXiv
  64. Adam Ibrahim, Benjamin Therien, Kshitij Gupta, Mats L. Richter, Quentin Anthony, Timon- thee Lesort, Eugene Belilovsky, and Irina Rish. Simple and scalable strategies to continually pre-train large language models. arXiv preprint arXiv:2403.08763, 2024. · arXiv
  65. Chengyue Wu, Yukang Gan, Yixiao Ge, Zeyu Lu, Jiahao Wang, Ye Feng, Ying Shan, and Ping Luo. Llama pro: Progressive llama with block expansion. arXiv preprint arXiv:2401.02415, 2024. · arXiv
  66. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feed- back. In Proceedings of the 36th International Conference on Neural Information Processing Systems, volume 2011, pages 27730-27744, 2022.
  67. Zhengyan shi, Adam X. Yang, Bin Wu, Laurence Aitchison, Emine YilmaZ, and Aldo Lipani. Instruction tuning with loss over instructions. arXiv preprint arXiv:2405.14394, 2024. · arXiv
  68. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25, 2024.
  69. Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. Hello again! llm-powered personalized agent for long-term dialogue. arXiv preprint arXiv:2406.05925, 2024. · arXiv
  70. Kai Zhang, Fubang Zhao, Yangyang Kang, and Xiaozhong Liu. Memory-augmented llm per- sonalization with short-and long-term memory coordination. arXiv preprint arXiv:2309.11696, 2023. · arXiv
  71. Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Xiang-Bo Mao, Sitaram Asur, et al. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. arXiv preprint arXiv:2407.16216, 2024. · arXiv
  72. Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267, 2023. · arXiv
  73. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017.
  74. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
  75. Jiwoo Hong, Noah Lee, and James Thorne. Orpo: Monolithic preference optimization without reference model. arXiv preprint arXiv:2403.07691, 2(4):5, 2024. · arXiv
  76. Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. arXiv preprint arXiv:2405.14734, 2024. · arXiv
  77. Swaroop Nath, Tejpalsingh Siledar, Sankara Sri Raghava Ravindra Muddu, Rupasai Rangaraju, Harshad Khadilkar, Pushpak Bhattacharyya, Suman Banerjee, Amey Patil, Sudhanshu Shekhar Singh, Muthusamy Chelliah, and Nikesh Garera. Leveraging domain knowledge for efficient reward modelling in rlhf: A case-study in e-commerce opinion summarization. arXiv preprint arXiv:2402.15473, 2024. · arXiv
  78. Tzu-Han Lin, Chen-An Li, Hung-yi Lee, and Yun-Nung Chen. Dogerm: Equipping reward models with domain knowledge through model merging. arXiv preprint arXiv:2407.01470, 2024. · arXiv
  79. Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model based agents. arXiv preprint arXiv:2404.13501, 2024. · arXiv
  80. Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context win- dow of large language models via positional interpolation. arXiv preprint arXiv:2306.15595, 2023. · arXiv
  81. Ofir Press, Noah A. Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length extrapolation. International Conference on Learning Representations, 2022.
  82. Yira Ding, Li Lyna Zhang, Chengruidong Zhang, Yuanyuan Xu, Ning Shang, Jiahang Xu, Fang Yang, and Yang Mao. Longrope: Extending llm context window beyond 2 million tokens. arXiv preprint arXiv:2402.13753, 2024. · arXiv
  83. Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071, 2023. · arXiv
  84. Dawei Zhu, Nan Yang, Liang Wang, Yifan Song, Wenhao Wu, Furu Wei, and Sujian Li. Pose: Efficient context window extension of llms via positional skip-wise training. arXiv preprint arXiv:2309.10400, 2024. · arXiv
  85. Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. Longlora: Efficient fine-tuning of long-context large language models. arXiv preprint arXiv:2309.12307, 2024. · arXiv
  86. Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, Madian Khabsa, Han Fang, Yashar Mehdad, Sharan Narang, Kshitiz Malik, Angela Fan, Shruti Bhosale, Sergey Edunov, Mike Lewis, Sinong Wang, and Hao Ma. Effective long-context scaling of foundation models. arXiv preprint arXiv:2309.16039, 2023. · arXiv
  87. Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, and Furu Wei. Structured prompting: Scaling in-context learning to 1,000 examples. arXiv preprint arXiv:2212.06713, 2022. · arXiv
  88. Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Ammon Shashua, Kevin Leyton-Brown, and Yoav Shoham. Parallel context windows for large language models. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, 1:6383--6402, 2023.
  89. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Lllia Polosukhin. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 6000-6010, 2017.
  90. Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19724-19731, 2024.
  91. Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023.
  92. Lei Wang, Jingsen Zhang, Hao Yang, Zhiyuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Ruihua Song, Zhao Xin Wayne, Jun Xu, Zhicheng Dou, Jun Wang, and Ji-Rong Wen. User behavior simulation with large language model based agents. arXiv preprint arXiv:2306.02552, 2023. · arXiv
  93. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. Character-llm: A trainable agent for role-playing. arXiv preprint arXiv:2310.10158, 2023. · arXiv
  94. Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Libiao Peng, Jiaming Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Jie Tang, and Minlie Huang. Characterglm: Customizing chinese conversational ai characters with large language models. arXiv preprint arXiv:2311.16832, 2023. · arXiv
  95. Li Cheng, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi Mi, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun. Chatharuhi: Reviving anime character in reality via large language model. arXiv preprint arXiv:2308.09597, 2023. · arXiv
  96. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Scharli, Aakanksha Chowdhery, Philip Mansfield, Blaise Aguera y Arcas, Dale Webster, Greg S. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Natarajan. Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138, 2022. · arXiv
  97. Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, Mike Schaekermann, Amy Wang, Mohamed Amin, Sami Lachgar, Philip Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Aguera y Arcas, Nenad Tomasev, Yun Liu, Renee Wong, Christopher Semturs, S. Sara Mahdavi, Joelle Barral, Dale Webster, Greg S. Corrado, Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam, and Vivek Natarajan. Large language models encode clinical knowledge. arXiv preprint arXiv:2305.09617, 2023. · arXiv
  98. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways. The Journal of Machine Learning Research, 24(240), 2024.
  99. Haoxi Zhong, Chaojun Xiao, CunChao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. How does nlp benefit legal system: A summary of legal artificial intelligence. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
  100. Quzhe Huang, Mingxu Tao, Chen Zhang, Zhenwei An, Cong Jiang, Zhibin Chen, Zirui Wu, and Yansong FEng. Lawyer llama technical report. arXiv preprint arXiv:2305.15062, 2023. · arXiv
  101. Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Yuan Li. Chatlaw: Open-source legal large language model with integrated external knowledge bases. arXiv preprint arXiv:2306.16092v1, 2023. · arXiv
  102. Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shunjun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang, and Zhongyu Wei. Disc-lawllm: Fine-tuning large language models for intelligent legal services. arXiv preprint arXiv:2309.11325, 2023. · arXiv
  103. Sjijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023. · arXiv
  104. Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. Fingpt: Open-source financial large language models. arXiv preprint arXiv:2306.06031, 2023. · arXiv
  105. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Wang Haofen. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2023. · arXiv
  106. Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652, 2023. · arXiv
  107. Chengyu Huang, Zeqiu Wu, Yushi Hu, and Wenya Wang. Training language models to generate text with citations via fine-grained rewards. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024.
  108. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzales. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334, 2023. · arXiv
  109. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
  110. Yanyang Li, Shuo Liang, Michael R. Lyu, and Liwei Wang. Making long-context language models better multi-hop reasoners. arXiv preprint arXiv:2408.03246, 2024. · arXiv
  111. Junkai Li, Siyu Wang, Meng Zhang, Weitao Li, Yunghwei Lai, Xinhui Kang, Weizhi Ma, and Yang Liu. Agent hospital: A simulacrum of hospital with evolvable medical agents. arXiv preprint arXiv:2405.02957, 2024. · arXiv
  112. Hajra Murtaza, Musharif Ahmed, Naurin Farooq Khan, Ghulam Murtaza, Saad Zafar, and Ambreen Bano. Synthetic data generation: State of the art in health care domain. Computer Science Review, 48:100546, 2023.
  113. Chenhao Zhang, Renhao Li, Minghuan Tan, Min Yang, Jingwei Zhu, Di Yang, Jiahao Zhao, Guancheng Ye, Chengming Li, Xiping Hu, et al. Cpsycoun: A report-based multi-turn dialogue reconstruction and evaluation framework for chinese psychological counseling. arXiv preprint arXiv:2405.16433, 2024. · arXiv
  114. Michael B First. Structured clinical interview for the dsm (scid). The encyclopedia of clinical psychology, pages 1-6, 2014.
  115. Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. In Proceedings of the 56th Annual Meeting of the Association for Com- putational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 2018.
  116. Xiao Liu, Hanyu Lai, Hao Yu, Yifan Xu, Aohan Zeng, Zhengxiao Du, Peng Zhang, Yuxiao Dong, and Jie Tang. Webglm: Towards an efficient web-enhanced question answering system with human preferences. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4549-4560, 2023.
  117. Kevin Clark and Christopher D. Manning. Deep reinforcement learning for mention-ranking coreference models. arXiv preprint arXiv:1609.08667, 2016. · arXiv
  118. K Lan, B Jin, Z Zhu, et al. Depression diagnosis dialogue simulation: Self-improving psychiatrist with tertiary memory. arXiv preprint arXiv:2409.15084, 2024. · arXiv
  119. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. · arXiv
  120. Zhili He and Yu-Hsing Wang. Mamba meets crack segmentation. arXiv preprint arXiv:2407.15714, 2024. · arXiv
  121. Bo Peng, Daniel Goldstein, Quentin Anthony, Alon Albalak, Eric Alcaide, Stella Biderman, Eugene Cheah, Xingjian Du, Teddy Ferdinan, Haowen Hou, Przemysław Kazienko, Kran- thi Kiran GV, Jan Kocoń, Bartłomiej Koptyra, Satyapriya Krishna, Ronald McClelland Jr., Niklas Muennighoff, Fares Obeid, Atsushi Saito, Guangyu Song, Haoqin Tu, Stanisław Woź- niak, Ruichong Zhang, Bingchen Zhao, Qihang Zhao, Peng Zhou, Jian Zhu, and Rui-Jie Zhu. Eagle and finch: Rwkv with matrix-valued states and dynamic recurrence. arXiv preprint arXiv:2404.05892, 2024. · arXiv
  122. Feng-Lei Fan, Yingxin Li, Hanchuan Peng, Tieyong Zeng, and Fei Wang. Towards neuroai: Introducing neuronal diversity into artificial neural networks. arXiv preprint arXiv:2301.09245, 2023. · arXiv
  123. Jason K. Eshraghian, Max Ward, Emre O. Neftci, Xinxin Wang, Gregor Lenz, Girish Dwivedi, Mohammed Bennamoun, Doo Seok Jeong, and Wei D. Lu. Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE, 111(9):1016-1054, 2023.
  124. Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. arXiv preprint arXiv:2311.12983, 2023. · arXiv

Metadata

Authors
Jiang, X., Li, F., Zhao, H., Qiu, J., Wang, J., Shao, J., Xu, S., Zhang, S., Chen, W., Tang, X., Chen, Y., Wu, M., Ma, W., Wang, M., & Chen, T.
Affiliations
  • Princeton University (US)
  • Tsinghua University (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2410.15665
arXiv
2410.15665
OpenAlex
W4404344231
Cited by
7
References
0
Citations by year
2026: 3 · 2025: 4
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
long-term memory, large language models, lifelong learning, multi-agent systems, personalized models, interaction-based learning, memory architecture

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review