SCM: Enhancing Large Language Model with Self-Controlled Memory Framework
Open accessAbstract
Large Language Models (LLMs) are constrained by their inability to process lengthy inputs, resulting in the loss of critical historical information. To address this limitation, in this paper, we propose the Self-Controlled Memory (SCM) framework to enhance the ability of LLMs to maintain long-term memory and recall relevant information. Our SCM framework comprises three key components: an LLM-based agent serving as the backbone of the framework, a memory stream storing agent memories, and a memory controller updating memories and determining when and how to utilize memories from memory stream. Additionally, the proposed SCM is able to process ultra-long texts without any modification or fine-tuning, which can integrate with any instruction following LLMs in a plug-and-play paradigm. Furthermore, we annotate a dataset to evaluate the effectiveness of SCM for handling lengthy inputs. The annotated dataset covers three tasks: long-term dialogues, book summarization, and meeting summarization. Experimental results demonstrate that our method achieves better retrieval recall and generates more informative responses compared to competitive baselines in long-term dialogues. (https://github.com/wbbeyourself/SCM4LLMs)
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
SCM proposes a flash memory based on the cache mechanism, which preserves observations from the recent t-1 time steps, aimed at enhancing the recency of information.
“For example, SCM[98] proposes a flash memory based on the cache mechanism, which preserves observations from the recent t -1 time steps, aimed at enhancing the recency of information.”
From A Survey on the Memory Mechanism of Large Language Model-based Agents (2025) · cites this paper as [98]
The Activation Memory passage (page unavailable) says flash memory stores interaction memories from the previous turn, T−1; it does not say it preserves observations from the recent t−1 time steps. Section 2.3 describes cache storage for the memory stream generally, while Section 2.4 treats recency as a factor in memory retrieval, rather than stating that flash memory was designed to enhance it. (LLM, conf=0.85)
Experimental — accuracy not yet measuredLLMs overlook important contextual information from long dialogue histories due to the accumulation of irrelevant historical data ('historical noise').
“research has shown that LLMs overlook important contextual information from long dialogue histories due to the accumulation of irrelevant historical data, referred to as”
From TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialogues (2025) · cites this paper as Wang et al., 2023
SCMemory (SCM) is a self-controlled memory model that can recall minimal but necessary information for inference.
“• SCMemory (SCM [10]): A self-controlled memory model that can recall minimal but necessary information for inference.”
From MemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agents (2025) · cites this paper as [10]
The Controller Agent helps reduce the amount of data and prevent decreased performance caused by large input.
“This helps to reduce the amount of data and prevent decreased performance caused by large input [53].”
From CAIM: Development and Evaluation of a Cognitive AI Memory Framework for Long-Term Interaction with Intelligent Agents (2026) · cites this paper as [53]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Lost in the Middle: How Language Models Use Long Contexts
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00638 · Cited by 1,612 · see citing works
Query Rewriting in Retrieval-Augmented Large Language Models
Ma, X., Gong, Y., He, P., Zhao, H., & Duan, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.322 · Cited by 391 · see citing works
MemoryBank: Enhancing Large Language Models with Long-Term Memory
Zhong, W., Guo, L., Gao, Q., Ye, H., & Wang, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.10250 · Cited by 253 · see citing works
LongT5: Efficient Text-To-Text Transformer for Long Sequences
Guo, M., Ainslie, J., Uthus, D., Ontanon, S., Ni, J., Sung, Y.-H., & Yang, Y. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.55 · Cited by 230 · see citing works
A Survey of Natural Language Generation
Dong, C., Li, Y., Gong, H., Chen, M., Li, J., Shen, Y., & Yang, M. (2022) · ACM Computing Surveys · doi:10.1145/3554727 · Cited by 169 · see citing works
Alternating Language Modeling for Cross-Lingual Pre-Training
Yang, J., Ma, S., Zhang, D.-D., Wu, S.-Z., Li, Z.-J., & Zhou, M. (2020) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6480 · Cited by 99 · see citing works
Self-citationZhang, Y., Ni, A., Mao, Z., Wu, C. H., Zhu, C., Deb, B., Awadallah, A., Radev, D., & Zhang, R. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.112 · Cited by 44 · see citing works
HIBRIDS: Attention with Hierarchical Biases for Structure-aware Long Document Summarization
Cao, S., & Wang, L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.58 · Cited by 43 · see citing works
XCOT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning
Chai, L., Yang, J. J., Sun, T., Guo, H., Liu, J., Wang, B., Liang, X., Bai, J., Li, T., Peng, Q., et al. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i22.34524 · Cited by 30 · see citing works
Self-citationKnow What I don’t Know: Handling Ambiguous and Unknown Questions for Text-to-SQL
Wang, B., Gao, Y. F., Li, Z., & Lou, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.352 · Cited by 26 · see citing works
Self-citationPi, X., Wang, B., Gao, Y., Guo, J., Li, Z., & Lou, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.142 · Cited by 25 · see citing works
Self-citationKINet: Incorporating Relevant Facts Into Knowledge-Grounded Dialog Generation
Bai, J., Yang, Z., Yang, J., Guo, H., & Li, Z. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3240654 · Cited by 13 · see citing works
Self-citationVCSUM: A Versatile Chinese Meeting Summarization Dataset
Wu, H., Zhan, M., Tan, H., Hou, Z., Ding, L., & Song, L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.377 · Cited by 13 · see citing works
Improving Unsupervised Extractive Summarization by Jointly Modeling Facet and Redundancy
Liang, X., Li, J., Wu, S., Li, M., & Li, Z. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3138673 · Cited by 10 · see citing works
Self-citationBai, J., Guo, H., Liu, J., Yang, J., Liang, X., Yan, Z., & Li, Z. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3614901 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 11
- Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020a. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Pro- cessing Systems 2020, NeurIPS 2020, December 6- 12, 2020, virtual.
- Ofir Press, Noah Smith, and Mike Lewis. 2022. Train short, test long: Attention with linear biases enables input length extrapolation. In International Confer- ence on Learning Representations.
- Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
- Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. 2020. Learn- ing to summarize from human feedback. CoRR, abs/2009.01325. · arXiv
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems, volume 30.
- Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022b. Emer- gent abilities of large language models. Transactions on Machine Learning Research. Survey Certifica- tion.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022c. Chain of thought prompt- ing elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
- BigScience Workshop, :, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luc- cioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, and etc. 2023. Bloom: A 176b- parameter open-access multilingual language model.
- Jian Yang, Shuming Ma, Haoyang Huang, Dongdong Zhang, Li Dong, Shaohan Huang, Alexandre Muzio, Saksham Singhal, Hany Hassan, Xia Song, and Furu Wei. 2021. Multilingual machine translation systems from microsoft for WMT21 shared task. In Proceed- ings of the Sixth Conference on Machine Translation, WMT@EMNLP 2021, Online Event, November 10- 11, 2021, pages 446-455. Association for Computa- tional Linguistics.
- Jian Yang, Yuwei Yin, Shuming Ma, Liqun Yang, Hongcheng Guo, Haoyang Huang, Dongdong Zhang, Yutao Zeng, Zhoujun Li, and Furu Wei. 2023. Hanoit: Enhancing context-aware translation via selective context. In Database Systems for Advanced Applica- tions -28th International Conference, DASFAA 2023, Tianjin, China, April 17-20, 2023, Proceedings, Part III, volume 13945 of Lecture Notes in Computer Sci- ence, pages 471-486. Springer. · DOI
- Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yuxiao Dong, and Jie Tang. 2023. GLM-130b: An open bilingual pre-trained model. In The Eleventh International Conference on Learning Representations (ICLR).
Cited by
Sorted by citations
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · see citing works
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · see citing works
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274 · see citing works
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · see citing works
A Survey on the Memory Mechanism of Large Language Model-based Agents
Zhang, Z., Dai, Q., Bo, X., Ma, C., Li, R., Chen, X., Zhu, J., Dong, Z., & Wen, J.-R. (2025) · ACM Transactions on Information Systems · doi:10.1145/3748302 · Cited by 118 · see citing works
Evaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.747 · Cited by 96 · see citing works
PANGeA: Procedural Artificial Narrative Using Generative AI for Turn-Based, Role-Playing Video Games
Buongiorno, S., Klinkert, L., Zhuang, Z., Chawla, T., & Clark, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v20i1.31876 · Cited by 37 · see citing works
A-Mem: Agentic Memory for LLM Agents
Xu, W., Liang, Z., Mei, K., Gao, H., Tan, J., & Zhang, Y. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-0593 · Cited by 51 · see citing works
Kang, J., Ji, M., Zhao, Z., & Bai, T. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.1318 · Cited by 23 · see citing works
A Survey on the Memory Mechanism of Large Language Model based Agents
Zhang, Z., Bo, X., Ma, C., Li, R., Chen, X., Dai, Q., Zhu, J., Dong, Z., & Wen, J.-R. (2024) · arXiv · doi:10.48550/arxiv.2404.13501 · Cited by 149 · see citing works
Enhancing memory retrieval in generative agents through LLM-trained cross attention networks
Hong, C., & He, Q. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1591618 · Cited by 7 · see citing works
KARMA: Augmenting Embodied AI Agents with Long-and-Short Term Memory Systems
Wang, Z., Yu, B., Zhao, J., Sun, W., Hou, S., Liang, S., Hu, X., Han, Y., & Gan, Y. (2025) · 2025 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra55743.2025.11128047 · Cited by 11 · see citing works
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
An, C., Gong, S., Zhong, M., Zhao, X., Li, M., Zhang, J., Kong, L., & Qiu, X. (2023) · arXiv · doi:10.48550/arxiv.2307.11088 · Cited by 6 · see citing works
Dong, Z., Tang, T., Li, J., Zhao, W. X., & Wen, J.-R. (2023) · arXiv · doi:10.48550/arxiv.2309.13345 · Cited by 7 · see citing works
“Unlimi-former (Bertsch et al., 2023) and Focused Trans-former (Tworkowski et al., 2023) retrieved top-k relevant tokens from cached external memories for each token, achieving near unlimited context length.”
BackgroundEvaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.17753 · Cited by 3 · see citing works
“…a range of previous dialogues and reasoning over retrieved segments in a temporal order (Lee et al., 2023b; Lu et al., 2023; Zhong et al., 2023; Liang et al., 2023) and/or using events to scaffold the dialogues (Jang et al., 2023b; Zhang et al., 2023) to enable consistency in long-term…”
BackgroundMemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agents
Tan, H., Zhang, Z., Ma, C., Chen, X., Dai, Q., & Dong, Z. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.989 · Cited by 7 · see citing works
H-MEM: Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents
Sun, H., Zeng, S., & Zhang, B. (2026) · Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.eacl-long.15 · Cited by 4 · see citing works
Westhäußer, R., Minker, W., & Zepf, S. (2026) · Proceedings of the 31st International Conference on Intelligent User Interfaces · doi:10.1145/3742413.3789222 · Cited by 1 · see citing works
Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
Long, L., He, Y., Ye, W., Pan, Y., Lin, Y., Li, H., Zhao, J., & Li, W. (2025) · arXiv · doi:10.48550/arxiv.2508.09736 · Cited by 2 · see citing works
MemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agents
Zhang, Z., Dai, Q., Chen, X., Li, R., Li, Z., & Dong, Z. (2025) · Companion Proceedings of the ACM on Web Conference 2025 · doi:10.1145/3701716.3715299 · Cited by 2 · see citing works
Self-citationFrom Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs
Wu, Y., Liang, S., Zhang, C., Wang, Y., Zhang, Y., Guo, H., Tang, R., & Liu, Y. (2025) · arXiv · doi:10.48550/arxiv.2504.15965 · Cited by 3 · see citing works
HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents
Zhu, J., Li, J., Zhang, C., Liu, J., & Yang, M. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.625 · Cited by 2 · see citing works
Memory in the Age of AI Agents
Hu, Y., Liu, S., Yue, Y., Zhang, G., Liu, B., Zhu, F., Lin, J., Guo, H., Dou, S., Xi, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.13564 · Cited by 1 · see citing works
Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents
Sun, H., & Zeng, S. (2025) · arXiv · doi:10.48550/arxiv.2507.22925 · Cited by 1 · see citing works
Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
Du, P. (2026) · arXiv · Cited by 1 · see citing works
“The Self-Controlled Memory system [Liang et al., 2023] hands this decision to the agent itself, letting it choose which segments deserve verbatim retention versus aggressive condensation.”
Membox: Weaving Topic Continuity into Long-Range Memory for LLM Agents
Tao, D., Ma, G., Huang, Y., & Jiang, M. (2026) · arXiv · doi:10.48550/arxiv.2601.03785
Ge, Y., Romeo, S., Cai, J., Shu, R., Sunkara, M., Benajiba, Y., & Zhang, Y. (2025) · arXiv · doi:10.48550/arxiv.2502.01630 · Cited by 3 · see citing works
Chain-of-Memory: Lightweight Memory Construction with Dynamic Evolution for LLM Agents
Xu, X., Xu, B., Xueyun, T., Huang, Z., Chen, R., Yunfan, L., & Shen, H. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.534
HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents
Zhu, J., Li, J., Zhang, C., Liu, J., & Yang, M. (2026) · arXiv · doi:10.48550/arxiv.2604.16839
Rapid Episodic-Merging Memory for LLM Agents
YANG, H., & Gopalakrishnan, A. K. (2026) · International Conference on Mechatronics, Robotics, and Artificial Intelligence (MRAI) · doi:10.1109/mrai70020.2026.11621122
- Authors
- Wang, B., Liang, X., Yang, J., Huang, H., Wu, S., Wu, P., Lu, L., Ma, Z., & Li, Z.
- Affiliations
- Beihang University (CN)
- Harbin Institute of Technology (CN)
- ByteDance (China) (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 188-203
- ISSN
- 0302-9743 (ISSN-L)
- arXiv
- 2304.13343
- OpenAlex
- W7117975211
- Cited by
- 5
- References
- 10
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 1 · 2023: 1
- FWCI
- 8.55
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, long-term memory, memory controller, LLM agents, meeting summarization, context maintenance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
