From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs
Open accessAbstract
Recent advancements in large language models have significantly improved their context windows, yet challenges in effective long-term memory management remain. We introduce MemTree, an algorithm that leverages a dynamic, tree-structured memory representation to optimize the organization, retrieval, and integration of information, akin to human cognitive schemas. MemTree organizes memory hierarchically, with each node encapsulating aggregated textual content, corresponding semantic embeddings, and varying abstraction levels across the tree's depths. Our algorithm dynamically adapts this memory structure by computing and comparing semantic embeddings of new and existing information to enrich the model's context-awareness. This approach allows MemTree to handle complex reasoning and extended interactions more effectively than traditional memory augmentation methods, which often rely on flat lookup tables. Evaluations on benchmarks for multi-turn dialogue understanding and document question answering show that MemTree significantly enhances performance in scenarios that demand structured memory management.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The deeper MemTree setting provides only modest gains over the flat setting.
“the deeper MemTree setting provides only modest gains over the flat setting”
From Are We Ready For An Agent-Native Memory System? (2026) · cites this paper as [27]
In the adaptive-threshold ablation (Section A.3.7, page unavailable), a lower threshold produces a deeper MemTree and yields only a slight, statistically nonsignificant accuracy improvement. The comparison is with a shallower tree, however, not an explicitly flat setting, so the stated comparator is not established by our copy. (LLM, conf=0.82)
Experimental — accuracy not yet measuredMTMemory (MemTree) is a dynamic memory model with a tree-structured semantic representation to organize information.
“• MTMemory (MemTree [8]): A dynamic memory model with a tree-structured semantic representation to organize information.”
From MemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agents (2025) · cites this paper as [8]
MemTree employs a hierarchical memory structure for organizing and retrieving information dynamically.
“MemTree employs a hierarchical memory structure for organizing and retrieving information dynamically [14]”
From Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control (2025) · cites this paper as [14]
MemTree organizes interaction history as a hierarchical dynamically evolving tree whose nodes hold abstracted summaries, enabling continual updates of memory.
“MemTree organizes interaction history as a hierarchical dynamically evolving tree whose nodes hold abstracted summaries, enabling continual updates of memory [14].”
From Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control (2025) · cites this paper as [14]
More advanced write policies may incorporate richer memory management structures to handle structured knowledge abstractions.
“More advanced approaches may incorporate richer memory management structures [14] to handle structured knowledge abstractions.”
From Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control (2025) · cites this paper as [14]
MemTree reaches 63.5 Normalized Utility at 15.9 s Avg. Operation Latency/Query.
“MemTree reaching 63.5 at 15.9 s”
From Are We Ready For An Agent-Native Memory System? (2026) · cites this paper as [27]
The Time Efficiency section reports an average of 10 seconds to insert new information, not average operation latency per query. Searches for 63.5, 15.9, and normalized utility found no passage reporting the claimed pair of measurements for MemTree. (LLM, conf=0.76)
Experimental — accuracy not yet measuredAll of these designs assume a single-user, single-agent setting with globally visible memory.
“Although highly effective, all of these designs assume a single-user, single-agent setting with globally visible memory.”
From Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control (2025) · cites this paper as [14]
The Introduction and Memory-Augmented LLMs section discuss memory structures and retrieval, but do not say that the designs assume a single user, a single agent, or globally visible memory. The Datasets section describes dialogue between two agents, but does not establish the memory-access assumptions of the designs. Searches for single-user, single-agent, shared-memory, and access-related terms did not settle the claim. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 34
- Petr Anokhin, Nikita Semenov, Artyom Sorokin, Dmitry Evseev, Mikhail Burtsev, and Evgeny Burnaev. Arigraph: Learning knowledge graph world models with episodic memory for llm agents. arXiv preprint arXiv:2407.04363, 2024. · arXiv
- Jinheon Baek, Alham Fikri Aji, and Amir Saffari. Knowledge-augmented language model prompting for zero-shot knowledge graph question answering. Proceedings of the First Work- shop on Matching From Unstructured and Structured Data (MATCHING 2023), 2023. URL https://api.semanticscholar.org/CorpusID:260063238.
- Taiyu Ban, Lyvzhou Chen, Xiangyu Wang, and Huanhuan Chen. From query tools to causal architects: Harnessing large language models for advanced causal discovery from data. ArXiv, abs/2306.16902, 2023. URL https://api.semanticscholar.org/CorpusID:259287331. · arXiv
- Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020. · arXiv
- Aydar Bulatov, Yuri Kuratov, Yermek Kapushev, and Mikhail S Burtsev. Scaling transformer to 1m tokens and beyond with rmt. arXiv preprint arXiv:2304.11062, 2023. · arXiv
- Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. Longlora: Efficient fine-tuning of long-context large language models. arXiv preprint arXiv:2309.12307, 2023. · arXiv
- Xin Cheng, Di Luo, Xiuying Chen, Lemao Liu, Dongyan Zhao, and Rui Yan. Lift yourself up: Retrieval-augmented text generation with self memory. ArXiv, abs/2305.02437, 2023. URL https://api.semanticscholar.org/CorpusID:258479968. · arXiv
- Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, Yuanyuan Xu, Ning Shang, Jiahang Xu, Fan Yang, and Mao Yang. Longrope: Extending llm context window beyond 2 million tokens. arXiv preprint arXiv:2402.13753, 2024. · arXiv
- Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. · arXiv
- Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130, 2024. · arXiv
- Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2023. · arXiv
- Asaf Gilboa and Hannah Marlatte. Neurobiology of schemas and schema-mediated memory. Trends in cognitive sciences, 21(8):618-631, 2017.
- Ari Kobren, Nicholas Monath, Akshay Krishnamurthy, and Andrew McCallum. A hierarchical algo- rithm for extreme clustering. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 255-264, 2017.
- Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Dmitry Sorokin, Artyom Sorokin, and Mikhail Burtsev. In search of needles in a 11m haystack: Recurrent memory finds what llms miss. ArXiv, abs/2402.10790, 2024. URL https://api.semanticscholar.org/CorpusID: 267740348. · arXiv
- Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157-173, 2024.
- Aditya Krishna Menon, Anand Rajagopalan, Baris Sumengen, Gui Citovsky, Qin Cao, and Sanjiv Kumar. Online hierarchical clustering approximations. arXiv preprint arXiv:1909.09667, 2019. · arXiv
- Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory-based model editing at scale. ArXiv, abs/2206.06520, 2022. URL https://api. semanticscholar.org/CorpusID:249642147. · arXiv
- Ali Modarressi, Ayyoob Imani, Mohsen Fayyaz, and Hinrich Schütze. Ret-llm: Towards a general read-write memory for large language models. arXiv preprint arXiv:2305.14322, 2023. · arXiv
- Benjamin Moseley and Joshua Wang. Approximation bounds for hierarchical clustering: Average linkage, bisecting k-means, and local search. Advances in Neural Information Processing Systems, 30, 2017.
- Charles Packer, Vivian Fang, Shishir G Patil, Kevin Lin, Sarah Wooders, and Joseph E Gonzalez. Memgpt: Towards llms as operating systems. arXiv preprint arXiv:2310.08560, 2023. · arXiv
- Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, et al. Quality: Question answering with long input texts, yes! arXiv preprint arXiv:2112.08608, 2021. · arXiv
- Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pp. 1-22, 2023.
- Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D Man- ning. Raptor: Recursive abstractive processing for tree-organized retrieval. arXiv preprint arXiv:2401.18059, 2024. · arXiv
- Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, and Mohit Iyyer. Do long-range language models actually use long-range context? ArXiv, abs/2109.09115, 2021. URL https://api. semanticscholar.org/CorpusID:237572264. · arXiv
- Yixuan Tang and Yi Yang. Multihop-rag: Benchmarking retrieval-augmented generation for multi- hop queries. arXiv preprint arXiv:2401.15391, 2024. Copyright © 2024. Licensed under CC BY-SA 4.0, including disclaimer of warranties. Available at https://arxiv.org/abs/2401.15391. · arXiv
- Milena Trajanoska, Riste Stojanov, and Dimitar Trajanov. Enhancing knowledge graph con- struction using large language models. ArXiv, abs/2305.04676, 2023. URL https://api. semanticscholar.org/CorpusID:258557103. · arXiv
- Szymon Tworkowski, Konrad Staniszewski, Mikołaj Pacek, Yuhuai Wu, Henryk Michalewski, and Piotr Miłoś. Focused transformer: Contrastive training for context scaling. Advances in Neural Information Processing Systems, 36, 2024.
- Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Improving text embeddings with large language models. arXiv preprint arXiv:2401.00368, 2023. · arXiv
- Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. arXiv preprint arXiv:1410.3916, 2014. · arXiv
- J Xu. Beyond goldfish memory: Long-term open-domain conversation. arXiv preprint arXiv:2107.07567, 2021. · arXiv
- Liang Yao, Jiazhen Peng, Chengsheng Mao, and Yuan Luo. Exploring large language mod- els for knowledge graph completion. ArXiv, abs/2308.13916, 2023. URL https://api. semanticscholar.org/CorpusID:261242758. · arXiv
- Tian Zhang, Raghu Ramakrishnan, and Miron Livny. Birch: an efficient data clustering method for very large databases. ACM sigmod record, 25(2):103-114, 1996.
- Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model based agents. arXiv preprint arXiv:2404.13501, 2024. · arXiv
- Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 19724-19731, 2024.
Cited by
Sorted by citations
A Survey on the Memory Mechanism of Large Language Model-based Agents
Zhang, Z., Dai, Q., Bo, X., Ma, C., Li, R., Chen, X., Zhu, J., Dong, Z., & Wen, J.-R. (2025) · ACM Transactions on Information Systems · doi:10.1145/3748302 · Cited by 118 · see citing works
MemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agents
Zhang, Z., Dai, Q., Chen, X., Li, R., Li, Z., & Dong, Z. (2025) · Companion Proceedings of the ACM on Web Conference 2025 · doi:10.1145/3701716.3715299 · Cited by 2 · see citing works
Self-citationShen, Y., Li, K., Zhou, W., & Hu, S. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.370 · Cited by 2 · see citing works
TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents
Li, K., Yu, X., Ni, Z., Zeng, Y., Xu, Y., Zhang, Z., Li, X., Sang, J., Duan, X., Wang, X., et al. (2026) · arXiv · doi:10.48550/arxiv.2601.02845 · Cited by 1 · see citing works
RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents
Dai, Z., Deng, S., Guan, S., Tian, Y., Yao, X., Yan, X., & Cheng, J. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1619
“TreeMem (Rezazadeh et al., 2025): Maintains a hierarchical summary tree.”
InfluentialMao, W., Tan, H., Liu, S., Liu, H., Xu, Y., Ji, H., & Wang, X. (2026) · arXiv · doi:10.48550/arxiv.2601.06490
“However, such fragmented factual memories struggle to capture inter-fact relationships or users’ high-level behavioral patterns (Rezazadeh et al., 2025; Pan et al., 2025).”
Collaborative Memory: Multi-User Memory Sharing in LLM Agents with Dynamic Access Control
Rezazadeh, A., Li, Z., Lou, A., Zhao, Y., Wei, W., & Bao, Y. (2025) · arXiv · doi:10.48550/arxiv.2505.18279
“MemTree organizes interaction history as a hierarchical dynamically evolving tree whose nodes hold abstracted summaries, enabling continual updates of memory [14].”
Self-citationChain-of-Memory: Lightweight Memory Construction with Dynamic Evolution for LLM Agents
Xu, X., Xu, B., Xueyun, T., Huang, Z., Chen, R., Yunfan, L., & Shen, H. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.534
Are We Ready For An Agent-Native Memory System?
Zhou, W., Zhou, X., Han, S., Xu, H., Li, G., Li, Z., Xiong, F., & Wu, F. (2026) · arXiv · doi:10.48550/arxiv.2606.24775
Self-citationMemory Matters More: Event-Centric Memory as a Logic Map for Agent Searching and Reasoning
Hu, Y., Liu, J., Tan, J., Zhu, Y., & Dou, Z. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1123
Qian, X., Xu, Y., Ma, C., & Chan, C. K. (2026) · Research Square · doi:10.21203/rs.3.rs-10522439/v1
- Authors
- Rezazadeh, A., Li, Z., Wei, W., & Bao, Y.
- Affiliations
- Accenture (Switzerland) (CH)
- ORCID
- Wei, Wei
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2410.14052
- OpenAlex
- W4403995458
- Cited by
- 1
- References
- 0
- Citations by year
- 2026: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, semantic embeddings, memory retrieval, document question answering, context awareness, memory enhancement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
