CAIM: Development and Evaluation of a Cognitive AI Memory Framework for Long-Term Interaction with Intelligent Agents
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index; 39 of the 58 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) have advanced the field of artificial intelligence (AI) and are a powerful enabler for interactive systems. However, they still face challenges in long-term interactions that require adaptation towards the user as well as contextual knowledge and an understanding of the ever-changing environment. To overcome these challenges, holistic memory modeling is required to efficiently retrieve and store relevant information across sessions for accurate responses. Cognitive AI, which aims to simulate the human thought process in a computerized model, highlights interesting aspects, such as thoughts, memory mechanisms, and decision making, that can contribute towards improved memory modeling for LLMs. Inspired by these principles, we propose CAIM, a cognitive AI memory framework that models key aspects of human memory through a multi-agent architecture. Specialized LLM-based agents handle memory-related functions such as retrieval, contextual relevance evaluation, and memory maintenance. We compare CAIM against existing approaches, focusing on metrics such as retrieval accuracy, response correctness, and contextual coherence. The results demonstrate that CAIM outperforms baseline frameworks in different metrics, highlighting its context awareness and demonstrating its contribution to memory mechanisms for improving long-term human-AI interactions.
Overview
CoChat Summary
TL;DR
CAIM is a cognitive-AI-inspired memory framework that uses eight specialized LLM-based agents to decide when to retrieve, filter, and maintain long-term memory for LLM agents in long-term interaction. Across the GVD and LoCoMo benchmarks it outperforms MemoryBank and Think-in-Memory on retrieval accuracy, response correctness, and contextual coherence, though it struggles with detailed queries and relative time references.
Key findings
- On GVD, CAIM reached 83.3% retrieval accuracy with GPT-3.5 turbo (vs. MemoryBank's 76.3%) and 88.7% with GPT-4o; with GLM it dropped to 67.6%, below TiM (82%) and MemoryBank (80.9%) because GLM produced Chinese tags and failed to follow the required tag format.
- Response correctness improved across all models: GLM 60% (vs. TiM 45%, MemoryBank 43.8%), GPT-3.5 turbo 81.3% (vs. MemoryBank 71.6%), GPT-4o 87.5%. Contextual coherence also improved (GLM 86.5% vs. TiM 73.5% and MemoryBank 68%; GPT-3.5 turbo 98.3% vs. MemoryBank 91.2%).
- On LoCoMo with GPT-4o, CAIM scored 66.8% retrieval accuracy, 63.8% response correctness, and 89.2% contextual coherence, versus MemoryBank's 53.9%, 42.6%, and 65.3%.
- Ablation (GPT-4o, GVD): removing the Controller Agent left retrieval accuracy similar (85.5% vs. 87.5%) but dropped response correctness to 78.2%; removing the Filtering Agent cut retrieval accuracy to 64.3% and response correctness to 63.5%, while contextual coherence stayed at 90.2%.
Methods
- Design: CAIM combines MemoryBank, Think-in-Memory, and Self-Controlled Memory concepts into a multi-agent architecture with STM/LTM, an LLM-generated three-level JSON ontology for tagging, and eight LLM-based agents (Controller, Tagging, Time, Filtering, Response Generator, Summary, Memory, Review).
- Data: the public Generated Virtual Dataset (GVD; 15 users, ten days, 100 English test questions) compared against MemoryBank and TiM, and LoCoMo (ten long-term conversations) compared against MemoryBank; SCM was excluded due to incompatible datasets and metrics.
- Analysis: fact-checking evaluation with three human annotators (GLM, GPT-3.5 turbo, GPT-4o; default parameters) using Retrieval Accuracy, Response Correctness, and Contextual Coherence (0/0.5/1 scales), with Percent Agreement and ICC for reliability; GPT-4o served as automatic evaluator for LoCoMo after validation against human ratings on GVD (93%/81%/94% agreement).
Limitations
- Stated: latency and computational cost of the multi-agent architecture and ontology-based processing were intentionally not evaluated; no long-term user study was conducted (experiments are on synthetic data); tag-based retrieval is fragile across LLMs; memory conciseness limits detailed queries; relative time references and split information across inductive thoughts cause errors; occasional hallucinations when relevant information is missing.
- Evident from the text: the GLM results show CAIM's effectiveness depends on a model's ability to follow predefined tag formats, and the low ICC values for Contextual Coherence on GPT models (5.8% and −0.6%) indicate that metric's reliability is questionable under low rating variance.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
21 Valid · 3 Not valid · 7 Not checked · 104 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite their capabilities, LLMs still have shortcomings, such as a limited context window, which restricts the amount of information that can be processed and thus the ability to understand the ever-changing environment [58, 60] and maintain effective long-term interactions [45, 50].”
What the sources say
- Partly supportedcites [58] [58] Yuanzhen Xie, Tao Xie, Mingxiong Lin, WenTao Wei, Chenglin Li, Beibei Kong, Lei Chen, Chengxiang Zhuo, Bo Hu, and Zang Li. 2023. Olagpt: Empowering llms with human-like problem-solving abilities. arXiv preprint arXiv:https://arXiv.org/abs/2305.16334 (2023). Google Scholar a [...] to understand the ever-changing environment b [...] activities to find the optimal solution c [...] access contextually relevant past knowledge d [...] negatively influence interaction quality sourceExperimental — accuracy not yet measured
The Introduction says LLMs struggle to understand an ever-changing environment, but attributes this to knowledge limited by their training data. The paper does not link that difficulty to a limited context window or a restriction on how much information can be processed. The separate claim about long-term interactions has different citations in the citing sentence. (LLM, conf=0.83)
- Supportedcites [60] [60] Xinchao Xu, Zhibin Gou, Wenquan Wu, Zheng-Yu Niu, Hua Wu, Haifeng Wang, and Shihang Wang. 2022. Long time no see! open-domain conversation with long-term persona memory. arXiv preprint arXiv:https://arXiv.org/abs/2203.05797 (2022). Google Scholar a [...] to understand the ever-changing environment b [...] due to their limited context window c [...] relevant knowledge for extended periods d [...] , following models of human memory source
The citation is attached to the difficulty of maintaining effective long-term interactions, not to the earlier claim about context-window limits. The Abstract reports that dialogue models perform poorly in long-term conversations, possibly because they cannot adequately understand and remember dialogue history; it does not establish a context-window mechanism. (LLM, conf=0.88)
- Partly supportedcites [45] [45] Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics 11 (2023), 1316–1331. Google Scholar a [...] maintain effective long-term interactions b [...] of past information into the input prompt sourceExperimental — accuracy not yet measured
Section 3.1 says Transformer-based language models accept limited-length inputs and that tokens are removed when an input exceeds the limit. The Introduction separately says models need training to incorporate up-to-date information. The paper does not establish that the context-window limit causes difficulty understanding a changing environment, and it does not address effective long-term interactions. (LLM, conf=0.85)
- Partly supportedcites [50] [50] Ron Sun. 2024. Can A Cognitive Architecture Fundamentally Enhance LLMs? Or Vice Versa?arXiv preprint arXiv:https://arXiv.org/abs/2401.10444 (2024). Google Scholar a [...] maintain effective long-term interactions b [...] to enhance reasoning and decision-making sourceExperimental — accuracy not yet measured
The Introduction says current LLM-centered systems have limited memory, and “Human-like Memory: Mnemonic Aid to LLMs” says episodic and working memory are lacking in most LLMs. Those passages support a general memory limitation, but they do not identify a limited context window or say that it restricts understanding of a changing environment or effective long-term interactions. The indexed passages provide no page numbers. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In particular, CAIM’s memory structure is divided into STM and LTM, following models of human memory [2, 3, 49, 60].”
§ 3 CAIM > 3.1 Memory Structure
What the sources say
- Not checked — no full text of the sourcecites [2] [2] Richard C Atkinson and Richard M Shiffrin. 1968. Human memory: A proposed system and its control processes. In Psychology of learning and motivation. Vol. 2. Elsevier, 89–195. Google Scholar a [...] relevant knowledge for extended periods b [...] to influence ongoing reasoning and behavior c [...] , following models of human memory d [...] periods and across multiple experiences e [...] into concise, generalized representations f [...] emerge implicitly through learning source
- Not checked — no full text of the sourcecites [3] [3] Alan Baddeley. 2020. Working memory. Memory (2020), 71–111. Google Scholar a [...] called Working Memory in cognitive science b [...] , following models of human memory c [...] of the current task or situation d [...] periods and across multiple experiences source
- Supportedcites [49] [49] Theodore R Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L Griffiths. 2023. Cognitive architectures for language agents. arXiv preprint arXiv:https://arXiv.org/abs/2309.02427 (2023). Google Scholar a [...] activities to find the optimal solution b [...] , following models of human memory c [...] based on the current context or task source
Section 4.1 Memory divides a language agent’s memory into short-term working memory and long-term memories. Section 2.3 says this architectural distinction builds on psychological theories. The source supports the human-memory-model rationale; it does not itself describe CAIM’s implementation, which is the citing authors’ own design. (LLM, conf=0.88)
- Partly supportedcites [60] [60] Xinchao Xu, Zhibin Gou, Wenquan Wu, Zheng-Yu Niu, Hua Wu, Haifeng Wang, and Shihang Wang. 2022. Long time no see! open-domain conversation with long-term persona memory. arXiv preprint arXiv:https://arXiv.org/abs/2203.05797 (2022). Google Scholar a [...] to understand the ever-changing environment b [...] due to their limited context window c [...] relevant knowledge for extended periods d [...] , following models of human memory sourceExperimental — accuracy not yet measured
Sections 4 and 4.2 describe a dialogue system with a long-term memory (LTM) module. They do not describe CAIM, an STM/LTM division, or a design based on models of human memory; the paper’s LTM instead stores user and chatbot persona information. No page numbers are available in our copy. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“To overcome the limited context window of LLMs, CAIM further utilizes RAG to store and access information outside the model’s internal context [22, 47, 62].”
§ 3 CAIM > 3.1 Memory Structure
What the sources say
- Supportedcites [22] [22] Kostas Hatalis, Despina Christou, Joshua Myers, Steven Jones, Keith Lambert, Adam Amos-Binks, Zohreh Dannenhauer, and Dustin Dannenhauer. 2023. Memory Matters: The Need to Improve Long-Term Memory in LLM-Agents. In Proceedings of the AAAI Symposium Series, Vol. 2. AAAI, Palo Alto, California, USA, 277–280. Google Scholar a [...] LLMs as the core of agent-based systems b [...] outside the model’s internal context source
The Introduction explains that fixed-size context windows limit how much information LLMs can retain. In “Long-Term Memory via Vector Databases,” the paper describes storing information in an external database and retrieving it to populate an LLM prompt, supporting the cited rationale for CAIM’s approach. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [47] [47] Seongbum Seo, Sangbong Yoo, and Yun Jang. 2025. A Prompt Chaining Framework for Long-Term Recall in LLM-Powered Intelligent Assistant. In Proceedings of the 30th International Conference on Intelligent User Interfaces. 89–105. Digital Library Google Scholar a [...] improve long-term interactions. Seo et al. b [...] outside the model’s internal context source
- Partly supportedcites [62] [62] Kai Zhang, Fubang Zhao, Yangyang Kang, and Xiaozhong Liu. 2023. Memory-augmented llm personalization with short-and long-term memory coordination. arXiv preprint. arxiv:https://arXiv.org/abs/2309.11696 Google Scholar a [...] outside the model’s internal context b [...] knowledge beyond its pretrained knowledge sourceExperimental — accuracy not yet measured
Section 2.4.2 (Memory Utilization; page unavailable) describes MaLP retrieving knowledge from stored memory and adding it to the LLM’s prompt, supporting the general idea of accessing information outside the model. The paper does not establish that CAIM uses this approach or that CAIM adopted it to overcome a limited context window. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Current LLMs are not designed to retain information persistently across interactions due to their limited context window [60].”
What the sources say
- Partly supportedcites [60] [60] Xinchao Xu, Zhibin Gou, Wenquan Wu, Zheng-Yu Niu, Hua Wu, Haifeng Wang, and Shihang Wang. 2022. Long time no see! open-domain conversation with long-term persona memory. arXiv preprint arXiv:https://arXiv.org/abs/2203.05797 (2022). Google Scholar a [...] to understand the ever-changing environment b [...] due to their limited context window c [...] relevant knowledge for extended periods d [...] , following models of human memory sourceExperimental — accuracy not yet measured
The Introduction says large-scale generative dialogue models perform poorly in long-term conversations, and the Conclusion describes adding long-term memory to retain persona information. Appendix B specifies a maximum context length, but the paper does not say that this limit is why current models fail to retain information across interactions, or that LLMs generally are not designed to do so. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Combining tags with timestamps enables efficient retrieval based on topics and recency [30, 31].”
§ 3 CAIM > 3.1 Memory Structure
What the sources say
- Not checked yetcites [30] [30] Hang Li, Tianlong Xu, Ethan Chang, and Qingsong Wen. 2025. Knowledge tagging with large language model based multi-agent system. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. AAAI, Palo Alto, California, USA, 28775–28782. Go to Citation Digital Library Google Scholar
- Supportedcites [31] [31] Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. 2024. Hello Again! LLM-powered Personalized Agent for Long-term Dialogue. arXiv preprint. arxiv:https://arXiv.org/abs/2406.05925 Google Scholar a [...] LLMs as the core of agent-based systems b [...] retrieval based on topics and recency source
The Memory Retrieval section describes using topic noun sets to score topic overlap and an exponential time-decay coefficient to weight retrieval by recency. Section 3.2.1 says the memories record occurrence times and are efficiently retrieved. The paper calls these topic noun sets rather than tags, but the claim captures the mechanism. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
48 more claims · 3 with their cited sources checked
Create a free account to see all 135 claims and to verify the 5 references still open.
- It may state two separate claims as one. 58
- Probably not a substantial claim. 35
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4 Experiments > 4.1 Setup
“Overall, the Percent Agreement and ICC show a strong level of inter-rater agreement across all models and metrics, demonstrating the reliability of our annotation process.”
§ 4 Experiments > 4.1 Setup
“However, the ICC values for the GPT models were unexpectedly low with 5.8% for GPT-3.5 turbo and –0.6% for GPT-4o.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4 Experiments > 4.2 Results
“Removing the Controller Agent results in a similar Retrieval Accuracy of 85.5% compared to the full system with 87.5%”
§ 4 Experiments > 4.2 Results
“Removing the Filtering Agent leads to a significant drop in Retrieval Accuracy from 88.7% to 64.3%”
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Toward principles for the design of ontologies used for knowledge sharing?
Gruber, T. R. (1995) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1995.1081 · Cited by 7,753 · cited 1×
“As illustrated in Figure 2, CAIM is structured into three modules designed to work together to address the out-lined challenges, including the lack of LTM in LLMs and the need for more accurate and contextually relevant responses.”
Human Memory: A Proposed System and its Control Processes
Atkinson, R. C., & Shiffrin, R. M. (1968) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60422-3 · Cited by 7,216 · cited 6×
“This memory management addresses information storage outside the LLMs’ context window, allowing the agent to retain and recall past information.”
InfluentialForming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903 · cited 10×
“Further, the Response Correctness is reduced from 87.5% to 63.5%.”
InfluentialRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
A survey on context-aware systems
Baldauf, M., Dustdar, S., & Rosenberg, F. (2007) · International Journal of Ad Hoc and Ubiquitous Computing · doi:10.1504/ijahuc.2007.014070 · Cited by 1,983
Guarino, N., Oberle, D., & Staab, S. (2009) · Handbook on Ontologies · doi:10.1007/978-3-540-92673-3_0 · Cited by 1,244 · cited 1×
“As illustrated in Figure 2, CAIM is structured into three modules designed to work together to address the out-lined challenges, including the lack of LTM in LLMs and the need for more accurate and contextually relevant responses.”
Interrater agreement and interrater reliability: Key concepts, approaches, and applications
Gisev, N., Bell, J. S., & Chen, T. F. (2013) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2012.04.004 · Cited by 924 · cited 3×
“To measure inter-rater agreement and the reliability of our annotations, we calculated Percent Agreement and Intra-class Correlation Coefficient (ICC) [4, 13] across all evaluation metrics and models.”
InfluentialCognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 798
Agentic AI: Autonomous intelligence for complex goals–a comprehensive survey
Acharya, D. B., Kuppan, K., & Divya, B. (2025) · IEEE Access · doi:10.1109/access.2025.3532853 · Cited by 717
Benchmarking Large Language Models in Retrieval-Augmented Generation
Chen, J., Lin, H., Han, X., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2309.01431 · Cited by 455
Benchmarking Large Language Models in Retrieval-Augmented Generation
Chen, J., Lin, H., Han, X., & Sun, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29728 · Cited by 454
In-Context Retrieval-Augmented Language Models
Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., & Shoham, Y. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00605 · Cited by 449
Research contributions in human-computer interaction
Wobbrock, J. O., & Kientz, J. A. (2016) · Interactions · doi:10.1145/2907069 · Cited by 416 · cited 1×
“In addition to Cognitive AI, CAIM builds upon three existing memory-augmented approaches - MemoryBank, Think-in-Memory, and Self-Controlled Memory - by combining and extending their concepts to create a more holistic memory mechanism.”
Detecting and Preventing Hallucinations in Large Vision Language ModelsPreprint
Gunjal, A., Yin, J., & Bas, E. (2023) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2308.06394 · Cited by 394
A Survey on Multimodal Large Language Models for Autonomous Driving
Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D., et al. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00106 · Cited by 369 · cited 1×
“These advancements led to a growing area of research focused on utilizing LLMs to develop AI agents with human-like capabilities, such as decision-making and planning [10, 52].”
When large language models meet personalization: perspectives of challenges and opportunities
Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., Pu, Y., Lei, Y., Chen, X., Wang, X., et al. (2024) · World Wide Web · doi:10.1007/s11280-024-01276-1 · Cited by 367
The role of agentic AI in shaping a smart future: A systematic review
Hosseini, S., & Seilani, H. (2025) · Array · doi:10.1016/j.array.2025.100399 · Cited by 280 · cited 1×
“Therefore, we believe that CAIM yields a promising and extendable approach toward holistic memory modeling for LLMs that enables personalized context-aware interactions.”
MemoryBank: Enhancing Large Language Models with Long-Term Memory
Zhong, W., Guo, L., Gao, Q., Ye, H., & Wang, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.10250 · Cited by 253
Cognitive Computing: Architecture, Technologies and Intelligent Applications
Chen, M., Herrera, F., & Hwang, K. (2018) · IEEE Access · doi:10.1109/access.2018.2791469 · Cited by 242
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.356 · Cited by 188
Personalization in HRI: a longitudinal field experiment
Lee, M. K., Forlizzi, J., Kiesler, S., Rybski, P., Antanitis, J., & Savetsila, S. (2012) · Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction · doi:10.1145/2157689.2157804 · Cited by 169 · cited 1×
“Furthermore, some LLMs are constrained by limited context windows, which negatively impacts their ability to understand and memorize historical information over extended periods [48], hindering their effectiveness in long-term interactions [26, 33, 36, 40, 52].”
Detecting and Preventing Hallucinations in Large Vision Language Models
Gunjal, A., Yin, J., & Bas, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29771 · Cited by 161
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Xu, J., Szlam, A., & Weston, J. (2021) · arXiv · doi:10.48550/arxiv.2107.07567 · Cited by 148 · cited 2×
“By utilizing external knowledge sources, memory-augmented meth-ods address the limitation of a limited context window and enable the integration of past information into the current input prompt [22, 26, 33, 36, 39, 42, 47, 50].”
From Emotion AI to Cognitive AI
Zhao, G., Li, Y., & Xu, Q. (2022) · International Journal of Network Dynamics and Intelligence · doi:10.53941/ijndi0101006 · Cited by 100 · cited 7×
“To achieve this, recent research states that bridging AI with cognitive science is required to deepen the understanding of human cognition for more human-centric AI systems [31, 51].”
InfluentialA Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future DirectionsPreprint
Du, H., Thudumu, S., Vasa, R., & Mouzakis, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.01968 · Cited by 98
Evaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.747 · Cited by 96
User-LLM: Efficient LLM Contextualization with User EmbeddingsPreprint
Ning, L., Liu, L., Wu, J., Wu, N., Berlowitz, D., Prakash, S., Green, B., O’Banion, S., & Xie, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.13598 · Cited by 91
Agent4Edu: Generating Learner Response Data by Generative Agents for Intelligent Education Systems
Gao, W., Liu, Q., Yue, L., Yao, F., Lv, R., Zhang, Z., Wang, H., & Huang, Z. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i22.34565 · Cited by 83
Multimodal Agent AI: A Survey of Recent Advances and Future Directions
Sun, Y., Sun, H., Ma, J., Zhang, P., & Huang, X. (2025) · Journal of Computer Science and Technology · doi:10.1007/s11390-025-4802-8 · Cited by 79
Agent4Edu: Generating Learner Response Data by Generative Agents for Intelligent Education Systems
Gao, W., Liu, Q., Yue, L.-N., Yao, F.-Z., Lv, R., Zhang, Z., Wang, H., & Huang, Z.-Y. (2025) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2501.10332 · Cited by 70
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2025) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia251160 · Cited by 69
(2020) · doi:10.4324/9780429449642-4 · Cited by 65
Long Time No See! Open-Domain Conversation with Long-Term Persona Memory
Xu, X., Gou, Z., Wu, W., Niu, Z.-Y., Wu, H., Wang, H., & Wang, S. (2022) · arXiv · doi:10.48550/arxiv.2203.05797 · Cited by 65
MemGPT: Towards LLMs as Operating Systems
Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., & Gonzalez, J. E. (2023) · arXiv · doi:10.48550/arxiv.2310.08560 · Cited by 64
Memory-Based Personalization for Fostering a Long-Term Child-Robot Relationship
Ligthart, M. E. U., Neerincx, M. A., & Hindriks, K. V. (2022) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889446 · Cited by 57 · cited 1×
“Furthermore, some LLMs are constrained by limited context windows, which negatively impacts their ability to understand and memorize historical information over extended periods [48], hindering their effectiveness in long-term interactions [26, 33, 36, 40, 52].”
Hello Again! LLM-powered Personalized Agent for Long-term Dialogue
Li, H., Yang, C., Zhang, A., Deng, Y., Wang, X., & Chua, T.-S. (2024) · arXiv · doi:10.48550/arxiv.2406.05925 · Cited by 53
Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation
Salemi, A., Kallumadi, S., & Zamani, H. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657783 · Cited by 52 · cited 2×
“In this context, methods such as reinforcement learning from human feedback [37, 44, 45] have shown promising approaches to improve the long-term capabilities of LLMs.”
A Survey of Personalization: From RAG to Agent
Li, X., Jia, P., Xu, D., Wen, Y., Zhang, Y., Zhang, W., Wang, W., Wang, Y., Du, Z., Li, X., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.10147 · Cited by 52
A-Mem: Agentic Memory for LLM Agents
Xu, W., Liang, Z., Mei, K., Gao, H., Tan, J., & Zhang, Y. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-0593 · Cited by 51 · cited 2×
“Therefore, we focused on MemoryBank and TiM to ensure a consistent comparison, as both use the same dataset and evaluation metrics.”
Chen, W., Yu, C., Wang, H., Wang, Z., Yang, L., Wang, Y., Shi, W., & Shi, Y. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606741 · Cited by 44
Memory Matters: The Need to Improve Long-Term Memory in LLM-Agents
Hatalis, K., Christou, D., Myers, J., Jones, S., Lambert, K., Amos-Binks, A., Dannenhauer, Z., & Dannenhauer, D. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v2i1.27688 · Cited by 42
Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG
Singh, A., Ehtesham, A., Kumar, S., Khoei, T. T., & Vasilakos, A. V. (2025) · arXiv · doi:10.48550/arxiv.2501.09136 · Cited by 36
Designing the Mind: How Agentic Frameworks Are Shaping the Future of AI Behavior
Garg, V. (2025) · Journal of Computer Science and Technology Studies · doi:10.32996/jcsts.2025.7.5.24 · Cited by 35 · cited 1×
“Additionally, we aim to face the limitation of CAIM’s memory, which is currently incapable of learning from queries.”
Prompted LLMs as Chatbot Modules for Long Open-domain Conversation
Lee, G., Hartmann, V., Park, J., Papailiopoulos, D., & Lee, K. (2023) · arXiv · doi:10.48550/arxiv.2305.04533 · Cited by 34
Prompted LLMs as Chatbot Modules for Long Open-domain Conversation
Lee, G., Hartmann, V., Park, J., Papailiopoulos, D., & Lee, K. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.277 · Cited by 34 · cited 3×
“Additionally, we use in-context learning to adapt the LLM to tasks based on brief instructions via prompts [6, 20].”
Personalization and Customization of LLM Responses
Eapen, J., & Adhithyan, V. S. (2023) · International Journal of Research Publication and Reviews · doi:10.55248/gengpi.4.1223.123512 · Cited by 31 · cited 1×
“Although models such as GPT-4o [17] or Gemini [41] offer larger context windows, it is crucial to keep the input concise to ensure high performance, as longer inputs may reduce efficiency and potentially overwhelm the model [28].”
Personalized Large Language Models
Woźniak, S., Koptyra, B., Janz, A., Kazienko, P., & Kocoń, J. (2024) · 2024 IEEE International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw65004.2024.00071 · Cited by 28 · cited 2×
“In this context, methods such as reinforcement learning from human feedback [37, 44, 45] have shown promising approaches to improve the long-term capabilities of LLMs.”
To Reuse or Not To Reuse?: A Framework and System for Evaluating Summarized Knowledge
Liu, M. X., Kittur, A., & Myers, B. A. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449240 · Cited by 23
Memolet: Reifying the Reuse of User-AI Conversational Memories
Yen, R., & Zhao, J. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676388 · Cited by 22
Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell
Lu, T., Gao, M., Yu, K., Byerly, A., & Khashabi, D. (2024) · arXiv · doi:10.48550/arxiv.2406.14673 · Cited by 21
Knowledge Tagging with Large Language Model Based Multi-Agent System
Li, H., Xu, T., Chang, E., & Wen, Q. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i28.35141 · Cited by 16
Niu, Q., Liu, J., Bi, Z., Feng, P., Peng, B., Chen, K., Li, M., Yan, L. K., Zhang, Y., Yin, C. H., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.02387 · Cited by 16
CarMem: Enhancing Long-Term Memory in LLM Voice Assistants through Category-BoundingPreprint
Kirmayr, J., Stappen, L., Schneider, P., Matthes, F., & André, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.09645 · Cited by 12
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Chhikara, P., Khant, D., Aryan, S., Singh, T., & Yadav, D. (2025) · arXiv · doi:10.48550/arxiv.2504.19413 · Cited by 10
Tang, Y., Vasileiou, S. L., & Yeoh, W. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i13.33578 · Cited by 8
Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memory
Liu, L., Yang, X., Shen, Y., Hu, B., Zhang, Z., Gu, J., & Zhang, G. (2023) · arXiv · doi:10.48550/arxiv.2311.08719 · Cited by 7
A Holistic Review of Agentic AI Frameworks, Applications, and Research Trajectories
Peykani, P., Ghanidel, S., Javadi-Sisi, I., Snåšel, V., & Mirjalili, S. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10675-8 · Cited by 7
Personalized Large Language ModelsPreprint
Woźniak, S., Koptyra, B., Janz, A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.09269 · Cited by 7
Tang, Y.-X., Vasileiou, S. L., & Yeoh, W. (2025) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2502.16376 · Cited by 6
Keep Me Updated! Memory Management in Long-term Conversations
Bae, S., Kwak, D., Kang, S., Lee, M. Y., Kim, S., Jeong, Y., Kim, H., Lee, S.-W., Park, W., & Sung, N. (2022) · arXiv · doi:10.48550/arxiv.2210.08750 · Cited by 5 · cited 3×
“Therefore, remembering information from the past and correctly referencing it in ongoing conversations is crucial in long-term interactions [3].”
SCM: Enhancing Large Language Model with Self-Controlled Memory Framework
Wang, B., Liang, X., Yang, J., Huang, H., Wu, S., Wu, P., Lu, L., Ma, Z., & Li, Z. (2023) · arXiv · doi:10.48550/arxiv.2304.13343 · Cited by 5
Evaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.17753 · Cited by 3
Can A Cognitive Architecture Fundamentally Enhance LLMs? Or Vice Versa?
Sun, R. (2024) · arXiv · doi:10.48550/arxiv.2401.10444 · Cited by 3 · cited 4×
“Remembering and retrieving information lay the foundation of a cognitive architecture [31, 40] and enable contextually aware interactions.”
InfluentialA Prompt Chaining Framework for Long-Term Recall in LLM-Powered Intelligent Assistant
Seo, S., Yoo, S., & Jang, Y. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712117 · Cited by 3 · cited 2×
“TiM consists of two key stages: recalling thoughts before generation and post-thinking after generation to update the memory.”
CAIM: A Cognitive AI Memory Framework for Long-term Interaction with LLMs
Westhäußer, R., Zepf, S., & Minker, W. (2025) · Companion Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708557.3716342 · Cited by 2 · cited 1×
“To achieve this, recent research states that bridging AI with cognitive science is required to deepen the understanding of human cognition for more human-centric AI systems [31, 51].”
Self-citationLLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordination
Zhang, K., Kang, Y., Zhao, F., & Liu, X. (2023) · arXiv · doi:10.48550/arxiv.2309.11696 · Cited by 2
OlaGPT: Empowering LLMs With Human-like Problem-Solving Abilities
Xie, Y., Xie, T., Lin, M., Wei, W., Li, C., Kong, B., Chen, L., Zhuo, C., Hu, B., & Li, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.16334 · Cited by 1
Understanding the Role of User Profile in the Personalization of Large Language ModelsPreprint
Wu, B., Shi, Z., Rahmani, H. A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.17803
Research works printed but not linked: 2
- [5] Shweta Bajpai, Ram Bajpai, and Himanshu Chaturvedi. 2015. Evaluation of inter-rater agreement and inter-rater reliability for observational data: an overview of concepts and methods. Journal of the Indian Academy of Applied Psychology 41, 3 (2015), 20–27. Google Scholar a [...] for GPT-4o. As discussed by Bajpai et al. b [...] agreed on the classification of each metric
- [24] Kai Hwang and Min Chen. 2017. Big-data analytics for cloud, IoT and cognitive computing. John Wiley & Sons. Digital Library Google Scholar a [...] language, perception, memory, and reasoning b [...] memorize, and respond to external changes c [...] concept of a memory mechanism is essential · doi:10.5555/3169338
Cited by
MIRIX: Multi-Agent Memory System for LLM-Based Agents
Wang, Y., & Chen, X. (2025) · arXiv · doi:10.48550/arxiv.2507.07957 · Cited by 201
“Most current systems focus primarily on Short-Term and Long-Term Memory [22, 32, 25], with some incorporating Mid-Term Memory [12].”
A Holistic Review of Agentic AI Frameworks, Applications, and Research Trajectories
Peykani, P., Ghanidel, S., Javadi-Sisi, I., Snåšel, V., & Mirjalili, S. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10675-8 · Cited by 7
Metadata
- Authors
- Westhäußer, R., Minker, W., & Zepf, S.
- Affiliations
- Mercedes-Benz (Germany) (DE)
- Universität Ulm (DE)
- Human-Computer Interaction, Mercedes-Benz AG, Böblingen, Germany
- Venue
- Proceedings of the 31st International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2026-03-22
- Pages
- 134-143
- OpenAlex
- W7133361058
- Cited by
- 1
- References
- 34
- Citations by year
- 2026: 1
- FWCI
- 8.44
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- cognitive AI, memory modeling, large language models, multi-agent architecture, contextual relevance, memory retrieval, memory maintenance, context awareness, intelligent agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.