Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index; 23 of the 29 sources its claims cite were checked for support.

From Goldfish to Elephant: Long-Term Memory for Large Language Models via Selective Storage and Hierarchical Compression

Open access
Kummara, Harshavardhan · 2026 · Cambridge Open Engage (preprint)

Abstract

Large language models (LLMs) demonstrate remarkable reasoning and generative capabilities but remain fundamentally limited by fixed context windows and the absence of persistent, structured memory. As a result, they struggle with long-term personalization, continual learning, and consistency across extended interactions. In this paper, we propose a modular memory-augmented architecture that integrates episodic and semantic memory with a consistency-aware retrieval mechanism and a learned memory-gating network. Episodic memory stores recent, high-fidelity interaction traces, while semantic memory maintains compressed, abstracted knowledge derived from historical experiences. A hierarchical compression module regulates memory growth by summarizing episodic content into semantic representations when capacity thresholds are exceeded. To retrieve relevant information, we introduce a consistency-aware scoring function that jointly optimizes semantic similarity, temporal relevance, and contradiction avoidance. This retrieval output is fused into the input context of a transformer-based LLM to support grounded and coherent generation. A lightweight gating network predicts the utility of storing new interactions, enabling selective memory updates and reducing catastrophic interference during continual learning. We show that this design supports long-horizon dialogue coherence, improved factual consistency, and adaptive personalization without requiring model retraining. The proposed framework is model-agnostic and can be integrated with existing retrieval-augmented generation pipelines. Our results highlight a practical pathway toward scalable long-term memory for LLMs, bridging the gap between stateless neural generation and persistent, structured knowledge systems.

Overview

CoChat Summary

TL;DR

MEMLLM is a memory-augmented LLM architecture combining selective gating, episodic/semantic hierarchical compression, consistency-aware retrieval, and elastic weight consolidation for long-term personalization. It reports 94.3% personalization accuracy, 73% memory reduction, and 68% fewer contradictions versus RAG baselines over 10,000+ interactions.

Key findings

  • Selective gating achieves 0.893 F1, 91.2% precision, 87.4% recall, and 27.3% selectivity, storing only about 1 in 4 interactions; without gating, accuracy drops to 89.1% and memory rises to 647.2 MB.
  • Hierarchical compression at ρ = 0.2 gives 91.3% reconstruction F1 and 5× size reduction; without compression, memory grows to 891.4 MB, and memory stabilizes around 1096 total entries (519 episodic + 577 semantic).
  • Consistency-aware retrieval reduces contradiction rate to 6.3% versus 28.2% for RAG-Dense; without consistency scoring, consistency falls to 79.4%.
  • Continual learning with EWC limits forgetting to 3.8% versus 15.3% without EWC and 18.9% for LoRA; long-horizon degradation is 1.9% versus 27.6% for Full-Context, with 198 ms latency.

Methods

  • Architecture: dual episodic/semantic memory, a gating MLP over recurrence, correction, task relevance, user feedback, and LM features; K-means plus LLM summarization for compression; retrieval scoring with cosine similarity, temporal decay, and NLI contradiction detection; EWC with utility-weighted experience replay.
  • Data and baselines: PersonaChat-Long (50 sessions/user, 10,234 turns, 87 users), MSC (5 sessions × 1,800 dialogues), ConvAI2-Continual (8,500 turns), and SyntheticPersonal (12,000 turns, 200 users); compared against Vanilla LLM, Full-Context, RAG-Dense, RAG-BM25, LoRA-FT, and Memorizing Transformer using LLaMA-2-7B, BERT-base encoder, and RoBERTa-large-MNLI.
  • Analysis: metrics include personalization accuracy, factual consistency, BLEU-4, ROUGE-L, memory efficiency, forgetting, and latency; theoretical bounds cover memory growth, consistency, rate-distortion, and convergence; experiments run on 4× NVIDIA A100 GPUs.

Limitations

  • Stated: compression irreversibility permanently loses fine-grained details; cold-start performance is suboptimal in sessions 0–5; adversarial users could inject contradictions; long-term storage raises privacy/data-retention concerns; the 72 ms retrieval overhead may be prohibitive for latency-critical applications; and NLI-based consistency detection can fail on nuanced or domain-specific contradictions.
  • Evident from the text: the main results table is fragmented across sections, with some baseline and MEMLLM rows appearing only in later table fragments, making direct comparison harder from the presented tables alone.
  • Evident from the text: some reported numbers vary slightly across sections (e.g., 0.89 vs. 0.893 gating F1; 72.7% vs. 73% storage reduction; 68% vs. 77.6% contradiction reduction versus RAG-Dense), likely due to rounding or inconsistent reporting.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.33774/coe-2026-xpkvb

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

10 Valid · 4 Not valid · 3 Not checked · 98 Own results and interpretations

76 more claims

Create a free account to see all 115 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 10
  • It may state two separate claims as one. 27

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 3

  1. [1][1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, 'Attention is all you need,' in Advances in Neural Information Processing Systems , 2017, pp. 5998-6008. -
  2. [17][17] E. J. Hu et al. , 'LoRA: Low-rank adaptation of large language models,' in ICLR , 2022. -
  3. [27][27] N. Mrkˇ si´ c et al. , 'Neural belief tracker: Data-driven dialogue state tracking,' in ACL , 2017, pp. 1777-1788. -

Metadata

Authors
Kummara, H.
Affiliations
University of Bath (GB)
Corresponding author
Kummara, Harshavardhan
Venue
Cambridge Open Engage (preprint)
Published
2026-02-01
OpenAlex
W7127051997
Cited by
0
References
0
Type
posted content
Language
en
Open access
Funders
University of Cambridge
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
long-term memory, large language models, episodic memory, semantic memory, continual learning, catastrophic forgetting, dialogue coherence, retrieval-augmented generation, personalization, factual consistency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: From Goldfish to Elephant: Long-Term Memory for Large Language Models via Selective Storage and Hierarchical Compression. https://app.cochat.ai/papers/10.33774/coe-2026-xpkvb
The paper
Kummara, H. (2026). From Goldfish to Elephant: Long-Term Memory for Large Language Models via Selective Storage and Hierarchical Compression. Cambridge Open Engage (preprint). https://doi.org/10.33774/coe-2026-xpkvb

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.