CoChat
Sign Up
PreprintCited by 32Topic ModelingMultimodal Machine Learning ApplicationsNatural Language Processing Techniques

RET-LLM: Towards a General Read-Write Memory for Large Language Models

Open access
Modarressi, Ali, Imani, Ayyoob, Fayyaz, Mohsen, Schütze, Hinrich·2023·arXiv
DOI 10.48550/arxiv.2305.14322arXiv 2305.14322
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) through their extensive parameters and comprehensive data utilization. However, existing LLMs lack a dedicated memory unit, limiting their ability to explicitly store and retrieve knowledge for various tasks. In this paper, we propose RET-LLM a novel framework that equips LLMs with a general write-read memory unit, allowing them to extract, store, and recall knowledge from the text as needed for task performance. Inspired by Davidsonian semantics theory, we extract and save knowledge in the form of triplets. The memory unit is designed to be scalable, aggregatable, updatable, and interpretable. Through qualitative evaluations, we demonstrate the superiority of our proposed framework over baseline approaches in question answering tasks. Moreover, our framework exhibits robust performance in handling temporal-based question answering tasks, showcasing its ability to effectively manage time-dependent information.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2305.14322

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • RET-LLM fine-tunes LLMs based on a human constructed dataset of triplet-natural language pairs to better convert natural languages into structured memory information.

    “In RET-LLM [42], in order to better convert natural languages into structured memory information, the authors fine-tune LLMs based on a human constructed dataset, where each sample is a”

    From A survey on large language model based autonomous agents (2024) · cites this paper as [42]

    Section 3.3, “Finetuning the LLM” (page unavailable), says the authors fine-tune an LLM on examples pairing relationship statements with memory-write API calls, supporting the conversion of natural language into structured memory information. It describes the training dataset as synthetic, generated from names, organizations, relationships, and templates—not as a human-constructed dataset of triplet–natural-language pairs. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Ret-LLM applies supervised fine-tuning to the LLM, enabling more robust read-write interactions with the relational graph.

    “Additionally, Ret-LLM (Modarressi et al., 2023) applies supervised fine-tuning to the LLM, enabling more robust read-write interactions with the relational graph.”

    From Memory in the Age of AI Agents (2025) · cites this paper as Modarressi et al., 2023

    Section 3.3 describes fine-tuning the LLM to generate read and write memory-API calls, and Section 4 gives qualitative examples of successful retrieval. But Section 3.1 describes the memory as a three-column triplet table, not a relational graph; the paper also does not establish that the interactions are more robust. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Existing long-term memory methods for LLM agents operate over turns, chunks, summaries, or mutable records and rely on update or retrieval to compensate for fragmentation.

    “These methods improve capacity and flexibility, but still operate over turns, chunks, summaries, or mutable records and rely on update or retrieval to compensate for fragmentation.”

    From Membox: Weaving Topic Continuity into Long-Range Memory for LLM Agents (2026) · cites this paper as Modarressi et al., 2023

    The Introduction and Related Works (pages not provided) describe earlier approaches that retrieve documents or agent observations, and present RET-LLM as able to update memory and aggregate information scattered across documents. They do not establish the broader characterization that existing methods operate over turns, chunks, summaries, or mutable records, or that those methods use updates to compensate for fragmentation. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Modarressi et al. [7] design both memory reading and writing operations to interact with environments for task solving.

    “Modarressi et al. [7] design both memory reading and writing operations to interact with environments for task solving.”

    From A Survey on the Memory Mechanism of Large Language Model-based Agents (2025) · cites this paper as [7]

    Section 3.2 describes the designed memory-write and memory-read operations, and Section 3 explains their use to store user-provided information and answer questions. The paper does not describe these operations as interacting with environments for task solving; its examples concern memory-assisted question answering. Page numbers are unavailable in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Modarressi et al. (2023) augments LLM with a write-read memory module by extracting and saving knowledge in the form of triplets, allowing for scalable, updatable, interpretable, and aggregatable memory storage, which is particularly beneficial for handling temporal-based question answering tasks.

    “Similarly, Modarressi et al. (2023) augments LLM with a write-read memory module by extracting and saving knowledge in the form of triplets, allowing for scalable, updatable, interpretable, and aggregatable memory storage, which is particularly beneficial for handling temporal-based question answering tasks.”

    From The AI Hippocampus: How Far are We From Human Memory? (2026) · cites this paper as Modarressi et al., 2023

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 7

  1. Sébastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lund- berg, et al. 2023. Sparks of artificial general intelli- gence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. · arXiv
  2. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  3. Ali Modarressi, Abdullatif Köksal, Ayyoob Imani, Mohsen Fayyaz, and Hinrich Schütze. 2024. Mem- llm: Finetuning llms to use an explicit read-write memory. arXiv preprint arXiv:2404.11672. · arXiv
  4. Joon Sung Park, Joseph C O'Brien, Carrie J Cai, Mered- ith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. · arXiv
  5. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761. · arXiv
  6. Yuhuai Wu, Markus N Rabe, DeLesley Hutchins, and Christian Szegedy. 2022. Memorizing transformers. arXiv preprint arXiv:2203.08913. · arXiv
  7. Zexuan Zhong, Tao Lei, and Danqi Chen. 2022. Train- ing language models with memory augmentation. In Proceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing, pages 5657-5673, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.

Cited by

Sorted by citations

Related

Authors
Modarressi, A., Imani, A., Fayyaz, M., & Schütze, H.
Affiliations
  • Munich Center for Machine Learning (DE)
  • Microsoft (Germany) (DE)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2305.14322
arXiv
2305.14322
OpenAlex
W4378509422
Cited by
8
References
0
Citations by year
2026: 1 · 2025: 5 · 2024: 1 · 2023: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
large language models, memory units, knowledge storage and retrieval, question answering, temporal question answering

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review