CoChat
Sign Up
PreprintCited by 32Topic ModelingInformation Retrieval and Search BehaviorMultimodal Machine Learning Applications

A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts

Open access
Lee, Kuang-Huei, Chen, Xinyun, Furuta, Hiroki, Canny, John, Fischer, Ian·2024·arXiv
DOI 10.48550/arxiv.2402.09727arXiv 2402.09727
Open this paper in CoChat — free accountOpen PDF

Abstract

Current Large Language Models (LLMs) are not only limited to some maximum context length, but also are not able to robustly consume long inputs. To address these limitations, we propose ReadAgent, an LLM agent system that increases effective context length up to 20x in our experiments. Inspired by how humans interactively read long documents, we implement ReadAgent as a simple prompting system that uses the advanced language capabilities of LLMs to (1) decide what content to store together in a memory episode, (2) compress those memory episodes into short episodic memories called gist memories, and (3) take actions to look up passages in the original text if ReadAgent needs to remind itself of relevant details to complete a task. We evaluate ReadAgent against baselines using retrieval methods, using the original long contexts, and using the gist memories. These evaluations are performed on three long-document reading comprehension tasks: QuALITY, NarrativeQA, and QMSum. ReadAgent outperforms the baselines on all three tasks while extending the effective context window by 3.5-20x.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.09727

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • MemoryBank and ReadAgent rely primarily on embedding-based retrieval.

    “This contrasts with systems like MemoryBank [Zhong et al., 2023] and ReadAgent [Lee et al., 2024] that rely primarily on embedding-based retrieval.”

    From Cognis: Context-Aware Memory for Conversational AI Agents (2026) · cites this paper as Lee et al., 2024

    For the ReadAgent citation, Section 4.4 explicitly describes ReadAgent’s retrieval as prompt-based and compares it with neural retrieval as a replacement. Section 4.2 likewise treats embedding-based retrieval as a baseline, not ReadAgent’s primary method. This verdict does not assess the separately cited MemoryBank claim. (LLM, conf=0.90)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Recent systems support dynamic memory evolution and attempt to establish connections between memories, but rely on implicit, unstructured associations rather than explicit schemas for modeling information evolution across sessions.

    “Recent systems (Lee et al., 2024; Xu et al., 2025; Yuan et al., 2025) support dynamic memory evolution and attempt to establish connections between memories. However, they rely on implicit, unstructured associations rather than explicit schemas for modeling information evolution across sessions.”

    From Pre-Storage Reasoning for Episodic Memory: Shifting Inference Burden to Memory for Personalized Dialogue (2025) · cites this paper as Lee et al., 2024

    Section 2 (Related Work; page unavailable) describes reasoning over a contextualized gist memory for densely correlated documents, which supports the claim's broad point about implicit connections. But Appendix G.3 (Iterative Gisting; page unavailable) says further compression of older memories as interactions continue is outside this work's scope. The paper therefore does not support attributing dynamic memory evolution across sessions to this system. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • There is an inherent trade-off between abstraction fidelity and long-term recall in summarization strategies for single-agent memory systems.

    “While these mechanisms improve context utilization and scalability, prior work highlights an inherent trade-off between abstraction fidelity and long-term recall, making summarization strategies a critical design consideration in single-agent memory systems (Lee et al., 2024; Wu et al., 2025d).”

    From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Lee et al., 2024

    The Example Pagination Prompt section says that compressing larger chunks tends to remove more details, and the QuALITY results show lower accuracy for more heavily compressed gist-only memory. This supports a compression-versus-detail trade-off, but the paper does not establish an inherent trade-off involving long-term recall in single-agent memory systems; Section G.3 describes iterative compression of older memories as outside the study’s scope. Page numbers are unavailable in our copy. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • LLM agents need memory to maintain cross-turn consistency and accumulate reusable information in long-horizon interactions.

    “Although LLM agents can leverage tools for complex tasks, they still need memory to maintain cross-turn consistency and accumulate reusable information in long-horizon interactions.”

    From Lightweight LLM Agent Memory with Small Language Models (2026) · cites this paper as Lee et al., 2024

    The Introduction describes gist memory for retaining and retrieving information from long contexts, and Appendix G.3 suggests that compressing older conversation history may help assistant agents as interactions grow over time. But that application is explicitly outside the study’s scope; the paper does not establish that memory maintains cross-turn consistency or accumulates reusable information across interactions. No page numbers are available in our copy. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 43

  1. Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023. · arXiv
  2. Chen, H., Pasunuru, R., Weston, J., and Celikyilmaz, A. Walking down the memory maze: Beyond con- text limit through interactive reading. arXiv preprint arXiv:2310.05029, 2023a. · arXiv
  3. Chen, Y., Qian, S., Tang, H., Lai, X., Liu, Z., Han, S., and Jia, J. Longlora: Efficient fine-tuning of long-context large language models. arXiv preprint arXiv:2309.12307, 2023c. · arXiv
  4. Deng, X., Gu, Y., Zheng, B., Chen, S., Stevens, S., Wang, B., Sun, H., and Su, Y. Mind2web: Towards a generalist agent for the web. arXiv preprint arXiv:2306.06070, 2023. · arXiv
  5. Furuta, H., Matsuo, Y., Faust, A., and Gur, I. Language model agents suffer from compositional generalization in web automation. arXiv preprint arXiv:2311.18751, 2023. · arXiv
  6. Furuta, H., Lee, K.-H., Nachum, O., Matsuo, Y., Faust, A., Gu, S. S., and Gur, I. Multimodal web navigation with instruction-finetuned foundation models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum? id=efFmBWioSc.
  7. Guo, M., Ainslie, J., Uthus, D., Ontanon, S., Ni, J., Sung, Y.-H., and Yang, Y. LongT5: Efficient text-to- text transformer for long sequences. In Carpuat, M., de Marneffe, M.-C., and Meza Ruiz, I. V. (eds.), Find- ings of the Association for Computational Linguistics: NAACL 2022, pp. 724-736, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.findings-naacl.55. Gur, I., Furuta, H., Huang, A., Safdari, M., Matsuo, Y., Eck, D., and Faust, A. A real-world webagent with planning, long context understanding, and program synthesis. arXiv preprint arxiv:2307.12856, 2023. · DOI
  8. Han, C., Wang, Q., Xiong, W., Chen, Y., Ji, H., and Wang, S. Lm-infinite: Simple on-the-fly length generalization for large language models. arXiv preprint arXiv:2308.16137, 2023. · arXiv
  9. He, P., Liu, X., Gao, J., and Chen, W. Deberta: Decoding- enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654, 2020. · arXiv
  10. Jin, H., Han, X., Yang, J., Jiang, Z., Liu, Z., Chang, C.- Y., Chen, H., and Hu, X. Llm maybe longlm: Self- extend llm context window without tuning. arXiv preprint arXiv:2401.01325, 2024. · arXiv
  11. Kim, G., Baldi, P., and McAleer, S. Language models can solve computer tasks. arXiv preprint arxiv:2303.17491, 2023. · arXiv
  12. Kočiskỳ, T., Schwarz, J., Blunsom, P., Dyer, C., Hermann, K. M., Melis, G., and Grefenstette, E. The narrativeqa reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317-328, 2018.
  13. Lanchantin, J., Toshniwal, S., Weston, J., Szlam, A., and Sukhbaatar, S. Learning to reason and memorize with self-notes. arXiv preprint arXiv:2305.00833, 2023. · arXiv
  14. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. Retrieval-augmented generation for knowledge- intensive NLP tasks. Advances in Neural Information Processing Systems, 33:9459-9474, 2020.
  15. Lin, C.-Y. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the ACL Workshop: Text Summarization Branches Out 2004, pp. 10, 01 2004.
  16. Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilac- qua, M., Petroni, F., and Liang, P. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172, 2023. · arXiv
  17. Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. Webgpt: Browser-assisted question-answering with hu- man feedback. arXiv preprint arXiv:2112.09332, 2021. · arXiv
  18. Pang, R. Y., Parrish, A., Joshi, N., Nangia, N., Phang, J., Chen, A., Padmakumar, V., Ma, J., Thompson, J., He, H., et al. Quality: Question answering with long input texts, yes! In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5336- 5358, 2022.
  19. Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pp. 1-22, 2023.
  20. Peng, B., Li, C., He, P., Galley, M., and Gao, J. Instruc- tion tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023. · arXiv
  21. Press, O., Smith, N., and Lewis, M. Train short, test long: Attention with linear biases enables input length extrapo- lation. In International Conference on Learning Repre- sentations, 2022.
  22. Reyna, V. and Brainerd, C. Fuzzy-trace theory: Some foundational issues. Learning and Individual differences, 7(2):145-162, 1995a.
  23. Reyna, V. F. A theory of medical decision making and health: fuzzy trace theory. Medical decision making, 28 (6):850-865, 2008.
  24. Reyna, V. F. A new intuitionism: Meaning, memory, and de- velopment in fuzzy-trace theory. Judgment and Decision making, 7(3):332-359, 2012.
  25. Reyna, V. F. and Brainerd, C. J. Fuzzy-trace theory: An interim synthesis. Learning and individual Differences, 7 (1):1-75, 1995b.
  26. Robertson, S., Zaragoza, H., et al. The probabilistic rele- vance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333-389, 2009.
  27. Shaham, U., Segal, E., Ivgi, M., Efrat, A., Yoran, O., Haviv, A., Gupta, A., Xiong, W., Geva, M., Berant, J., et al. Scrolls: Standardized comparison over long language sequences. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 12007-12021, 2022.
  28. Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E. H., Schärli, N., and Zhou, D. Large language models can be easily distracted by irrelevant context. In Inter- national Conference on Machine Learning, pp. 31210- 31227. PMLR, 2023.
  29. Shi, T., Karpathy, A., Fan, L., Hernandez, J., and Liang, P. World of bits: An open-domain platform for web- based agents. In International Conference on Machine Learning, 2017.
  30. Sun, S., Liu, Y., Wang, S., Zhu, C., and Iyyer, M. Pearl: Prompting large language models to plan and execute actions over long documents. arXiv preprint arXiv:2305.14564, 2023. · arXiv
  31. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. At- tention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30.
  32. Weston, J. and Sukhbaatar, S. System 2 attention (is something you might need too). arXiv preprint arXiv:2311.11829, 2023. · arXiv
  33. Wu, J., Ouyang, L., Ziegler, D. M., Stiennon, N., Lowe, R., Leike, J., and Christiano, P. Recursively summa- rizing books with human feedback. arXiv preprint arXiv:2109.10862, 2021. · arXiv
  34. Wu, Y., Rabe, M. N., Hutchins, D., and Szegedy, C. Memorizing transformers. In International Conference on Learning Representations, 2022. URL https:// openreview.net/forum?id=TrjbxzRcnf-.
  35. Xiao, G., Tian, Y., Chen, B., Han, S., and Lewis, M. Ef- ficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453, 2023. · arXiv
  36. Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. arXiv preprint arXiv:2309.03409, 2023. · arXiv
  37. Yang, K., Tian, Y., Peng, N., and Klein, D. Re3: Generating longer stories with recursive reprompting and revision. In Proceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pp. 4393-4479, 2022.
  38. Yao, S., Chen, H., Yang, J., and Narasimhan, K. Web- shop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Informa- tion Processing Systems, 35:20744-20757, 2022.
  39. Yuan, X., Fu, J., Côté, M.-A., Tay, Y., Pal, C., and Trischler, A. Interactive machine comprehension with information seeking agents. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pp. 2325-2338, 2020.
  40. Zaheer, M., Guruganesh, G., Dubey, K. A., Ainslie, J., Al- berti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., Yang, L., and Ahmed, A. Big bird: Transformers for longer sequences. In Larochelle, H., Ranzato, M., Had- sell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 17283-17297.
  41. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., Zhang, H., Gonzalez, J. E., and Stoica, I. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
  42. Zhong, M., Yin, D., Yu, T., Zaidi, A., Mutuma, M., Jha, R., Hassan, A., Celikyilmaz, A., Liu, Y., Qiu, X., et al. Qm- sum: A new benchmark for query-based multi-domain meeting summarization. In Proceedings of the 2021 Con- ference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pp. 5905-5921, 2021.
  43. Zhong, W., Guo, L., Gao, Q., and Wang, Y. Memorybank: Enhancing large language models with long-term mem- ory. arXiv preprint arXiv:2305.10250, 2023. · arXiv

Cited by

Sorted by citations

Related

Authors
Lee, K.-H., Chen, X., Furuta, H., Canny, J., & Fischer, I.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.09727
arXiv
2402.09727
OpenAlex
W4391912783
Cited by
1
References
0
Citations by year
2025: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
LLM agents, long context, gist memory, episodic memory, memory compression, quality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review