The DOI matches its DataCite record; 40 of the 41 references it cites were found in a registry or an index.
Beyond Goldfish Memory: Long-Term Open-Domain Conversation
Open accessAbstract
Despite recent improvements in open-domain dialogue models, state of the art models are trained and evaluated on short conversations with little context. In contrast, the long-term conversation setting has hardly been studied. In this work we collect and release a human-human dataset consisting of multiple chat sessions whereby the speaking partners learn about each other's interests and discuss the things they have learnt from past sessions. We show how existing models trained on existing datasets perform poorly in this long-term conversation setting in both automatic and human evaluations, and we study long-context models that can perform much better. In particular, we find retrieval-augmented methods and methods with an ability to summarize and recall previous conversations outperform the standard encoder-decoder architectures currently considered state of the art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Numerous approaches have demonstrated that memory-augmented methods improve the memory capabilities of LLMs without requiring fine-tuning, which is often impossible for API-based models [25, 42, 46].”
From CAIM: Development and Evaluation of a Cognitive AI Memory Framework for Long-Term Interaction with Intelligent Agents (2026) · cites this paper as [25]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“MSC [27] is a human-human multi-session open-domain dialogue dataset in which speakers learn about each other across repeated conversations.”
From MRMMIA: Membership Inference Attacks on Memory in Chat Agents (2026) · cites this paper as [27]
The cited source says what the claim attributes to it.
In the citing paper
“Memory-augmented methods extend this concept by retrieving user-specific information from past interactions. This addresses the challenge of a limited context window and enables the integration of past information into the input prompt [25, 29, 45].”
From CAIM: Development and Evaluation of a Cognitive AI Memory Framework for Long-Term Interaction with Intelligent Agents (2026) · cites this paper as [25]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Thus, OpenAI released a dataset, namely, PRM800K, which resulted in a SOTA reward model [153].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [153]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Since the effectiveness of the system is constrained by the performance of the reward model itself, emphasis has been placed on determining the most effective training methods [153].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [153]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281
Reading Wikipedia to answer open-domain questions
Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1171 · Cited by 1,546
Self-citationReformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations
Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., & Mihalcea, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1050 · Cited by 1,416
Personalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1205 · Cited by 1,177
Self-citationLeveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.74 · Cited by 1,109
Generating Long Sequences with Sparse Transformers
Child, R., Gray, S., Radford, A., & Sutskever, I. (2019) · arXiv · doi:10.48550/arxiv.1904.10509 · Cited by 1,063
Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset
Rashkin, H., Smith, E. M., Li, M., & Boureau, Y.-L. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1534 · Cited by 1,020
A Persona-Based Neural Conversation Model
Li, J., Galley, M., Brockett, C., Spithourakis, G., Gao, J., & Dolan, B. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1094 · Cited by 898
Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7347 · Cited by 881
Neural Summarization by Extracting Sentences and Words
Cheng, J., & Lapata, M. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1046 · Cited by 865
Recipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784
Self-citationDailyDialog: A Manually Labelled Multi-turn Dialogue DatasetPreprint
Li, Y., Su, H., Shen, X., Li, W., Cao, Z., & Niu, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03957 · Cited by 757
Wizard of Wikipedia: Knowledge-Powered Conversational agentsPreprint
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., & Weston, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01241 · Cited by 755
Self-citationTowards a Human-like Open-Domain Chatbot
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614
Retrieval Augmentation Reduces Hallucination in Conversation
Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.320 · Cited by 575
Self-citationSAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization
Gliwa, B., Mochol, I., Biesek, M., & Wawer, A. (2019) · Proceedings of the 2nd Workshop on New Frontiers in Summarization · doi:10.18653/v1/d19-5409 · Cited by 424
Neural Text Summarization: A Critical Evaluation
Kryscinski, W., Keskar, N. S., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1051 · Cited by 335
Challenges in building intelligent open-domain dialog systems
Huang, M., Zhu, X., & Gao, J. (2020) · ACM Transactions on Information Systems · doi:10.1145/3383123 · Cited by 288
DailyDialog: A Manually Labelled Multi-Turn Dialogue Dataset
Li, Y. (2025) · TIB Data Manager · doi:10.57702/l9x4bl6j · Cited by 262
Recipes for Safety in Open-domain ChatbotsPreprint
Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07079 · Cited by 236
Self-citationTraining Millions of Personalized Dialogue Agents
Mazaré, P.-E., Humeau, S., Raison, M., & Bordes, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1298 · Cited by 226
Memory fixation in the goldfish.
Agranoff, B. W., Davis, R. E., & Brink, J. J. (1965) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.54.3.788 · Cited by 217
Learning to Speak and Act in a Fantasy Text Adventure Game
Urbanek, J., Fan, A., Karamcheti, S., Jain, S., Humeau, S., Dinan, E., Rocktäschel, T., Kiela, D., Szlam, A. D., & Weston, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1062 · Cited by 195
Self-citationConversational AI: Dialogue Systems, Conversational Agents, and Chatbots
McTear, M. (2021) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02176-3 · Cited by 155
A Persona-Based Neural Conversation ModelPreprint
Li, J., Galley, M., Brockett, C., Spithourakis, G., Gao, J., & Dolan, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06155 · Cited by 148
Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts
Goo, C.-W., & Chen, Y.-N. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639531 · Cited by 119
Automatic text summarization with neural networks
Kaikhah, K. (2004) · IEEE International Conference on Intelligent Systems · doi:10.1109/is.2004.1344634 · Cited by 89
Improving Neural Conversational Models with Entropy-Based Data Filtering
Csáky, R., Purgai, P., & Recski, G. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1567 · Cited by 61
West, P., Holtzman, A., Buys, J., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1389 · Cited by 58
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2021) · arXiv · doi:10.48550/arxiv.2107.03451 · Cited by 44
Storytelling with Dialogue: A Critical Role Dungeons and Dragons Dataset
Rameshkumar, R., & Bailey, P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.459 · Cited by 37
Dial2Desc: End-to-end Dialogue Description GenerationPreprint
Pan, H., Zhou, J., Zhao, Z., Liu, Y., Cai, D., & Yang, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00185 · Cited by 17
The Gutenberg Dialogue Dataset
Csáky, R., & Recski, G. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.11 · Cited by 13
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
“[153] introduce a dataset consisting of over 40 utterances per episode on average.”
BackgroundUnleashing the potential of prompt engineering for large language models
CHEN, B. ‐., Zhang, Z., Langrené, N., & Zhu, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101260 · Cited by 1,097
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00530 · Cited by 337
“Using both parametric and non-parametric memory generally leads to reduced hallucinations and higher interpretability in tasks like question answering and summarization (Xu et al., 2021; Komeili et al., 2021; Guu et al., 2020; Lewis et al., 2020b).”
BackgroundMethodInfluentialJo, E., Epstein, D. A., Jung, H., & Kim, Y.-H. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581503 · Cited by 174
A Survey on the Memory Mechanism of Large Language Model-based Agents
Zhang, Z., Dai, Q., Bo, X., Ma, C., Li, R., Chen, X., Zhu, J., Dong, Z., & Wen, J.-R. (2025) · ACM Transactions on Information Systems · doi:10.1145/3748302 · Cited by 592
“Some previous works have constructed several datasets to support the evaluation of LLM-based agents [26, 191, 194], and some benchmarks also conduct evaluations on the memory capability of LLM-based agents [102, 182, 219].”
Metadata
- Authors
- Xu, J., Szlam, A., & Weston, J.
- Affiliations
- Meta (United States) (US)
- ORCID
- Xu, Jing
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 5180-5197
- ISSN
- 0736-587X (ISSN-L)
- arXiv
- 2107.07567
- OpenAlex
- W3186804217
- Cited by
- 148
- References
- 41
- Citations by year
- 2026: 21 · 2025: 25 · 2024: 22 · 2023: 45 · 2022: 30 · 2021: 5
- FWCI
- 16.87
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- open-domain dialogue, long-context models, retrieval-augmented generation, dialogue summarization, chatbot evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
