The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Mind the Gap Between Conversations for Improved Long-Term Dialogue Generation
Open accessAbstract
Knowing how to end and resume conversations over time is a natural part of communication, allowing for discussions to span weeks, months, or years. The duration of gaps between conversations dictates which topics are relevant and which questions to ask, and dialogue systems which do not explicitly model time may generate responses that are unnatural. In this work we explore the idea of making dialogue models aware of time, and present GapChat, a multi-session dialogue dataset in which the time between each session varies. While the dataset is constructed in real-time, progress on events in speakers' lives is simulated in order to create realistic dialogues occurring across a long timespan. We expose time information to the model and compare different representations of time and event progress. In human evaluation we show that time-aware models perform better in metrics that judge the relevance of the chosen topics and the information gained from the conversation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing works on long-term open-domain dialogues focus on evaluating model responses within contexts spanning no more than five chat sessions.”
From Evaluating Very Long-Term Conversational Memory of LLM Agents (2024) · cites this paper as Zhang et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These datasets only cover a few to dozens of dialogue turns, lacking sufficient historical dialogue content and time span to adequately assess the long-term memory capabilities of LLMs.”
From Evaluating the Long-Term Memory of Large Language Models (2025) · cites this paper as Zhang et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing works on long-term open-domain dialogues focus on evaluating model responses within contexts spanning no more than five chat sessions.”
From Evaluating Very Long-Term Conversational Memory of LLM Agents (2024) · cites this paper as Zhang et al., 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Reasoning over time intervals presents challenges. For example, the way a system responds about past events can vary depending on the amount of time that has passed since the last conversation (Zhang et al., 2023; Jang et al., 2023b).”
From Evaluating Very Long-Term Conversational Memory of LLM Agents (2024) · cites this paper as Zhang et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Reasoning over time intervals presents challenges. For example, the way a system responds about past events can vary depending on the amount of time that has passed since the last conversation (Zhang et al., 2023; Jang et al., 2023).”
From Evaluating Very Long-Term Conversational Memory of LLM Agents (2024) · cites this paper as Zhang et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2021) · arXiv · doi:10.48550/arxiv.2104.08786 · Cited by 615 · cited 1×
“We try different types of prompts following previous works researching on the factors that could affect the performance of prompting results (Schick and Schütze, 2021; Lu et al., 2022; Mishra et al., 2022).”
Result
Research works printed but not linked: 21
- Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multi- task, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023. · arXiv
- Michelle Eskritt, Juanita Whalen, and Kang Lee. 2008. Preschoolers can recognize violations of the gricean maxims. British Journal of Developmental Psychol- ogy, 26(3):435-443.
- Yubin Ge, Ziang Xiao, Jana Diesner, Heng Ji, Karrie Karahalios, and Hari Sundaram. 2022. What should i ask: A knowledge-driven approach for follow-up questions generation in conversational surveys. arXiv preprint arXiv:2205.10977. · arXiv
- Herbert P Grice. 1975. Logic and conversation. In Speech acts, pages 41-58.
- Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, Online. Association for Computational Linguistics. · DOI · Paper page
- Sonja Kleinke. 2010. Speaker activity and grice's max- ims of conversation at the interface of pragmatics and cognitive linguistics. Journal of pragmatics, 42(12):3345-3366.
- Tian Lan, Xian-Ling Mao, Wei Wei, Xiaoyan Gao, and Heyan Huang. 2020. Pone: A novel automatic eval- uation metric for open-domain generative dialogue systems. ACM Transactions on Information Systems (TOIS), 39(1):1-37.
- Raina Langevin, Ross J Lordon, Thi Avrahami, Ben- jamin R Cowan, Tad Hirsch, and Gary Hsieh. 2021. Heuristic evaluation of conversational agents. In Pro- ceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1-15.
- Margaret Li, Jason Weston, and Stephen Roller. 2019. Acute-eval: Improved dialogue evaluation with opti- mized questions and multi-turn comparisons. arXiv preprint arXiv:1909.03087. · arXiv
- Yanxiang Ling, Fei Cai, Xuejun Hu, Jun Liu, Wanyu Chen, and Honghui Chen. 2021. Context-controlled topic-aware neural response generation for open- domain dialog systems. Information Processing & Management, 58(1):102392.
- Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. 2022. Reframing instructional prompts to GPTk's language. In Find- ings of the Association for Computational Linguistics: ACL 2022, pages 589-612, Dublin, Ireland. Associa- tion for Computational Linguistics. · DOI
- Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Asso- ciation for Computational Linguistics: Main Volume, pages 255-269, Online. Association for Computa- tional Linguistics. · DOI · Paper page
- Eric Smith, Orion Hsu, Rebecca Qian, Stephen Roller, Y-Lan Boureau, and Jason Weston. 2022. Human evaluation of conversations is an open problem: com- paring the sensitivity of various methods for eval- uating dialogue agents. In Proceedings of the 4th Workshop on NLP for Conversational AI, pages 77- 97, Dublin, Ireland. Association for Computational Linguistics. · DOI · Paper page
- Swapna Somasundaran et al. 2020. Two-level trans- former and auxiliary coherence modeling for im- proved text segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 34, pages 7797-7804.
- Huiyuan Xie, Zhenghao Liu, Chenyan Xiong, Zhiyuan Liu, and Ann Copestake. 2021. Tiage: A benchmark for topic-shift aware dialog modeling. arXiv preprint arXiv:2109.04562. · arXiv
- Jing Xu, Arthur Szlam, and Jason Weston. 2022a. Be- yond goldfish memory: Long-term open-domain con- versation. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5180-5197, Dublin, Ireland. Association for Computational Linguistics. · DOI · Paper page
- Xinchao Xu, Zhibin Gou, Wenquan Wu, Zheng-Yu Niu, Hua Wu, Haifeng Wang, and Shihang Wang. 2022b. Long time no see! open-domain conversation with long-term persona memory. In Findings of the As- sociation for Computational Linguistics: ACL 2022, pages 2639-2650, Dublin, Ireland. Association for Computational Linguistics. · DOI
- Yi Xu, Hai Zhao, and Zhuosheng Zhang. 2021. Topic- aware multi-turn dialogue modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14176-14184.
- Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Per- sonalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 2204-2213, Melbourne, Australia. Association for Computational Linguistics. · DOI · Paper page
- Tong Zhang, Yong Liu, Boyang Li, Zhiwei Zeng, Peng- wei Wang, Yuan You, Chunyan Miao, and Lizhen Cui. 2022. History-aware hierarchical transformer for multi-session open-domain dialogue system. In Find- ings of the Association for Computational Linguistics: EMNLP 2022, pages 3395-3407, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
- Ben Zhou, Daniel Khashabi, Qiang Ning, and Dan Roth. 2019. "going on a vacation" takes longer than "go- ing for a walk": A study of temporal commonsense understanding. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3363-3369, Hong Kong, China. Association for Computational Linguistics. · DOI · Paper page
Cited by
Sorted by citations
Evaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.747 · Cited by 96
Evaluating the Long-Term Memory of Large Language Models
Jia, Z., Liu, Q., Li, H., Chen, Y., & Liu, J. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.1014 · Cited by 5
“To evaluate the performance of dialogue agents in long-term conversations, some datasets have been proposed(Jang et al., 2023; Zhang et al., 2023).”
Evaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.17753 · Cited by 3
“Indeed, studies in long-term open-domain dialogues have concentrated on assessing model responses within limited contexts e.g., ∼ 1K tokens over five chat sessions (Xu et al., 2022; Jang et al., 2023b; Zhang et al., 2023).”
Background
Metadata
- Authors
- Zhang, Q., Naradowsky, J., & Miyao, Y.
- Affiliations
- The University of Tokyo (JP)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 10735-10762
- arXiv
- 2310.15415
- OpenAlex
- W4389520258
- Cited by
- 4
- References
- 1
- Citations by year
- 2026: 3 · 2025: 1
- FWCI
- 0.71
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- dx.doi.org
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Funders
- Japan Society for the Promotion of Science (19H05692)
- Moonshot Research and Development Program
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- representation of time, topic relevance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
