Augmenting Neural Response Generation with Context-Aware Topical Attention
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Sequence-to-Sequence (Seq2Seq) models have witnessed a notable success in generating natural conversational exchanges.Notwithstanding the syntactically well-formed responses generated by these neural network models, they are prone to be acontextual, short and generic.In this work, we introduce a Topical Hierarchical Recurrent Encoder Decoder (THRED), a novel, fully data-driven, multi-turn response generation system intended to produce contextual and topic-aware responses.Our model is built upon the basic Seq2Seq model by augmenting it with a hierarchical joint attention mechanism that incorporates topical concepts and previous interactions into the response generation.To train our model, we provide a clean and high-quality conversational dataset mined from Reddit comments.We evaluate THRED on two novel automated metrics, dubbed Semantic Similarity and Response Echo Index, as well as with human evaluation.Our experiments demonstrate that the proposed model is able to generate more diverse and contextually relevant responses compared to the strong baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 1×
“…success of deep neural networks in natural language processing tasks such as machine translation (Sutskever et al., 2014) and language modeling (Mikolov et al., 2010), there has been growing research interest in building data-driven dialogue systems that are capable of conducting conversations…”
BackgroundVinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400
Cer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., John, R. S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.11175 · Cited by 1,267 · cited 1×
“To render the semantic representation of an utterance, we leverage Universal Sentence En-coder (Cer et al., 2018) wherein a sentence is projected to a fixed dimensional embedding vector.”
MethodComparing Twitter and Traditional Media Using Topic Models
Zhao, W. X., Jiang, J., Weng, J., He, J., Lim, E.-P., Yan, H., & Li, X. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-20161-5_34 · Cited by 1,214 · cited 1×
“The LDA parameters were estimated using the collapsed Gibbs sampling algorithm (Zhao et al., 2011).”
MethodPersonalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1205 · Cited by 1,177
A note on the evaluation of generative models
Theis, L., Oord, A. V. D., & Bethge, M. (2015) · arXiv · doi:10.48550/arxiv.1511.01844 · Cited by 1,026
Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses
Lowe, R., Noseworthy, M., Serban, I. V., Angelard-Gontier, N., Bengio, Y., & Pineau, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1103 · Cited by 341 · cited 4×
“Examples include ADEM (Lowe et al., 2017), an evaluation model that learns to score responses from an annotated dataset of human responses scores.”
BackgroundResultInfluentialPersonalizing Dialogue Agents: I have a dog, do you have pets too?
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., & Weston, J. (2018) · arXiv · doi:10.48550/arxiv.1801.07243 · Cited by 165
Cited by
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding
Dziri, N., Madotto, A., Zaïane, O., & Bose, A. J. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.168 · Cited by 81
Self-citation(2021) · arXiv (Cornell University) · Cited by 13
Human Improvised Theatre Augmented with Artificial Intelligence
Mirowski, P., & Mathewson, K. W. (2019) · Proceedings of the 2019 on Creativity and Cognition · doi:10.1145/3325480.3326547 · Cited by 6
Self-citation
Metadata
- Authors
- Dziri, N., Kamalloo, E., Mathewson, K., & Zaiane, O.
- Venue
- Proceedings of the First Workshop on NLP for Conversational AI
- Published
- 2019
- Pages
- 18-31
- OpenAlex
- W2899231639
- Cited by
- 75
- References
- 9
- FWCI
- 5.21
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Augmenting Neural Response Generation with Context-Aware Topical Attention. https://app.cochat.ai/papers/10.18653/v1/w19-4103
- The paper
- Dziri, N., Kamalloo, E., Mathewson, K., & Zaiane, O. (2019). Augmenting Neural Response Generation with Context-Aware Topical Attention. Proceedings of the First Workshop on NLP for Conversational AI. https://doi.org/10.18653/v1/w19-4103
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.