Large scale multi-actor generative dialog modeling,
Open accessAbstract
Non-goal oriented dialog agents (i.e. chatbots) aim to produce varying and engaging conversations with a user; however, they typically exhibit either inconsistent personality across conversations or the average personality of all users. This paper addresses these issues by controlling an agent's persona upon generation via conditioning on prior conversations of a target actor. In doing so, we are able to utilize more abstract patterns within a person's speech and better emulate them in generated responses. This work introduces the Generative Conversation Control model, an augmented and fine-tuned GPT-2 language model that conditions on past reference conversations to probabilistically model multi-turn conversations in the actor's persona. We introduce an accompanying data collection procedure to obtain 10.3M conversations from 6 months worth of Reddit comments. We demonstrate that scaling model sizes from 117M to 8.3B parameters yields an improvement from 23.14 to 13.14 perplexity on 1.7M held out Reddit conversations. Increasing model scale yielded similar improvements in human evaluations that measure preference of model samples to the held out target distribution in terms of realism (31% increased to 37% preference), style matching (37% to 42%), grammar and content quality (29% to 42%), and conversation coherency (32% to 40%). We find that conditionally modeling past conversations improves perplexity by 0.47 in automatic evaluations. Through human trials we identify positive trends between conditional modeling and style matching and outline steps to further improve persona control.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
End-to-end models can be trained to contain internal states keeping track of the user model, learning these states from large amounts of dialogue examples.
“End-to-end models can be trained to contain internal states keeping track of the user model, learning these states simply from large amounts of dialogue examples (Crook and Marin, 2017; Boyd et al., 2020; Li et al., 2020).”
From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Boyd et al., 2020
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · see citing works
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works
A Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1014 · Cited by 1,981 · see citing works
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,639 · see citing works
Multi-Task Deep Neural Networks for Natural Language Understanding
Liu, X., He, P., Chen, W., & Gao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1441 · Cited by 1,216 · see citing works
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., & Dolan, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.acl-demos.30 · Cited by 1,076 · see citing works
A Persona-Based Neural Conversation Model
Li, J., Galley, M., Brockett, C., Spithourakis, G., Gao, J., & Dolan, B. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1094 · Cited by 898 · see citing works
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., & Catanzaro, B. (2019) · arXiv · doi:10.48550/arxiv.1909.08053 · Cited by 893 · see citing works
Self-citationA Neural Network Approach to Context-Sensitive Generation of Conversational Responses
Sordoni, A., Galley, M., Auli, M., Brockett, C., Ji, Y., Mitchell, M., Nie, J., Gao, J., & Dolan, B. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1020 · Cited by 768 · see citing works
Towards a Human-like Open-Domain Chatbot
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614 · see citing works
A Knowledge-Grounded Neural Conversation Model
Ghazvininejad, M., Brockett, C., Chang, M.-W., Dolan, B., Gao, J., Yih, W.-T., & Galley, M. (2018) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v32i1.11977 · Cited by 457 · see citing works
TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational AgentsPreprint
Wolf, T., Sanh, V., Chaumond, J., & Delangue, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08149 · Cited by 435 · see citing works
Augmenting End-to-End Dialogue Systems With Commonsense Knowledge
Young, T., Cambria, E., Chaturvedi, I., Zhou, H., Biswas, S., & Huang, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11923 · Cited by 305 · see citing works
The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
Lowe, R., Pow, N., Serban, I. V., & Pineau, J. (2015) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w15-4640 · Cited by 282 · see citing works
Training Millions of Personalized Dialogue Agents
Mazaré, P.-E., Humeau, S., Raison, M., & Bordes, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1298 · Cited by 226 · see citing works
Generative Deep Neural Networks for Dialogue: A Short ReviewPreprint
Serban, I. V., Lowe, R., Charlin, L., & Pineau, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06216 · Cited by 82 · see citing works
Personalization in Goal-Oriented DialogPreprint
Joshi, C. K., Mi, F., & Faltings, B. V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.07503 · Cited by 75 · see citing works
Chan, Z., Li, J., Yang, X., Chen, X., Hu, W., Zhao, D., & Yan, R. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1201 · Cited by 45 · see citing works
Variational Hierarchical User-based Conversation Model
Bak, J., & Oh, A. H. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1202 · Cited by 23 · see citing works
An Adversarial Learning Framework For A Persona-Based Multi-Turn Dialogue Model
Olabiyi, O., Khazane, A., Salimov, A., & Mueller, E. T. (2019) · doi:10.18653/v1/w19-2301 · Cited by 19 · see citing works
Flagged first, then by citations
3 of the 27 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Recent advances in deep learning based dialogue systems: a systematic survey
Ni, J., Young, T., Pandelea, V., Xue, F., & Cambria, E. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10248-8 · Cited by 279 · see citing works
Xu, P., Patwary, M. A., Shoeybi, M., Puri, R., Fung, P., Anandkumar, A., & Catanzaro, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.226 · Cited by 112 · see citing works
Self-citationModeling Feedback in Interaction With Conversational Agents—A Review
Axelsson, A., Buschmeier, H., & Skantze, G. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.744574 · Cited by 33 · see citing works
“End-to-end models can be trained to contain internal states keeping track of the user model, learning these states simply from large amounts of dialogue examples (Crook and Marin, 2017; Boyd et al., 2020; Li et al., 2020).”
BackgroundControlling Style in Generated DialoguePreprint
Smith, E. M., Gonzalez-Rico, D., Dinan, E., & Boureau, Y.-L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.10855 · Cited by 38 · see citing works
Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledge
Jang, Y., Lim, J., Hur, Y., Oh, D., Son, S., Lee, Y., Shin, D., Kim, S., & Lim, H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21326 · Cited by 34 · see citing works
Open-domain Dialogue Generation: What We Can Do, Cannot Do, And Should Do Next
Kann, K., Ebrahimi, A., Koh, J. J., Dudy, S., & Roncone, A. (2022) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2022.nlp4convai-1.13 · Cited by 25 · see citing works
Stylized Story Generation with Style-Guided Planning
Kong, X., Huang, J., Tung, Z., Guan, J., & Huang, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.215 · Cited by 26 · see citing works
Han, S., Kim, B., Yoo, J. Y., Seo, S., Kim, S., Erdenee, E., & Chang, B. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.377 · Cited by 21 · see citing works
Unstructured Text Enhanced Open-Domain Dialogue System: A Systematic Survey
Ma, L., Li, M., Zhang, W., Li, J., & Liu, T. (2021) · ACM Transactions on Information Systems · doi:10.1145/3464377 · Cited by 23 · see citing works
Survey on dialogue systems including slavic languages
Wołk, K., Wołk, A., Wnuk, D. C., Grześ, T., & Skubis, I. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.076 · Cited by 20 · see citing works
Conversation Style Transfer using Few-Shot Learning
Roy, S., Shu, R., Παππάς, Ν., Mansimov, E., Zhang, Y., Mansour, S., & Roth, D. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.9 · Cited by 8 · see citing works
Zhou, J., Pang, L., Shen, H., & Cheng, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.553 · Cited by 7 · see citing works
Leveraging speaker-aware structure and factual knowledge for faithful dialogue summarization
Zhao, L., Xu, W., Zhang, C., & Guo, J. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108550 · Cited by 7 · see citing works
Generating Video Game Scripts with Style
Latouche, G. L., Marcotte, L., & Swanson, B. (2023) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2023.nlp4convai-1.11 · Cited by 7 · see citing works
An Empirical Study on Deep Neural Network Models for Chinese Dialogue Generation
Li, Z., Maimaiti, M., Sheng, J., Ke, Z., Silamu, W., Wang, Q., & Li, X. (2020) · Symmetry · doi:10.3390/sym12111756 · Cited by 5 · see citing works
A Background Knowledge Revising and Incorporating Dialogue Model
Zhao, X., Feng, X., & Chen, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3123128 · Cited by 5 · see citing works
Parallel Decoders Guided Lexically Constrained Response Generation
Cheng, D., Sun, H., Qi, Q., Wang, J., & Qi, Y. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671855 · Cited by 1 · see citing works
Local Knowledge Powered Conversational AgentsPreprint
Santhanam, S., Ping, W., Puri, R., Shoeybi, M., Patwary, M. A., & Catanzaro, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10150 · Cited by 4 · see citing works
Self-citationSingle User-Item Profile Matrix in Recommendations System
Dalai, N. (2023) · Journal of Propulsion Technology · doi:10.52783/tjjpt.v44.i3.260
MERCI: A Multimodal Dataset for Personalised and Emotionally-Aware Dialogues
Althubyani, M., Meng, Z., Xie, S., Cruz, F., Razzak, I., Prasad, M., Sandoval, E. B., & Kocaballi, B. (2025) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1109/cbmi66578.2025.11339324
Xu, P., Patwary, M., Shoeybi, M., Puri, R., Fung, P., Anandkumar, A., & Catanzaro, B. (2020) · arXiv · doi:10.48550/arxiv.2010.00840 · Cited by 12 · see citing works
Self-citation
- Authors
- Boyd, A., Puri, R., Shoeybi, M., Patwary, M., & Catanzaro, B.
- Affiliations
- University of California, Irvine (US)
- Nvidia (United Kingdom) (GB)
- ORCID
- Catanzaro, Bryan
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 66-84
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3025249494
- Cited by
- 22
- References
- 27
- Citations by year
- 2025: 1 · 2023: 5 · 2022: 6 · 2021: 5 · 2020: 5
- FWCI
- 2.01
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
- Keywords
- chatbots, GPT-2, model scaling, perplexity, human evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
