CoChat

Challenges in Building Intelligent Open-domain Dialog Systems

Open access
Huang, Minlie, Zhu, Xiaoyan, Gao, Jianfeng · 2020 · ACM Transactions on Information Systems

The DOI matches its Crossref record; 149 of the 149 references it cites were found in a registry or an index.

Abstract

There is a resurgent interest in developing intelligent open-domain dialog systems due to the availability of large amounts of conversational data and the recent progress on neural approaches to conversational AI [33]. Unlike traditional task-oriented bots, an open-domain dialog system aims to establish long-term connections with users by satisfying the human need for communication, affection, and social belonging. This article reviews the recent work on neural approaches that are devoted to addressing three challenges in developing such systems: semantics , consistency , and interactiveness . Semantics requires a dialog system to not only understand the content of the dialog but also identify users’ emotional and social needs during the conversation. Consistency requires the system to demonstrate a consistent personality to win users’ trust and gain their long-term confidence. Interactiveness refers to the system’s ability to generate interpersonal responses to achieve particular social goals such as entertainment and conforming. The studies we select to present in this survey are based on our unique views and are by no means complete. Nevertheless, we hope that the discussion will inspire new research in developing more intelligent open-domain dialog systems.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3383123

Cites

149 of 149 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 149 references

Research works printed but not linked: 62

  1. [2] Joshua Albrecht and Rebecca Hwa. 2007. A Re-examination of Machine Learning Approaches for Sentence-Level MT Evaluation. In Proceedings of ACL 2007, June 23-30, 2007, Prague, Czech Republic . 880-887. -
  2. [5] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In ICLR 2015, San Diego, CA, USA, May 7-9, 2015 . -
  3. [9] Antoine Bordes, Y.-Lan Boureau, and Jason Weston. 2017. Learning End-to-End Goal-Oriented Dialog. In ICLR 2017, Toulon, France, April 24-26, 2017 . -
  4. [14] Iñigo Casanueva, Thomas Hain, Heidi Christensen, Ricard Marxer, and Phil D. Green. 2015. Knowledge transfer between speakers for personalised dialogue management. In Proceedings of SIGDIAL 2015, September 2-4, 2015, Prague, Czech Republic . 12-21. -
  5. Chun-Yen Chen Dian Yu Weiming Wen Yi Mang Yang Jiaping Zhang Mingyang Zhou Kevin Jesse Austin Chau Antara Bhowmick Shreenath Iyer Giritheja Sreenivasulu Sam Davidson Ashwin Bhandare and Zhou Yu. 2018. Gunrock: Building a human-like social bot by leveraging large scale real user data. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstration. Association for Computational Linguistics 79--84. DOI:10.18653/v1/D19-3014 · DOI
  6. [17] Yi-Lin Tuan Yun-Nung Chen and Hung-yi Lee. 2019. DyKgChat: Benchmarking Dialogue Generation Grounding on Dynamic Knowledge Graphs. In Proceedings of EMNLP 2019, Hong Kong, China, November 3-7, 2019 . -
  7. [22] Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The PASCAL recognising textual entailment challenge. In Machine Learning Challenges Workshop . Springer, 177-190. -
  8. [27] Nouha Dziri, Ehsan Kamalloo, Kory Mathewson, and Osmar R Zaiane. 2019. Evaluating Coherence in Dialogue Systems using Entailment. In NAACL 2019, Minneapolis, USA, June 2-7, 2019 . 3806-3812. -
  9. [29] Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtzman, Maarten Sap, Mari Ostendorf, Yejin Choi, and Noah A Smith. 2017. Sounding board-university of washington's alexa prize submission. Alexa prize proceedings (2017). -
  10. [31] Michel Galley, Chris Brockett, Xiang Gao, Jianfeng Gao, and Bill Dolan. 2019. Grounded Response Generation Task at DSTC7. In AAAI Dialog System Technology Challenges Workshop . -
  11. [36] Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. 2018. A Knowledge-Grounded Neural Conversation Model. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 5110-5117. -
  12. [43] Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. 2016. Incorporating Copying Mechanism in Sequence-toSequence Learning. In Proceedings of ACL 2016, Berlin, Germany, August 7-12, 2016 . -
  13. [47] Matthew Henderson, Blaise Thomson, and Steve J. Young. 2013. Deep Neural Network Approach for the Dialog State Tracking Challenge. In Proceedings of SIGDIAL 2013, August 22-24, 2013, SUPELEC, Metz, France . 467-471. -
  14. [48] Ryuichiro Higashinaka, Kenji Imamura, Toyomi Meguro, Chiaki Miyazaki, Nozomi Kobayashi, Hiroaki Sugiyama, Toru Hirano, Toshiro Makino, and Yoshihiro Matsuo. 2014. Towards an open-domain conversational system fully based on natural language processing. In COLING 2014, August 23-29, 2014, Dublin, Ireland . 928-939. -
  15. [49] Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. 2014. Convolutional Neural Network Architectures for Matching Natural Language Sentences. In NIPS 2014, Montreal, Quebec, Canada, December 8-13, 2014 . 2042-2050. -
  16. [51] Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry P. Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In 22nd ACM International Conference on Information and Knowledge Management, CIKM'13, San Francisco, CA, USA, October 27 - November 1, 2013 . 2333-2338. -
  17. [57] Yonghee Kim, Jeesoo Bang, Junhwi Choi, Seonghan Ryu, Sangjun Koo, and Gary Geunbae Lee. 2014. Acquisition and Use of Long-Term Memory for Personalized Dialog Systems. In Multimodal Analyses enabling Artificial Agents in Human-Machine Interaction - Second International Workshop, MA3HMI 2014, Held in Conjunction with INTERSPEECH 2014, Singapore, Singapore, September 14, 2014 , Vol. 8757. 78-87. -
  18. [58] Alex Kulesza and Stuart M. Shieber. 2004. A Learning Approach to Improving Sentence-Level MT Evaluation. In Proceedings of the 10th International Conference on Theoretical and Methodological Issues in Machine Translation . Baltimore, MD. -
  19. [62] Jiwei Li, Michel Galley, Chris Brockett, Georgios P. Spithourakis, Jianfeng Gao, and William B. Dolan. 2016. A Persona-Based Neural Conversation Model. In Proceedings of ACL 2016, Berlin, Germany, August 7-12, 2016 . -
  20. [65] Xiang Li, Lili Mou, Rui Yan, and Ming Zhang. 2016. StalemateBreaker: A Proactive Content-Introducing Approach to Automatic Human-Computer Conversation. In Proceedings of IJCAI 2016, New York, NY, USA, 9-15 July 2016 . 2845-2851. -
  21. [66] Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . 986-995. -
  22. [70] Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out (2004). -
  23. [71] Zachary C. Lipton, Xiujun Li, Jianfeng Gao, Lihong Li, Faisal Ahmed, and Li Deng. 2018. BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 5237-5244. -
  24. [80] Zhengdong Lu and Hang Li. 2013. A Deep Architecture for Matching Short Texts. In NIPS 2013, December 5-8, 2013, Lake Tahoe, Nevada, United States. 1367-1375. -
  25. [81] Yi Luan, Chris Brockett, Bill Dolan, Jianfeng Gao, and Michel Galley. 2017. Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models. In Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27 - December 1, 2017 - Volume 1: Long Papers . 605-614. -
  26. [85] Kaixiang Mo, Yu Zhang, Shuangyin Li, Jiajun Li, and Qiang Yang. 2018. Personalizing a Dialogue System With Transfer Reinforcement Learning. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 5317-5324. -
  27. [86] Nikita Moghe, Siddhartha Arora, Suman Banerjee, and Mitesh M Khapra. 2018. Towards Exploiting Background Knowledge for Building Conversation Systems. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . 2322-2332. -
  28. [88] Nasrin Mostafazadeh, Chris Brockett, Bill Dolan, Michel Galley, Jianfeng Gao, Georgios Spithourakis, and Lucy Vanderwende. 2017. Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation. In IJCNLP . 462-472. -
  29. [89] Nikola Mrksic, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve J. Young. 2017. Neural Belief Tracker: Data-Driven Dialogue State Tracking. In Proceedings of ACL 2017, Vancouver, Canada, July 30-August 4, 2017 . 1777-1788. -
  30. [91] Florian Nothdurft, Stefan Ultes, and Wolfgang Minker. 2015. Finding appropriate interaction strategies for proactive dialogue systems-an open quest. In Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication, August 6-8, 2014, Tartu, Estonia . 73-80. -
  31. [92] Shereen Oraby, Lena Reed, Shubhangi Tandon, Sharath T. S., Stephanie M. Lukin, and Marilyn A. Walker. 2018. Controlling Personality-Based Stylistic Variation with Neural Natural Language Generators. In Proceedings of SIGDIAL 2018, July 12-14, 2018, Melbourne, Australia . 180-190. -
  32. [97] Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Shengxian Wan, and Xueqi Cheng. 2016. Text Matching as Image Recognition. In Proceedings of AAAI 2016, February 12-17, 2016, Phoenix, Arizona, USA. 2793-2799. -
  33. [98] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of ACL 2002, July 6-12, 2002, Philadelphia, PA, USA. 311-318. -
  34. [100] James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates 71, 2001 (2001), 2001. -
  35. [111] Alan Ritter, Colin Cherry, and William B. Dolan. 2011. Data-Driven Response Generation in Social Media. In Proceedings of EMNLP 2011, John McIntyre Conference Centre, Edinburgh, UK, 27-31 July 2011 . 583-593. -
  36. [112] Lina Maria Rojas-Barahona, Milica Gasic, Nikola Mrksic, Pei-Hao Su, Stefan Ultes, Tsung-Hsien Wen, Steve J. Young, and David Vandyke. 2017. A Network-based End-to-End Trainable Task-oriented Dialogue System. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017 . 438-449. -
  37. [114] Ananya Sai, Mithun Das Gupta, Mitesh M. Khapra, and Mukundhan Srinivasan. 2019. Response Generation by Context-aware Prototype Editing. In Proceedings of AAAI 2019, Honolulu, Hawaii, USA, Jan 27-Feb 1, 2019 . -
  38. [116] Abigail See, Stephen Roller, Douwe Kiela, and Jason Weston. 2019. What makes a good conversation? How controllable attributes affect human judgments. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . 1702-1723. -
  39. [120] Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C. Courville, and Joelle Pineau. 2016. Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models. In Proceedings of AAAI 2016, February 12-17, 2016, Phoenix, Arizona, USA. 3776-3784. -
  40. [121] Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. 2017. A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues. In AAAI . 3295-3301. -
  41. [124] Xiaoyu Shen, Hui Su, Shuzi Niu, and Vera Demberg. 2018. Improving Variational Encoder-Decoders in Dialogue Generation. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 5456-5463. -
  42. [127] Kihyuk Sohn, Honglak Lee, and Xinchen Yan. 2015. Learning Structured Output Representation using Deep Conditional Generative Models. In NIPS 2015, December 7-12, 2015, Montreal, Quebec, Canada . 3483-3491. -
  43. [130] Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. In Proceedings of AAAI 2017, San Francisco, California, USA., February 4-9, 2017 . 4444-4451. -
  44. [132] Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Maria Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve J. Young. 2016. On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems. In Proceedings of ACL 2016, Berlin, Germany, August 7-12, 2016 . -
  45. [133] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In NIPS 2014, Montreal, Quebec, Canada, December 8-13, 2014 . 3104-3112. -
  46. [136] Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2018. RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 722-729. -
  47. [139] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems . 5998-6008. -
  48. [142] Richard S Wallace. 2009. The anatomy of ALICE. In Parsing the Turing Test . Springer, 181-210. -
  49. [144] Jianan Wang, Xin Wang, Fang Li, Zhen Xu, Zhuoran Wang, and Baoxun Wang. 2017. Group Linguistic Bias Aware Neural Response Generation. In Proceedings of the 9th SIGHAN Workshop on Chinese Language Processing, SIGHAN@IJCNLP 2017, Taipei, Taiwan, December 1, 2017 . 1-10. -
  50. [153] Jason Weston, Emily Dinan, and Alexander H Miller. 2018. Retrieve and Refine: Improved Sequence Generation Models For Dialogue. EMNLP 2018 (2018), 87. -
  51. [154] Genta Indra Winata, Onno Kampman, Yang Yang, Anik Dey, and Pascale Fung. 2017. Nora the empathetic psychologist. In Proc. Interspeech . 3437-3438. -
  52. [155] Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2018. TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents. In NIPS2018 CAI Workshop . -
  53. [160] Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic Aware Neural Response Generation. In Proceedings of AAAI 2017, San Francisco, California, USA., February 4-9, 2017 . 3351-3357. -
  54. [168] Tom Young, Erik Cambria, Iti Chaturvedi, Hao Zhou, Subham Biswas, and Minlie Huang. 2018. Augmenting End-toEnd Dialogue Systems With Commonsense Knowledge. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 4970-4977. -
  55. [169] Zhou Yu, Ziyu Xu, Alan W. Black, and Alexander I. Rudnicky. 2016. Strategy and Policy Learning for Non-TaskOriented Conversational Systems. In Proceedings of SIGDIAL 2016, Los Angeles, CA, USA, September 13-15, 2016 . 404-412. -
  56. [174] Rui Zhang, Honglak Lee, Lazaros Polymenakos, and Dragomir R. Radev. 2018. Addressee and Response Selection in Multi-Party Conversations With Speaker Interaction RNNs. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 5690-5697. -
  57. [176] Wei-Nan Zhang, Qingfu Zhu, Yifa Wang, Yanyan Zhao, and Ting Liu. 2017. Neural personalized response generation as domain adaptation. World Wide Web (2017), 1-20. -
  58. [181] Zhuosheng Zhang, Jiangtong Li, Pengfei Zhu, Hai Zhao, and Gongshen Liu. 2018. Modeling Multi-turn Conversation with Deep Utterance Aggregation. In Proceedings of the 27th International Conference on Computational Linguistics . 3740-3752. -
  59. [187] Ganbin Zhou, Ping Luo, Rongyu Cao, Fen Lin, Bo Chen, and Qing He. 2017. Mechanism-Aware Neural Machine for Dialogue Response Generation. In Proceedings of AAAI 2017, San Francisco, California, USA., February 4-9, 2017 . 3400-3407. -
  60. [188] Ganbin Zhou, Ping Luo, Yijun Xiao, Fen Lin, Bo Chen, and Qing He. 2018. Elastic Responding Machine for Dialog Generation with Dynamically Mechanism Selecting. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 5730-5737. -
  61. [189] Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018. Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory. In Proceedings of AAAI 2018, New Orleans, Louisiana, USA, February 2-7, 2018 . 730-739. -
  62. [191] Kangyan Zhou, Shrimai Prabhumoye, and Alan W. Black. 2018. A Dataset for Document Grounded Conversations. In Proceedings of EMNLP 2018, Brussels, Belgium, October 31 - November 4, 2018 . 708-713. -

Cited by

See all 424

Metadata

Authors
Huang, M., Zhu, X., & Gao, J.
Affiliations
  • Beijing National Research Center for Information Science and Technology (CN)
  • Microsoft (United States) (US)
Venue
ACM Transactions on Information Systems
Published
2020-04-09
Volume
38
Issue
3
Pages
1-32
ISSN
  • 1046-8188 (ISSN-L)
  • 1558-2868
OpenAlex
W3015322406
Cited by
288
References
124
Citations by year
2026: 12 · 2025: 29 · 2024: 36 · 2023: 63 · 2022: 53 · 2021: 66 · 2020: 26
FWCI
20.43
Type
journal article
Language
en
Open access
Locations
  • ACM Transactions on Information Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Key R&D Program of China (2018YFC0830200)
  • National Natural Science Foundation of China (61936010/61876096)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
conversational AI, personality consistency, user trust, social-emotional needs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.