The DOI matches its Crossref record; 149 of the 149 references it cites were found in a registry or an index.
Challenges in Building Intelligent Open-domain Dialog Systems
Open accessAbstract
There is a resurgent interest in developing intelligent open-domain dialog systems due to the availability of large amounts of conversational data and the recent progress on neural approaches to conversational AI [33]. Unlike traditional task-oriented bots, an open-domain dialog system aims to establish long-term connections with users by satisfying the human need for communication, affection, and social belonging. This article reviews the recent work on neural approaches that are devoted to addressing three challenges in developing such systems: semantics , consistency , and interactiveness . Semantics requires a dialog system to not only understand the content of the dialog but also identify users’ emotional and social needs during the conversation. Consistency requires the system to demonstrate a consistent personality to win users’ trust and gain their long-term confidence. Interactiveness refers to the system’s ability to generate interpersonal responses to achieve particular social goals such as entertainment and conforming. The studies we select to present in this survey are based on our unique views and are by no means complete. Nevertheless, we hope that the discussion will inspire new research in developing more intelligent open-domain dialog systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“More importantly, they tend to produce factual inaccuracies and errors (Lin et al., 2022;Maynez et al., 2020;Huang et al., 2020).”
From In-Context Retrieval-Augmented Language Models (2023) · cites this paper as Huang et al., 2020
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775
Norman, W. T. (1963) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0040291 · Cited by 2,485
Norms of valence, arousal, and dominance for 13,915 English lemmas
Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0314-x · Cited by 1,959
Research works printed but not linked: 65
- Albrecht Joshua, 2007, Proceedings of the ACL. 880--887
- Bahdanau Dzmitry, 2015, Proceedings of the ICLR.
- Bordes Antoine, 2017, Proceedings of the ICLR.
- Casanueva Iñigo, Proceedings of the SIGDIAL. 12--21
- Chun-Yen Chen Dian Yu Weiming Wen Yi Mang Yang Jiaping Zhang Mingyang Zhou Kevin Jesse Austin Chau Antara Bhowmick Shreenath Iyer Giritheja Sreenivasulu Sam Davidson Ashwin Bhandare and Zhou Yu. 2018. Gunrock: Building a human-like social bot by leveraging large scale real user data. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstration. Association for Computational Linguistics 79--84. DOI:10.18653/v1/D19-3014 · DOI
- Tuan Yun-Nung Chen Yi-Lin, 2019, Proceedings of the EMNLP.
- Dagan Ido, 2005, Proceedings of the MLCW. Springer, 177--190
- Devlin Jacob, 2019, Proceedings of the NAACL. 4171--4186
- Dziri Nouha, Proceedings of the NAACL. 3806--3812
- Fang Hao, 2017, Alexa Prize Proceedings
- Galley Michel, 2019, Proceedings of the AAAI Dialog System Technology Challenges Workshop.
- Ghazvininejad Marjan, 2018, Proceedings of the AAAI. 5110--5117
- Gu Jiatao, Proceedings of the ACL.
- Henderson Matthew, Proceedings of the SIGDIAL. 467--471
- Higashinaka Ryuichiro, 2014, Proceedings of the COLING. 928--939
- Hu Baotian, 2014, Proceedings of the NIPS. 2042--2050
- Huang Po-Sen, Proceedings of the CIKM. 2333--2338
- Kim Yonghee, 2014, Acquisition and use of long-term memory for personalized dialog systems. In MA3HMI, Held in Conjunction with INTERSPEECH, 8757, 78
- Kulesza Alex, Proceedings of the TMI.
- Li Jiwei, Proceedings of the ACL.
- Li Xiang, 2016, Proceedings of the IJCAI. 2845--2851
- Li Yanran, 2017, Proceedings of the IJCNLP (Volume 1: Long Papers). 986--995
- Lin Chin-Yew, 2004, Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out
- Lipton Zachary C., 2018, Proceedings of the AAAI. 5237--5244
- Lu Zhengdong, 2013, Proceedings of the NIPS. 1367--1375
- Luan Yi, 2017, Proceedings of the IJCNLP (Volume 1: Long Papers). 605--614
- Mo Kaixiang, 2018, Proceedings of the AAAI. 5317--5324
- Moghe Nikita, Proceedings of the EMNLP. 2322--2332
- Mostafazadeh Nasrin, 2017, Proceedings of the IJCNLP. 462--472
- Mrksic Nikola, Proceedings of the ACL. 1777--1788
- Nothdurft Florian, 2015, Proceedings of the NSMMC. 73--80
- Oraby Shereen, Proceedings of the SIGDIAL. 180--190
- Pang Liang, 2016, Proceedings of the AAAI. 2793--2799
- Papineni Kishore, 2002, Proceedings of the ACL. 311--318
- Pennebaker James W., 2001, Linguistic inquiry and word count: LIWC 2001. Mahway, Lawrence Erlbaum Associates, 71, 2001
- Ritter Alan, Proceedings of the EMNLP. 583--593
- Rojas-Barahona Lina Maria, 2017, Proceedings of the EACL. 438--449
- Sai Ananya, 2019, Proceedings of the AAAI.
- Liu Peter J., Proceedings of the ACL (Volume 1: Long Papers). 1073--1083
- Roller Stephen, 2019, Proceedings of the ACL (Vol. 1 Long and Short Papers). 1702--1723
- Serban Iulian Vlad, 2016, Proceedings of the AAAI. 3776--3784
- Serban Iulian Vlad, 2017, Proceedings of the AAAI. 3295--3301
- Shen Xiaoyu, 2018, Proceedings of the AAAI. 5456--5463
- Sohn Kihyuk, 2015, Proceedings of the NIPS. 3483--3491
- Speer Robyn, 2017, Proceedings of the AAAI. 4444--4451
- Su Pei-Hao, Proceedings of the ACL.
- Sutskever Ilya, Proceedings of the NIPS. 3104--3112
- Tao Chongyang, 2018, Proceedings of the AAAI. 722--729
- Vaswani Ashish, 2017, Proceedings of the NIPS. 5998--6008
- Vijayakumar Ashwin K., 2018, Proceedings of the AAAI. 7371--7379
- Wallace Richard S., Parsing the Turing Test
- Wang Jianan, 2017, Proceedings of the SIGHAN@IJCNLP. 1--10
- Weston Jason, Proceedings of the EMNLP. 87
- Winata Genta Indra, 2017, Proceedings of the INTERSPEECH. 3437--3438
- Wolf Thomas, 2018, Proceedings of the NIPS’18 CAI Workshop.
- Xing Chen, 2017, Proceedings of the AAAI. 3351--3357
- Young Tom, 2018, Proceedings of the AAAI. 4970--4977
- Yu Zhou, Proceedings of the SIGDIAL. 404--412
- Zhang Rui, Proceedings of the AAAI. 5690--5697
- Zhang Wei-Nan, 2017, Proceedings of the WWW. 1--20
- Zhang Zhuosheng, 2018, Proceedings of the COLING. 3740--3752
- Zhou Ganbin, 2017, Proceedings of the AAAI. 3400--3407
- Zhou Ganbin, 2018, Proceedings of the AAAI. 5730--5737
- Zhou Hao, 2018, Proceedings of the AAAI. 730--739
- Zhou Kangyan, Proceedings of the EMNLP. 708--713
Cited by
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
“In contrast, the tolerance is relatively higher in dialogue systems because the desired characteristics are not only faithfulness but also user engagement, especially in open-domain dialogue systems [73].”
BackgroundChatbots applications in education: A systematic review
Okonkwo, C. W., & Ade-Ibijola, A. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100033 · Cited by 879
The design and implementation of Xiaoice, an empathetic social chatbot
Zhou, L., Gao, J., Li, D., & Shum, H.-Y. (2020) · Computational Linguistics · doi:10.1162/coli_a_00368 · Cited by 777
Self-citationThe Design and Implementation of XiaoIce, an Empathetic Social Chatbot
Zhou, L., Gao, J., Li, D., & Shum, H.-Y. (2018) · arXiv · doi:10.48550/arxiv.1812.08989 · Cited by 777
Self-citationMarked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
Cheng, M., Durmus, E., & Jurafsky, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.84 · Cited by 464
Metadata
- Authors
- Huang, M., Zhu, X., & Gao, J.
- Affiliations
- Beijing National Research Center for Information Science and Technology (CN)
- Microsoft (United States) (US)
- Venue
- ACM Transactions on Information Systems
- Published
- 2020-04-09
- Volume
- 38
- Issue
- 3
- Pages
- 1-32
- ISSN
- 1046-8188 (ISSN-L)
- 1558-2868
- OpenAlex
- W3015322406
- Cited by
- 288
- References
- 124
- Citations by year
- 2026: 12 · 2025: 29 · 2024: 36 · 2023: 63 · 2022: 53 · 2021: 66 · 2020: 26
- FWCI
- 20.43
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Transactions on Information Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key R&D Program of China (2018YFC0830200)
- National Natural Science Foundation of China (61936010/61876096)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- conversational AI, personality consistency, user trust, social-emotional needs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Challenges in Building Intelligent Open-domain Dialog Systems. https://app.cochat.ai/papers/10.1145/3383123
- The paper
- Huang, M., Zhu, X., & Gao, J. (2020). Challenges in Building Intelligent Open-domain Dialog Systems. ACM Transactions on Information Systems. https://doi.org/10.1145/3383123
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.