CoChat
Sign Up
PreprintCited by 777Speech and dialogue systemsAI in Service InteractionsSocial Robot Interaction and HRI

The Design and Implementation of XiaoIce, an Empathetic Social Chatbot

Open access
Zhou, Li, Gao, Jianfeng, Li, Di, Shum, Heung-Yeung·2018·arXiv
DOI 10.48550/arxiv.1812.08989arXiv 1812.08989
Open this paper in CoChat — free account

Abstract

This paper describes the development of Microsoft XiaoIce, the most popular social chatbot in the world. XiaoIce is uniquely designed as an AI companion with an emotional connection to satisfy the human need for communication, affection, and social belonging. We take into account both intelligent quotient (IQ) and emotional quotient (EQ) in system design, cast human-machine social chat as decision-making over Markov Decision Processes (MDPs), and optimize XiaoIce for long-term user engagement, measured in expected Conversation-turns Per Session (CPS). We detail the system architecture and key components including dialogue manager, core chat, skills, and an empathetic computing module. We show how XiaoIce dynamically recognizes human feelings and states, understands user intent, and responds to user needs throughout long conversations. Since her launch in 2014, XiaoIce has communicated with over 660 million active users and succeeded in establishing long-term relationships with many of them. Analysis of large scale online logs shows that XiaoIce has achieved an average CPS of 23, which is significantly higher than that of other chatbots and even human conversations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1812.08989

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Empathic responses and social skills in the social chatbot XiaoIce are key drivers of users engaging in long-term relationships with it.

    “Empathic responses and social skills in the social chatbot XiaoIce are argued to be key drivers of users engaging in long-term relationships with it (Zhou et al., 2018).”

    From My Chatbot Companion - a Study of Human-Chatbot Relationships (2021) · cites this paper as Zhou et al., 2018

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 62

  1. Joshua Albrecht and Rebecca Hwa. A re-examination of machine learning approaches for sentencelevel mt evaluation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics , pages 880-887, Prague, Czech Republic, June 2007.
  2. Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image caption evaluation. In European Conference on Computer Vision , pages 382-398. Springer, 2016.
  3. Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 6077-6086, 2018.
  4. Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization , pages 65-72, 2005.
  5. Sheryl Brahnam. Strategies for handling customer abuse of ecas. Abuse: The darker side of humancomputer interaction , pages 62-67, 2005.
  6. Yang Cai. Empathic computing. In Ambient Intelligence in Everyday Life , pages 67-85. Springer, 2006.
  7. Wen-Feng Cheng, Chao-Chung Wu, Ruihua Song, Jianlong Fu, Xing Xie, and Jian-Yun Nie. Image inspired poetry generation in xiaoice. arXiv preprint arXiv:1808.03090 , 2018. · arXiv
  8. Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation , pages 103-111, Doha, Qatar, October 2014. URL http://www.aclweb.org/anthology/W14-4012 .
  9. Kenneth Mark Colby, Sylvia Weber, and Franklin Dennis Hilf. Artificial paranoia. Artificial Intelligence , 2(1):1-25, 1971.
  10. Heriberto Cuayáhuitl, Seonghan Ryu, Donghyeon Lee, and Jihie Kim. A study on dialogue reward prediction for open-ended conversational agents. NeurIPS Workshop on Conversational AI , 2018.
  11. Amanda Cercas Curry and Verena Rieser. # metoo alexa: How conversational systems respond to sexual harassment. In Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing , pages 7-14, 2018.
  12. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. Wizard of wikipedia: Knowledge-powered conversational agents. CoRR , abs/1811.01241, 2018. · arXiv
  13. Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh K Srivastava, Li Deng, Piotr Dollár, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C Platt, et al. From captions to visual concepts and back. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1473-1482, 2015.
  14. Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtzman, Maarten Sap, Mari Ostendorf, Yejin Choi, and Noah A Smith. Sounding board-university of washington's alexa prize submission. Alexa prize proceedings , 2017.
  15. Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A Smith, and Mari Ostendorf. Sounding board: A user-centric and content-driven social chatbot. NAACL HLT 2018 , page 96, 2018.
  16. Denis Fedorenko, Nikita Smetanin, and Artem Rodichev. Avoiding echo-responses in a retrievalbased conversation system. In Conference on Artificial Intelligence and Natural Language , pages 91-97. Springer, 2018.
  17. Pascale Fung, Dario Bertero, Yan Wan, Anik Dey, Ricky Ho Yin Chan, Farhad Bin Siddique, Yang Yang, Chien-Sheng Wu, and Ruixi Lin. Towards empathetic human-robot interactions. CoRR , abs/1605.04072, 2016. URL http://arxiv.org/abs/1605.04072 . · arXiv
  18. Michel Galley, Chris Brockett, Alessandro Sordoni, Yangfeng Ji, Michael Auli, Chris Quirk, Margaret Mitchell, Jianfeng Gao, and Bill Dolan. deltaBLEU: A discriminative metric for generation tasks with intrinsically diverse targets. In ACL-IJCNLP , page 445-450, 2015.
  19. Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, and Li Deng. Stylenet: Generating attractive visual captions with styles. In Proc IEEE Conf on Computer Vision and Pattern Recognition , pages 3137-3146, 2017.
  20. Jianfeng Gao, Patrick Pantel, Michael Gamon, Xiaodong He, and Li Deng. Modeling interestingness with deep neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 2-13, 2014.
  21. Jianfeng Gao, Michel Galley, and Lihong Li. Neural approaches to conversational ai. Foundations and Trends R © in Information Retrieval , 13(2-3):127-298, 2019.
  22. Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. A knowledge-grounded neural conversation model. In Proc. of AAAI , 2018.
  23. Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. Challenges in building intelligent open-domain dialog systems. arXiv preprint arXiv:1905.05709 , 2019a. · arXiv
  24. Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using clickthrough data. In CIKM , pages 2333-2338. ACM, 2013.
  25. Chandra Khatri, Behnam Hedayatnia, Anu Venkatesh, Jeff Nunn, Yi Pan, Qing Liu, Han Song, Anna Gottardi, Sanjeev Kwatra, Sanju Pancholi, et al. Advancing the state of the art in open domain dialog systems through the alexa prize. arXiv preprint arXiv:1812.10757 , 2018. · arXiv
  26. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting objective function for neural conversation models. In NAACL-HLT , 2016a.
  27. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A persona-based neural conversation model. In ACL , 2016b.
  28. Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages 1192-1202, 2016c.
  29. Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Proceedings of the ACL workshop . Association for Computational Linguistics, 2004.
  30. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of EMNLP 2016, Austin, Texas, USA, November 1-4, 2016 , pages 2122-2132, 2016.
  31. Ryan Lowe, Michael Noseworthy, Iulian Vlad Serban, Nicolas Angelard-Gontier, Yoshua Bengio, and Joelle Pineau. Towards an automatic turing test: Learning to evaluate dialogue responses. In Proceedings of ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers , pages 1116-1126, 2017. doi: 10.18653/v1/P17-1103. · DOI · Paper page
  32. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to information retrieval . Cambridge University Press, 2008. ISBN 978-0-521-86571-5.
  33. Abraham Harold Maslow. A theory of human motivation. Psychological review , 50(4):370, 1943.
  34. Alexander Patrick Mathews, Lexing Xie, and Xuming He. Senticap: Generating image descriptions with sentiments. In AAAI , pages 3574-3580, 2016.
  35. Teruhisa Misu, Kallirroi Georgila, Anton Leuski, and David Traum. Reinforcement learning of question-answering dialogue policies for virtual museum guides. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue , pages 84-93. Association for Computational Linguistics, 2012.
  36. Meredith Ringel Morris, Annuska Zolyomi, Catherine Yao, Sina Bahram, Jeffrey P Bigham, and Shaun K Kane. With most of it being pictures now, i rarely use it: Understanding twitter's evolving accessibility to blind users. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems , pages 5506-5516. ACM, 2016.
  37. Nasrin Mostafazadeh, Chris Brockett, Bill Dolan, Michel Galley, Jianfeng Gao, Georgios Spithourakis, and Lucy Vanderwende. Image-grounded conversations: Multimodal context for natural question and response generation. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 462-472, 2017.
  38. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics , pages 311-318. Association for Computational Linguistics, 2002.
  39. Baolin Peng, Xiujun Li, Lihong Li, Jianfeng Gao, Asli Celikyilmaz, Sungjin Lee, and Kam-Fai Wong. Composite task-completion dialogue policy learning via hierarchical deep reinforcement learning. In EMNLP , pages 2231-2240, 2017.
  40. Rosalind W Picard. Affective computing . MIT press, 2000.
  41. Ashwin Ram, Rohit Prasad, Chandra Khatri, Anu Venkatesh, Raefer Gabriel, Qing Liu, Jeff Nunn, Behnam Hedayatnia, Ming Cheng, Ashish Nagar, et al. Conversational ai: The science behind the alexa prize. arXiv preprint arXiv:1801.03604 , 2018. · arXiv
  42. Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. Self-critical sequence training for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017.
  43. Ananya Sai, Mithun Das Gupta, Mitesh M. Khapra, and Mukundhan Srinivasan. Response generation by context-aware prototype editing. In Proceedings of AAAI 2019, Honolulu, Hawaii, USA, January 27-February 1, 2019 , 2019.
  44. Anna Schmidt and Michael Wiegand. A survey on hate speech detection using natural language processing. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media , pages 1-10, 2017.
  45. Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C Courville, and Joelle Pineau. Building end-to-end dialogue systems using generative hierarchical neural network models. In AAAI , volume 16, pages 3776-3784, 2016.
  46. Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. A hierarchical latent variable encoder-decoder model for generating dialogues. In AAAI , pages 3295-3301, 2017.
  47. Lifeng Shang, Zhengdong Lu, and Hang Li. Neural responding machine for short-text conversation. In ACL-IJCNLP , pages 1577-1586, July 2015.
  48. Bayan Abu Shawar and Eric Atwell. Different measurements metrics to evaluate a chatbot system. In Proceedings of the workshop on bridging the gap: Academic and industrial research in dialog technologies , pages 89-96. Association for Computational Linguistics, 2007.
  49. Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. A latent semantic model with convolutional-pooling structure for information retrieval. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management , pages 101-110. ACM, 2014.
  50. Heung-Yeung Shum, Xiaodong He, and Di Li. From eliza to xiaoice: Challenges and opportunities with social chatbots. CoRR , abs/1801.01957, 2018. URL http://arxiv.org/abs/1801.01957 . · arXiv
  51. Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. A neural network approach to context-sensitive generation of conversational responses. In NAACL-HLT , May 2015.
  52. Ilya Sutskever, Oriol Vinyals, and Quoc Le. Sequence to sequence learning with neural networks. In NIPS , pages 3104-3112, 2014.
  53. Richard S. Sutton, Doina Precup, and Satinder P. Singh. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence , 112(1-2): 181-211, 1999. An earlier version appeared as Technical Report 98-74, Department of Computer Science, University of Massachusetts, Amherst, MA 01003. April, 1998.
  54. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4566-4575, 2015.
  55. Oriol Vinyals and Quoc Le. A neural conversational model. In ICML Deep Learning Workshop , July 2015.
  56. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3156-3164, 2015.
  57. Richard S Wallace. The anatomy of alice. In Parsing the Turing Test , pages 181-210. Springer, 2009.
  58. Joseph Weizenbaum. Eliza-a computer program for the study of natural language communication between man and machine. Communications of the ACM , 9(1):36-45, 1966.
  59. Bowen Wu, Baoxun Wang, and Hui Xue. Ranking responses oriented to conversational relevance in chat-bots. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers , pages 652-662, 2016.
  60. Qiang Wu, Christopher JC Burges, Krysta M Svore, and Jianfeng Gao. Adapting boosting for information retrieval measures. Information Retrieval , 13(3):254-270, 2010.
  61. Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, and Wei-Ying Ma. Topic aware neural response generation. In AAAI , volume 17, pages 3351-3357, 2017.
  62. Kai Zhang, Wei Wu, Fang Wang, Ming Zhou, and Zhoujun Li. Learning distributed representations of data in community question answering for question retrieval. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining , pages 533-542. ACM, 2016.

Cited by

Sorted by citations

Authors
Zhou, L., Gao, J., Li, D., & Shum, H.-Y.
Affiliations
Microsoft (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
Volume
46
Issue
1
Pages
53-93
ISSN
0891-2017 (ISSN-L)
DOI
10.48550/arxiv.1812.08989
arXiv
1812.08989
OpenAlex
W2906579211
Cited by
777
References
0
Citations by year
2026: 97 · 2025: 109 · 2024: 106 · 2023: 113 · 2022: 120 · 2021: 112 · 2020: 94 · 2019: 26
FWCI
43.15
Type
journal article
Language
en
Open access
Licence
Locations
  • Computational Linguistics (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • Microsoft Research
  • Microsoft Research Asia
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
social chatbots, Markov decision process, dialogue management, user engagement, emotion recognition, user intent understanding

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review