The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
The Design and Implementation of XiaoIce, an Empathetic Social Chatbot
Open accessAbstract
This article describes the development of Microsoft XiaoIce, the most popular social chatbot in the world. XiaoIce is uniquely designed as an artifical intelligence companion with an emotional connection to satisfy the human need for communication, affection, and social belonging. We take into account both intelligent quotient and emotional quotient in system design, cast human–machine social chat as decision-making over Markov Decision Processes, and optimize XiaoIce for long-term user engagement, measured in expected Conversation-turns Per Session (CPS). We detail the system architecture and key components, including dialogue manager, core chat, skills, and an empathetic computing module. We show how XiaoIce dynamically recognizes human feelings and states, understands user intent, and responds to user needs throughout long conversations. Since the release in 2014, XiaoIce has communicated with over 660 million active users and succeeded in establishing long-term relationships with many of them. Analysis of large-scale online logs shows that XiaoIce has achieved an average CPS of 23, which is significantly higher than that of other chatbots and even human conversations.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This paper describes Microsoft XiaoIce, an empathetic social chatbot designed as an AI companion with IQ, EQ, and personality, optimized for long-term user engagement measured by expected Conversation-turns Per Session (CPS). It reports system architecture, dialogue manager, empathetic computing, Core Chat, skills, and large-scale deployment results, including 660 million active users and an average CPS of 23.
Key findings
- XiaoIce was launched in May 2014 and by July 2018 had over 660 million active users, was deployed in five countries on more than 40 platforms, and achieved an average CPS of 23 in its 6th generation; the paper compares this with CPS 9 for human conversations in its user study and CPS 14.6 for the latest Amazon Alexa systems per Khatri et al. [2018].
- The empathetic computing module’s A/B test on Weibo showed no significant CPS change but increased NAU from 0.5 million to 5.1 million in 3 months; it was released in July 2018 and became the most important feature in the 6th generation.
- Core Chat’s hybrid approach improved expected CPS by 0.5 in two weeks over a retrieval-only baseline; incorporating the neural generator and unpaired-data retrieval improved response coverage by 20% and 10%, respectively. Topic Manager improved Core Chat expected CPS by 0.5, and Image Commenting doubled expected CPS across all dialogues containing images.
- The persona-based neural response generator achieved perplexity 25.0 (−8.4% vs. seq2seq) and BLEU 1.9% (+18.8% vs. seq2seq, +11.8% vs. LSTM-MMI) on the TV series dataset; the XiaoIce Image Commenting prototype scored BLEU-4 4.53, ROUGE-L 15.33, CIDEr-D 3.21, METEOR 15.51, and SPICE 4.82, outperforming four state-of-the-art image captioning systems.
Methods
- Design: XiaoIce is a modular system with user experience, conversation engine, and data layers; social chat is cast as hierarchical decision-making over options in Markov Decision Processes, with a dialogue manager, empathetic computing module, Core Chat, and 230 dialogue skills, optimized for expected CPS.
- Data and training: paired conversational data included more than 30 billion conversation pairs as of May 2018, with 70% of XiaoIce’s responses retrieved from her own past conversations; classifiers for user understanding were trained on 10K manually labeled dialogue sessions, the response ranker on 50K manually labeled dialogues, neural generation and ranking features on 50 million human dialogues, and Image Commenting on 28 million images each paired with 6 text comments.
- Evaluation: A/B tests used expected CPS and NAU; pilot studies used human ratings on a 3-level quality scale and benchmark metrics such as perplexity, BLEU, METEOR, CIDEr, ROUGE-L, and SPICE; large-scale online logs were analyzed over long periods.
Limitations
- The paper states that automatic evaluation of open-domain dialogue systems is “by no means a solved problem,” that common automatic metrics have limitations, and that a unified modeling framework for XiaoIce “remains to be proved” effective.
- The paper states that fully grounding all conversations in the physical world for goal-oriented interactions remains a non-trivial challenge, and that the fundamental mechanism of human-level intelligence is not yet fully understood.
- The paper discusses ethical concerns including privacy, who is in control, unrealistic user expectations, and potential misuse; evident from the text, many key results come from proprietary A/B tests and online logs without reported confidence intervals or full statistical details, and the comparison to human conversations relies on an unspecified user study.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 27 cited sources checked · 0 supported · 0 flagged
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although the overall quality of the candidates generated from the unpaired database is lower than those retrieved from the paired database, with unpaired database XiaoIce can cover a much broader range of topics.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although the overall quality of the candidates generated from the unpaired database is lower than those retrieved from the paired database, with unpaired database XiaoIce can cover a much broader range of topics.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The unpaired database sentences from public lectures and quotes are suitable as candidate responses for XiaoIce.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The unpaired database we have used in XiaoIce consists of sentences collected from public lectures and quotes in news articles and reports.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The module was released in July 2018, and became the most important feature in the 6th generation of XiaoIce that has substantially strengthened XiaoIce's emotional connections to human users and increased XiaoIce's NAU.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although we do not observe any significant change in CPS, NAU is increased from 0.5 million to 5.1 million in 3 months.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although we do not observe any significant change in CPS, NAU is increased from 0.5 million to 5.1 million in 3 months.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our finding is that the majority of the 'desired' users are young, female users.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nowadays, 70% of XiaoIce's responses are retrieved from her own past conversations.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although the response candidates retrieved from the paired database is of high quality, the coverage is low because many new or less frequently discussed topics on the Internet forums are not included in the database.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After the launch of XiaoIce in May 2014, we also started collecting human-machine conversations generated by XiaoIce and her users, which amounted to more than 30 billion conversation pairs as of May 2018.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
- Probably not a substantial claim. 66
- It may state two separate claims as one. 43
- The quote only partly states this claim. 2
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Over the last 5 years XiaoIce has developed more than 230 skills, ranging from answering questions and recommending movies or restaurants to comforting and storytelling.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The module was released in July 2018, and became the most important feature in the 6th generation of XiaoIce that has substantially strengthened XiaoIce's emotional connections to human users and increased XiaoIce's NAU.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We show how XiaoIce dynamically recognizes human feelings and states, understands user intent, and responds to user needs throughout long conversations.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite being extremely knowledgeable due to her access to large amounts of data and knowledge, XiaoIce never comes across as egotistical and only demonstrates her wit and creativity when appropriate.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As shown in Figure 2, XiaoIce demonstrates sufficient EQ as it generates socially acceptable responses (e.g., having a sense of humor, comforting, etc.), and can determine whether to drive the conversation to a new topic when e.g., the conversation has stalled, or whether or not to be actively listening when the user herself is engaged in the conversation.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most important and sophisticated skill is Core Chat, which can engage long and open-domain conversations with users.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of large-scale online logs shows that XiaoIce has achieved an average CPS of 23, which is significantly higher than that of other chatbots and even human conversations.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This is possible because XiaoIce has attracted a large number of active users since her launch in 2014.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 7, while a typical seq2seq model which is not grounded in any persona often outputs inconsistent responses Li et al. [2016b], XiaoIce is able to generate consistent and humorous responses.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in the example in Figure 3, XiaoIce switches to a new but related topic (i.e., a song titled 'the time machine' by Ashin in Turn 13) when she detects that the user is not familiar with 'Ashin' and about to terminate the conversation by responding 'Ah! Boring' and 'Okay, I am going to bed'.”
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Since the release in 2014, XiaoIce has communicated with over 660 million active users and succeeded in establishing long-term relationships with many of them.”
- Also stated in: § 1 Introduction · p. 1 · plain
“Since her launch in China in May 2014, XiaoIce has attracted over 660 million active users (i.e., subscribed users).”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Also stated in: § 1 Introduction · p. 1 · plain
In the paper
“As shown in Table 1, the persona model significantly outperforms both baselines, achieving a lower perplexity (-8.4%) and a higher BLEU score (+18.8% and +11.8%) Papineni et al. [2002].”
What the sources say
- Not checked yetcites Papineni et al. [2002]
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce has been deployed on many platforms. As a result, we have witnessed the creation and growth of a XiaoIce ecosystem since 2016.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce-generated TV and Radio programs have covered 9 top satellite TV stations, and have attracted over 800 million weekly active audience.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 5 (a), CQU rewrites user queries to include necessary context, e.g., replacing 'him' in Turn 12 with 'Ashin', 'that' with 'The Time Machine' in Turn 14, and adding 'send The Time Machine' in Turn 15.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Every poem in the album is jointly written by XiaoIce and human poets.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce, however, generates a factual response based on the user's geographical location and the corresponding database, as shown in Figure 19 (a).”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“On May 15, 2018, XiaoIce published the first AI-created Chinese poem album in history 11 .”
What the sources say
- Not checked yetcites [11] Cuayáhuitl Heriberto, 2018
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce is equipped with 230 dialogue skills, which, together with Core Chat and Image Commenting, form the IQ component of XiaoIce.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since July 2014, XiaoIce has released 230 skills, which amounts to nearly one new skill every week, as shown in Figure 21.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Depending on the availability of training data and knowledge bases for each individual task, either a rule-based method or a data-driven method, or a hybrid of both is adopted for the task.”
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce, on the other hand, is designed as an AI companion which integrates both EQ and IQ skills that are needed to help users complete specific tasks.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Up to August 2015, XiaoIce has had more than 10 billion conversations with humans.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that the pairs that are shared among acquaintances (e.g., coworkers, classmates and friends) are of good quality, and amount to a large portion in the database.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These characters include more than 60,000 official accounts, e.g., Lawson and Tokopedia's customer service bots, Pokemon, Tecent and Netease's chatbots, and even real human celebrities such as the singers of Guoyun Entertainment.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“From 2015 on, XiaoIce started powering third party characters, personal assistants and real human's virtual avatars.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In two months, XiaoIce successfully became a cross-platform social chatbot.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As of July 2018, XiaoIce has been deployed on more than 40 platforms, and has attracted 660 million active users.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The longest conversation lasts for more than 6 hours, covering 53 different topics across 8 domains and using 16 task-completion skills.”
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The XiaoIce Poetry Generation skill has helped over four million users to generate poems.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce was first launched on May 29, 2014, and went viral immediately.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“By then, users have proactively posted more than 6 million conversation sessions to public.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Within 72 hours, XiaoIce was looped into 1.5 million chat groups.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The CPS is close to the 3rd generation of XiaoIce, as shown in Figure 20.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Compared to the neural response generator which often generates well-form but short responses, the candidates from unpaired database are much longer with more useful content.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Through a long conversation, XiaoIce has demonstrated human-like empathy and social skills, and eventually helped the user regain her confidence and move forward with a positive attitude.”
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“| XiaoIce (prototype) | 4.53 | 15.33 | 3.21 | 15.51 | 4.82 |”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For example, although XiaoIce can provide answers to many questions due to the access to the large scale knowledge graph, these answers are not always accurate.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“XiaoIce cheers users up, encourages them, helps them accomplish tasks, and holds their attention throughout the conversation.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As an AI companion of human users, XiaoIce can recognize user interests and intents much more accurately than traditional intelligent personal assistants.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While XiaoIce carefully leverages this information to serve users and build emotional bonds over a long period of time”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“When working on the modules or tasks where there are benchmarks used in the research community, such as the neural response generator (Section 4.3), we often compare our approaches to other state-of-the-art methods using these benchmarks.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Neural-model-based and retrieval-based generators are complementary: the neural-model-based generator offers robustness and high coverage, while the retrieval-based provides high-quality responses for popular topics.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The dialogue policy is designed to optimize long-term user engagement through an iterative, trialand-error process based on the feedback of XiaoIce's users.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- The authors state this as an assumption: Human-machine social chat can be modeled as a hierarchical decision-making process over Markov Decision Processes.
- The authors state this as an assumption: Expected CPS is a valid measure of long-term user engagement for social chatbots.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A social chatbot needs to present a consistent personality to set the right expectations for users in the conversation and gain their long-term confidence and trust.”
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This paper describes the development of Microsoft XiaoIce , the most popular social chatbot in the world.”
- Also stated in: § 1 Introduction · p. 1 · plain
“In this paper we present the design and implementation of Microsoft XiaoIce ('Little Ice' literally in Chinese), the most popular social chatbot in the world.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Also stated in: § 1 Introduction · p. 1 · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Social chatbots require a sufficiently high IQ to acquire a range of skills to keep up with the users and help them complete specific tasks.”
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Topic Manager simulates human behavior of changing topics during a conversation.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Empathetic computing reflects XiaoIce's EQ and models the empathetic aspects of the humanmachine conversation.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
In the paper
“More importantly, social chatbots also require a sufficient EQ to meet users' emotional needs, such as emotional affection and social belonging, which are among the fundamental needs for human beings Maslow [1943].”
What the sources say
- Not checked yetcites Maslow [1943]
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Implementation of the conversation engine relies heavily on A/B test to evaluate if a new module or a new dialogue skill is going to improve an existing component.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the past, the Turing Test has been used to evaluate chitchat performance. But it is not sufficient to measure the success of long-term, emotional engagement with users.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To ensure a smooth conversation, we avoid switching among different skills too often. We prefer keeping the running skill activated until it terminates to activating a new skill.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Figure 3, XiaoIce tries a new topic (i.e., a popular singer named Ashin) in Turn 5 and recommends a song in Turn 15, and thereby learns the user's preferences (e.g., the music topic and the singer he loves), knowledge that would lead to more engagement in the future.”
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“XiaoIce wins the user's trust and friendship by her wonderful sense of humor and empathetic responses to all sorts of questions, some of which are quite challenging, such as “you are all lies”, and “who is your daddy”.”
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, XiaoIce has to interact with the user's environment and access real-world knowledge e.g., via API calls.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Traditionally, social chatbots are designed for chitchat scenarios where bots are expected to mimic human user conversations but not to interact with the user's environment.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Each generated response is labeled on the 3-level quality scale by three human judges.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Rests on an assumption: The 3-level quality scale labels assigned by human judges reliably measure response quality.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the short term, incorporating many task-completion skills can reduce the CPS since these skills help users accomplish tasks more efficiently by minimizing the CPS.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Through a long conversation, XiaoIce has demonstrated human-like empathy and social skills, and eventually helped the user regain her confidence and move forward with a positive attitude.”
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To human users, it is these latter aspects that are more important to drive a meaningful and interesting conversation.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Compared with traditional personal assistants, XiaoIce's task-completion skills offer more perspectives and empathy in generating interpersonal responses.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“According to Khatri et al. [2018], the latest Alexa systems have achieved the CPS of 14.6, an increase of 54% since the launch of the 2018 competition.”
What the sources say
- Not checked yetcites Khatri et al. [2018]
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To control the data quality, a pipeline similar to that for Core Chat is applied to retain only the pairs whose text comments fit XiaoIce's persona 9 .”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It can be observed that the XiaoIce-produced comments are emotional, subjective, imaginative, and are very likely to inspire meaningful human-machine interactions, while the comments generated by the other image captioning models are reasonable in content but boring in the context of social chats, and thus less likely to improve user engagement.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The response candidate ranker is trained using 50K manually labeled dialogues.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The use of neural response generator in Core Chat, starting from the 5th generation, significantly improves the coverage and diversity of XiaoIce's responses.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The full duplex voice mode released in the 5th generation has made the humanmachine communication substantially more natural, thus significantly increasing the length of conversation sessions.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is the latter, the social skill aspects, that distinguishe image commenting from other traditional vision tasks such as image tagging and image description, as illustrated in Figure 11.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a result, it helps drive the number of active users from 500 million to 660 million, and keep the CPS to 23 in spite of the incorporation of many task-completion tasks that are designed to minimize the CPS such as those that control the smart devices.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But in the long run, these new skills not only help grow XiaoIce's NAU by meeting user needs and strengthening the emotional bond with human users, but also provide large amounts of training data to improve the core conversation engine e.g., by optimizing the neural response generation models, empathy models, and the dialogue manager, etc.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
In the paper
“The second is the LSTM-MMI model Li et al. [2016a], which is one of the state-of-the-art neural response generation models.”
What the sources say
- Not checked yetcites Li et al. [2016a]
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, low BLEU (or other metrics) scores do not necessarily indicate low quality as the number of reference responses is always limited in the test set.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
- Rests on an assumption: In open-domain dialog systems, the same input may have many plausible responses that differ in topics or contents significantly.
- Rests on an assumption: The number of reference responses in the test set is always limited.
In the paper
“Galley et al. [2015] showed that the correlation of string-based metrics (e.g., BLEU and deltaBLEU) significantly increases with the units of measurement bigger than a sentence.”
What the sources say
- Not checked yetcites Galley et al. [2015]
Strength:boosted· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“But as pointed out in Gao et al. [2019], the correlation analysis by Liu et al. [2016] is performed at the sentence level while BLEU is designed from the outset to be used as a corpus-level metric.”
What the sources say
- Not checked yetcites Gao et al. [2019]
- Not checked yetcites Liu et al. [2016]
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Retrieval-based methods are often evaluated using traditional information retrieval metrics Manning et al. [2008] such as as Precision@K, Mean Average Precision (MAP), and normalized Discounted Cumulative Gain (nDCG).”
What the sources say
- Not checked yetcites Manning et al. [2008]
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, even a completely deterministic function can lead to unpredictable behavior. For example, a simple answer 'Yes' by XiaoIce could be perceived offensive in a given context.”
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is not surprising, then, that with vastly more people being digitally connected in the social media age, social chatbots have become an important alternative means for engagement.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: Happiness and meaningful conversations often go hand in hand.
None of the sources cited for this claim have been checked yet.
In the paper
“Generation-based methods are evaluated using those metrics borrowed from text generation tasks like machine translation and text summarization, using string and n-gram matching metrics such as BLEU Papineni et al. [2002], METEOR Banerjee and Lavie [2005] and ROUGE Lin [2004].”
What the sources say
- Not checked yetcites Papineni et al. [2002]
- Not checked yetcites Banerjee and Lavie [2005]
- Not checked yetcites Lin [2004]
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This could mislead the XiaoIce users by setting an unrealistic expectation. As a result, the users might become addicted after chatting with XiaoIce for a very long time.”
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While manual evaluation is reliable, it is very expensive and chatbot developers often have to resort to automatic metrics for quantifying day-to-day progress and for performing automatic system optimization.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a score that is substantially better than that of other chatbots and even human conversations”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Commonly used automatic evaluation metrics for open-domain dialogue systems existing today all have their own limitations.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite the success of XiaoIce, the fundamental mechanism of human-level intelligence, as demonstrated in human conversations, is not yet fully understood.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“However, as argued by Gao et al. [2019], machine-learned metrics lead to potential problems such as overfitting and 'gaming of the metric' Albrecht and Hwa [2007].”
What the sources say
- Not checked yetcites Gao et al. [2019]
- Not checked yetcites Albrecht and Hwa [2007]
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“deltaBLEU Galley et al. [2015] is an extension to BLEU that exploits numerical ratings associated with conversational responses.”
What the sources say
- Not checked yetcites Galley et al. [2015]
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The databases used by the retrieval-based candidate generators and for training the neural response generator have been carefully cleaned, and a hand-crafted editorial response is used to avoid any improper or offensive responses.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Most open-domain dialogue systems, such as XiaoIce, generate responses using either retrieval-based methods or generation-based methods, or hybrid methods.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Unlike early chatbots designed for chitchat, XiaoIce is designed as a social chatbot intended to serve users' needs for communication, affection, and social belonging, and is endowed with empathy, personality and skills, integrating both EQ and IQ to optimize for long-term user engagement, measured in expected CPS.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As we have shown in Core Chat and Image Commenting, XiaoIce always preserves the right of not discussing or commenting on inappropriate topics and contents.”
Strength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The authors state this as an assumption.
Human-machine social chat can be modeled as a hierarchical decision-making process over Markov Decision Processes.
“we mathematically cast human-machine social chat as a hierarchical decision-making process, and optimize XiaoIce for long-term user engagement, measured in expected CPS.”
“cast human-machine social chat as decision-making over Markov Decision Processes (MDPs)”
The authors state this as an assumption.
Expected CPS is a valid measure of long-term user engagement for social chatbots.
“we propose to use expected Conversation-turns Per Session (CPS) as the success metric for social chatbots.”
“The larger the CPS is, the better engaged the social chatbot is.”
The paper relies on this without support.
An AI companion can satisfy the human need for communication, affection, and social belonging.
“XiaoIce is uniquely designed as an AI companion with an emotional connection to satisfy the human need for communication, affection, and social belonging.”
The paper relies on this without support.
The machine learning classifiers used for user understanding accurately detect user intents, emotions, topics, opinions, and persona from dialogue.
“These key-value pairs are generated using a set of machine learning classifiers as follows.”
The paper relies on this without support.
The TV series dataset used in the first pilot study is representative enough to validate the persona-based neural response generator for XiaoIce.
“In the first pilot study reported in Li et al. [2016b], we compare the persona model against two baseline models, using the TV series dataset for model training and evaluation.”
The paper relies on this without support.
The 3-level quality scale labels assigned by human judges reliably measure response quality.
“Each generated response is labeled on the 3-level quality scale by three human judges.”
The paper relies on this without support.
The KG triples that co-occur often in the same conversations represent related topics humans often discuss.
“We include in the XiaoIce KG a Satori triple ( h, r, t ) only if h and t co-occur often enough in the same conversations. Such a triple contains a pair of related topics ( h, t ) that humans often discuss in one conversation”
The paper relies on this without support.
The unpaired database sentences from public lectures and quotes are suitable as candidate responses for XiaoIce.
“The unpaired database we have used in XiaoIce consists of sentences collected from public lectures and quotes in news articles and reports. These sentences are considered candidate responses R .”
The paper relies on this without support.
Any individual dialogue skill is unlikely to have a significant impact on CPS.
“Since any individual skill is unlikely to have a significant impact on CPS, we measure the user satisfaction of a skill by monitoring its active users and skill triggering rate (i.e., the number of times a skill is activated by users within a day or a week).”
The paper relies on this without support.
Traditionally, social chatbots are designed for chitchat scenarios where bots are expected to mimic human user conversations but not to interact with the user's environment.
“Traditionally, social chatbots are designed for chitchat scenarios where the bots are expected to mimic human user conversations but not to interact with the user's environment.”
The paper relies on this without support.
Happiness and meaningful conversations often go hand in hand.
“Psychological studies show that happiness and meaningful conversations often go hand in hand.”
The paper relies on this without support.
The long-term success of a social chatbot needs to be measured by its user engagement.
“the long-term success of a social chatbot needs to be measured by its user engagement”
The paper relies on this without support.
The most reliable evaluation is to deploy the chatbot to users and monitor user feedback and engagement over a long period.
“There is no doubt that the most reliable evaluation is to deploy the chatbot to users and monitor the user feedback and engagement, measured by user ratings, NAU, CPS, and so on, over a long period of time.”
The paper relies on this without support.
In open-domain dialog systems, the same input may have many plausible responses that differ in topics or contents significantly.
“Nevertheless, in open-domain dialog systems, the same input may have many plausible responses that differ in topics or contents significantly.”
The paper relies on this without support.
The number of reference responses in the test set is always limited.
“as the number of reference responses is always limited in the test set”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Popular social chatbots have millions of users (Zhou et al., 2020), and their user bases are expanding.”
From A longitudinal study of human–chatbot relationships (2022) · cites this paper as Zhou et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Pilot-study dataset is reported as both 4K and 5K sessions
MinorThe description of the second pilot study says its evaluation data consists of 4K dialogue sessions, but the caption of the resulting Table 3 calls it a 5K dialogue dataset.
“Our evaluation data consists of 4K dialogue sessions.”
“Table 3: Ratings of three response generation systems on a 5K dialogue dataset.”
Image Commenting generator direction is reversed
MinorThe Image Commenting section describes a generator that produces text from images, but its evaluation calls the trained component a text-to-image generator.
“The generation-based approach uses an image-to-text generator”
“The components of Image Commenting, including the text-to-image generator and boosted tree ranker, are trained on a dataset consisting of 28 million images”
Citations
Cites
Maslow, A. H. (1943) · Psychological Review · doi:10.1037/h0054346 · Cited by 21,563
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Axford, N. (2017) · Routledge eBooks · doi:10.4324/9781315258324-16 · Cited by 11,329
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
(2004) · Cited by 9,718
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Flagged first, then by citations
Research works printed but not linked: 23
- Albrecht Joshua, 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, 880
- Asimov I., 1984, “The Bicentennial Man” in I. Asimov, The Bicentennial Man and Other Stories
- Banerjee Satanjeev, 2005, Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 65
- Brahnam Sheryl, 2005, Abuse: The Darker Side of Human Computer Interaction, 62
- Cuayáhuitl Heriberto, 2018, NeurIPS Workshop on Conversational AI
- Dinan Emily, 2018, CoRR
- Fang Hao, 2017, Alexa Prize Proceedings
- Fung Pascale, 2016, CoRR
- Gan Chuang, 2017, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3137
- Ghazvininejad Marjan, 2018, Proceedings of AAAI, 5111
- Lin Chin-Yew, 2004, Proceedings of the ACL workshop, 74
- Mathews Alexander Patrick, 2016, AAAI, 3574
- Misu Teruhisa, 2012, Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 84
- Mostafazadeh Nasrin, 2017, Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 462
- Papineni Kishore, 2002, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 311
- Sai Ananya, 2019, Proceedings of AAAI 2019, 33, 7281
- Serban Iulian Vlad, 2016, AAAI, 16, 3776
- Serban Iulian Vlad, 2017, AAAI, 3295
- Shum Heung-Yeung, 2018, CoRR
- Sutskever Ilya, 2014, NIPS, 3104
- Vinyals Oriol, 2015, ICML Deep Learning Workshop
- Wu Bowen, 2016, Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 652
- Xing Chen, 2017, AAAI, 17, 3351
Cited by
Sorted by citations
Items 501–600 of 815
Children in-conflict chatbot system using natural language processing technique
Romilla, P. M., Matuguinas, J., Santiago, A. J., Cortez, D. M. A., Centeno, C. J., Sison, A. A. R. C., & MERCADO, M. A. S. (2023) · World Journal of Advanced Research and Reviews · doi:10.30574/wjarr.2023.18.3.1107 · Cited by 3
Monolingual and Cross-Lingual Knowledge Transfer for Topic Classification
Karpov, D., & Burtsev, M. (2024) · Journal of Mathematical Sciences · doi:10.1007/s10958-024-07421-5 · Cited by 3
Shi, N., Zeng, Q. D., & Lee, R. (2020) · International Journal on Natural Language Computing · doi:10.5121/ijnlc.2020.9501 · Cited by 3
Shi, N., Zeng, Q., & Lee, R. S. T. (2021) · Advances in Science Technology and Engineering Systems Journal · doi:10.25046/aj060505 · Cited by 3
Affective feature knowledge interaction for empathetic conversation generation
Chen, E., Zhao, H., Li, B., Zha, X., Wang, H., & Wang, S. (2022) · Connection Science · doi:10.1080/09540091.2022.2134301 · Cited by 3
An Adversarially-Learned Turing Test for Dialog Generation ModelsPreprint
Gao, X., Zhang, Y., Galley, M., & Dolan, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08231 · Cited by 4
AI-Individualism: Understanding New Sociality in the Age of Social Artificial IntelligencePreprint
Brandtzæg, P. B., Skjuve, M. B., & Følstad, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4804274 · Cited by 3
Using Generative AI Chatbot for enhancing people's digital participation
Pinto, F., & Macadar, M. A. (2024) · International Conference on Theory and Practice of Electronic Governance · doi:10.1145/3680127.3680194 · Cited by 2
Lee, R. (2025) · Natural Language Processing · doi:10.1007/978-981-96-3208-4_9 · Cited by 2
Zhou, S. (2025) · International Journal for the Semiotics of Law - Revue internationale de Sémiotique juridique · doi:10.1007/s11196-025-10388-4 · Cited by 2
Emotional Intelligence: A Human-Centred AI Perspective
Gámez, M. E. (2026) · Springer eBooks · doi:10.1007/978-981-97-8440-0_109-1 · Cited by 2
Yi, J., Du, F., Nie, Y., Liang, W., Zhou, X., Chen, J., Li, G., Liu, M., Lv, Y., Zhao, W., et al. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104651 · Cited by 3
Wei, H., Xiang, B., Zheng, X., & Yang, J. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2630076 · Cited by 2
History of chatbots and their emerging applications in textiles, polymers, and sustainability
Al-Amin, M., Hossain, M. E., Ali, M. S., Salam, A., Ali, A., Ullah, A., Alam, N., Akhter, M. N., & Mahmood, S. A. (2026) · Sustainable materials and technologies · doi:10.1016/j.susmat.2026.e01967 · Cited by 3
Generative AI for Mental Well-Being and Insights From High and Low Resource Languages
Sitoula, S., Bhatt, L. P., Pokhrel, A., & Neupane, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7340-9.ch012 · Cited by 2
Conversational AI and Personal Growth: Insights from a Critical Integrative Review
Sharma, S., Vohra, P., & Vowels, L. M. (2026) · Behavioral Sciences · doi:10.3390/bs16050756 · Cited by 3
Li, Q., Geng, H., Hu, X., Pan, D., Liu, H., Li, Y., & Guo, J. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1810097 · Cited by 2
Revisiting Human Identity vis-à-vis AI Evolution
Chen, F., & Tsai, C. (2021) · Caietele Echinox · doi:10.24193/cechinox.2021.40.26 · Cited by 2
Пантелеева, Т. А. (2021) · Market economy problems · doi:10.33051/2500-2325-2021-1-131-148 · Cited by 2
Question Answering System Based on University Knowledge Graph
Leng, J., Yang, Y., Lin, R., & Tang, Y. Z. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-2385-4_12 · Cited by 2
MADNet: Maximizing Addressee Deduction Expectation for Multi-Party Conversation Generation
Gu, J.-C., Tan, C.-H., Chu, C., Ling, Z.-H., Tao, C., Liu, Q., & Liu, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.476 · Cited by 2
Social Features of Artificial Intelligence Companions for Children
Chang, S. J., Han, S.-H., & Park, K. C. (2023) · Journal of Digital Contents Society · doi:10.9728/dcs.2023.24.12.3199 · Cited by 2
A Framework for Understanding the Weaponisation of the Internet
McLoughlin, I. V. (2023) · IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI) · doi:10.1109/soli60636.2023.10425340 · Cited by 2
Enhancing Chatbot Intelligence Through Narrative Memory Structures
Azam, M., Zafar, L., Rafiq, T., Zafar, S., Rafiq, U., & Adnan, M. (2024) · The Asian Bulletin of Big Data Management · doi:10.62019/abbdm.v4i02.154 · Cited by 2
Building Empathic Chabot using LSTM and NLP
Jeyalakshmi, J., Srinidhi, S. K., & Balaprathap, C. (2024) · International Conference on Innovative Practices in Technology and Management (ICIPTM) · doi:10.1109/iciptm59628.2024.10563763 · Cited by 2
TriKF: Triple-Perspective Knowledge Fusion Network for Empathetic Question Generation
Chen, T., Shen, Y., Chen, X., Zhang, L., & Zhao, S. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3418820 · Cited by 2
Why Conversational AI won’t replace healthcare providers
Gangadharaiah, R., Shivade, C., Bhatia, P., Zhang, Y., & Kass‐Hout, T. (2020) · Cited by 2
Semantic Data Discovery from Social Big Data
Abu-Salih, B., Wongthongtham, P., Zhu, D., Chan, K. Y., & Rudra, A. (2021) · Springer eBooks · doi:10.1007/978-981-33-6652-7_4 · Cited by 2
What Did You Refer to? Evaluating Co-References in Dialogue
Zhang, W., Zhang, Y., Tang, H., Zhao, Z., Zhu, C., & Liu, T. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.450 · Cited by 2
Combining Visual and Social Dialogue for Human-Robot Interaction
Gunson, N., García, D. H., Part, J. L., Yu, Y., Sieińska, W., Dondrup, C., & Lemon, O. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3481303 · Cited by 2
Xu, F., Zhou, S., Wang, X., & Zhang, W. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.06.105 · Cited by 3
Using Extracted Emotion Cause to Improve Content-Relevance for Empathetic Conversation Generation
Zou, M., Pan, R., Zhang, S., & Zhang, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18315-7_8 · Cited by 2
People and machines in communication / Personas y máquinas en comunicación
Jacomuzzi, A. C., & Alioto, B. P. (2024) · Studies in Psychology Estudios de Psicología · doi:10.1177/02109395241241380 · Cited by 2
Intelligent Conversational Chatbots: History, Taxonomy, Classification with PRISMA Model
Gnanaprakasam, J., & Lourdusamy, R. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-71481-8_3 · Cited by 2
Unveiling Trends of Chatbot and Conversational Agents: A Bibliometric Study
Alasali, T., Dakkak, O., & Türker, İ. (2024) · Applied Computer Systems · doi:10.2478/acss-2024-0019 · Cited by 4
Emotional State Estimation by Dialogue History and Sentence Distributed Representation
Matsumoto, K., Sasayama, M., Yoshida, M., & Kita, K. (2019) · IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS) · doi:10.1109/ccis48116.2019.9073750 · Cited by 2
Han, X., & YEH, T. P. (2020) · Cited by 2
When Creative AI Meets Conversational AI
Wu, X. (2021) · Journal of Natural Language Processing · doi:10.5715/jnlp.28.881 · Cited by 2
Experiences on Creating Personal Study Plans with Chatbots
Koivisto, M. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-21569-8_18 · Cited by 2
Attacks on ML Systems: From Security Analysis to Attack Mitigation
Zou, Q., Zhang, L., Singhal, A., Sun, X., & Liu, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-23690-7_7 · Cited by 2
Online Psychological Counseling Chatbot for Seniors
Kim, B., Kim, E., & Rhee, J. (2022) · International Conference on Advances in Artificial Intelligence (ICAAI) · doi:10.1145/3571560.3571583 · Cited by 3
Zhang, H., Wang, X., Qin, B., & Wang, H. (2022) · IEEE International Conference on e-Business Engineering (ICEBE) · doi:10.1109/icebe55470.2022.00037 · Cited by 2
Foundation Models for Text Generation
Paaß, G., & Giesselbach, S. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-3-031-23190-2_6 · Cited by 3
Kath, H., Lüers, B., Gouvêa, T. S., & Sonntag, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42608-7_9 · Cited by 3
Conversational Bibliographic Search
Nilles, M. (2024) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3627508.3638337 · Cited by 2
A Multi-talent Healthcare AI Bot Platform
Horn, M., Li, X., Lin, C., & Kafle, S. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-53352-6_15 · Cited by 2
Investigating person-specific errors in chat-oriented dialogue systems
Mitsuda, K., Higashinaka, R., Li, T., & Yoshida, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.50 · Cited by 2
Proposal Towards a Personalized Knowledge-powered Self-play Based Ensemble Dialog SystemPreprint
Csáky, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05016 · Cited by 2
Cue-word Driven Neural Response Generation with a Shrinking VocabularyPreprint
Wang, Q., Liu, Y., Lv, C., Wang, Z., & Fu, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04927 · Cited by 2
Data-Efficient Methods for Dialogue SystemsPreprint
Shalyminov, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02929 · Cited by 3
Graph-Based Transfer Learning for Conversational Agents
Rajan, M. H., Rebello, K. J., Sood, Y., & Wankhade, S. B. (2021) · International Conference on Communication and Electronics Systems (ICCES) · doi:10.1109/icces51350.2021.9489179 · Cited by 1
Zenimoto, Y., Komata, S., & Utsuro, T. (2023) · doi:10.18653/v1/2023.ijcnlp-srw.5 · Cited by 1
Enhancing Patient Intake Process in Mental Health Consultations Using RAG-Driven Chatbot
Shayaninasab, M., Zahoor, M., & Yalçın, Ö. N. (2024) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw63320.2024.00053 · Cited by 2
Vrins, A. (2025) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3713043.3731603 · Cited by 1
Ezzat, A. H., Osama, A., Elsherif, F. A., Eldin, M. H., Hesham, K., & Mansour, A. (2025) · International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC) · doi:10.1109/miucc66482.2025.11196837 · Cited by 1
Demystifying the mist: Why do individuals hesitate to accept AI educational services?
Shao, A., Lü, Z., Liu, S. Q., Shi, Y., & Lu, W. (2025) · British Journal of Psychology · doi:10.1111/bjop.70040 · Cited by 1
Leveraging Label Potential for Enhanced Multimodal Emotion Recognition
Shao, X., Yu, Y., & Wang, L. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228349 · Cited by 1
Towards Intelligent Tax Systems: Evaluating NLP Approaches for Pakistan’s Fiscal Needs
Arsal, M., Ashfaq, H., Mahmood, M. A., & Ashfaq, A. (2025) · International Conference on Frontiers of Information Technology (FIT) · doi:10.1109/fit67061.2025.11333725 · Cited by 1
Sun, Y., Liu, J., Liu, T., Wei, Z., & Shen, H. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1841223 · Cited by 1
Understanding AI-Generated Personal Narratives as Design Material for Socially Engaging Things
Cho, H., Yun, G., Lee, J., & Nam, T.-J. (2022) · [ ] With Design: Reinventing Design Modes · doi:10.1007/978-981-19-4472-7_75 · Cited by 1
Gao, J., Xiong, C., Bennett, P. N., & Craswell, N. (2023) · The information retrieval series · doi:10.1007/978-3-031-23080-6_1 · Cited by 1
Self-citationResearch on Intelligent Dialogue Systems
Hu, S. (2023) · Theoretical and Natural Science · doi:10.54254/2753-8818/2/20220175 · Cited by 1
Predicting the Subjective Responses’ Emotion in Dialogues with Multi-Task Learning
Hayat, H., Ventura, C., & Lapedriza, À. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36616-1_55 · Cited by 1
Multi-modal sentiment analysis based on expression and text
Kao, W., Yang, H., Liu, C., & Bai, J. (2023) · International Conference on Advanced Algorithms and Neural Networks (AANN) · doi:10.1117/12.3004687 · Cited by 1
Liu, K., Song, M., Xu, W., Li, K. E., Peng, M., & Tian, G. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0573-6_3 · Cited by 1
Siddiquee, M., & -, M. J. (2025) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2025.v07i02.40428 · Cited by 1
Sanjeev, A., & Sharma, R. (2025) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83549-852-120251006 · Cited by 1
Technological Evolution and Research Trends of Intelligent Question-Answering Systems in Healthcare
Lei, B., & Yin, P. (2025) · Healthcare · doi:10.3390/healthcare13182269 · Cited by 1
Châm, N. T. P. (2025) · Architecture Image Studies · doi:10.62754/ais.v6i3.289 · Cited by 1
Is attributional style a marker of personification? A case study with ChatGPT
Norcia, M., & Colí, E. (2025) · Journal of Psychology and AI · doi:10.1080/29974100.2025.2569932 · Cited by 1
International Al Safety Report: First Key Update Capabilities and Risk Implications
Bengio, Y., Bucknall, B., Clare, S., Prunkl, C., Andriushchenko, M., Fox, P., Hu, T., Jones, C., Manning, S., Maslej, N., et al. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i6.16253 · Cited by 1
AI-Based Chatbots for Mental Health: Comparative Study
Ech-Cheikh, S., Kharmoum, N., & Ziti, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-01970-7_6 · Cited by 1
Jain, R., & Rani, S. (2026) · Shodh Manjusha An International Multidisciplinary Journal · doi:10.70388/sm250192 · Cited by 1
Hwang, L. J., Li, H., & Feng, B. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2633768 · Cited by 1
Will customers buy financial products impulsively? AI-powered chatbots’ recommendations
Cheng, Y.-M. (2026) · Journal of Asia Business Studies · doi:10.1108/jabs-06-2025-0340 · Cited by 1
From Jugaad to Generative AI: A Framework for Culturally Responsive Human-Centered AI
Rathore, N. S., & Thakkar, S. (2025) · International Association of Societies of Design Research Congress (IASDR) · doi:10.21606/iasdr.2025.537 · Cited by 1
Wu, Y. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6623-4.ch007 · Cited by 1
Dawood, H., Liew, C. Y., & Rajan, M. E. S. (2026) · International Journal of Intelligent Computing and Cybernetics · doi:10.1108/ijicc-11-2025-0771 · Cited by 1
Digital Darwinism: steering the evolution of artificial life in socio-technical systems
Ulrich, K. T. (2026) · AI and Ethics · doi:10.1007/s43681-026-01057-8 · Cited by 1
Harnessing Artificial Intelligence to Facilitate Difficult Conversations
Maheshka, C., Folk, D. P., & Dunn, E. (2026) · Social Psychological and Personality Science · doi:10.1177/19485506261444068 · Cited by 1
Bridging the silos in affective AI: a critical perspective from data to society
Inoshita, K. (2026) · AI & Society · doi:10.1007/s00146-026-03324-y · Cited by 1
Liu, P., Zhu, P., Yuan, Y. E., Zhao, F., & Liu, B. (2026) · Current Psychology · doi:10.1007/s12144-026-09403-z · Cited by 1
Empathy through the Lens of Conversational Agents: A Systematic Review
Khadka, S., & Desai, S. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816220 · Cited by 1
Li, H., Xing, W., Li, C., & Lyu, B. (2026) · Computers & Education · doi:10.1016/j.compedu.2026.105725 · Cited by 1
Platform-Facilitated Grooming and AI Chatbots: Rethinking Criminal Liability and Regulation
Chawki, M. (2026) · Laws · doi:10.3390/laws15040093 · Cited by 1
Ribas, L. R., Gomes-Jr, L., & Silva, T. H. (2020) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia_estendido.2020.13067 · Cited by 1
How to Evaluate a Good Conversation? An Evaluation Framework for Chat Experience in Smart Home
Chen, X., Ma, L., Jia, M., Han, Y., Mi, J., & Xu, M. Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-78462-1_27 · Cited by 1
The practice of speech and language processing in China
Jia, J., Chen, W., Yu, K., He, X., Du, J., & Shum, H. (2021) · Communications of the ACM · doi:10.1145/3481625 · Cited by 1
Self-citationDialogue Response Generation for Text-Based Dialogue Systems with Emotion Regulation
Katayama, S., Yonezawa, T., & Kawaguchi, N. (2022) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.34.3_568 · Cited by 1
An Automatic Evaluation Method for Open-domain Dialogue Based on BLEURT
Wu, S.-H., & Lee, J.-J. (2022) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri54793.2022.00030 · Cited by 1
The Gendered Nature of Chatbots
Ashcroft, A., & Ashcroft, A. (2023) · Advances in web technologies and engineering book series · doi:10.4018/978-1-6684-6234-8.ch003 · Cited by 1
Proactivity in Conversational Assistants: The mPLiCA Model Based on a Systematic Literature Review
Johnson, E., Villa, L., Mondéjar, T., & Hervás, R. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-48306-6_28 · Cited by 1
Liu, Z., Chen, Z., Ren, Z., Gao, S., Ma, J., & Ren, P. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103578 · Cited by 1
Chatbot using Artificial Intelligence Markup Language
(2024) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets48132 · Cited by 1
Empathy-Driven Chatbots for the Arabic Language: A Transformer Based Approach
Rabii, I., Boussakssou, M., & Erritali, M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-53082-1_5 · Cited by 1
Toward an end-to-end implicit addressee modeling for dialogue disentanglement
Gao, J., Li, Z., Xiang, S., Wang, Z., Liu, T., & Fu, Y. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18428-0 · Cited by 1
Self-citationLearning WHO Saying WHAT to WHOM in Multi-Party Conversations
Gu, J.-C., Zhang, Z., & Ling, Z.-H. (2023) · doi:10.18653/v1/2023.ijcnlp-tutorials.5 · Cited by 2
Hao, J., Kong, F., & Kang, J. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5672-8_22 · Cited by 1
Constructing an English Language Chatbot Using Reinforcement Learning Algorithms
Xu, B. (2024) · International Conference on Artificial Intelligence and Autonomous Robot Systems (AIARS) · doi:10.1109/aiars63200.2024.00070 · Cited by 1
Rahman, A. T. A. S., & Kaur, A. (2024) · IEEE International Conference on Advances in Computing Research on Science Engineering and Technology (ACROSET) · doi:10.1109/acroset62108.2024.10743336 · Cited by 1
Metadata
- Authors
- Zhou, L., Gao, J., Li, D., & Shum, H.-Y.
- Affiliations
- Microsoft (United States) (US)
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-03
- Volume
- 46
- Issue
- 1
- Pages
- 53-93
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W2906579211
- Semantic Scholar
- 66b7d31527f980bb2eecc23629f08ba6037facc4
- Cited by
- 777
- References
- 66
- Citations by year
- 2026: 97 · 2025: 109 · 2024: 106 · 2023: 113 · 2022: 120 · 2021: 112 · 2020: 94
- FWCI
- 43.11
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- Microsoft Research
- Microsoft Research Asia
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- social chatbots, Markov decision process, dialogue management, user engagement, emotion recognition, user intent understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review