The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
The Design and Implementation of XiaoIce, an Empathetic Social Chatbot
Open accessAbstract
This article describes the development of Microsoft XiaoIce, the most popular social chatbot in the world. XiaoIce is uniquely designed as an artifical intelligence companion with an emotional connection to satisfy the human need for communication, affection, and social belonging. We take into account both intelligent quotient and emotional quotient in system design, cast human–machine social chat as decision-making over Markov Decision Processes, and optimize XiaoIce for long-term user engagement, measured in expected Conversation-turns Per Session (CPS). We detail the system architecture and key components, including dialogue manager, core chat, skills, and an empathetic computing module. We show how XiaoIce dynamically recognizes human feelings and states, understands user intent, and responds to user needs throughout long conversations. Since the release in 2014, XiaoIce has communicated with over 660 million active users and succeeded in establishing long-term relationships with many of them. Analysis of large-scale online logs shows that XiaoIce has achieved an average CPS of 23, which is significantly higher than that of other chatbots and even human conversations.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This paper describes Microsoft XiaoIce, an empathetic social chatbot designed as an AI companion with IQ, EQ, and personality, optimized for long-term user engagement measured by expected Conversation-turns Per Session (CPS). It reports system architecture, dialogue manager, empathetic computing, Core Chat, skills, and large-scale deployment results, including 660 million active users and an average CPS of 23.
Key findings
- XiaoIce was launched in May 2014 and by July 2018 had over 660 million active users, was deployed in five countries on more than 40 platforms, and achieved an average CPS of 23 in its 6th generation; the paper compares this with CPS 9 for human conversations in its user study and CPS 14.6 for the latest Amazon Alexa systems per Khatri et al. [2018].
- The empathetic computing module’s A/B test on Weibo showed no significant CPS change but increased NAU from 0.5 million to 5.1 million in 3 months; it was released in July 2018 and became the most important feature in the 6th generation.
- Core Chat’s hybrid approach improved expected CPS by 0.5 in two weeks over a retrieval-only baseline; incorporating the neural generator and unpaired-data retrieval improved response coverage by 20% and 10%, respectively. Topic Manager improved Core Chat expected CPS by 0.5, and Image Commenting doubled expected CPS across all dialogues containing images.
- The persona-based neural response generator achieved perplexity 25.0 (−8.4% vs. seq2seq) and BLEU 1.9% (+18.8% vs. seq2seq, +11.8% vs. LSTM-MMI) on the TV series dataset; the XiaoIce Image Commenting prototype scored BLEU-4 4.53, ROUGE-L 15.33, CIDEr-D 3.21, METEOR 15.51, and SPICE 4.82, outperforming four state-of-the-art image captioning systems.
Methods
- Design: XiaoIce is a modular system with user experience, conversation engine, and data layers; social chat is cast as hierarchical decision-making over options in Markov Decision Processes, with a dialogue manager, empathetic computing module, Core Chat, and 230 dialogue skills, optimized for expected CPS.
- Data and training: paired conversational data included more than 30 billion conversation pairs as of May 2018, with 70% of XiaoIce’s responses retrieved from her own past conversations; classifiers for user understanding were trained on 10K manually labeled dialogue sessions, the response ranker on 50K manually labeled dialogues, neural generation and ranking features on 50 million human dialogues, and Image Commenting on 28 million images each paired with 6 text comments.
- Evaluation: A/B tests used expected CPS and NAU; pilot studies used human ratings on a 3-level quality scale and benchmark metrics such as perplexity, BLEU, METEOR, CIDEr, ROUGE-L, and SPICE; large-scale online logs were analyzed over long periods.
Limitations
- The paper states that automatic evaluation of open-domain dialogue systems is “by no means a solved problem,” that common automatic metrics have limitations, and that a unified modeling framework for XiaoIce “remains to be proved” effective.
- The paper states that fully grounding all conversations in the physical world for goal-oriented interactions remains a non-trivial challenge, and that the fundamental mechanism of human-level intelligence is not yet fully understood.
- The paper discusses ethical concerns including privacy, who is in control, unrealistic user expectations, and potential misuse; evident from the text, many key results come from proprietary A/B tests and online logs without reported confidence intervals or full statistical details, and the comparison to human conversations relies on an unspecified user study.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
45 No basis given · 41 Own interpretation · 34 Own result · 22 Not checked yet · 20 Own method
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although the overall quality of the candidates generated from the unpaired database is lower than those retrieved from the paired database, with unpaired database XiaoIce can cover a much broader range of topics.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although the overall quality of the candidates generated from the unpaired database is lower than those retrieved from the paired database, with unpaired database XiaoIce can cover a much broader range of topics.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The unpaired database sentences from public lectures and quotes are suitable as candidate responses for XiaoIce.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The unpaired database we have used in XiaoIce consists of sentences collected from public lectures and quotes in news articles and reports.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The module was released in July 2018, and became the most important feature in the 6th generation of XiaoIce that has substantially strengthened XiaoIce's emotional connections to human users and increased XiaoIce's NAU.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although we do not observe any significant change in CPS, NAU is increased from 0.5 million to 5.1 million in 3 months.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although we do not observe any significant change in CPS, NAU is increased from 0.5 million to 5.1 million in 3 months.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our finding is that the majority of the 'desired' users are young, female users.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nowadays, 70% of XiaoIce's responses are retrieved from her own past conversations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although the response candidates retrieved from the paired database is of high quality, the coverage is low because many new or less frequently discussed topics on the Internet forums are not included in the database.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After the launch of XiaoIce in May 2014, we also started collecting human-machine conversations generated by XiaoIce and her users, which amounted to more than 30 billion conversation pairs as of May 2018.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our A/B test over a 1-month period shows that incorporating topic manager improves the expected CPS of Core Chat by 0.5.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Since the release in 2014, XiaoIce has communicated with over 660 million active users and succeeded in establishing long-term relationships with many of them.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of large-scale online logs shows that XiaoIce has achieved an average CPS of 23, which is significantly higher than that of other chatbots and even human conversations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Figure 1 shows how an emotional connection between a user and XiaoIce has been established over a 2-month period.”
- Also stated in: § Session 71 · p. 2 · plain
“showing how an emotional connection between the user and XiaoIce has been established over a 2-month period.”
plaincorrelationalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Session 71 · p. 2 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in the examples in Table 2, the persona model is sensitive to the identity of the user (addressee), generating specific words (e.g., the user names) in responses targeted at different users.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“In July 2018, XiaoIce has evolved to the 6th generation with an impressive average CPS of 23, which is significantly higher than the CPS of 9 for human conversations based on our user study and the CPS of 14.6 for the latest Amazon Alexa systems according to Khatri et al. [2018].”
What the sources say
- Not checked yetcites Khatri et al. [2018]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Comparing to the baseline which uses only the retrieval-based generator using paired database for candidate generation, incorporating the neural response generator and the retrieval-based generator using unpaired database at the candidate generation stage improves the expected CPS of Core Chat by 0.5 in two weeks.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results in Table 3 show that incorporating the neural generator, as in the hybrid system, significantly improves the human rating over the retrieval-based system.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find that that incorporating the neural-based generator and the retrieval-based generator using unpaired database improve the coverage over the baseline by 20% and 10%, respectively.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce is designed as a modular system based on a hybrid AI engine that combines rule-based and data-driven approaches, as presented in Figure 4 and Section 4.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The 1st generation achieved an average CPS of 5, which already outperforms other dialogue systems such as digital personal assistants whose CPS ranges from 1 to 3.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Table 4 presents the result of a pilot study, Huang et al. [2019b] showing that the XiaoIce Image Commenting skill outperforms several state-of-the-art image captioning systems on a test set consisting of 5K image-comment pairs whose ratings are 2, in terms of BLEU-4 Papineni et al. [2002], METEOR Banerjee and Lavie [2005], CIDEr Vedantam et al. [2015], ROUGE-L Lin [2004], and SPICE Anderson et al”
What the sources say
- Not checked yetcites Huang et al. [2019b]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the A/B test we observe that Image Commenting doubles the expect CPS across all dialogues that contain images.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since its launch, it has been triggered over 50 million dialogue sessions where an extremely negative user sentiment is detected (by XiaoIce's empathetic computing module).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In July 2018, XiaoIce has evolved to the 6th generation with an impressive average CPS of 23, which is significantly higher than the CPS of 9 for human conversations based on our user study and the CPS of 14.6 for the latest Amazon Alexa systems according to Khatri et al. [2018].”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find that that incorporating the neural-based generator and the retrieval-based generator using unpaired database improve the coverage over the baseline by 20% and 10%, respectively.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The 1st generation achieved an average CPS of 5, which already outperforms other dialogue systems such as digital personal assistants whose CPS ranges from 1 to 3.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In July 2018, XiaoIce has evolved to the 6th generation with an impressive average CPS of 23, which is significantly higher than the CPS of 9 for human conversations based on our user study and the CPS of 14.6 for the latest Amazon Alexa systems according to Khatri et al. [2018].”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The qualitative analysis confirms that the persona model indeed generates more interpersonal responses than the baselines.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The user feedback log shows that the products recommended by Rinna are very well received”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of large-scale online logs collected since the launch of XiaoIce in May 2014 shows that XiaoIce is capable of interpreting users' emotional needs and engaging in interpersonal communications in a manner analogous with a reliable, sympathetic and affectionate friend.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and as a result Rinna has delivered a much higher conversion rate than that achieved using other traditional channels such as coupon markets or ad campaigns.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce can gain access to users' emotional lives - to information that is highly personal, intimate and private, such as the user's opinion on (sensitive) topics, her friends and colleagues.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a result, XiaoIce has succeeded in establishing long-term relationships with millions of users worldwide”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For example, we have incorporated the Coupon skill in the Rinna system (Japanese XiaoIce) which can send a user the coupons of a grocery store if user needs are detected during the conversation.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“achieving an average CPS of 23”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“As shown in Figure 7, while a typical seq2seq model which is not grounded in any persona often outputs inconsistent responses Li et al. [2016b], XiaoIce is able to generate consistent and humorous responses.”
What the sources say
- Not checked yetcites Li et al. [2016b]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To maximize long-term user engagement, measured in expected CPS, we take an iterative, trial-and-error approach to developing XiaoIce, and always try to balance the exploration-exploitation tradeoff.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: Human-machine social chat can be modeled as a hierarchical decision-making process over Markov Decision Processes.
- The authors state this as an assumption: Expected CPS is a valid measure of long-term user engagement for social chatbots.
In the paper
“More importantly, social chatbots also require a sufficient EQ to meet users' emotional needs, such as emotional affection and social belonging, which are among the fundamental needs for human beings Maslow [1943].”
What the sources say
- Not checked yetcites Maslow [1943]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The formulation of dialogue as a hierarchical decision-making process guides the design and implementation of XiaoIce.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: Human-machine social chat can be modeled as a hierarchical decision-making process over Markov Decision Processes.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 3, XiaoIce tries to maintain user interest by promoting diversity of conversation modes.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The module was released in July 2018, and became the most important feature in the 6th generation of XiaoIce that has substantially strengthened XiaoIce's emotional connections to human users and increased XiaoIce's NAU.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But these skills establish XiaoIce as an efficient personal assistant and more importantly trustworthy personal companion, thus strengthening the emotional bond with human users in the long run.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is worth noting that we optimize XiaoIce for expected CPS which corresponds to long-term, rather than short-term, engagement.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: Expected CPS is a valid measure of long-term user engagement for social chatbots.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, incorporating many task-completion skills often reduces the CPS in the short term since these skills help users accomplish tasks more efficiently by minimizing the CPS.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“XiaoIce is developed on an empathetic computing framework Cai [2006], Fung et al. [2016] that enables the machine (social chatbot in our case) to recognize human feelings and states, understand user intents and respond to user needs dynamically.”
What the sources say
- Not checked yetcites Cai [2006]
- Not checked yetcites Fung et al. [2016]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Unlike task-oriented bots where their performance is measured by task success rate, measuring the performance of social chatbots is difficult Shawar and Atwell [2007].”
What the sources say
- Not checked yetcites Shawar and Atwell [2007]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The evaluation methodology eliminates many possibilities of gaming the metric.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: Expected CPS is a valid measure of long-term user engagement for social chatbots.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Being able to establish such long-term relationships with human users as an open-domain social chatbot distinguishes XiaoIce from not only early social chatbots but also other recently developed conversational AI personal assistants such as Apple Siri, Amazon Alexa, Google Assistant and Microsoft Cortana.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Both the topic switching classifier and the topic ranker are trained using 50K dialogue sessions whose topics are manually labeled.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although the response candidates retrieved from the paired database is of high quality, the coverage is low because many new or less frequently discussed topics on the Internet forums are not included in the database.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But this hurts the CPS and NAU in the long run since few users are willing to talk (again) to a bot who always gives bland responses no matter how interactive these responses are, not to mention establishing long-term, emotional connections.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike the retrieval-based generator, the neural response generator is trained using the paired database to learn to simulate human conversations, and is able to generate responses for any topics including those that are unseen in human conversational data, so that a user can chat about any topic she likes.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The effectiveness of the empathetic computing module is verified in the A/B test on Weibo users.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The machine learning classifiers used for user understanding accurately detect user intents, emotions, topics, opinions, and persona from dialogue.
In the paper
“Such hierarchical decision-making processes can be cast in the mathematical framework of options over Markov Decision Processes (MDPs) [Sutton et al., 1999], where options generalize primitive actions to higher-level actions.”
What the sources say
- Not checked yetcites Sutton et al., 1999
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The module was released in July 2018, and became the most important feature in the 6th generation of XiaoIce that has substantially strengthened XiaoIce's emotional connections to human users and increased XiaoIce's NAU.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“train a group of classifiers for user understanding (i.e., topic detection, intent detection, opinion detection and sentiment analysis) using 10K manually labeled dialogue sessions.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The machine learning classifiers used for user understanding accurately detect user intents, emotions, topics, opinions, and persona from dialogue.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A detailed analysis shows that the gain is mainly attributed to the fact that the neural response generator and the retrieval-based generator using unpaired database significantly improve the coverage of responses.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since the coherence features use global context information C , they are particularly useful when Q c is a bland query whose topic is hard to detect without context, such as 'OK', 'why', 'I don't know'.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Evaluating the quality of open-domain social chatbots is challenging because social chats are inherently open-ended Ram et al. [2018], Gao et al. [2019], Huang et al. [2019a]”
What the sources say
- Not checked yetcites Ram et al. [2018]
- Not checked yetcites Gao et al. [2019]
- Not checked yetcites Huang et al. [2019a]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, XiaoIce uses a modular architecture similar to task-oriented dialogue systems, with different modules dealing with different tasks.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Traditionally, social chatbots are designed for chitchat scenarios where bots are expected to mimic human user conversations but not to interact with the user's environment.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This attributes to a large agree to those task-completion skills that enable XiaoIce to control approximately 80 IoT smart devices in around 300 scenarios.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our evaluation data consists of 4K dialogue sessions.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The improvement on the empathetic computing module, especially the integration of the specific empathy models in the 6th generation, substantially strengthens XiaoIce's emotional connections to human users.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The neural response generator and the set of classifiers and models that are used to generate the ranking features for the candidate ranker (e.g., local cohesion and global coherence features) are trained using 50 million human dialogues.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The dataset consists of scripts of 69,565 dialogue turns of 13 main characters from the American TV comedies Friends 6 and The Big Bang Theory 7 , available from IMSDB 8 .”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our analysis of the user log shows that we are achieving the goal.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our A/B test confirms the conclusions we draw from the pilot studies.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We have verified carefully with these users that that these long conversations are generated by XiaoIce and human users, not another bot.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This enables new scenarios that are of significant commercial value.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All prior work suggests that automatic evaluation of open-domain dialog systems is by no means a solved problem.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“For example, Sai et al. [2019] showed that ADEM can be easily fooled with a variation as simple as reversing the word order in the text.”
What the sources say
- Not checked yetcites Sai et al. [2019]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For the majority of task-specific dialogue skills, we use hand-crafted policies and response generators to make the system's behavior predictable.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 66
- It may state two separate claims as one. 43
- The quote only partly states this claim. 2
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Over the last 5 years XiaoIce has developed more than 230 skills, ranging from answering questions and recommending movies or restaurants to comforting and storytelling.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The module was released in July 2018, and became the most important feature in the 6th generation of XiaoIce that has substantially strengthened XiaoIce's emotional connections to human users and increased XiaoIce's NAU.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We show how XiaoIce dynamically recognizes human feelings and states, understands user intent, and responds to user needs throughout long conversations.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite being extremely knowledgeable due to her access to large amounts of data and knowledge, XiaoIce never comes across as egotistical and only demonstrates her wit and creativity when appropriate.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As shown in Figure 2, XiaoIce demonstrates sufficient EQ as it generates socially acceptable responses (e.g., having a sense of humor, comforting, etc.), and can determine whether to drive the conversation to a new topic when e.g., the conversation has stalled, or whether or not to be actively listening when the user herself is engaged in the conversation.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most important and sophisticated skill is Core Chat, which can engage long and open-domain conversations with users.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Analysis of large-scale online logs shows that XiaoIce has achieved an average CPS of 23, which is significantly higher than that of other chatbots and even human conversations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This is possible because XiaoIce has attracted a large number of active users since her launch in 2014.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 7, while a typical seq2seq model which is not grounded in any persona often outputs inconsistent responses Li et al. [2016b], XiaoIce is able to generate consistent and humorous responses.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in the example in Figure 3, XiaoIce switches to a new but related topic (i.e., a song titled 'the time machine' by Ashin in Turn 13) when she detects that the user is not familiar with 'Ashin' and about to terminate the conversation by responding 'Ah! Boring' and 'Okay, I am going to bed'.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Since the release in 2014, XiaoIce has communicated with over 660 million active users and succeeded in establishing long-term relationships with many of them.”
- Also stated in: § 1 Introduction · p. 1 · plain
“Since her launch in China in May 2014, XiaoIce has attracted over 660 million active users (i.e., subscribed users).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Also stated in: § 1 Introduction · p. 1 · plain
In the paper
“As shown in Table 1, the persona model significantly outperforms both baselines, achieving a lower perplexity (-8.4%) and a higher BLEU score (+18.8% and +11.8%) Papineni et al. [2002].”
What the sources say
- Not checked yetcites Papineni et al. [2002]
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce has been deployed on many platforms. As a result, we have witnessed the creation and growth of a XiaoIce ecosystem since 2016.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce-generated TV and Radio programs have covered 9 top satellite TV stations, and have attracted over 800 million weekly active audience.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 5 (a), CQU rewrites user queries to include necessary context, e.g., replacing 'him' in Turn 12 with 'Ashin', 'that' with 'The Time Machine' in Turn 14, and adding 'send The Time Machine' in Turn 15.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Every poem in the album is jointly written by XiaoIce and human poets.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce, however, generates a factual response based on the user's geographical location and the corresponding database, as shown in Figure 19 (a).”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“On May 15, 2018, XiaoIce published the first AI-created Chinese poem album in history 11 .”
What the sources say
- Not checked yetcites [11] Cuayáhuitl Heriberto, 2018
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce is equipped with 230 dialogue skills, which, together with Core Chat and Image Commenting, form the IQ component of XiaoIce.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since July 2014, XiaoIce has released 230 skills, which amounts to nearly one new skill every week, as shown in Figure 21.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Depending on the availability of training data and knowledge bases for each individual task, either a rule-based method or a data-driven method, or a hybrid of both is adopted for the task.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce, on the other hand, is designed as an AI companion which integrates both EQ and IQ skills that are needed to help users complete specific tasks.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Up to August 2015, XiaoIce has had more than 10 billion conversations with humans.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that the pairs that are shared among acquaintances (e.g., coworkers, classmates and friends) are of good quality, and amount to a large portion in the database.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These characters include more than 60,000 official accounts, e.g., Lawson and Tokopedia's customer service bots, Pokemon, Tecent and Netease's chatbots, and even real human celebrities such as the singers of Guoyun Entertainment.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“From 2015 on, XiaoIce started powering third party characters, personal assistants and real human's virtual avatars.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In two months, XiaoIce successfully became a cross-platform social chatbot.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As of July 2018, XiaoIce has been deployed on more than 40 platforms, and has attracted 660 million active users.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The longest conversation lasts for more than 6 hours, covering 53 different topics across 8 domains and using 16 task-completion skills.”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The XiaoIce Poetry Generation skill has helped over four million users to generate poems.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“XiaoIce was first launched on May 29, 2014, and went viral immediately.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“By then, users have proactively posted more than 6 million conversation sessions to public.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Within 72 hours, XiaoIce was looped into 1.5 million chat groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The CPS is close to the 3rd generation of XiaoIce, as shown in Figure 20.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Compared to the neural response generator which often generates well-form but short responses, the candidates from unpaired database are much longer with more useful content.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Through a long conversation, XiaoIce has demonstrated human-like empathy and social skills, and eventually helped the user regain her confidence and move forward with a positive attitude.”
boosteddirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“| XiaoIce (prototype) | 4.53 | 15.33 | 3.21 | 15.51 | 4.82 |”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For example, although XiaoIce can provide answers to many questions due to the access to the large scale knowledge graph, these answers are not always accurate.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“XiaoIce cheers users up, encourages them, helps them accomplish tasks, and holds their attention throughout the conversation.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As an AI companion of human users, XiaoIce can recognize user interests and intents much more accurately than traditional intelligent personal assistants.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While XiaoIce carefully leverages this information to serve users and build emotional bonds over a long period of time”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“When working on the modules or tasks where there are benchmarks used in the research community, such as the neural response generator (Section 4.3), we often compare our approaches to other state-of-the-art methods using these benchmarks.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Neural-model-based and retrieval-based generators are complementary: the neural-model-based generator offers robustness and high coverage, while the retrieval-based provides high-quality responses for popular topics.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The dialogue policy is designed to optimize long-term user engagement through an iterative, trialand-error process based on the feedback of XiaoIce's users.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- The authors state this as an assumption: Human-machine social chat can be modeled as a hierarchical decision-making process over Markov Decision Processes.
- The authors state this as an assumption: Expected CPS is a valid measure of long-term user engagement for social chatbots.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A social chatbot needs to present a consistent personality to set the right expectations for users in the conversation and gain their long-term confidence and trust.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This paper describes the development of Microsoft XiaoIce , the most popular social chatbot in the world.”
- Also stated in: § 1 Introduction · p. 1 · plain
“In this paper we present the design and implementation of Microsoft XiaoIce ('Little Ice' literally in Chinese), the most popular social chatbot in the world.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Also stated in: § 1 Introduction · p. 1 · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Social chatbots require a sufficiently high IQ to acquire a range of skills to keep up with the users and help them complete specific tasks.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Topic Manager simulates human behavior of changing topics during a conversation.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Empathetic computing reflects XiaoIce's EQ and models the empathetic aspects of the humanmachine conversation.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“More importantly, social chatbots also require a sufficient EQ to meet users' emotional needs, such as emotional affection and social belonging, which are among the fundamental needs for human beings Maslow [1943].”
What the sources say
- Not checked yetcites Maslow [1943]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Implementation of the conversation engine relies heavily on A/B test to evaluate if a new module or a new dialogue skill is going to improve an existing component.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the past, the Turing Test has been used to evaluate chitchat performance. But it is not sufficient to measure the success of long-term, emotional engagement with users.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To ensure a smooth conversation, we avoid switching among different skills too often. We prefer keeping the running skill activated until it terminates to activating a new skill.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Figure 3, XiaoIce tries a new topic (i.e., a popular singer named Ashin) in Turn 5 and recommends a song in Turn 15, and thereby learns the user's preferences (e.g., the music topic and the singer he loves), knowledge that would lead to more engagement in the future.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“XiaoIce wins the user's trust and friendship by her wonderful sense of humor and empathetic responses to all sorts of questions, some of which are quite challenging, such as “you are all lies”, and “who is your daddy”.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, XiaoIce has to interact with the user's environment and access real-world knowledge e.g., via API calls.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Traditionally, social chatbots are designed for chitchat scenarios where bots are expected to mimic human user conversations but not to interact with the user's environment.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Each generated response is labeled on the 3-level quality scale by three human judges.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Rests on an assumption: The 3-level quality scale labels assigned by human judges reliably measure response quality.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the short term, incorporating many task-completion skills can reduce the CPS since these skills help users accomplish tasks more efficiently by minimizing the CPS.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Through a long conversation, XiaoIce has demonstrated human-like empathy and social skills, and eventually helped the user regain her confidence and move forward with a positive attitude.”
boosteddirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To human users, it is these latter aspects that are more important to drive a meaningful and interesting conversation.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Compared with traditional personal assistants, XiaoIce's task-completion skills offer more perspectives and empathy in generating interpersonal responses.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“According to Khatri et al. [2018], the latest Alexa systems have achieved the CPS of 14.6, an increase of 54% since the launch of the 2018 competition.”
What the sources say
- Not checked yetcites Khatri et al. [2018]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To control the data quality, a pipeline similar to that for Core Chat is applied to retain only the pairs whose text comments fit XiaoIce's persona 9 .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It can be observed that the XiaoIce-produced comments are emotional, subjective, imaginative, and are very likely to inspire meaningful human-machine interactions, while the comments generated by the other image captioning models are reasonable in content but boring in the context of social chats, and thus less likely to improve user engagement.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The response candidate ranker is trained using 50K manually labeled dialogues.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The use of neural response generator in Core Chat, starting from the 5th generation, significantly improves the coverage and diversity of XiaoIce's responses.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The full duplex voice mode released in the 5th generation has made the humanmachine communication substantially more natural, thus significantly increasing the length of conversation sessions.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is the latter, the social skill aspects, that distinguishe image commenting from other traditional vision tasks such as image tagging and image description, as illustrated in Figure 11.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As a result, it helps drive the number of active users from 500 million to 660 million, and keep the CPS to 23 in spite of the incorporation of many task-completion tasks that are designed to minimize the CPS such as those that control the smart devices.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But in the long run, these new skills not only help grow XiaoIce's NAU by meeting user needs and strengthening the emotional bond with human users, but also provide large amounts of training data to improve the core conversation engine e.g., by optimizing the neural response generation models, empathy models, and the dialogue manager, etc.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“The second is the LSTM-MMI model Li et al. [2016a], which is one of the state-of-the-art neural response generation models.”
What the sources say
- Not checked yetcites Li et al. [2016a]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, low BLEU (or other metrics) scores do not necessarily indicate low quality as the number of reference responses is always limited in the test set.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
- Rests on an assumption: In open-domain dialog systems, the same input may have many plausible responses that differ in topics or contents significantly.
- Rests on an assumption: The number of reference responses in the test set is always limited.
In the paper
“Galley et al. [2015] showed that the correlation of string-based metrics (e.g., BLEU and deltaBLEU) significantly increases with the units of measurement bigger than a sentence.”
What the sources say
- Not checked yetcites Galley et al. [2015]
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“But as pointed out in Gao et al. [2019], the correlation analysis by Liu et al. [2016] is performed at the sentence level while BLEU is designed from the outset to be used as a corpus-level metric.”
What the sources say
- Not checked yetcites Gao et al. [2019]
- Not checked yetcites Liu et al. [2016]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Retrieval-based methods are often evaluated using traditional information retrieval metrics Manning et al. [2008] such as as Precision@K, Mean Average Precision (MAP), and normalized Discounted Cumulative Gain (nDCG).”
What the sources say
- Not checked yetcites Manning et al. [2008]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, even a completely deterministic function can lead to unpredictable behavior. For example, a simple answer 'Yes' by XiaoIce could be perceived offensive in a given context.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is not surprising, then, that with vastly more people being digitally connected in the social media age, social chatbots have become an important alternative means for engagement.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: Happiness and meaningful conversations often go hand in hand.
None of the sources cited for this claim have been checked yet.
In the paper
“Generation-based methods are evaluated using those metrics borrowed from text generation tasks like machine translation and text summarization, using string and n-gram matching metrics such as BLEU Papineni et al. [2002], METEOR Banerjee and Lavie [2005] and ROUGE Lin [2004].”
What the sources say
- Not checked yetcites Papineni et al. [2002]
- Not checked yetcites Banerjee and Lavie [2005]
- Not checked yetcites Lin [2004]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This could mislead the XiaoIce users by setting an unrealistic expectation. As a result, the users might become addicted after chatting with XiaoIce for a very long time.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While manual evaluation is reliable, it is very expensive and chatbot developers often have to resort to automatic metrics for quantifying day-to-day progress and for performing automatic system optimization.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a score that is substantially better than that of other chatbots and even human conversations”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Commonly used automatic evaluation metrics for open-domain dialogue systems existing today all have their own limitations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite the success of XiaoIce, the fundamental mechanism of human-level intelligence, as demonstrated in human conversations, is not yet fully understood.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“However, as argued by Gao et al. [2019], machine-learned metrics lead to potential problems such as overfitting and 'gaming of the metric' Albrecht and Hwa [2007].”
What the sources say
- Not checked yetcites Gao et al. [2019]
- Not checked yetcites Albrecht and Hwa [2007]
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“deltaBLEU Galley et al. [2015] is an extension to BLEU that exploits numerical ratings associated with conversational responses.”
What the sources say
- Not checked yetcites Galley et al. [2015]
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The databases used by the retrieval-based candidate generators and for training the neural response generator have been carefully cleaned, and a hand-crafted editorial response is used to avoid any improper or offensive responses.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Most open-domain dialogue systems, such as XiaoIce, generate responses using either retrieval-based methods or generation-based methods, or hybrid methods.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Unlike early chatbots designed for chitchat, XiaoIce is designed as a social chatbot intended to serve users' needs for communication, affection, and social belonging, and is endowed with empathy, personality and skills, integrating both EQ and IQ to optimize for long-term user engagement, measured in expected CPS.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As we have shown in Core Chat and Image Commenting, XiaoIce always preserves the right of not discussing or commenting on inappropriate topics and contents.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The authors state this as an assumption.
Human-machine social chat can be modeled as a hierarchical decision-making process over Markov Decision Processes.
“we mathematically cast human-machine social chat as a hierarchical decision-making process, and optimize XiaoIce for long-term user engagement, measured in expected CPS.”
“cast human-machine social chat as decision-making over Markov Decision Processes (MDPs)”
The authors state this as an assumption.
Expected CPS is a valid measure of long-term user engagement for social chatbots.
“we propose to use expected Conversation-turns Per Session (CPS) as the success metric for social chatbots.”
“The larger the CPS is, the better engaged the social chatbot is.”
The paper relies on this without support.
An AI companion can satisfy the human need for communication, affection, and social belonging.
“XiaoIce is uniquely designed as an AI companion with an emotional connection to satisfy the human need for communication, affection, and social belonging.”
The paper relies on this without support.
The machine learning classifiers used for user understanding accurately detect user intents, emotions, topics, opinions, and persona from dialogue.
“These key-value pairs are generated using a set of machine learning classifiers as follows.”
The paper relies on this without support.
The TV series dataset used in the first pilot study is representative enough to validate the persona-based neural response generator for XiaoIce.
“In the first pilot study reported in Li et al. [2016b], we compare the persona model against two baseline models, using the TV series dataset for model training and evaluation.”
The paper relies on this without support.
The 3-level quality scale labels assigned by human judges reliably measure response quality.
“Each generated response is labeled on the 3-level quality scale by three human judges.”
The paper relies on this without support.
The KG triples that co-occur often in the same conversations represent related topics humans often discuss.
“We include in the XiaoIce KG a Satori triple ( h, r, t ) only if h and t co-occur often enough in the same conversations. Such a triple contains a pair of related topics ( h, t ) that humans often discuss in one conversation”
The paper relies on this without support.
The unpaired database sentences from public lectures and quotes are suitable as candidate responses for XiaoIce.
“The unpaired database we have used in XiaoIce consists of sentences collected from public lectures and quotes in news articles and reports. These sentences are considered candidate responses R .”
The paper relies on this without support.
Any individual dialogue skill is unlikely to have a significant impact on CPS.
“Since any individual skill is unlikely to have a significant impact on CPS, we measure the user satisfaction of a skill by monitoring its active users and skill triggering rate (i.e., the number of times a skill is activated by users within a day or a week).”
The paper relies on this without support.
Traditionally, social chatbots are designed for chitchat scenarios where bots are expected to mimic human user conversations but not to interact with the user's environment.
“Traditionally, social chatbots are designed for chitchat scenarios where the bots are expected to mimic human user conversations but not to interact with the user's environment.”
The paper relies on this without support.
Happiness and meaningful conversations often go hand in hand.
“Psychological studies show that happiness and meaningful conversations often go hand in hand.”
The paper relies on this without support.
The long-term success of a social chatbot needs to be measured by its user engagement.
“the long-term success of a social chatbot needs to be measured by its user engagement”
The paper relies on this without support.
The most reliable evaluation is to deploy the chatbot to users and monitor user feedback and engagement over a long period.
“There is no doubt that the most reliable evaluation is to deploy the chatbot to users and monitor the user feedback and engagement, measured by user ratings, NAU, CPS, and so on, over a long period of time.”
The paper relies on this without support.
In open-domain dialog systems, the same input may have many plausible responses that differ in topics or contents significantly.
“Nevertheless, in open-domain dialog systems, the same input may have many plausible responses that differ in topics or contents significantly.”
The paper relies on this without support.
The number of reference responses in the test set is always limited.
“as the number of reference responses is always limited in the test set”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Popular social chatbots have millions of users (Zhou et al., 2020), and their user bases are expanding.”
From A longitudinal study of human–chatbot relationships (2022) · cites this paper as Zhou et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.3 Core Chat · p. 16
“Our evaluation data consists of 4K dialogue sessions.”
§ 4.3 Core Chat · p. 16
“Table 3: Ratings of three response generation systems on a 5K dialogue dataset.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.4 Image Commenting · p. 17
“The generation-based approach uses an image-to-text generator”
§ 4.4 Image Commenting · p. 18
“The components of Image Commenting, including the text-to-image generator and boosted tree ranker, are trained on a dataset consisting of 28 million images”
Citations
Cites
Maslow, A. H. (1943) · Psychological Review · doi:10.1037/h0054346 · Cited by 21,563
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Axford, N. (2017) · Routledge eBooks · doi:10.4324/9781315258324-16 · Cited by 11,329
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
(2004) · Cited by 9,718
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Research works printed but not linked: 23
- Albrecht Joshua, 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, 880
- Asimov I., 1984, “The Bicentennial Man” in I. Asimov, The Bicentennial Man and Other Stories
- Banerjee Satanjeev, 2005, Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 65
- Brahnam Sheryl, 2005, Abuse: The Darker Side of Human Computer Interaction, 62
- Cuayáhuitl Heriberto, 2018, NeurIPS Workshop on Conversational AI
- Dinan Emily, 2018, CoRR
- Fang Hao, 2017, Alexa Prize Proceedings
- Fung Pascale, 2016, CoRR
- Gan Chuang, 2017, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3137
- Ghazvininejad Marjan, 2018, Proceedings of AAAI, 5111
- Lin Chin-Yew, 2004, Proceedings of the ACL workshop, 74
- Mathews Alexander Patrick, 2016, AAAI, 3574
- Misu Teruhisa, 2012, Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 84
- Mostafazadeh Nasrin, 2017, Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 462
- Papineni Kishore, 2002, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 311
- Sai Ananya, 2019, Proceedings of AAAI 2019, 33, 7281
- Serban Iulian Vlad, 2016, AAAI, 16, 3776
- Serban Iulian Vlad, 2017, AAAI, 3295
- Shum Heung-Yeung, 2018, CoRR
- Sutskever Ilya, 2014, NIPS, 3104
- Vinyals Oriol, 2015, ICML Deep Learning Workshop
- Wu Bowen, 2016, Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 652
- Xing Chen, 2017, AAAI, 17, 3351
Cited by
Sorted by citations
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
A Survey of AI-Generated Content (AIGC)
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. (2024) · ACM Computing Surveys · doi:10.1145/3704262 · Cited by 1,457
Chatbots: History, technology, and applications
Adamopoulou, E., & Moussiades, L. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100006 · Cited by 1,090
My Chatbot Companion - a Study of Human-Chatbot Relationships
Skjuve, M., Følstad, A., Fostervold, K. I., & Brandtzaeg, P. B. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102601 · Cited by 797
“These chatbots can converse in an empathetic way with the users (Zhou et al., 2018), often with an aim to become companions, friends or even romantic partners.”
BackgroundInfluentialRecipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784
Metadata
- Authors
- Zhou, L., Gao, J., Li, D., & Shum, H.-Y.
- Affiliations
- Microsoft (United States) (US)
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-03
- Volume
- 46
- Issue
- 1
- Pages
- 53-93
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W2906579211
- Semantic Scholar
- 66b7d31527f980bb2eecc23629f08ba6037facc4
- Cited by
- 777
- References
- 66
- Citations by year
- 2026: 97 · 2025: 109 · 2024: 106 · 2023: 113 · 2022: 120 · 2021: 112 · 2020: 94
- FWCI
- 43.11
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- Microsoft Research
- Microsoft Research Asia
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- social chatbots, Markov decision process, dialogue management, user engagement, emotion recognition, user intent understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
