Future directions for chatbot research: An interdisciplinary research agenda

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Chatbots have emerged as a substantial research area in recent years, with a marked increase of interest in chatbot research within academia and industry from 2016 onwards.

“In response, chatbots have emerged as a substantial research area in recent years.”

• Also stated in: § Introduction · plain

  “There has lately been a marked increase of interest in chatbot research within academia and industry, specifically from 2016 and onwards [86].”

• cites [86] Piccolo LS, Mensio M, Alani H (2018) Chasing the chatbots. In Internet Science. INSCI 2018 international workshops. Springer, Cham, pp 157–169 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

The rapidly growing body of chatbot research has a marked interdisciplinary character, spanning fields such as informatics, management and marketing, media and communication science, linguistics and philosophy, psychology and sociology, engineering, design, and human-computer interaction.

“The rapidly growing body of chatbot research has a marked interdisciplinary character—spanning fields such as informatics, management and marketing, media and communication science, linguistics and philosophy, psychology and sociology, engineering, design, and human-computer interaction.”

• Also stated in: § Research communities · boosted

  “Chatbot research is currently evolving within and across a range of disciplines and has a strong interdisciplinary character.”

May combine two claims.

Research has helped identify factors contributing to positive or negative user experience and has addressed specific aspects such as trust, perceived social support, human likeness, and how these aspects are impacted by chatbot design.

“This has helped identify factors contributing to positive or negative user experience [118] and has addressed specific aspects such as trust [119], perceived social support [71], human likeness [4], and how these aspects are impacted by chatbot design [42].”

• cites [118] Zarouali B, Van den Broeck E, Walrave M, Poels K (2018) Predicting consumer responses to a chatbot on Facebook. Cyberpsychol Behav Soc Netw 21(8):491–497 sourceNot checked yet

  Not checked in the free run.

• cites [119] Zierau N, Engel C, Söllner M, Leimeister JM (2020) Trust in smart personal assistants: A systematic literature review and development of a research agenda. In: Proceedings of the 15th international conference on Wirtschaftsinformatik—WI2020. Association for Information Systems,pp 99–11 sourceNot checked yet

  Not checked in the free run.

• cites [71] Lee S, Choi J (2017) Enhancing user experience with conversational agent for movie recommendation: effects of self-disclosure and reciprocity. Int J Hum Comput Stud 103:95–105 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Users still experience issues in chatbot interaction, both in terms of pragmatic experiences-where chatbots fail to understand or to help users achieve their intended goals-and in terms of hedonic experiences-where chatbots fail to engage users over time.

“Users still experience issues in chatbot interaction, both in terms of pragmatic experiences—where chatbots fail to understand or to help users achieve their intended goals [75]—and in terms of hedonic experiences—where chatbots fail to engage users over time [117].”

• cites [75] Luger E, Sellen A (2016) “Like having a really bad PA”—the gulf between user expectation and experience of conversational agents. In: Proceedings of CHI 2016. ACM, New York, pp 5286–5297 sourceNot checked yet

  Not checked in the free run.

• cites [117] Zamora J (2017) I’m sorry, Dave, I’m afraid I can’t do that: Chatbot perception and expectations. In: Proceedings of the 5th international conference on human agent Interaction. ACM,New York,pp 253–260 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Chatbot user experience has been a key theme in recent research efforts for voice-based virtual agents and text-based applications.

“Chatbot user experience has been a key theme in recent research efforts, for voice-based virtual agents [75] and text-based applications [4].”

• cites [75] Luger E, Sellen A (2016) “Like having a really bad PA”—the gulf between user expectation and experience of conversational agents. In: Proceedings of CHI 2016. ACM, New York, pp 5286–5297 sourceNot checked yet

  Not checked in the free run.

• cites [4] Araujo T (2018) Living up to the chatbot hype: the influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Comput Hum Behav 85:183–189 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Through the CONVERSATIONS workshop series, six overarching topics for future chatbot research have been identified.

“Through the CONVERSATIONS workshop series, six overarching topics for future chatbot research have been identified.”

May combine two claims.

Evaluation in chatbot research is conducted through instruments for users' self reports of user experience, user observation and interviews, analyses of chatbot interaction, and physiological measurements.

“Evaluation in chatbot research is conducted by instruments for users' self reports of user experience [63], through user observation and interviews [75] and analyses of chatbot interaction [66], and also by physiological measurements [22].”

• cites [63] Kocaballi AB, Laranjo L, Coiera E (2019) Understanding and measuring user experience in conversational interfaces. Interact Comput 31(2):192–207 sourceNot checked yet

  Not checked in the free run.

• cites [75] Luger E, Sellen A (2016) “Like having a really bad PA”—the gulf between user expectation and experience of conversational agents. In: Proceedings of CHI 2016. ACM, New York, pp 5286–5297 sourceNot checked yet

  Not checked in the free run.

• cites [66] Kvale K, Sell OA, Hodnebrog S, Følstad A (2019) Improving conversations: lessons learnt from manual analysis of chatbot dialogues. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 187–200 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Important topics in chatbot user research lack sufficient coverage, particularly broader chatbot uses and user groups and implications of chatbot use.

“While we have substantial knowledge on current chatbot users, important topics lack sufficient coverage. Two warrant particular mention: (a) broader chatbot uses and user groups and (b) implications of chatbot use, both detailed below.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Chatbots are taken up in application areas such as customer service, health, education, and office work.

“Chatbots are increasingly becoming important gateways to digital services and information—taken up within domains such as customer service, health, education, and work support.”

• Also stated in: § Introduction · plain

  “Chatbots are currently taken up in application areas as diverse as customer service [1], health [105], education [53], and office work [78].”

• cites [1] Adam M, Wessel M, Benlian A (2020) AI-based chatbots in customer service and their effects on user compliance. Electron Markets. https://doi.org/10.1007/s12525-020-00414-7 sourceNot checked yet

  Not checked in the free run.

• cites [105] Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB (2019) Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry 64(7):456–464. https://doi.org/10.1177/0706743719828977 sourceNot checked yet

  Not checked in the free run.

• cites [53] Hobert S, Berens F (2019) Small talk conversations and the long-term use of chatbots in educational settings–experiences from a field study. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 260–272 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Chatbot use is becoming commonplace; in 2019, over 50% of US and German consumers were estimated to have used chatbots at least once, with even higher numbers in the UK or France.

“For example, in 2019, over 50 % of US and German consumers were estimated to have used chatbots at least once—with even higher numbers in the UK or France [88].”

• Also stated in: § Users and implications · plain

  “Chatbot use is becoming commonplace.”

• cites [88] Press G (2019) AI Stats News: 62 % of US Consumers like using chatbots to interact with businesses. Forbes. Retrieved from https://www.forbes.com/sites/gilpress/2019/10/25/ai-stats-news-us-consumers-interest-in-using-chatbots-to-interact-with-businesses-rise-to-62/Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

We lack the needed technological underpinnings to support some key aspects of chatbot applications.

“we lack the needed technological underpinnings to support some key aspects of chatbot applications.”

Probably not a substantial claim: may not pass S3 (load-bearing).

General guidelines for human-AI interaction are emerging.

“it is promising to see that general guidelines for human-AI interaction are emerging [3] and more of these are needed.”

• cites [3] Amershi S, Weld D, Vorvoreanu M, Fourney A, Nushi B, Collisson P, Teevan J (2019) Guidelines for human–AI interaction. In: Proceedings of the CHI 2019 (paper no. 3). ACM, New York sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Current research on chatbots typically addresses dyadic interactions between one chatbot and one user.

“While the current research on chatbots typically addresses dyadic interactions between one chatbot and one user”

May combine two claims.

High-quality open-source libraries have lowered the barrier to entry and make it possible to build conversational models that exhibit high generalization and coherence.

“along with high-quality open-source libraries [113] have lowered the barrier to entry and make it possible to build conversational models that exhibit high generalization and coherence [90].”

• cites [113] Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Brew J (2019) HuggingFace’s transformers: state-of-the-art natural language processing. arXiv preprint, arxiv:1910.03771 sourceNot checked yet

  Not checked in the free run.

• cites [90] Roller S, Dinan E, Goyal N, Ju D, Williamson M, Liu Y, Boureau YL (2020) Recipes for building an open-domain chatbot. arXiv preprint. arXiv:2004.13637 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Advances in chatbot frameworks and platforms are key enablers of the current interest in chatbot applications.

“The advances in chatbot frameworks and platforms are key enablers of the current interest in chatbot applications.”

Probably not a substantial claim: may not pass S2 (checkable).

There is an essential challenge in studying and designing chatbots for collaboration due to the multifaceted character of such interaction and the range of potential theoretical perspectives.

“There is an essential challenge in studying and designing chatbots for collaboration due to the multifaceted character of such interaction, and the range of potential theoretical perspectives to apply.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Human-chatbot teams (humbots) handle challenging service queries better than chatbots alone and more efficiently than humans alone.

“human-chatbot teams which handle challenging service queries better than chatbots alone and more efficiently than humans alone.”

• cites [45] Grudin J, Jacques R (2019) Chatbots, humbots, and the quest for artificial general intelligence. In: Proceedings of CHI 2019 (paper no. 209). ACM, New York sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Further progress in chatbots will depend on advances in natural language understanding.

“further progress in the field of chatbots will depend on advances in natural language understanding, which will remain a key area of research interest.”

May combine two claims.

Criticism around large models has been growing, especially ethical concerns regarding societal biases, carbon footprint, misuse and misinterpretation, privatization of AI research, and research opportunity costs.

“criticism around large models has been growing lately [9], especially ethical concerns regarding undesirable and often inscrutable societal biases percolating the models [9, 120], carbon footprint [9, 99], misuse and misinterpretation [9], privatization of AI research [99], and even research opportunity costs [49].”

• cites [9] Bender EM, Gebru T, McMillan-Major A, Mitchell M (2021) On the dangers of stochastic parrots: can language models be too big? In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. ACM, New York, pp 610–623 sourceNot checked yet

  Not checked in the free run.

• cites [120] Zou J, Schiebinger L (2018) AI can be sexist and racist—it’s time to make it fair. Nature 559:324–326 sourceNot checked yet

  Not checked in the free run.

• cites [99] Strubell E, Ganesh A, McCallum A (2019) Energy and policy considerations for deep learning in NLP. arXiv preprint, arXiv:1906.02243 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The deliberative process at the CONVERSATIONS workshop series was key to identify and propose research directions in a true interdisciplinary fashion.

“The deliberative process at the workshop series was key to identify and propose research directions in a true interdisciplinary fashion.”

• Also stated in: § Approach · plain

  “As a result, the proposed research directions reflect the interdisciplinary position of a group of collaborating researchers within this emerging field.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The Transformer is a dominant and highly effective architecture for natural language processing.

“the introduction of the Transformer [107] as a dominant and highly effective architecture for natural language processing”

• cites [107] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Conference on neural information processing systems—NIPS 2017, pp 5998–6008Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

End-to-end dialogue systems are applied to predict suitable responses to user input based on models built from large conversational datasets.

“the use of generative approaches has seen substantial progress, where end-to-end dialogue systems are applied to predict suitable responses to user input based on models built from large conversational datasets [2, 109].”

• cites [2] Adiwardana D, Luong MT, So DR, Hall J, Fiedel N, Thoppilan R, Le QV (2020) Towards a human-like open-domain chatbot. arXiv preprint. arXiv:2001.09977Not checked yet

  Not checked in the free run.

• cites [109] Vinyals O, Le Q (2015) A neural conversational model. arXiv preprint, arXiv:1506.05869Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Challenges remain in solutions for supporting chatbot development and standardised testing, for example in studies simulating production environments and approaches to improve chatbots more easily in production.

“Third, challenges remain in solutions for supporting chatbot development and standardised testing for example in terms of studies simulating production environments and approaches to improve chatbots more easily in production.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Opening up powerful large-scale models to the public has the potential to accelerate chatbot development.

“Thus, opening up these powerful models to the public has the potential to accelerate chatbot development even further.”

May combine two claims.

Supervised machine learning for classification and information retrieval enables intent prediction and identification of user sentiment, which are critical to support task-oriented conversations.

“Key advances include the application of supervised machine learning for classification and information retrieval—enabling, for example, intent prediction and identification of user sentiment [20], which are critical to support task-oriented conversations.”

• cites [20] Chen H, Liu X, Yin D, Tang J (2017) A survey on dialogue systems: recent advances and new frontiers. ACM SIGKDD Explor Newsl 19(2):25–35 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Voice-based conversational agents have been shown to more easily interpret particular English dialects, potentially reducing their utility for users of specific areas.

“voice-based conversational agents have been shown to more easily interpret particular English dialects potentially reducing their utility for users of specific areas [51]”

• cites [51] Harwell D (2018) The accent gap. The Washington Post. Retrieved from https://www.washingtonpost.com/graphics/2018/business/alexa-does-not-understand-your-accent/Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Current efforts are moving toward providing development resources that promise anyone with enough motivation, regardless of background, to deliver human-like interactive experiences.

“Current efforts are already moving in that direction, providing development resources that promise anyone with enough motivation, regardless of their background, to deliver human-like interactive experiences.”

Probably not a substantial claim: may not pass S2 (checkable).

Chatbots hold implications for the democratization of technology.

“Chatbots will both raise critical ethical challenges and hold implications for democratization of technology”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Chatbots in collaborative relations involving more people and bots will become more prominent as chatbots mature further.

“we foresee that chatbots in collaborative relations involving more people and bots will become more prominent as chatbots mature further.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

According to European guidelines, trustworthy AI must be aligned with legal regulations and ethical principles and values, and be robust from a technical perspective given its social context.

“According to these guidelines, it is of paramount importance for trustworthy AI to be aligned with (a) legal regulations and (b) ethical principles and values, and also (c) be robust from a technical perspective given its particular social context.”

• cites [30] European Commission (2019) Ethics guidelines for trustworthy AI. Retrieved from: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-aiNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Chatbots have been suggested as accessible interactive systems for visually impaired users, as conversational support for users with special needs, and to support youth engagement in societal issues.

“Chatbots have been suggested as accessible interactive systems for visually impaired in need of an easily navigable user interface [7], as conversational support for users with special needs [19], and to support youth to engage in societal issues [110].”

• cites [7] Baez M, Daniel F, Casati F (2019) Conversational web interaction: proposal of a dialog-based natural language interaction paradigm for the Web. In: Chatbot research and design. Third international workshop, CONVERSATIONS. Springer, Cham, pp 94–11 sourceNot checked yet

  Not checked in the free run.

• cites [19] Catania F, Di Nardo N, Garzotto F, Occhiuto D (2019) Emoty: an emotionally sensitive conversational agent for people with neurodevelopmental disorders. In Proceedings of the 52nd Hawaii international conference on system sciences. Scholar Space, Honolulu, pp 2014–2023 sourceNot checked yet

  Not checked in the free run.

• cites [110] Väänänen K, Hiltunen A, Varsaluoma J, Pietilä I (2019) CivicBots–Chatbots for supporting youth in societal participation. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 143–157 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

A European set of ethics guidelines for trustworthy AI has been presented based on the advice of a high-level expert group on AI.

“based on the advice of a high-level expert group on AI, a European set of ethics guidelines for trustworthy AI has been presented [30].”

• cites [30] European Commission (2019) Ethics guidelines for trustworthy AI. Retrieved from: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-aiNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Concern for ethics in AI is motivated by its disruptive character, potential for job market changes, misuse by malevolent actors, and issues of accountability and bias.

“This concern for ethics in AI is motivated by its disruptive character and potential for changes in the job market and misuse by malevolent actors, as well as issues pertaining to accountability and bias [58].”

• cites [58] McTear MF (2002) Spoken dialogue technology: enabling the conversational user interface. ACM Comput Surv (CSUR) 34(1):90–169 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

There is a lack of knowledge regarding how available low-threshold platforms and frameworks are actually employed to democratize chatbot development and design.

“while available platforms and frameworks are promoted as low-threshold means of chatbot design and development, there is a lack of knowledge regarding how these are actually employed to democratize chatbot development and design.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Tasks assigned to chatbots in collaborative interactions can vary in complexity and involvement.

“tasks assigned to chatbots in these collaborative interactions can vary in complexity and involvement.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Such tasks can range from providing individual notifications to communicating processed and analysed data to different stakeholders.

“Such tasks can be as simple as providing individual notifications, or as complicated as communicating processed and analysed data to different stakeholders.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The proposed European AI Act will help strengthen aspects of ethical concern in AI systems, including legal requirements for human oversight, accuracy, robustness, and security.

“a proposed European set of regulations for AI, the AI Act, will help strengthen aspects of ethical concern in AI systems, including legal requirements for human oversight, accuracy, robustness, and security.”

May combine two claims.

There has been an initial lack of ethical discussion in chatbot research, though exceptions exist.

“as noted in a review of the chatbot literature, there has been an initial lack of ethical discussion in chatbot research [102]—though noteworthy exceptions to this exist, such as the exploration of ethical and social considerations for conversational AI by Ruane et al. [91].”

• cites [102] Syvänen S, Valentini C (2020) Conversational agents in online organization–stakeholder interactions: a state-of-the-art analysis and implications for further research. J Commun Manag 339–362 sourceNot checked yet

  Not checked in the free run.

• cites [91] Ruane E, Birhane A, Ventresque A (2019) Conversational AI: social and ethical considerations. In: Proceedings for the 27th AIAI Irish conference on artificial intelligence and cognitive science—AICS 2019, pp 104–115. CEUR Workshop Proceedings, Vol-2563. http://ceur-ws.org/Vol-2563/Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Twitter bots utilize COVID-19 content to spread political conspiracies.

“Twitter bots utilizing COVID-19 content to spread political conspiracies [34]”

• cites [34] Ferrara E (2020) What types of COVID-19 conspiracies are populated by Twitter bots? First Monday. https://doi.org/10.5210/fm.v25i6.10633 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Due to technological limitations and design choices, chatbots can perpetuate and reinforce existing biases in society, exclude or discriminate against some user groups, and over-represent or enshrine specific values.

“due to technological limitations and design choices, they can be the means to perpetuate and even reinforce existing biases in society, exclude or discriminate against some user groups (e.g. [33, 51]) and over-represent or enshrine specific values”

• cites [33] Feine J, Gnewuch U, Morana S, Maedche A (2019b) Gender bias in chatbot design. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 79–93 sourceNot checked yet

  Not checked in the free run.

• cites [51] Harwell D (2018) The accent gap. The Washington Post. Retrieved from https://www.washingtonpost.com/graphics/2018/business/alexa-does-not-understand-your-accent/Not checked yet

  Not checked in the free run.

May combine two claims.

Chatbots are potentially a low-threshold means to access information and services and may serve to bridge digital divides and strengthen inclusion.

“By allowing simple natural language dialogues, chatbots are potentially a low-threshold means to access information and services and may as such serve to bridge digital divides and strengthen inclusion [14].”

• cites [14] Brandtzaeg PB, Følstad A (2017) Why people use chatbots. In: Proceeedings of the international conference on internet science—INSCI 2017. Springer, Cham, pp 377–392 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

AI has recently been the objective of substantial interest from policy-making and regulatory bodies, as well as in discussions and reflections on ethics, privacy management and trust.

“AI has recently been the objective of substantial interest from policy-making and regulatory bodies, as well as in discussions and reflections on ethics, privacy management and trust [21].”

• cites [21] Chung H, Iorga M, Voas J, Lee S (2017) Alexa, can I trust you? Computer 50(9):100–104 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The GDPR is now used to govern privacy in technology-based systems and services in Europe.

“the General Data Protection Regulation (GDPR) is now used to govern privacy in technology-based systems and services.”

Classified as probably not a substantial claim. May combine two claims.

Chatbots are a potential low-threshold means to access information and services and to promote inclusion.

“consequently are a potential low threshold means to access information and services and promote inclusion”

• AssumptionRests on an assumption: Chatbots permit users to interact through natural language.

Probably not a substantial claim: may not pass S3 (load-bearing).

The aim of using chatbots for strengthening democratization, reducing bias, and facilitating universal design has been included in the vision of chatbots for social good.

“The aim of using chatbots for strengthening democratization, reducing bias, and facilitating universal design has been included in the vision of chatbots for social good [40]”

• cites [40] Følstad A, Brandtzaeg PB, Feltwell T, Law ELC, Tscheligi M, Luger E (2018) Chatbots for social good. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems (paper no. SIG06). ACM, New YorkNot checked yet

  Not checked in the free run.

May combine two claims.

There is a lack of models to characterize conversational collaboration involving agents, although human-machine collaboration is addressed in more generic terms such as actor-network theory.

“While this problem of human–machine collaboration is addressed in more generic terms, for example in actor-network theory [70], there is a lack of models to characterize conversational collaboration involving agents.”

• cites [70] Latour B (2005) Reassembling the Social - An introduction to actor-network-theory. Oxford University Press, Oxford, UK sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The European Commission expert group has identified seven key requirements for ethical AI applications, including human agency and oversight, privacy and data governance, and diversity, non-discrimination, and fairness.

“From these principles, the European Commission expert group has identified seven key requirements for ethical AI applications, including human agency and oversight, privacy and data governance, and diversity, non-discrimination, and fairness.”

• cites [30] European Commission (2019) Ethics guidelines for trustworthy AI. Retrieved from: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-aiNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The conversational character of chatbots may introduce specific ethical problems, including implications of human-like and socially present interaction, consent, third-party privacy, and emotional effects on children and vulnerable users.

“the conversational character of chatbots may conceivably introduce a range of specific ethical problems, for example the ethical implications of human-like and socially present chatbot interaction, issues of consent, the privacy implications of third-party interactions and the implications of emotional effect on children and vulnerable users.”

Probably not a substantial claim: may not pass S2 (checkable).

Chatbots will raise critical ethical challenges.

“Chatbots will both raise critical ethical challenges and hold implications for democratization of technology”

Probably not a substantial claim: may not pass S4 (based).

There is a lack of more systematic or structured investigations of universal and inclusive design of chatbots.

“there is a lack of more systematic or structured investigations of universal and inclusive design of chatbots.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Bots are being deployed at large scale for political interference and influence.

“the general trend of deploying bots in large scale for political interference and influence [44]”

• cites [44] Gorwa R, Guilbeault D (2020) Unpacking the social media bot: a typology to guide research and policy. Policy Internet 12(2):225–248 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Human-chatbot collaboration is an increasingly important aspect of chatbot research and applications.

“Human–chatbot collaboration is foreseen as an increasingly important aspect of chatbot research and applications.”

May combine two claims.

Chatbots may improve access to health care services, support health-promoting behavior change, and supplement educational programs.

“Chatbots may improve access to health care services (e.g. [105]) and support health-promoting behavior change (e.g. [85]), and supplement educational programs [53].”

• cites [105] Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB (2019) Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry 64(7):456–464. https://doi.org/10.1177/0706743719828977 sourceNot checked yet

  Not checked in the free run.

• cites [85] Perski O, Crane D, Beard E, Brown J (2019) Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study. Digital Health 5:2055207619880676 sourceNot checked yet

  Not checked in the free run.

• cites [53] Hobert S, Berens F (2019) Small talk conversations and the long-term use of chatbots in educational settings–experiences from a field study. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 260–272 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

There is a large and growing body of ethical and privacy knowledge to draw on, and an emerging set of guidelines and regulations on ethics and privacy for digital and AI-based systems.

“There is a large and growing body of ethical and privacy knowledge to draw on, and an emerging set of guidelines and regulations on ethics and privacy for digital systems in general, and AI-based systems in particular.”

Classified as probably not a substantial claim.

Some studies have highlighted that chatbots may sustain and even strengthen existing biases in society.

“some studies highlighted critical aspects in using chatbots since they may sustain and even strengthen existing biases in society.”

Probably not a substantial claim: may not pass S4 (based).

There is a lack of research on the usability, accessibility, and effectiveness of low-code chatbot platforms.

“However, to our knowledge, there is a lack of research on the usability, accessibility, and effectiveness of such platforms.”

Probably not a substantial claim: may not pass S4 (based).

There is relatively scarce research on the characteristics of collaboration with chatbots.

“While a range of chatbots for collaboration have been developed, there is relatively scarce research on the characteristics of collaboration with chatbots.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Recent studies suggest that chatbots may serve as a low-threshold interface to information, services, and societal participation but may also face challenges regarding bias and inclusion.

“Recent studies suggest that while chatbots may indeed serve as a low-threshold interface to information, services, and societal participation, they may also face challenges regarding bias and inclusion.”

May combine two claims.

Recent work has discovered substantial divergence in how ethical AI guidelines are written and interpreted, highlighting the complexity of designing guidelines for systems with complex social impact.

“recent work [58] has discovered substantial divergence in how these are written and interpreted, highlighting the complexity of designing guidelines for systems with complex social impact.”

• cites [58] McTear MF (2002) Spoken dialogue technology: enabling the conversational user interface. ACM Comput Surv (CSUR) 34(1):90–169 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Chatbots are a prominent AI-based technology and are in principle addressed by broader concerns for ethics and privacy in technology and AI research.

“Chatbots are a prominent AI-based technology, and as such in principle addressed by the broader concern for ethics and privacy in technology research in general and in AI-based technology in particular.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Chatbot research and design are still in their relative infancy.

“chatbot research and design are still in its relative infancy”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

If the main characteristics of two application areas are similar, transfer of research results between them seems viable.

“If the main characteristics of both are similar, transfer of the research results seems viable.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Ethical reflections and discussion on chatbots and chatbot applications are already emerging.

“Ethical reflections and discussion on chatbots and chatbot applications are already emerging (e.g. [65, 91]).”

• cites [65] Kretzschmar K, Tyroll H, Pavarini G, Manzini A, Singh I (2019) Can your phone be your therapist? Young people’s ethical perspectives on the use of fully automated conversational agents (chatbots) in mental health support. Biomed Inform Insights. https://doi.org/10.1177/1178222619829083 sourceNot checked yet

  Not checked in the free run.

• cites [91] Ruane E, Birhane A, Ventresque A (2019) Conversational AI: social and ethical considerations. In: Proceedings for the 27th AIAI Irish conference on artificial intelligence and cognitive science—AICS 2019, pp 104–115. CEUR Workshop Proceedings, Vol-2563. http://ceur-ws.org/Vol-2563/Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

In personal settings, benefits are often not measured in monetary terms; the main focus is the improvement of daily life or wellbeing.

“personal benefits are often not measured in monetary terms, but the main focus of personal usage is the improvement of daily life or wellbeing”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Advances in the democratization of chatbots will increasingly inspire ethical discourse tied to higher-level discussions about chatbot applications.

“We anticipate that advances in the democratization of chatbots will increasingly inspire ethical discourse that ties in with higher-level discussions about chatbot applications.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In business contexts such as corporate customer support, the potential impacts of chatbots may be less severe than in health, though specific corporate regulations and norms need to be considered.

“In business contexts such as corporate customer support scenarios, the potential impacts may be less severe, but specific corporate regulations and norms need to be considered.”

Classified as probably not a substantial claim.

The emerging field of chatbot research currently lacks coherence, as observed in the broad range of journals and conferences presenting chatbot findings and the lack of commonly agreed key constructs, models, and measurement instruments.

“the current lack of coherence in the emerging field of chatbot research, which may in part be observed by the broad range of journals and conferences in which findings from chatbot research are presented, and also the lack of commonly agreed key constructs, models, and measurement instruments”

Classified as probably not a substantial claim.

Many research results from existing pilot studies and prototypes for specific application areas exist and may be reused as the basis for transfer and generalization to further application areas.

“many research results from existing pilot studies and prototypes for specific application areas exist and may be reused as the basis for transfer and generalization”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Chatbots are not conventional technologies but are developing as agents operating in social contexts.

“chatbots are not conventional technologies, but they are developing as agents operating in social contexts”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The use of chatbots in personal settings is often mainly driven by motivations for engagement and meaning-making.

“the use of chatbots for social relationship, is often mainly driven by motivations for engagement and meaning-making”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The findings are based on discussions at the CONVERSATIONS workshop and mainly involve researchers from European organizations.

“The findings are based on discussions at the CONVERSATIONS workshop and mainly involve researchers from European organizations.”

Probably not a substantial claim: may not pass S2 (checkable).

Future research will benefit from a greater degree of coherence and cohesiveness in the field of chatbot research.

“future research will benefit from a greater degree of coherence and cohesiveness in the field”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Results concerning chatbot communication, user experience, design, and technology are the basis for applying chatbots in specific application areas.

“results concerning chatbot communication, user experience, design, and technology are the basis for applying chatbots in specific application areas”

Probably not a substantial claim: may not pass S4 (based).

Many research studies exist in specific application domains that could possibly be transferred to other domains.

“many research studies exist in specific domains that could possibly be transferred to others”

Probably not a substantial claim: may not pass S3 (load-bearing).

Requirements in the health sector concerning privacy, ethics, and trust may be significantly more demanding than similar requirements in other sectors.

“requirements in the health sector concerning privacy aspects, ethics, and trust may be significantly more demanding than similar requirements in other sectors”

Probably not a substantial claim: may not pass S3 (load-bearing).

Chatbot development can be made accessible to non-programmers, for example through visual programming approaches such as Blockly.

“without requiring chatbot developers to have in-depth software engineering knowledge—as exemplified in using visual programming approaches such as Blockly to chatbot development [89]”

• cites [89] Rodríguez-Gil L, García-Zubia J, Orduña P, Villar-Martinez A, López-De-Ipiña D (2019) New approach for conversational agent definition by non-programmers: a visual domain-specific language. IEEE Access 7:5262–5276 sourceNot checked yet

  Not checked in the free run.