A critical review of state‐of‐the‐art chatbot designs and applications
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Chatbots are intelligent conversational agents that can interact with users through natural languages. As chatbots can perform a variety of tasks, many companies have committed numerous resources to develop and deploy chatbots to enhance various business processes. However, we lack an up‐to‐date critical review that thoroughly examines both state‐of‐the‐art technologies and innovative applications of chatbots. In this review, we not only critically analyze the various computational approaches used to develop state‐of‐the‐art chatbots, but also thoroughly review the usability and applications of chatbots for various business sectors. We also identify gaps in chatbot‐related studies and propose new research directions to address the shortcomings of existing studies and applications. Our review advances both academic research and practical business applications of state‐of‐the‐art chatbots. We provide guidance for practitioners to fully realize the business value of chatbots and assist in making sensible decisions related to the development and deployment of chatbots in various business contexts. Researchers interested in the design and development of chatbots can also gain useful insights from our critical review and identify fruitful research topics and future research directions based on the research gaps discussed herein. This article is categorized under: Technologies > Machine Learning Application Areas > Business and Industry
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 2×
“Following a recurrent mechanism similar to the hidden layers in RNN, the output sequence y 1 to y 3 is generated as a complete sentence (Cho et al., 2014; Sutskever et al., 2014).”
BackgroundNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 1×
“Based on the report by Mikolov et al. (2010) on RNN language modeling, which is an RNN-based algorithm for speech recognition, Wen et al. (2015) introduced a novel neural-based approach for NLG.”
MethodGenerative Adversarial Networks: An Overview
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765202 · Cited by 4,540 · cited 1×
“This method is widely adopted in the field of computer vision for image synthesis, semantic image editing, and image super-resolution (Creswell et al., 2018).”
MethodUser Acceptance of Hedonic Information Systems1
van der Heijden, H. (2004) · MIS Quarterly · doi:10.2307/25148660 · Cited by 3,850 · cited 1×
“…other factors determine the acceptance of chatbot systems, such as perceived ease of use, perceived usefulness (Davis, 1989), esthetics (Cyr et al., 2006), perceived enjoyment (van der Heijden, 2004), and anthropomorphism (Schuetzler et al., 2020), and all of which are related to interface design.”
BackgroundFitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388 · cited 1×
“Laranjo et al. (2018) also noted that the evaluation of patient safety was a rarity.”
BackgroundNadarzynski, T., Miles, O., Cowie, A., & Ridge, D. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619871808 · Cited by 810
A Survey on Dialogue Systems: Recent Advances and New Frontiers
Chen, H., Liu, X., Yin, D., & Tang, J. (2017) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3166054.3166058 · Cited by 579
Lau, J., Zimmerman, B., & Schaub, F. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274371 · Cited by 568
User Experiences of Social Support From Companion Chatbots in Everyday Contexts: Thematic Analysis
Ta, V., Griffith, C., Boatfield, C., Wang, X., Civitello, M., Bader, H., DeCero, E., & Loggarakis, A. (2020) · Journal of Medical Internet Research · doi:10.2196/16235 · Cited by 561
Survey on Chatbot Design Techniques in Speech Conversation Systems
A., S., & John, D. (2015) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2015.060712 · Cited by 479
Physicians’ perceptions of chatbots in health Care: cross-sectional web-based survey
Palanica, A., Flaschner, P., Thommandram, A., Li, M., & Fossat, Y. (2019) · Journal of Medical Internet Research · doi:10.2196/12887 · Cited by 384
The impact of chatbot conversational skill on engagement and perceived humanness
Schuetzler, R. M., Grimes, G. M., & Scott Giboney, J. (2020) · Journal of Management Information Systems · doi:10.1080/07421222.2020.1790204 · Cited by 357
Survey of conversational agents in health
Montenegro, J. L. Z., da Costa, C. A., & da Rosa Righi, R. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.03.054 · Cited by 322
Yen, C., & Chiang, M.-C. (2021) · Behaviour & Information Technology · doi:10.1080/0144929x.2020.1743362 · Cited by 297
Institutional Violence Against Users of the Family Law Courts and the Legal Harassment Scale
Clemente, M., Padilla-Racero, D., Espinosa, P., Reig-Botella, A., & Gandoy-Crego, M. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00001 · Cited by 267
Greer, S., Ramo, D., Chang, Y.-J., Fu, M., Moskowitz, J., & Haritatos, J. (2019) · JMIR mHealth and uHealth · doi:10.2196/15018 · Cited by 265
Effectiveness of an Empathic Chatbot in Combating Adverse Effects of Social Exclusion on Mood
de Gennaro, M., Krumhuber, E. G., & Lucas, G. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.03061 · Cited by 265
An Initial Model of Trust in Chatbots for Customer Service—Findings from a Questionnaire Study
Nordheim, C. B., Følstad, A., & Bjørkli, C. A. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz022 · Cited by 233
Hauser-Ulrich, S., Künzli, H., Meier-Peterhans, D., & Kowatsch, T. (2020) · JMIR mHealth and uHealth · doi:10.2196/15806 · Cited by 210
Technical metrics used to evaluate health care chatbots: scoping review
Abd-Alrazaq, A., Safi, Z., Alajlani, M., Warren, J., Househ, M., & Denecke, K. (2020) · Journal of Medical Internet Research · doi:10.2196/18301 · Cited by 150
Gardiner, P. M., McCue, K. D., Negash, L. M., Cheng, T., White, L. F., Yinusa-Nyahkoon, L., Jack, B. W., & Bickmore, T. W. (2017) · Patient Education and Counseling · doi:10.1016/j.pec.2017.04.015 · Cited by 139
Case study—Embodied virtual agents: An analysis on reasons for failure
Ben Mimoun, M. S., Poncin, I., & Garnier, M. (2012) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2012.07.006 · Cited by 136 · cited 1×
“Although chatbots have been widely adopted for customer service, there is still a notable gap between customer expectation and chatbot performance (Mimoun et al., 2012).”
BackgroundPerski, O., Crane, D., Beard, E., & Brown, J. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619880676 · Cited by 135
Suganuma, S., Sakamoto, D., & Shimoyama, H. (2018) · JMIR Mental Health · doi:10.2196/10454 · Cited by 112
Ethical implications of conversational agents in global public health
Luxton, D. D. (2020) · Bulletin of the World Health Organization · doi:10.2471/blt.19.237636 · Cited by 108 · cited 1×
“Luxton (2020) laid out several potential ethical risks that need to be addressed, including risk of bias, risk of harm, privacy, and inequitable access.”
BackgroundTegos, S., Demetriadis, S., & Karakostas, A. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.07.014 · Cited by 90
Martínez-Miranda, J., Martínez, A., Ramos, R., Aguilar, H., Jiménez, L., Arias, H., Rosales, G., & Valencia, E. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1387-1 · Cited by 79
Assistive Conversational Agent for Health Coaching: A Validation Study
Fadhil, A., Wang, Y., & Reiterer, H. (2019) · Methods of Information in Medicine · doi:10.1055/s-0039-1688757 · Cited by 77 · cited 1×
“Fadhil et al. (2019) introduced CoachAI, an intent-based chatbot that guides users on following healthy lifestyles and reducing the risk of chronic illness.”
BackgroundKocaballi, A. B., Quiroz, J. C., Rezazadegan, D., Berkovsky, S., Magrabi, F., Coiera, E., & Laranjo, L. (2020) · Journal of Medical Internet Research · doi:10.2196/15823 · Cited by 73
The potential of conversational agents to provide a rapid HIV counseling and testing services
van Heerden, A., Ntinga, X., & Vilakazi, K. (2017) · 2017 International Conference on the Frontiers and Advances in Data Science (FADS) · doi:10.1109/fads.2017.8253198 · Cited by 57 · cited 1×
“When a virtual agent developed by Van Heerden et al. (2017) was used to guide counseling sessions before HIV tests, the test takers said they felt comfortable talking with chatbots as the chatbots offered more privacy than humans.”
MethodImproving Conversations: Lessons Learnt from Manual Analysis of Chatbot Dialogues
Kvale, K., Sell, O. A., Hodnebrog, S., & Følstad, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-39540-7_13 · Cited by 51
Aquabot: A Diagnostic Chatbot for Achluophobia and Autism
Mujeeb, S., Hafeez, M., & Arshad, T. (2017) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2017.080930 · Cited by 44 · cited 1×
“Mujeeb et al. (2017) introduced a chatbot named Aquabot that efficiently diagnoses autism and achluophobia (fear of darkness) with an accuracy of 88%.”
Background
Cited by
Do AI chatbots improve students learning outcomes? Evidence from a meta‐analysis
Wu, R., & Yu, Z. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13334 · Cited by 576
Menon, D., & Shilpa, K. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e20962 · Cited by 359
Using Chatbots as AI Conversational Partners in Language Learning
Belda-Medina, J., & Calvo-Ferrer, J. R. (2022) · Applied Sciences · doi:10.3390/app12178427 · Cited by 306
Let's have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations
Shahriar, S., & Hayawi, K. (2023) · arXiv · doi:10.48550/arxiv.2302.13817 · Cited by 167
Let’s Have a Chat! A Conversation with ChatGPT: Technology, Applications, and Limitations
Shahriar, S., & Hayawi, K. (2023) · Artificial Intelligence and Applications · doi:10.47852/bonviewaia3202939 · Cited by 167
Metadata
- Authors
- Luo, B., Lau, R. Y. K., Li, C., & Si, Y.
- Venue
- WIREs Data Mining and Knowledge Discovery
- Publisher
- Wiley
- Published
- 2021-10-25
- Volume
- 12
- Issue
- 1
- ISSN
- 1942-4787 (ISSN-L)
- 1942-4795
- OpenAlex
- W3210314098
- Cited by
- 224
- References
- 40
- FWCI
- 14.38
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.