Software-Based Dialogue Systems: Survey, Taxonomy, and Challenges
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
The use of natural language interfaces in the field of human-computer interaction (HCI) is undergoing intense study through dedicated scientific and industrial research. The latest contributions in the field, including deep learning approaches like recurrent neural networks (RNNs), the potential of context-aware strategies and user-centred design approaches, have brought back the attention of the community to software-based dialogue systems, generally known as conversational agents or chatbots. Nonetheless, and given the novelty of the field, a generic, context-independent overview of the current state of research on conversational agents covering all research perspectives involved is missing. Motivated by this context, this article reports a survey of the current state of research of conversational agents through a systematic literature review of secondary studies. The conducted research is designed to develop an exhaustive perspective through a clear presentation of the aggregated knowledge published by recent literature within a variety of domains, research focuses and contexts. As a result, this research proposes a holistic taxonomy of the different dimensions involved in the conversational agents’ field, which is expected to help researchers and to lay the groundwork for future research in the field of natural language interfaces.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · cited 1×
“…[23, 93, 177], the quality assessment plan was designed based on the quality assessment criteria (QAs) included in the DARE-5 extended proposal from the York University Centre for Reviews and Dissemination [47] as refined by Kitchenham et al. in systematic reviews [86] and tertiary studies [87].”
MethodELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“ELIZA [171], which has been historically considered as the first chatbot, was designed and developed by the Massachusetts Institute of Technology (MIT) more than half a century ago, between 1964 and 1966.”
MethodFitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 1×
“On the other hand, user satisfaction questionnaires cover the topics related to the usability quality characteristics reported in Figure 15, including examples like emotional awareness, learning and relevancy of content [45].”
BackgroundSystematic literature reviews in software engineering – A tertiary study
Kitchenham, B., Pretorius, R., Budgen, D., Pearl Brereton, O., Turner, M., Niazi, M., & Linkman, S. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.006 · Cited by 1,230 · cited 3×
“To guarantee that the search exhaustively covers all relevant literature in the field, we use a set of synonyms as proposed by Kitchenham et al. [87] which encompass the majority of structured literature reviews and secondary studies (e.g., literature reviews, mapping studies, literature…”
MethodBlut, M., Wang, C., Wünderlich, N. V., & Brock, C. (2021) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-020-00762-y · Cited by 1,194
Chatbots: History, technology, and applications
Adamopoulou, E., & Moussiades, L. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100006 · Cited by 1,090 · cited 10×
“Generic and cross-domain chatbots typically require auxiliary NLP techniques to process and contextualize user input into a specific topic or domain, like co-reference resolution techniques [5].”
BackgroundMethodInfluentialInkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063
Alexa, Siri, Cortana, and more: An introduction to voice assistants.
Hoy, M. B. (2018) · Medical Reference Services Quarterly · doi:10.1080/02763869.2018.1404391 · Cited by 965
Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., & Rauws, M. (2018) · doi:10.2196/preprints.9782 · Cited by 715
The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review
Milne-Ives, M., de Cock, C., Lim, E., Shehadeh, M. H., de Pennington, N., Mole, G., Normando, E., & Meinert, E. (2020) · Journal of Medical Internet Research · doi:10.2196/20346 · Cited by 626 · cited 10×
“[107], which is generally summarized into safety/privacy/security concerns.”
BackgroundMethodInfluentialSurvey on Chatbot Design Techniques in Speech Conversation Systems
A., S., & John, D. (2015) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2015.060712 · Cited by 479 · cited 4×
“Enumerations of NLP techniques are present in research literature, but they are frequently presented as a simplified, preliminary stage [13] or even as a secondary task [3].”
BackgroundMethodInfluentialUsability Meanings and Interpretations in ISO Standards
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003) · Software Quality Journal · doi:10.1023/a:1025869312943 · Cited by 451 · cited 2×
“Some studies [133, 136] propose to align the quality characteristics collected from primary studies with the concept of usability reported in ISO 9241 [4] in terms of efficiency [52], effectiveness and satisfaction [60].”
BackgroundAI-Based Digital Assistants: Opportunities, Threats, and Research Perspectives
Maedche, A., Legner, C., Benlian, A., Berger, B., Gimpel, H., Hess, T., Hinz, O., Morana, S., & Söllner, M. (2019) · Business & Information Systems Engineering · doi:10.1007/s12599-019-00600-8 · Cited by 432
Evaluating and Informing the Design of Chatbots
Jain, M., Kumar, P., Kota, R., & Patel, S. N. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196735 · Cited by 376
Systematic Review: Trust-Building Factors and Implications for Conversational Agent Design
Rheu, M., Shin, J. Y., Peng, W., & Huh-Yoo, J. (2021) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1807710 · Cited by 360
The Personalization of Conversational Agents in Health Care: Systematic Review
Kocaballi, A. B., Berkovsky, S., Quiroz, J. C., Laranjo, L., Tong, H. L., Rezazadegan, D., Briatore, A., & Coiera, E. (2019) · Journal of Medical Internet Research · doi:10.2196/15360 · Cited by 332 · cited 10×
“, user profiles [88], third-party services [90]); (2) exploring and categorizing adaptive mechanisms (e.”
BackgroundMethodInfluentialA Survey on Conversational Agents/Chatbots Classification and Design Techniques
Hussain, S., Ameri Sianaki, O., & Ababneh, N. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-15035-8_93 · Cited by 279
Reflection Companion: A Conversational System for Engaging Users in Reflection on Physical Activity
Kocielnik, R., Xiao, L., Avrahami, D., & Hsieh, G. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3214273 · Cited by 221
Design Features of Embodied Conversational Agents in eHealth: a Literature Review
ter Stal, S., Kramer, L. L., Tabak, M., op den Akker, H., & Hermens, H. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102409 · Cited by 188
Learning from Dialogue after Deployment: Feed Yourself, Chatbot!
Hancock, B., Bordes, A., Mazare, P.-E., & Weston, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1358 · Cited by 161
Conversational agents in business: A systematic literature review and future research directions
Bavaresco, R., Silveira, D., Reis, E., Barbosa, J., Righi, R., Costa, C., Antunes, R., Gomes, M., Gatti, C., Vanzin, M., et al. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100239 · Cited by 143 · cited 10×
“Moreover, adaptation in terms of contextualized and personalized verbal content [10] is one of the main sources of adaptation reported in recent literature (F7).”
BackgroundMethodInfluentialThe virtual patient as a learning tool: a mixed quantitative qualitative study
Isaza-Restrepo, A., Gómez, M. T., Cifuentes, G., & Argüello, A. (2018) · BMC Medical Education · doi:10.1186/s12909-018-1395-8 · Cited by 129 · cited 1×
“But user training is not restricted to education: healthcare and medical research focus on training in the means of social skills training for mental spectrum disorders [160], training of psychotherapies like mindfulness [72] and even using virtual patients as a learning tool [75].”
BackgroundPhilip, P., Micoulaud-Franchi, J.-A., Sagaspe, P., Sevin, E. D., Olive, J., Bioulac, S., & Sauteraud, A. (2017) · Scientific Reports · doi:10.1038/srep42656 · Cited by 128 · cited 1×
“Conversational agents for information request include Q&A chatbots [37] and utility experts for complex domains like autonomous diagnosis or access to health-related information [128].”
BackgroundVirtual training and coaching of health behavior: Example from mindfulness meditation training
Hudlicka, E. (2013) · Patient Education and Counseling · doi:10.1016/j.pec.2013.05.007 · Cited by 104 · cited 2×
“For the latter, some examples include improving user engagement and delivering adaptive training for mental health activities [72], adaptive support tasks like analysis of self-reported symptoms of depression [74], improve dialogue quality for a better user experience [74], and provide timely feedback through consulta…”
BackgroundMobile phone-based asthma self-management aid for adolescents (mASMAA): a feasibility study
Rhee, H., Allen, J., Mammen, J., & Swift, M. (2014) · Patient Preference and Adherence · doi:10.2147/ppa.s53504 · Cited by 96
Martínez-Miranda, J., Martínez, A., Ramos, R., Aguilar, H., Jiménez, L., Arias, H., Rosales, G., & Valencia, E. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1387-1 · Cited by 79 · cited 1×
“Some examples include the use of demographic data [81] and patient’s electronic health record [103].”
BackgroundSillice, M. A., Morokoff, P. J., Ferszt, G., Bickmore, T., Bock, B. C., Lantini, R., & Velicer, W. F. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.7640 · Cited by 56 · cited 3×
“The most frequent metric is the use of a Likert scale measure for objective questions (e.g., measuring the level of satisfaction of a specific response [150]), alongside discourse analysis on interview transcripts [19], restrospective think-aloud sessions [28] or cognitive walkthrough [142].”
BackgroundSyvänen, S., & Valentini, C. (2020) · Journal of Communication Management · doi:10.1108/jcom-11-2019-0145 · Cited by 56
Evaluation of Spoken Dialogue Technology for Real-Time Health Data Collection
Levin, E., & Levin, A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.4.e30 · Cited by 54 · cited 1×
“, topic suitability and verbal HCI features) is evaluated to achieve appropriateness in terms of users’ goals achievement to match each user’s specific needs [96].”
BackgroundSocial Media–based Conversational Agents for Health Management and Interventions
Wang, H., Zhang, Q., Ip, M., & Fai Lau, J. T. (2018) · Computer · doi:10.1109/mc.2018.3191249 · Cited by 53
A Systematic Literature Review of Medical Chatbot Research from a Behavior Change Perspective
Gentner, T., Neitzel, T., Schulze, J., & Buettner, R. (2020) · 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC) · doi:10.1109/compsac48688.2020.0-172 · Cited by 45 · cited 2×
“Nevertheless, these secondary studies are typically limited in terms of context or domain of application [53], the focus of research field [120] or even research method [6].”
BackgroundDeveloping enhanced conversational agents for social virtual worlds
Griol, D., Sanchis, A., Molina, J. M., & Callejas, Z. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.09.099 · Cited by 32
Automated Phone Capture of Diabetes Patients Readings with Consultant Monitoring via the Web
Harper, R., Nicholl, P., McTear, M., Wallace, J., Black, L.-A., & Kearney, P. (2008) · 15th Annual IEEE International Conference and Workshop on the Engineering of Computer Based Systems (ecbs 2008) · doi:10.1109/ecbs.2008.31 · Cited by 29
Chat with Bots Intelligently: A Critical Review & Analysis
Dsouza, R., Sahu, S., Patil, R., & Kalbande, D. R. (2019) · 2019 International Conference on Advances in Computing, Communication and Control (ICAC3) · doi:10.1109/icac347590.2019.9036844 · Cited by 27 · cited 6×
“[39], healthcarefocused primary studies use public or private medical knowledge repositories like scrapped data from medical-related forums [11] or disease-symptom mapping knowledge databases [134].”
BackgroundInfluentialHow Personal Experience and Technical Knowledge Affect Using Conversational Agents
Chen, M.-L., & Wang, H.-C. (2018) · Companion Proceedings of the 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3180308.3180362 · Cited by 26 · cited 1×
“The most frequent metric is the use of a Likert scale measure for objective questions (e.g., measuring the level of satisfaction of a specific response [150]), alongside discourse analysis on interview transcripts [19], restrospective think-aloud sessions [28] or cognitive walkthrough [142].”
BackgroundAllen, A. A., Shane, H. C., & Schlosser, R. W. (2018) · Advances in Neurodevelopmental Disorders · doi:10.1007/s41252-017-0041-5 · Cited by 23 · cited 2×
“The most common areas of application in this domain include support tools for healthcare professionals in medical activities like medication prescription [6] or vital sign control and monitoring [145] and support tools for patient assistance like therapy management [8].”
BackgroundA Conversational Interface for Self-screening for ADHD in Adults
Håvik, R., Wake, J. D., Flobak, E., Lundervold, A., & Guribye, F. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-17705-8_12 · Cited by 17 · cited 2×
“, the resources) and the effectiveness characteristic depicted before, which can be evaluated through a compared experimental analysis between chatbot-assisted and traditional tasks in terms of task completion time [66].”
Background
Cited by
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
Omarov, B., Zhumanov, Z., Gumar, A., & Kuntunova, L. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0140616 · Cited by 60
“While the AI-enabled mobile chatbot psychologist offers a convenient and accessible platform for psychological support, there is a risk that users may become overly reliant on the chatbot, neglecting the importance of face-to-face interactions and professional psychological care [49].”
BackgroundPublic opinion on types of voice systems for older adults
Qu, N. Z., Li, J., Kongmanee, J., & Chignell, M. (2024) · Journal of Rehabilitation and Assistive Technologies Engineering · doi:10.1177/20556683241287414 · Cited by 1
Metadata
- Authors
- Motger, Q., Franch, X., & Marco, J.
- Venue
- ACM Computing Surveys
- Published
- 2022-12-03
- Volume
- 55
- Issue
- 5
- Pages
- 1-42
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4223518348
- Cited by
- 68
- References
- 38
- FWCI
- 4.93
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.