Evaluating and Informing the Design of Chatbots
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Text messaging-based conversational agents (CAs), popularly called chatbots, received significant attention in the last two years. However, chatbots are still in their nascent stage: They have a low penetration rate as 84% of the Internet users have not used a chatbot yet. Hence, understanding the usage patterns of first-time users can potentially inform and guide the design of future chatbots. In this paper, we report the findings of a study with 16 first-time chatbot users interacting with eight chatbots over multiple sessions on the Facebook Messenger platform. Analysis of chat logs and user interviews revealed that users preferred chatbots that provided either a 'human-like' natural language conversation ability, or an engaging experience that exploited the benefits of the familiar turn-based messaging interface. We conclude with implications to evolve the design of chatbots, such as: clarify chatbot capabilities, sustain conversation context, handle dialog failures, and end conversations gracefully.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous studies have suggested that chatbots should aim to sustain contexts to improve the dialogue efficiency [37].”
From Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data (2024) · cites this paper as [37]
In the citing paper
“There is also a growing base of research to inform design of chatbot interactions, whether this concerns conversational design [6], personalization of chatbots [69], the use of interactive elements in chatbots [57], or the use of social cues to indicate social status and capabilities [32].”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [57]
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Norman, D. (2016) · Vahlen eBooks · doi:10.15358/9783800648108 · Cited by 6,374
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“The first chatbot emerged in 1966 from MIT, called ELIZA [44], which emulated a Rogerian psychotherapist.”
BackgroundELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,738
Licklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938
Embodied Conversational Agents
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/2697.001.0001 · Cited by 1,799
Establishing and maintaining long-term human-computer relationships
Bickmore, T. W., & Picard, R. W. (2005) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1067860.1067867 · Cited by 1,333
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gasic, M., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1199 · Cited by 874
A diary study of task switching and interruptions
Czerwinski, M., Horvitz, E., & Wilhite, S. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985715 · Cited by 830
Research works printed but not linked: 16
- 2002. A.L.I.C. E. Foundation website. (2002). Retrieved January 4, 2017 from http://alicebot.org 2002. A.L.I.C.E. Foundation website. (2002). Retrieved January 4, 2017 from http://alicebot.org
- 2013. Mitsuku. ( 2013 ). Retrieved January 4, 2017 from http://www.mitsuku.com 2013. Mitsuku. (2013). Retrieved January 4, 2017 from http://www.mitsuku.com
- 2013. Rose. ( 2013 ). Retrieved January 4, 2017 from http://brilligunderstanding.com/rosedemo.html 2013. Rose. (2013). Retrieved January 4, 2017 from http://brilligunderstanding.com/rosedemo.html
- 2016. Facebook Messenger bots. (2016). Retrieved Dec 1, 2016 from https://chatbottle.co/bots/messenger 2016. Facebook Messenger bots. (2016). Retrieved Dec 1, 2016 from https://chatbottle.co/bots/messenger
- 2017. Facebook Messenger Alterra. (2017) . Retrieved January 30, 2017 from https://www.messenger.com/t/alterra.cc 2017. Facebook Messenger Alterra. (2017). Retrieved January 30, 2017 from https://www.messenger.com/t/alterra.cc
- 2017. Facebook Messenger Call of Duty. (2017) . Retrieved January 30, 2017 from https://www.messenger.com/t/CallofDuty 2017. Facebook Messenger Call of Duty. (2017). Retrieved January 30, 2017 from https://www.messenger.com/t/CallofDuty
- 2017. Facebook Messenger chatShopper. (2017) . Retrieved January 30, 2017 from https://www.messenger.com/t/chatShopper 2017. Facebook Messenger chatShopper. (2017). Retrieved January 30, 2017 from https://www.messenger.com/t/chatShopper
- 2017. Facebook Messenger CNN. (2017) . Retrieved January 30, 2017 from https://www.messenger.com/t/cnn 2017. Facebook Messenger CNN. (2017). Retrieved January 30, 2017 from https://www.messenger.com/t/cnn
- 2017. Facebook Messenger Hi Poncho. (2017) . Retrieved January 30, 2017 from https://www.messenger.com/t/hiponcho 2017. Facebook Messenger Hi Poncho. (2017). Retrieved January 30, 2017 from https://www.messenger.com/t/hiponcho
- 2017. Facebook Messenger Pandorabots. (2017) . Retrieved January 30, 2017 from https://www.messenger.com/t/chatbots.io 2017. Facebook Messenger Pandorabots. (2017). Retrieved January 30, 2017 from https://www.messenger.com/t/chatbots.io
- 2017. Facebook Messenger Swelly. (2017) . Retrieved January 30, 2017 from https://www.messenger.com/t/swell.bot 2017. Facebook Messenger Swelly. (2017). Retrieved January 30, 2017 from https://www.messenger.com/t/swell.bot
- 2017. Facebook Messenger Trivia Blast. (2017) . Retrieved January 30, 2017 from https://www.messenger.com/t/triviablast1 2017. Facebook Messenger Trivia Blast. (2017). Retrieved January 30, 2017 from https://www.messenger.com/t/triviablast1
- Susan Brennan . 1990. Conversation as Direct Manipulation: An Iconoclastic View. The Art of Human-Computer Interface Design ( 1990 ). Susan Brennan. 1990. Conversation as Direct Manipulation: An Iconoclastic View. The Art of Human-Computer Interface Design (1990).
- Orange Hive . 2017. First time bot users deserve good bots. (2017). Retrieved Jan 5, 2018 from https://unfiltered.orangehive.de/ first-time-bot-users-deserve-good-bots/ Orange Hive. 2017. First time bot users deserve good bots. (2017). Retrieved Jan 5, 2018 from https://unfiltered.orangehive.de/ first-time-bot-users-deserve-good-bots/
- J. C. R. Licklider . 1960. IRE Transactions on Human Factors in Electronics HFE-1 (March 1960 ), 4--11. J. C. R. Licklider. 1960. IRE Transactions on Human Factors in Electronics HFE-1 (March 1960), 4--11.
- Statista. 2017. Most popular global mobile messenger apps as of January 2017 . (2017). Retrieved February 7, 2017 from https://www.statista.com/statistics/258749/ most-popular-global-mobile-messenger-apps/ Statista. 2017. Most popular global mobile messenger apps as of January 2017. (2017). Retrieved February 7, 2017 from https://www.statista.com/statistics/258749/ most-popular-global-mobile-messenger-apps/
Cited by
Rapp, A., Curti, L., & Boldi, A. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102630 · Cited by 643
Chaves, A. P., & Gerosa, M. A. (2021) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1841438 · Cited by 572
“Therefore, conversational Intelligence refers to the ability of a chatbot to effectively converse beyond the technical capability of achieving a conversational goal [Jain et al. 2018b].”
BackgroundMethodResultInfluentialWhat makes you continuously use chatbot services? Evidence from chinese online travel agencies
Li, L., Lee, K. Y., Emokpae, E., & Yang, S.-B. (2021) · Electronic Markets · doi:10.1007/s12525-020-00454-z · Cited by 289
Chatbots, Humbots, and the Quest for Artificial General Intelligence
Grudin, J., & Jacques, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300439 · Cited by 275
“Jain et al. [13] enlisted 13 relatively tech - savvy first - time users to try each of eight Facebook Messenger chatbots daily for 3 days.”
BackgroundBorsci, S., Malizia, A., Schmettow, M., van der Velde, F., Tariverdiyeva, G., Balaji, D., & Chamberlain, A. (2022) · Personal and Ubiquitous Computing · doi:10.1007/s00779-021-01582-9 · Cited by 266
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247
“There is also a growing base of research to inform design of chatbot interactions, whether this concerns conversational design [6], personalization of chatbots [69], the use of interactive elements in chatbots [57], or the use of social cues to indicate social status and capabilities [32].”
BackgroundHaugeland, I. K. F., Følstad, A., Taylor, C., & Bjørkli, C. A. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102788 · Cited by 235
Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224
“Both breakdowns and current recovery processes decrease peoples’ satisfaction, trust, and willingness to continue using a chatbot [19, 20, 28].”
BackgroundSelf-citationHow does artificial intelligence create business agility? Evidence from chatbots
Wang, X., Lin, X., & Shao, B. (2022) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2022.102535 · Cited by 197
Conceptual Metaphors Impact Perceptions of Human-AI Collaboration
Khadpe, P., Krishna, R., Fei-Fei, L., Hancock, J. T., & Bernstein, M. S. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415234 · Cited by 183
“e shown a negative review before using it [58]. Likewise, humor and other human-like characteristics that create high social intelligence expectations can be crucial in producing positive evaluations [36, 48]. Assimilation theory would predict that a metaphor signaling high competence will set positive expectations an…”
BackgroundInfluentialComparing artificial intelligence and human coaching goal attainment efficacy
Terblanche, N. H. D., Molyn, J., de Haan, E., & Nilsson, V. O. (2022) · PLoS ONE · doi:10.1371/journal.pone.0270255 · Cited by 194
Heuristic Evaluation of Conversational Agents
Langevin, R., Lordon, R. J., Avrahami, T., Cowan, B. R., Hirsch, T., & Hsieh, G. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445312 · Cited by 159
Are chatbots really useful for human resource management?
Majumder, S., & Mondal, A. (2021) · International Journal of Speech Technology · doi:10.1007/s10772-021-09834-y · Cited by 189
The effect of implementing chatbot customer service on stock returns: an event study analysis
Fotheringham, D., & Wiles, M. A. (2022) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-022-00841-2 · Cited by 180
Sedkaoui, S., & Benaichouba, R. (2026) · European Journal of Innovation Management · doi:10.1108/ejim-02-2024-0129 · Cited by 142
UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital Library
Zheng, Q., Tang, Y., Liu, Y., Liu, W., & Huang, Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501855 · Cited by 100
Chatbots as Conversational Healthcare Services
Jovanovic, M., Baez, M., & Casati, F. (2021) · IEEE Internet Computing · doi:10.1109/mic.2020.3037151 · Cited by 123
A Pedagogical Chatbot: A Supplemental Language Learning Tool
Kohnke, L. (2023) · RELC Journal · doi:10.1177/00336882211067054 · Cited by 100
Gnewuch, U., Morana, S., Adam, M. T. P., & Maedche, A. (2022) · Business & Information Systems Engineering · doi:10.1007/s12599-022-00755-x · Cited by 101
Følstad, A., & Taylor, C. (2021) · Quality and User Experience · doi:10.1007/s41233-021-00046-5 · Cited by 97
FarmChat: A Conversational Agent to Answer Farmer Queries
Jain, M., Kumar, P., Bhansali, I., Liao, Q. V., Truong, K., & Patel, S. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3287048 · Cited by 117
Self-citationToward a Chatbot for Financial Sustainability
Hwang, S., & Kim, J. (2021) · Sustainability · doi:10.3390/su13063173 · Cited by 95
Social interactions and relationships with an intelligent virtual agent
Lee, S. K., Kavya, P., & Lasser, S. C. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102608 · Cited by 108
Employees' perceptions of chatbots in B2B marketing: Affordances vs. disaffordances
Lin, X., Shao, B., & Wang, X. (2021) · Industrial Marketing Management · doi:10.1016/j.indmarman.2021.11.016 · Cited by 123
Multi-Modal Repairs of Conversational Breakdowns in Task-Oriented Dialogs
Li, T. J.-J., Chen, J., Xia, H., Mitchell, T. M., & Myers, B. A. (2020) · Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3379337.3415820 · Cited by 98
Rheu, M., Dai, Y. H., Meng, J., & Peng, W. (2024) · Communication Research · doi:10.1177/00936502231221669 · Cited by 110
Chaves, A. P., Egbert, J., Hocking, T., Doerry, E., & Gerosa, M. A. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3487193 · Cited by 76
Chaves, A. P., & Gerosa, M. A. (2020) · Figshare · doi:10.6084/m9.figshare.13207221 · Cited by 96
Virtual Assistance in Any Context: A Taxonomy of Design Elements for Domain-Specific Chatbots
Janssen, A., Passlick, J., Cardona, D. R., & Breitner, M. H. (2020) · Business & Information Systems Engineering · doi:10.1007/s12599-020-00644-1 · Cited by 117
Chatbots in Marketing: A Literature Review Using Morphological and Co-Occurrence Analyses
Ramesh, A., & Chawla, V. (2022) · Journal of Interactive Marketing · doi:10.1177/10949968221095549 · Cited by 111
Ju, J., Meng, Q., Sun, F., Liu, L., & Singh, S. R. (2023) · Government Information Quarterly · doi:10.1016/j.giq.2022.101785 · Cited by 105
Artificial intelligence changes the way we work: A close look at innovating with chatbots
Wang, X., Lin, X., & Shao, B. (2022) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24621 · Cited by 112
Schillaci, C. E., de Cosmo, L. M., Piper, L., Nicotra, M., & Guido, G. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.123025 · Cited by 98
Software-Based Dialogue Systems: Survey, Taxonomy, and Challenges
Motger, Q., Franch, X., & Marco, J. (2023) · ACM Computing Surveys · doi:10.1145/3527450 · Cited by 68
Rodriguez, S., & Mune, C. D. (2022) · Reference Services Review · doi:10.1108/rsr-05-2022-0020 · Cited by 96
Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data
Wei, J., Kim, S., Jung, H., & Kim, Y.-H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637364 · Cited by 84
“[34] revealed that many first-time chatbot users had disappointment and frustration with the selected chatbots: most chatbots lacked the ability to fully comprehend user messages or intentions.”
BackgroundInfluentialTalaei‐Khoei, A., Yang, A., & Masialeti, M. (2024) · Technovation · doi:10.1016/j.technovation.2024.102975 · Cited by 92
Engaging Students With a Chatbot-Based Academic Advising System
Kuhail, M. A., Al Katheeri, H., Negreiros, J., Seffah, A., & Alfandi, O. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2074645 · Cited by 64
Koivunen, S., Ala-Luopa, S., Olsson, T., & Haapakorpi, A. (2022) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-022-09429-4 · Cited by 95
Celino, I., & Re Calegari, G. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102410 · Cited by 63
de Sá Siqueira, M. A., Müller, B. C. N., & Bosse, T. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2175158 · Cited by 58
Exploring how politeness impacts the user experience of chatbots for mental health support
Bowman, R., Cooney, O., Newbold, J. W., Thieme, A., Clark, L., Doherty, G. J., & Cowan, B. R. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103181 · Cited by 81
Kocaballi, A. B., Sezgin, E., Clark, L., Carroll, J. M., Huang, Y., Huh-Yoo, J., Kim, J., Kocielnik, R., Lee, Y.-C., Mamykina, L., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/38525 · Cited by 62
Rapp, A., Lodovico, C. D., & Di, L. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103471 · Cited by 79
Artificial Intelligence and NLP -Based Chatbot for Islamic Banking and Finance
Khan, S., & Rabbani, M. R. (2021) · International Journal of Information Retrieval Research · doi:10.4018/ijirr.2021070105 · Cited by 64
Beyond self-diagnosis: how a chatbot-based symptom checker should respond
You, Y., Tsai, C.-H., Li, Y., Ma, F., Heron, C., & Gui, X. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589959 · Cited by 60
“Instead, a humanlike design should augment the efficiency by asking intelligent questions and increasing the diversity of chatbot responses [70].”
BackgroundVilaza, G. N., & McCashin, D. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.689736 · Cited by 60
Rapp, A., Boldi, A., Curti, L., Perrucci, A., & Simeoni, R. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2247596 · Cited by 71
A Semantic Web Framework for Automated Smart Assistants: A Case Study for Public Health
Sermet, Y., & Demir, I. (2021) · Big Data and Cognitive Computing · doi:10.3390/bdcc5040057 · Cited by 59
A Conversation Analysis of Non-Progress and Coping Strategies with a Banking Task-Oriented Chatbot
Li, C.-H., Yeh, S.-F., Chang, T.-J., Tsai, M.-H., Chen, K., & Chang, Y.-J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376209 · Cited by 57
Xygkou, A., Siriaraya, P., She, W. J., Covaci, A., & Ang, C. S. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2292880 · Cited by 61
ComPeer: A Generative Conversational Agent for Proactive Peer Support
Liu, T., Zhao, H., Liu, Y., Wang, X., & Peng, Z. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676430 · Cited by 77
Svenningsson, N., & Faraon, M. (2019) · Proceedings of the 2019 2nd Artificial Intelligence and Cloud Computing Conference · doi:10.1145/3375959.3375973 · Cited by 56
Trends & Methods in Chatbot Evaluation
Casas, J., Tricot, M.-O., Khaled, O. A., Mugellini, E., & Cudré-Mauroux, P. (2020) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3395035.3425319 · Cited by 76
Yim, M. C. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2234114 · Cited by 59
Rese, A., & Tränkner, P. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102699 · Cited by 70
A Matter of Trust? Examination of Chatbot Usage in Insurance Business
Cardona, D. R., Janssen, A., Guhr, N., Breitner, M. H., & Milde, J. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.068 · Cited by 67
Designing PairBuddy—A Conversational Agent for Pair Programming
Robe, P., & Kuttal, S. K. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3498326 · Cited by 44
Terblanche, N. H. D., van Heerden, M., & Hunt, R. (2024) · Coaching An International Journal of Theory Research and Practice · doi:10.1080/17521882.2024.2304792 · Cited by 51
Liebrecht, C., & van Hooijdonk, C. M. J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39540-7_4 · Cited by 62
Chatbots for active learning: A case of phishing email identification
Hobert, S., Følstad, A., & Law, E. L.-C. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103108 · Cited by 37
Do Multilingual Users Prefer Chat-bots that Code-mix? Let's Nudge and Find Out!
Bawa, A., Khadpe, P., Joshi, P., Bali, K., & Choudhury, M. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392846 · Cited by 82
Balaji, D., He, L., Giani, S., Bosse, T., Wiers, R., & de Bruijn, G.-J. (2022) · Sexual Health · doi:10.1071/sh22016 · Cited by 44
Yeh, S.-F., Wu, M.-H., Chen, T.-Y., Lin, Y.-C., Chang, X., Chiang, Y.-H., & Chang, Y.-J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501941 · Cited by 35
Li, X., & Wang, J. (2024) · Government Information Quarterly · doi:10.1016/j.giq.2024.101957 · Cited by 36
Park, J., Min, B., Son, K., Song, J. Y., Ma, X., & Kim, J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789107 · Cited by 53
ASHABot: An LLM-Powered Chatbot to Support the Informational Needs of Community Health Workers
Ramjee, P., Chhokar, M., Sachdeva, B., Meena, M., Abdullah, H., Vashistha, A., Nagar, R., & Jain, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713680 · Cited by 69
Self-citationKuttal, S. K., Ong, B., Kwasny, K., & Robe, P. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445659 · Cited by 44
Zarouali, B., Araujo, T., Ohme, J., & de Vreese, C. H. (2023) · Communication Methods and Measures · doi:10.1080/19312458.2022.2156489 · Cited by 55
How to Make chatbots productive – A user-oriented implementation framework
Janssen, A., Cardona, D. R., Passlick, J., & Breitner, M. H. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102921 · Cited by 51
te Pas, M. E., Rutten, W. G. M. M., Bouwman, R. A., & Buise, M. P. (2020) · JMIR Medical Informatics · doi:10.2196/21982 · Cited by 43
Khan, M. I., Fatima, J. K., Bahmannia, S., Chatrath, S. K., Dale, N. F., & Johns, R. (2025) · Journal of Service Theory and Practice · doi:10.1108/jstp-10-2023-0282 · Cited by 43
Hernandez-Bocanegra, D. C., & Ziegler, J. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3579541 · Cited by 50
Design Foundations for AI Assisted Decision Making: A Self Determination Theory Approach
De Vreede, T., Raghavan, M., & De Vreede, G.-J. (2021) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2021.019 · Cited by 42
“I’m Here for You”: Can Social Chatbots Truly Support Their Users? A Literature Review
van Wezel, M. M. C., Croes, E. A. J., & Antheunis, M. L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68288-0_7 · Cited by 54
Chatbots and virtual assistant in Indian banks
Pal, S., & Singh, D. (2019) · Industrija · doi:10.5937/industrija47-24578 · Cited by 40
Issom, D.-Z., Hardy-Dessources, M.-D., Romana, M., Hartvigsen, G., & Lovis, C. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.600333 · Cited by 39
“This could be achieved by comparing metrics such as usability, esthetics, speed, or by investigating alternative methods for conversational user interface evaluation (88, 89).”
MethodConceptualizing ethical AI-enabled marketing: Current state and agenda for future research
Eid, M. A. H., Abuhashesh, M., Sharabati, A. A., Khraiwish, A., Al-Haddad, S., & Abusaimeh, H. (2024) · International Journal of Data and Network Science · doi:10.5267/j.ijdns.2024.6.002 · Cited by 43
Lin, M. P.-C., Chang, D., & Winne, P. H. (2024) · Educational Technology Research and Development · doi:10.1007/s11423-024-10408-3 · Cited by 40
BigBlueBot: teaching strategies for successful human-agent interactions
Weisz, J. D., Jain, M., Joshi, N. N., Johnson, J., & Lange, I. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302290 · Cited by 38
Self-citationKocielnik, R., Langevin, R., George, J. S., Akenaga, S., Wang, A., Jones, D. P., Argyle, A., Fockele, C., Anderson, L., Hsieh, D. T., et al. (2021) · CUI 2021 - 3rd Conference on Conversational User Interfaces · doi:10.1145/3469595.3469599 · Cited by 37
“ceived efficiency in the broader context of CUI-based interaction is an important underlying theme of many task-oriented uses of such systems [15, 20, 33].”
BackgroundIt's How You Say It: Identifying Appropriate Register for Chatbot Language Design
Chaves, A. P., Doerry, E., Egbert, J., & Gerosa, M. A. (2019) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3349537.3351901 · Cited by 50
Klein, S. H. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-05618-w · Cited by 36
Chatbots as Cognitive, Educational, Advisory & Coaching Systems
Karyotaki, M., Drigas, A., & Skianis, C. (2022) · Technium Social Sciences Journal · doi:10.47577/tssj.v30i1.6277 · Cited by 29
Afjal, M. (2023) · Library Hi Tech · doi:10.1108/lht-07-2023-0322 · Cited by 38
Musi, E., Carmi, E., Reed, C., Yates, S., & O’Halloran, K. (2023) · Social Media + Society · doi:10.1177/20563051221150407 · Cited by 34
Sugumar, M., & Chandra, S. (2021) · Journal of International Technology and Information Management · doi:10.58729/1941-6679.1501 · Cited by 34
Chats with Bots: Balancing Imitation and Engagement
Muresan, A., & Pohl, H. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3313084 · Cited by 34
“[4] observed a tendency to gender chatbots and react with annoyance when presented with scripted replies.”
BackgroundArtificial Intelligence (AI)-Powered Chatbots for Marketing and Online Shopping
Waghambare, M. A., Prabhu, S., Ashok, P., & А, Н. Р. (2023) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/978-1-6684-9576-6.ch002 · Cited by 41
PlanFitting: Personalized Exercise Planning with Large Language Model-driven Conversational Agent
Shin, D.-H., Hsieh, G., & Kim, Y. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736607 · Cited by 42
Weber, P. P., & Ludwig, T. G. (2020) · Mensch und Computer (MuC) · doi:10.1145/3404983.3405513 · Cited by 52
"Forgot your password again?": acceptance and user experience of a chatbot for in-company IT support
Fiore, D., Baldauf, M., & Thiel, C. (2019) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3365610.3365617 · Cited by 31
Gessinger, I., Seaborn, K., Steeds, M., & Cowan, B. R. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103400 · Cited by 33
Terblanche, N. H. D., Wallis, G., & Kidd, M. (2023) · Interacting with Computers · doi:10.1093/iwc/iwad039 · Cited by 35
A Formal Understanding of Computational Empathy in Interactive Agents
Brännström, A., Wester, J., & Nieves, J. C. (2022) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101203 · Cited by 34
Implementasi Chatbot Pembelajaran Bahasa Inggris menggunakan Media Sosial
Sarosa, M., Kusumawardani, M., Suyono, A., & Sari, Z. (2020) · Jurnal Edukasi dan Penelitian Informatika (JEPIN) · doi:10.26418/jp.v6i3.43191 · Cited by 29
Poser, M., Küstermann, G. C., Tavanapour, N., & Bittner, E. A. C. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10265-6 · Cited by 35
Chasing the Chatbots: Directions for Interaction and Design Research
Piccolo, L. S. G., Mensio, M., & Alani, H. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-17705-8_14 · Cited by 28
Chatbot personality preferences in Global South urban English speakers
Mehra, B. (2021) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2021.100131 · Cited by 23
Conversational Physical Activity Coaches for Spanish and English Speaking Women: A User Design Study
Figueroa, C. A., Luo, T. C., Jacobo, A., Munoz, A., Manuel, M., Chan, D., Canny, J., & Aguilera, A. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.747153 · Cited by 29
Metadata
- Authors
- Jain, M., Kumar, P., Kota, R., & Patel, S. N.
- Affiliations
- University of Washington (US)
- IBM (India) (IN)
- IBM (Canada) (CA)
- Venue
- Proceedings of the 2018 Designing Interactive Systems Conference
- Publisher
- ACM
- Published
- 2018-06-08
- Pages
- 895-906
- OpenAlex
- W2808216081
- Cited by
- 376
- References
- 32
- Citations by year
- 2026: 18 · 2025: 55 · 2024: 48 · 2023: 61 · 2022: 65 · 2021: 73 · 2020: 32 · 2019: 23 · 2018: 1
- FWCI
- 13.11
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Designing Interactive Systems Conference (publishedVersion)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- chatbots, chatbot design, first-time users, user behavior patterns, Facebook Messenger, text messaging, natural language conversation, conversation context, chat logs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.