BigBlueBot: teaching strategies for successful human-agent interactions
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Chatbots are becoming quite popular, with many brands developing conversational experiences using platforms such as IBM's Watson Assistant and Facebook Messenger. However, previous research reveals that users' expectations of what conversational agents can understand and do far outpace their actual technical capabilities. Our work seeks to bridge the gap between these expectations and reality by designing a fun learning experience with several goals: explaining how chatbots work by mapping utterances to a set of intents, teaching strategies for avoiding conversational breakdowns, and increasing desire to use chatbots by creating feelings of empathy toward them. Our experience, called BigBlueBot, consists of interactions with two chatbots in which breakdowns occur and the user (or chatbot) must recover using one or more repair strategies. In a Mechanical Turk evaluation (N=88), participants learned strategies for having successful human-agent interactions, reported feelings of empathy toward the chatbots, and expressed a desire to interact with chatbots in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“Early visions of how humans would interact with machines in a natural manner [24] led to the first demonstrations of how the application of simple, declarative rules could emulate a Rogerian psychotherapist [41].”
BackgroundLicklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
Novak, J. D. (2002) · Science Education · doi:10.1002/sce.10032 · Cited by 914 · cited 2×
“Conceptual learning is often applied to science education (e.g., [35]) and has been shown to improve learning outcomes (e.g., [28]).”
BackgroundAbdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Voice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 1×
“Regarding successful interactions with conversational systems, Porcheron et al. conducted a field study of interactions with Amazon Alexa inside the home and found a significant rate of dialog failures [30].”
BackgroundEvaluating and Informing the Design of Chatbots
Jain, M., Kumar, P., Kota, R., & Patel, S. N. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196735 · Cited by 376
Self-citation"Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage
Sciuto, A., Saini, A., Forlizzi, J., & Hong, J. I. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196772 · Cited by 327 · cited 1×
“In other studies of conversational systems, people expressed apprehension toward future use due to uncertainties of the chatbot’s capabilities and issues with intent classification [18, 22, 25, 34].”
BackgroundLearning from a learning thermostat: lessons for intelligent systems for the home
Yang, R., & Newman, M. W. (2013) · Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing · doi:10.1145/2493432.2493489 · Cited by 268
Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224 · cited 10×
“Previous research on conversational agents has identified a mismatch between peoples’ expectations of chatbots and their real-world capabilities [2, 15, 25].”
BackgroundInfluentialSelf-citationExploring human error recovery strategies: Implications for spoken dialogue systems
Skantze, G. (2005) · Speech Communication · doi:10.1016/j.specom.2004.11.005 · Cited by 137
FarmChat: A Conversational Agent to Answer Farmer Queries
Jain, M., Kumar, P., Bhansali, I., Liao, Q. V., Truong, K., & Patel, S. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3287048 · Cited by 117
Self-citationLiao, Q. V., Mas-ud Hussain, M., Chandar, P., Davis, M., Khazaeni, Y., Crasso, M. P., Wang, D., Muller, M., Shami, N. S., & Geyer, W. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173577 · Cited by 116
Convey: Exploring the Use of a Context View for Chatbots
Jain, M., Kota, R., Kumar, P., & Patel, S. N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174042 · Cited by 74
Self-citationHow Personal Experience and Technical Knowledge Affect Using Conversational Agents
Chen, M.-L., & Wang, H.-C. (2018) · Companion Proceedings of the 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3180308.3180362 · Cited by 26 · cited 3×
“Unlike other work that showed that perceptions of usability were largely dependent upon prior experience with a technology [5], in our case, prior experience with chatbots was not a predictor of task success, learning outcomes, or feelings of empathy.”
BackgroundResultInfluential
Cited by
Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224
Self-citationBeyond self-diagnosis: how a chatbot-based symptom checker should respond
You, Y., Tsai, C.-H., Li, Y., Ma, F., Heron, C., & Gui, X. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589959 · Cited by 60
“As prior Human-Computer Interaction (HCI) studies have argued, healthcare systems should include human-like features [29, 107, 155] and improve their transparency and explainability [62, 101, 127, 140, 154], begging the question of how to tailor and improve CSC conversational design in consideration of these qualities.”
BackgroundA Conversation Analysis of Non-Progress and Coping Strategies with a Banking Task-Oriented Chatbot
Li, C.-H., Yeh, S.-F., Chang, T.-J., Tsai, M.-H., Chen, K., & Chang, Y.-J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376209 · Cited by 57
Xing, W., Song, Y., Li, C., Liu, Z., Zhu, W., & Oh, H. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13586 · Cited by 50
Dippold, D. (2023) · Journal of Pragmatics · doi:10.1016/j.pragma.2022.12.004 · Cited by 37
Metadata
- Authors
- Weisz, J. D., Jain, M., Joshi, N. N., Johnson, J., & Lange, I.
- Venue
- Proceedings of the 24th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2019-03-17
- Pages
- 448-459
- OpenAlex
- W2915664243
- Cited by
- 38
- References
- 16
- FWCI
- 2.10
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.