Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Abstract
Text-based conversational systems, also referred to as chatbots, have grown widely popular. Current natural language understanding technologies are not yet ready to tackle the complexities in conversational interactions. Breakdowns are common, leading to negative user experiences. Guided by communication theories, we explore user preferences for eight repair strategies, including ones that are common in commercially-deployed chatbots (e.g., confirmation, providing options), as well as novel strategies that explain characteristics of the underlying machine learning algorithms. We conducted a scenario-based study to compare repair strategies with Mechanical Turk workers (N=203). We found that providing options and explanations were generally favored, as they manifest initiative from the chatbot and are actionable to recover from breakdowns. Through detailed analysis of participants' responses, we provide a nuanced understanding on the strengths and weaknesses of each repair strategy.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“negatively impacting user experience and damaging the overall effectiveness of SDS performance [5, 18, 41, 44].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [5]
In the citing paper
“Furthermore, they often prevent users from completing tasks, leading to task abandonment [5]”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [5]
In the citing paper
“Nevertheless, the widespread adoption of SDSs and their consistent and successful long-term use remain challenging [5].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [5]
In the citing paper
“User-assisted repairs could also lead to a more tailored or accurate resolution if the user correctly assists the CA in resolving the error (Ashktorab et al., 2019;Collier et al., 2017;McGrath et al., 2025).”
From Explainability in AI: Comparing Human-Like and System-Like Trust Repair Strategies (2026) · cites this paper as Ashktorab et al., 2019
In the citing paper
“Also, they may even mis-recognize in-scope messages due to the complexity of natural language [9].”
From Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data (2024) · cites this paper as [9]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617 · cited 1×
“The authors individually reviewed this data and used open coding [13] to extract themes in the open-ended answers.”
Method"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746
Designing the user interface: Strategies for effective human-computer interaction
(1993) · Applied Ergonomics · doi:10.1016/0003-6870(93)90471-k · Cited by 5,587
The Preference for Self-Correction in the Organization of Repair in Conversation
Schegloff, E. A., Jefferson, G., & Sacks, H. (1977) · Language · doi:10.2307/413107 · Cited by 5,150 · cited 5×
“We used several concepts from communication theories on grounding [10] and repair [36] to drive the choices of repair strategies we studied.”
BackgroundMethodResultInfluentialClark, H. H., & Brennan, S. E. (1991) · Perspectives on socially shared cognition. · doi:10.1037/10096-006 · Cited by 4,555 · cited 6×
“Repair is also frequently studied under the framework of grounding in communication, proposed by Clark and Brennan [10].”
BackgroundMethodResultInfluentialELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“In 1966, Eliza simulated dialogue as a Rogerian psychothera-pist [47].”
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,738
Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons
Bradley, R. A., & Terry, M. E. (1952) · Biometrika · doi:10.2307/2334029 · Cited by 2,412 · cited 2×
“The Bradley-Terry model [4] is a mathematical model that estimates a vector of “ability scores” for a set of paired object comparisons, which yields an ultimate ranking of all objects.”
BackgroundMethodPrinciples of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 1×
“A distinction should be made between agent-initiative and user-initiative systems [17].”
BackgroundThe Method of Paired Comparisons.
Curnow, R. N., & David, H. A. (1964) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344014 · Cited by 1,271
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
(2016) · Biostatistics in Public Health Using STATA · doi:10.1201/b19547-8 · Cited by 1,037
Voice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 2×
“Though these interfaces are deemed “conversational,” they may not be repaired in the same way as talking to another person [32].”
BackgroundDirect manipulation vs. interface agents
Shneiderman, B., & Maes, P. (1997) · Interactions · doi:10.1145/267505.267514 · Cited by 518
The SAGE Dictionary of Statistics
Cramer, D., & Howitt, D. (2004) · SAGE Publications Ltd eBooks · doi:10.4135/9780857020123 · Cited by 475
Evaluating and Informing the Design of Chatbots
Jain, M., Kumar, P., Kota, R., & Patel, S. N. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196735 · Cited by 376 · cited 1×
“Both breakdowns and current recovery processes decrease peoples’ satisfaction, trust, and willingness to continue using a chatbot [19, 20, 28].”
BackgroundSelf-citationGracefully mitigating breakdowns in robotic services
Lee, M. K., Kielser, S., Forlizzi, J., Srinivasa, S. S., & Rybski, P. E. (2010) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/1734454.1734544 · Cited by 249 · cited 6×
“Previous work [19, 24, 42, 48] studied strategies that aim to alleviate peoples’ negative emotional response from agent or robot breakdowns, such as showing politeness and apologetic behaviors.”
BackgroundMethodInfluentialBradley-Terry Models in R: The BradleyTerry2 Package
Turner, H., & Firth, D. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i09 · Cited by 227
Patterns for How Users Overcome Obstacles in Voice User Interfaces
Myers, C., Furqan, A., Nebolsky, J., Caro, K., & Zhu, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173580 · Cited by 213 · cited 1×
“A common theme in work studying everyday use of conversational agents is users’ struggle with natural language interactions [26–29 , 32 ].”
BackgroundThe Sage dictionary of statistics : a practical resource for students in the social sciences
Cramer, D., & Howitt, D. (2004) · Cited by 197
LARSON‐POWERS, N., & PANGBORN, R. M. (1978) · Journal of Food Science · doi:10.1111/j.1365-2621.1978.tb09732.x · Cited by 196 · cited 1×
“Pairwise experiments are commonly used in various fields of research to determine participant judgments [9, 22].”
BackgroundA crowdsourceable QoE evaluation framework for multimedia content
Chen, K.-T., Wu, C.-C., Chang, Y.-C., & Lei, C.-L. (2009) · Proceedings of the 17th ACM international conference on Multimedia · doi:10.1145/1631272.1631339 · Cited by 192 · cited 1×
“Pairwise comparisons could yield more realistic results than Likert scales [1] because they take advantage of simple judgments and prioritize a small set of stimuli to learn people’s preferences [8, 12].”
Background(2010) · doi:10.5555/1734454.1734544 · Cited by 163
How Do You Want Your Chatbot? An Exploratory Wizard-of-Oz Study with Young, Urban Indians
Thies, I. M., Menon, N., Magapu, S., Subramony, M., & O’Neill, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67744-6_28 · Cited by 151 · cited 1×
“Previous work [19, 24, 42, 48] studied strategies that aim to alleviate peoples’ negative emotional response from agent or robot breakdowns, such as showing politeness and apologetic behaviors.”
BackgroundReceptionist or information kiosk: how do people talk with a robot?
Lee, M. K., Kiesler, S., & Forlizzi, J. (2010) · Proceedings of the 2010 ACM conference on Computer supported cooperative work · doi:10.1145/1718918.1718927 · Cited by 145
Srinivasan (2016) · doi:10.1145/2858036.2858217 · Cited by 143
Dey, M. T. (2024) · Advances in environmental engineering and green technologies book series · doi:10.4018/979-8-3693-7822-9.ch012 · Cited by 137
Toward harnessing user feedback for machine learning
Stumpf, S., Rajaram, V., Li, L., Burnett, M., Dietterich, T., Sullivan, E., Drummond, R., & Herlocker, J. (2007) · Proceedings of the 12th international conference on Intelligent user interfaces · doi:10.1145/1216295.1216316 · Cited by 132 · cited 3×
“Inspired by keyword-based explanations for text classifiers [41], we introduce a strategy that reveals why an intent was mistakenly recognized by highlighting keywords in the user’s utterance that contribute to the classifier’s decision.”
BackgroundMethodInfluentialClose Engagements with Artificial Companions: Key social, psychological, ethical and design issues
(2010) · Natural language processing · doi:10.1075/nlp.8 · Cited by 132
Liao, Q. V., Davis, M. J., Geyer, W., Muller, M., & Shami, N. S. (2016) · Designing Interactive Systems Conference · doi:10.1145/2901790.2901842 · Cited by 132
Self-citationSage Dictionary of Statistics: A Practical Resource for Students in the Social Sciences
Broughton, V. (2005) · Reference Reviews · doi:10.1108/09504120510580208 · Cited by 125
FarmChat: A Conversational Agent to Answer Farmer Queries
Jain, M., Kumar, P., Bhansali, I., Liao, Q. V., Truong, K., & Patel, S. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3287048 · Cited by 117 · cited 2×
“Both breakdowns and current recovery processes decrease peoples’ satisfaction, trust, and willingness to continue using a chatbot [19, 20, 28].”
BackgroundSelf-citationLiao, Q. V., Mas-ud Hussain, M., Chandar, P., Davis, M., Khazaeni, Y., Crasso, M. P., Wang, D., Muller, M., Shami, N. S., & Geyer, W. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173577 · Cited by 116
Self-citationA Psychological Model of Grounding and Repair in Dialog
Cahn, J. E., & Brennan, S. E. (1999) · Cited by 113
Development of Conversational Repair Strategies in Response to Requests for Clarification
Brinton, B., Fujiki, M., Loeb, D. F., & Winkler, E. (1986) · Journal of Speech Language and Hearing Research · doi:10.1044/jshr.2901.75 · Cited by 97 · cited 1×
“When talking to another person, repairs are almost subconscious acts, which may include a combination of speech, gesture, and facial expression [6].”
BackgroundComputational Models of Grounding in Collaborative Systems
Traum, D. R. (1999) · Cited by 96
Choisel, S., & Wickelmaier, F. (2007) · The Journal of the Acoustical Society of America · doi:10.1121/1.2385043 · Cited by 86 · cited 1×
“Pairwise experiments are commonly used in various fields of research to determine participant judgments [9, 22].”
BackgroundConvey: Exploring the Use of a Context View for Chatbots
Jain, M., Kota, R., Kumar, P., & Patel, S. N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174042 · Cited by 74
Self-citationFrozen Suit: Designing a Changeable Stiffness Suit and its Application to Haptic Games
Maimani, A. A., & Roudaut, A. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025655 · Cited by 66
Paek, T., & Horvitz, E. (1999) · National Conference on Artificial Intelligence · Cited by 65
BigBlueBot: teaching strategies for successful human-agent interactions
Weisz, J. D., Jain, M., Joshi, N. N., Johnson, J. M., & Lange, I. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3301275.3302290 · Cited by 47
Self-citationA Wizard-of-Oz Study on A Non-Task-Oriented Dialog Systems That Reacts to User Engagement
Yu, Z., Nicolich-Henkin, L., Black, A. W., & Rudnicky, A. I. (2016) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w16-3608 · Cited by 41
Rapidly Scaling Dialog Systems with Interactive Learning
Williams, J. D., Niraula, N. B., Dasigi, P., Lakshmiratan, A., Suarez, C. G. J., Reddy, M., & Zweig, G. (2015) · Springer eBooks · doi:10.1007/978-3-319-19291-8_1 · Cited by 39 · cited 2×
“Current chatbots often work in a question-and-answer format relying on an intent model [49], which uses machine learning classifiers to map a user utterance to one of many pre-defined intents (e.g., “hello” and “hi” would be classified as the greeting intent).”
MethodEngelhardt, S., Hansson, E., & Leite, I. (2017) · Cited by 38
On the Method of Paired Comparisons
Moran, P. A. P. (1947) · Biometrika · doi:10.2307/2332449 · Cited by 37
Design recommendations to support automated explanation and tutoring
Gomboc, D., Solomon, S., Core, M. G., Lane, H. C., & VanLent, M. (2005) · Cited by 31
Visual Composition of Graphical Elements on Non-Rectangular Displays
Serrano, M., Roudaut, A., & Irani, P. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025677 · Cited by 30 · cited 1×
“This model has been used in previous HCI studies that conducted pairwise comparison experiments (e.g. [ 3, 37]).”
MethodSarmukaddam, S. B. (2006) · Jaypee Brothers Medical Publishers (P) Ltd. eBooks · doi:10.5005/jp/books/10313_9 · Cited by 19
Grounding Criterion: Toward a Formal Theory of Grounding
Horvitz, E., & Paek, T. (2000) · Cited by 16
Research works printed but not linked: 3
- Susan E Brennan. 1998. The grounding problem in conversations with and through computers. Social and cognitive approaches to interpersonal communication (1998), 201--225.
- David Gunning. 2017. Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web (2017).
- Yeoreum Lee, Jae-eul Bae, Sona S Kwak, and Myung-Suk Kim. 2011. The effect of politeness strategy on human-robot collaborative interaction on malfunction of robot vacuum cleaner. In RSS Workshop on HRI.
Cited by
Rapp, A., Curti, L., & Boldi, A. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102630 · Cited by 643
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247
Lee, Y.-C., Yamashita, N., & Huang, Y. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392836 · Cited by 185
“Thus, research has been focusing on advancing technological contributions for making computer agents to naturally chat and understand people; thus, some studies examined different strategies [6, 36] to enhance users’ experience when talking with a chatbot.”
BackgroundZhang, Z., Xu, Y., Wang, Y., Yao, B., Ritchie, D., Wu, T., Yu, M., Wang, D., & Li, T. J.-J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517479 · Cited by 174
AI as Social Glue: Uncovering the Roles of Deep Generative AI during Social Music Composition
Suh, M. (., Youngblom, E., Terry, M., & Cai, C. J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445219 · Cited by 138
“These studies inform how AI could affect collaborative dimensions such as interpersonal dynamics, impression formation, trust, and conflict [3, 30, 41, 74].”
Background
Metadata
- Authors
- Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D.
- Affiliations
- IBM (United States) (US)
- IBM (India) (IN)
- Venue
- Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2019-05-02
- Pages
- 1-12
- OpenAlex
- W2942367614
- Cited by
- 224
- References
- 47
- Citations by year
- 2026: 31 · 2025: 32 · 2024: 34 · 2023: 33 · 2022: 28 · 2021: 42 · 2020: 20 · 2019: 4
- FWCI
- 8.80
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- chatbots, dialogue systems, repair strategies, user preferences, natural language understanding, explanation, confirmation, user experience
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.