Conversational Breakdown in a Customer Service Chatbot: Impact of Task Order and Criticality on User Trust and Emotion
Open accessAbstract
While chatbots are increasingly used for customer service, there is a knowledge gap concerning the impact of Conversational Breakdown in such chatbot interactions. In a 2 \({\times}\) 4 factorial design online experiment, we studied how Conversational Breakdown impacts user emotion and trust in a chatbot for customer service, given variations in task criticality and breakdown task order. Here, 257 participants were randomly assigned to complete high- or low-criticality tasks with a prototype chatbot for customer service, experiencing Conversational Breakdown for the first, second, third or none of their tasks. The task set was decided from a 63-participant pre-study. We found significant impact of Conversational Breakdown, including a marked order effect on overall trust, as well as a bounce-back effect on task-specific trust and emotion after subsequent successful task completion. We found no post-interaction effect of Task Criticality. Based on our findings, we discuss theoretical and practical implications and suggest future research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Research in human-computer interaction suggests that humans may be less willing to expend collaborative effort when interacting with entities they know to be artificial agents.
“Furthermore, research in human-computer interaction suggests that humans may be less willing to expend collaborative effort when interacting with entities they know to be artificial agents (Diederich et al., 2021;Følstad et al., 2024).”
From Explainability in AI: Comparing Human-Like and System-Like Trust Repair Strategies (2026) · cites this paper as Følstad et al., 2024
Cites
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433 · see citing works
A Theory of Cognitive Dissonance
Morvan, C., & O’Connor, A. (2017) · Macat Library eBooks · doi:10.4324/9781912282432 · Cited by 18,284 · see citing works
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Not So Different After All: A Cross-Discipline View Of Trust
Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. (1998) · Academy of Management Review · doi:10.5465/amr.1998.926617 · Cited by 10,274 · see citing works
Measuring emotion: The self-assessment manikin and the semantic differential
Bradley, M. M., & Lang, P. J. (1994) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)90063-9 · Cited by 9,538 · see citing works
Understanding Information Systems Continuance: An Expectation-Confirmation Model1
Bhattacherjee, A. (2001) · MIS Quarterly · doi:10.2307/3250921 · Cited by 8,767 · see citing works
The Preference for Self-Correction in the Organization of Repair in Conversation
Schegloff, E. A., Jefferson, G., & Sacks, H. (1977) · Language · doi:10.2307/413107 · Cited by 5,150 · see citing works
What are emotions? And how can they be measured?
Scherer, K. R. (2005) · Social Science Information · doi:10.1177/0539018405058216 · Cited by 4,264 · see citing works
Patterns of cognitive appraisal in emotion.
Smith, C. A., & Ellsworth, P. C. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.48.4.813 · Cited by 4,068 · see citing works
Lerner, J. S., & Keltner, D. (2001) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.81.1.146 · Cited by 3,325 · see citing works
Russell, J. A., & Barrett, L. F. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.76.5.805 · Cited by 3,121 · see citing works
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · see citing works
Construction and Evaluation of a User Experience Questionnaire
Laugwitz, B., Held, T., & Schrepp, M. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89350-9_6 · Cited by 2,114 · see citing works
Cognition and motivation in emotion.
Lazarus, R. S. (1991) · American Psychologist · doi:10.1037/0003-066x.46.4.352 · Cited by 2,096 · see citing works
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · see citing works
Evidence for a three-factor theory of emotions
Russell, J. A., & Mehrabian, A. (1977) · Journal of Research in Personality · doi:10.1016/0092-6566(77)90037-x · Cited by 1,940 · see citing works
The serial position effect of free recall.
Murdock, B. B. (1962) · Journal of Experimental Psychology · doi:10.1037/h0045106 · Cited by 1,717 · see citing works
Consumers acceptance of artificially intelligent (AI) device use in service delivery
Gursoy, D., Chi, O. H., Lu, L., & Nunkoo, R. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.03.008 · Cited by 1,437 · see citing works
The Effects of Personalization and Familiarity on Trust and Adoption of Recommendation Agents1
Komiak, S. Y. X., & Benbasat, I. (2006) · MIS Quarterly · doi:10.2307/25148760 · Cited by 1,374 · see citing works
AI-based chatbots in customer service and their effects on user compliance
Adam, M., Wessel, M., & Benlian, A. (2021) · Electronic Markets · doi:10.1007/s12525-020-00414-7 · Cited by 1,195 · see citing works
Trust in a specific technology: An investigation of its components and measures
Mcknight, D. H., Carter, M., Thatcher, J. B., & Clay, P. F. (2011) · ACM Transactions on Management Information Systems · doi:10.1145/1985347.1985353 · Cited by 1,185 · see citing works
Four systems for emotion activation: Cognitive and noncognitive processes.
Izard, C. E. (1993) · Psychological Review · doi:10.1037/0033-295x.100.1.68 · Cited by 848 · see citing works
Consumer Trade-Offs and the Evaluation of Services
Ostrom, A. L., & lacobucci, D. (1995) · Journal of Marketing · doi:10.1177/002224299505900102 · Cited by 834 · see citing works
Blame the Bot: Anthropomorphism and Anger in Customer–Chatbot Interactions
Crolic, C., Thomaz, F., Hadi, R., & Stephen, A. T. (2021) · Journal of Marketing · doi:10.1177/00222429211045687 · Cited by 817 · see citing works
Brandtzaeg, P. B., & Følstad, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-70284-1_30 · Cited by 788 · see citing works
Self-citationTaxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · see citing works
Feelings and Consumer Decision Making: The Appraisal‐Tendency Framework
Han, S., Lerner, J. S., & Keltner, D. J. (2007) · Journal of Consumer Psychology · doi:10.1016/s1057-7408(07)70023-2 · Cited by 701 · see citing works
Rapp, A., Curti, L., & Boldi, A. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102630 · Cited by 679 · see citing works
Plutchik, R. (2001) · American Scientist · doi:10.1511/2001.28.344 · Cited by 666 · see citing works
Studies on the generality of affective meaning systems.
Osgood, C. E. (1962) · American Psychologist · doi:10.1037/h0045146 · Cited by 595 · see citing works
Word Cloud Explorer: Text Analytics Based on Word Clouds
Heimerl, F., Lohmann, S., Lange, S., & Ertl, T. (2014) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2014.231 · Cited by 573 · see citing works
Technology, Humanness, and Trust: Rethinking Trust in Technology
Marshall University, Lankton, N., McKnight, D. H., Michigan State University, Tripp, J., & Baylor University (2015) · Journal of the Association for Information Systems · doi:10.17705/1jais.00411 · Cited by 553 · see citing works
Evidence of convergent validity on the dimensions of affect
Russell, J. A. (1978) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.36.10.1152 · Cited by 495 · see citing works
Towards a Theory of Longitudinal Trust Calibration in Human–Robot Teams
de Visser, E. J., Peeters, M. M. M., Jung, M. F., Kohn, S., Shaw, T. H., Pak, R., & Neerincx, M. A. (2020) · International Journal of Social Robotics · doi:10.1007/s12369-019-00596-x · Cited by 447 · see citing works
Some sources of misunderstanding in talk-in-interaction
Schegloff, E. A. (1987) · Linguistics · doi:10.1515/ling.1987.25.1.201 · Cited by 400 · see citing works
In bot we trust: A new methodology of chatbot performance measures
Przegalińska, A., Ciechanowski, L., Stróż, A., Gloor, P. A., & Mazurek, G. (2019) · Business Horizons · doi:10.1016/j.bushor.2019.08.005 · Cited by 355 · see citing works
Desmet, P. (2003) · Human-computer interaction series · doi:10.1007/1-4020-2967-5_12 · Cited by 352 · see citing works
Appraisal Theory: Old and New Questions
Ellsworth, P. C. (2013) · Emotion Review · doi:10.1177/1754073912463617 · Cited by 327 · see citing works
Creepiness: Its antecedents and impact on loyalty when interacting with a chatbot
Rajaobelina, L., Prom Tep, S., Arcand, M., & Ricard, L. (2021) · Psychology & Marketing · doi:10.1002/mar.21548 · Cited by 252 · see citing works
Service consumption criticality in failure recovery
Webster, C., & Sundaram, D. S. (1998) · Journal of Business Research · doi:10.1016/s0148-2963(97)00004-0 · Cited by 248 · see citing works
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247 · see citing works
Self-citationTrust me, I'm a bot – repercussions of chatbot disclosure in different service frontline settings
Mozafari, N., Weiger, W. H., & Hammerschmidt, M. (2021) · Journal of service management · doi:10.1108/josm-10-2020-0380 · Cited by 246 · see citing works
A critical review of state‐of‐the‐art chatbot designs and applications
Luo, B., Lau, R. Y. K., Li, C., & Si, Y. (2021) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1434 · Cited by 244 · see citing works
Evolving Trust in Robots: Specification Through Sequential and Comparative Meta-Analyses
Hancock, P. A., Kessler, T. T., Kaplan, A. D., Brill, J. C., & Szalma, J. L. (2020) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720820922080 · Cited by 240 · see citing works
An Initial Model of Trust in Chatbots for Customer Service—Findings from a Questionnaire Study
Nordheim, C. B., Følstad, A., & Bjørkli, C. A. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz022 · Cited by 233 · see citing works
Self-citationResilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224 · see citing works
Følstad, A., & Skjuve, M. B. (2019) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3342775.3342784 · Cited by 209 · see citing works
Self-citationManaging the human–chatbot divide: how service scripts influence service experience
Sands, S., Ferraro, C., Campbell, C., & Tsao, H. (2020) · Journal of service management · doi:10.1108/josm-06-2019-0203 · Cited by 181 · see citing works
Automated diagnostic aids: The effects of aid reliability on users' trust and reliance
Wiegmann, D. A., Rich, A., & Zhang, H. (2001) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220110110306 · Cited by 176 · see citing works
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146 · see citing works
Wang, C., Li, Y., Fu, W., & Jin, J. (2023) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2023.103325 · Cited by 126 · see citing works
Trust and digital privacy: willingness to disclose personal information to banking chatbot services
Lappeman, J., Marlie, S., Johnson, T., & Poggenpoel, S. (2022) · Journal of Financial Services Marketing · doi:10.1057/s41264-022-00154-z · Cited by 124 · see citing works
Følstad, A., & Taylor, C. (2021) · Quality and User Experience · doi:10.1007/s41233-021-00046-5 · Cited by 109 · see citing works
Self-citationThe Influence of Emotion on Trust
Myers, C. D., & Tingley, D. H. (2016) · Political Analysis · doi:10.1093/pan/mpw026 · Cited by 95 · see citing works
Organizational Factors Affecting Successful Implementation of Chatbots for Customer Service
Zhang, J. J. Y., Følstad, A., & Bjørkli, C. A. (2021) · Journal of Internet Commerce · doi:10.1080/15332861.2021.1966723 · Cited by 82 · see citing works
Self-citationConsideration of Multiple Components of Emotions in Human-Technology Interaction
Mahlke, S., & Minge, M. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85099-1_5 · Cited by 71 · see citing works
A Conversation Analysis of Non-Progress and Coping Strategies with a Banking Task-Oriented Chatbot
Li, C.-H., Yeh, S.-F., Chang, T.-J., Tsai, M.-H., Chen, K., & Chang, Y.-J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376209 · Cited by 57 · see citing works
Improving Conversations: Lessons Learnt from Manual Analysis of Chatbot Dialogues
Kvale, K., Sell, O. A., Hodnebrog, S., & Følstad, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-39540-7_13 · Cited by 51 · see citing works
Self-citationKvale, K., Freddi, E., Hodnebrog, S., Sell, O. A., & Følstad, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68288-0_14 · Cited by 44 · see citing works
Self-citationNavigating the science of emotion
Barrett, L. F., & Westlin, C. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-821124-3.00002-8 · Cited by 44 · see citing works
Chatbots for active learning: A case of phishing email identification
Hobert, S., Følstad, A., & Law, E. L. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2023.103108 · Cited by 43 · see citing works
Self-citationCustomer Service Chatbots: A Qualitative Interview Study into the Communication Journey of Customers
van der Goot, M. J., Hafkamp, L., & Dankfort, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68288-0_13 · Cited by 41 · see citing works
Chen, J., Mishler, S., & Hu, B. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3051137 · Cited by 40 · see citing works
“As an AI language model, I cannot”: Investigating LLM Denials of User Requests
Wester, J., Schrills, T., Pohl, H., & van Berkel, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642135 · Cited by 40 · see citing works
AniSAM & AniAvatar: Animated Visualizations of Affective States
Sonderegger, A., Heyden, K., Chavaillaz, A., & Sauer, J. S. (2016) · International Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858365 · Cited by 37 · see citing works
Effects of Humanlikeness and Conversational Breakdown on Trust in Chatbots for Customer Service
Law, E. L., Følstad, A., & van As, N. (2022) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3546155.3546665 · Cited by 37 · see citing works
Self-citationFølstad, A., & Taylor, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39540-7_14 · Cited by 32 · see citing works
Self-citationSubramanian, S., Moor, K. D., Fiedler, M., Koniuch, K., & Janowski, L. (2023) · Quality and User Experience · doi:10.1007/s41233-023-00059-2 · Cited by 30 · see citing works
The IBM natural conversation framework: a new paradigm for conversational UX design
Moore, R. J., An, S., & Ren, G.-J. (2023) · Human–Computer Interaction · doi:10.1080/07370024.2022.2081571 · Cited by 26 · see citing works
How Social Robots Influence People’s Trust in Critical Situations
Rossi, A., Dautenhahn, K., Koay, K. L., & Walters, M. L. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223471 · Cited by 25 · see citing works
Andújar, C. S. D. B., Gutiérrez-Martín, L., Miranda, J., Blanco-Ruíz, M., & López-Ongil, C. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.955530 · Cited by 19 · see citing works
Bartošová, M., Světlák, M., Kukletová, M., et al. (2019) · BMC Oral Health · doi:10.1186/s12903-019-0718-4 · Cited by 15 · see citing works
(2021) · doi:10.1145/3461778.3461997 · Cited by 14 · see citing works
Law, E. L., van As, N., & Følstad, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42283-6_16 · Cited by 9 · see citing works
Self-citationDoes the Appearance of a Robot Influence People's Perception of Task Criticality?
Chanseau, A., Dautenhahn, K., Walters, M. L., et al. (2018) · IEEE RO-MAN · doi:10.1109/roman.2018.8525663 · Cited by 8 · see citing works
Affect and Trust in Technology in Teams
Xu, J., & Montague, E. N. H. (2015) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931215591042 · Cited by 7 · see citing works
Liu, P., Wang, Y., Hu, J., Qing, L., & Zhao, K. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1078691 · Cited by 7 · see citing works
Lu, Y., & Sarter, N. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631345 · Cited by 5 · see citing works
Deriving and validating emotional dimensions from textual data
Grgić, D., Podobnik, V., & Carvalho, A. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116721 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Explainability in AI: Comparing Human-Like and System-Like Trust Repair Strategies
Konopka, B., & Wiesche, M. (2026) · Information Systems Frontiers · doi:10.1007/s10796-026-10751-1 · Cited by 1 · see citing works
- Authors
- Følstad, A., Law, E. L.-C., & van As, N.
- Affiliations
- SINTEF (NO)
- Durham University (GB)
- boost.ai, Sandnes, Norway
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2024-10-31
- Volume
- 31
- Issue
- 5
- Pages
- 1-52
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W4402168831
- Cited by
- 17
- References
- 79
- Citations by year
- 2026: 13 · 2025: 4
- FWCI
- 2.52
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Transactions on Computer-Human Interaction (publishedVersion, cc-by)
- Durham Research Online (Durham University) (acceptedVersion, other-oa)
- Funders
- Research Council of Norway (270940, 346762)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- conversational breakdown, chatbots, customer service, task criticality, task order, user trust, user emotions, order effects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
