To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions
Open accessThe DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Abstract
Conversational agents (CAs) can analyze online conversations using natural language techniques and effectively facilitate group discussions by sending supervisory messages. However, if a CA makes imperfect interventions, users may stop trusting the CA and discontinue using it. In this study, we demonstrate how inaccurate interventions of a CA and a conversational repair strategy can influence user acceptance of the CA, members' participation in the discussion, perceived discussion experience between the members, and group performance. We built a CA that encourages the participation of members with low contributions in an online chat discussion in which a small group (3-6 members) performs a decision-making task. Two types of errors can occur when detecting under-contributing members: 1) false-positive (FP) errors happen when the CA falsely identifies a member as under-contributing and 2) false-negative (FN) errors occur when the CA misses detecting an under-contributing member. We designed a conversational repair strategy that gives users a chance to contest the detection results and the agent sends a correctional message if an error is detected. Through an online study with 175 participants, we found that participants who received FN error messages reported higher acceptance of the CA and better discussion experience, but participated less compared to those who received FP error messages. The conversational repair strategy moderated the effect of errors such as improving the perceived discussion experience of participants who received FP error messages. Based on our findings, we offer design implications for which model should be selected by practitioners between high precision (i.e., fewer FP errors) and high recall (i.e., fewer FN errors) models depending on the desired effects. When frequent FP errors are expected, we suggest using the conversational repair strategy to improve the perceived discussion experience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, our findings are similar to Do et al. [16], which found that people who interacted with a conversational assistant that made factual errors were less likely to continue participating in group discussions.”
From The AI Double Standard: Humans Judge All AIs for the Actions of One (2025) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mixed-Effects Models in Sand S-PLUS
Pinheiro, J. C., & Bates, D. M. (2000) · Statistics and Computing · doi:10.1007/978-1-4419-0318-1 · Cited by 11,187
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307
The Preference for Self-Correction in the Organization of Repair in Conversation
Schegloff, E. A., Jefferson, G., & Sacks, H. (1977) · Language · doi:10.2307/413107 · Cited by 5,150
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 2×
“However, Dietvorst et al. argued that a system acknowledging mistakes can make errors stand out and people may lose trust and likability in algorithmic decisions (i.e., algorithmic aversion) [24].”
BackgroundMany Hands Make Light the Work: The Causes and Consequences of Social Loafing
Latané, B., Williams, K., & Harkins, S. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.6.822 · Cited by 2,216
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 2×
“This perspective supports the Computers Are Social Actors (CASA) paradigm [65] where people respond in the same manner regardless of whether they are interacting with a human facilitator or a computer.”
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
Araujo, T. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.03.051 · Cited by 1,156
Nowak, K. L., & Biocca, F. (2003) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474603322761289 · Cited by 1,141
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Research works printed but not linked: 19
- Martin Adam, Michael Wessel, and Alexander Benlian. 2020. AI-based chatbots in customer service and their effects on user compliance. Electronic Markets (2020), 1-19.
- Saul Albert and Jan P de Ruiter. 2018. Repair: the interface between interaction and cognition. Topics in cognitive science 10, 2 (2018), 279-313.
- Dennis Benner, Edona Elshan, Sofia Schöbel, and Andreas Janson. 2021. What do you mean? a review on recovery strategies to overcome conversational breakdowns of conversational agents. In International Conference on Information Systems (ICIS).
- Anthony D Bergstrom. 2011. Social mirrors: visualization as conversation feedback. Ph.D. Dissertation. University of Illinois at Urbana-Champaign.
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77-101.
- The Concord Consortium. 2017. Teaching Teamwork. Retrieved Jan 22, 2021 from https://learn.concord.org/resources/ 565/teaching-teamwork-adder
- Fred D Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly (1989), 319-340.
- Gerardine DeSanctis and R Brent Gallupe. 1987. A foundation for the study of group decision support systems. Management science 33, 5 (1987), 589-609.
- James Fishkin, Nikhil Garg, Lodewijk Gelauff, Ashish Goel, Kamesh Munagala, Sukolsak Sakshuwong, Alice Siu, and Sravya Yandamuri. 2018. Deliberative democracy with the Online Deliberation platform. In The 7th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2019). HCOMP.
- Asbjørn Følstad and Cameron Taylor. 2019. Conversational repair in chatbots for customer service: the effect of expressing uncertainty and suggesting alternatives. In International Workshop on Chatbot Research and Design. Springer, 201-214.
- Taemie Kim, Agnes Chang, Lindsey Holland, and Alex Sandy Pentland. 2008. Meeting mediator: enhancing group collaborationusing sociometric feedback. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. 457-466.
- Ben Lafreniere, Tanya R. Jonker, Stephanie Santosa, Mark Parent, Michael Glueck, Tovi Grossman, Hrvoje Benko, and Daniel Wigdor. 2021. False Positives vs. False Negatives: The Effects of Recovery Time and Cognitive Costs on Input Error Preference. In The 34th Annual ACM Symposium on User Interface Software and Technology. 54-68.
- J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159-174.
- Christopher H Morrell. 1998. Likelihood ratio testing of variance components in the linear mixed-effects model using restricted maximum likelihood. Biometrics (1998), 1560-1568.
- Sherry L Piezon and William D Ferree. 2007. Perceptions of social loafing in online learning groups. In 23 rd Annual Conference on Distance Teaching & Learning, http://www. uwex. edu.
- Rachel Quednau. 2016. Top Strong Citizen Discussions this Week. Retrieved Jan 22, 2021 from https://www.strongtowns. org/journal/2016/4/13/top-strong-citizens-discussions?rq=slack
- Sandhya Saisubramanian, Shannon C Roberts, and Shlomo Zilberstein. 2021. Understanding User Attitudes Towards Negative Side Effects of AI Systems. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1-6.
- James A Shepperd. 1993. Productivity loss in performance groups: A motivation analysis. Psychological bulletin 113, 1 (1993), 67.
- Sam R Wilson, William C Barley, Luisa Ruge-Jones, and Marshall Scott Poole. 2020. Tacking Amid Tensions: Using Oscillation to Enable Creativity in Diverse Teams. The Journal of Applied Behavioral Science (2020), 0021886320960245.
Cited by
“As an AI language model, I cannot”: Investigating LLM Denials of User Requests
Wester, J., Schrills, T., Pohl, H., & van Berkel, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642135 · Cited by 36
Cai, Z., Park, S., Nixon, N., & Doroudi, S. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650868 · Cited by 56
Are We On Track? AI-Assisted Active and Passive Goal Reflection During Meetings
Chen, X., Tankelevitch, L., Vanukuru, R., Scott, A. E., Panda, P., & Rintel, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714052 · Cited by 52
Fok, R., Lipka, N., Sun, T., & Siu, A. F. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641969 · Cited by 37
Rayan, J. A., Kanetkar, D., Gong, Y., Yang, Y., Palani, S., Xia, H., & Dow, S. P. (2024) · Creativity and Cognition · doi:10.1145/3635636.3656184 · Cited by 37
MeetMap: Real-Time Collaborative Dialogue Mapping with LLMs in Online Meetings
Chen, X., Yap, N., Lu, X., Gunal, A. E., & Wang, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711030 · Cited by 33
Social-RAG: Retrieving from Group Interactions to Socially Ground AI Generation
Wang, R., Zhou, X., Qiu, L., Chang, J. C., Bragg, J., & Zhang, A. X. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713749 · Cited by 35
Ba, S., Xuechun, S. H. I., Wu, S., & Lu, G. (2026) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2026.100579 · Cited by 23
Assessing Cognitive and Social Awareness among Group Members in AI-assisted Collaboration
de Jong, S., Wester, J., Schrills, T., Secher, K. S., Griggio, C. F., & van Berkel, N. (2024) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3701571.3701582 · Cited by 19
The AI Double Standard: Humans Judge All AIs for the Actions of One
Manoli, A., Pauketat, J. V. T., & Anthis, J. R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711083 · Cited by 12
Chen, W., Yu, C., Wang, Y., Chen, M., Xu, Y., & Shi, Y. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757587 · Cited by 15
Human-AI collaboration or obedient and often clueless AI in instruct, serve, repeat dynamics?
Saqr, M., Misiejuk, K., & López‐Pernas, S. (2026) · The Internet and Higher Education · doi:10.1016/j.iheduc.2026.101087 · Cited by 9
Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics PerspectivePreprint
Wang, M., Wu, J., Ma, S., Li, N., Zhang, P., Gu, N., & Lu, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.10918 · Cited by 8
Kianpisheh, M., Mariakakis, A. T., & Truong, K. N. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3643500 · Cited by 8
CLARA: AI-Mediated Facilitation for Enhancing Group Cognition and Cohesion in Remote Collaboration
Gunasekaran, T. S., Doosti, M. S., Gupta, K., Bai, H., Pai, Y. S., & Billinghurst, M. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3786325 · Cited by 5
Benner, D., Rauch, J., Janson, A., & Leimeister, J. M. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719957 · Cited by 4
Wong, A., Spyrou, J., & Fredericks, J. (2024) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3726986.3727047 · Cited by 6
Wen, Q., Kochhar, P., Zeyada, S., Javaheri, T., & Rawassizadeh, R. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2561760 · Cited by 5
Wang, R., Zhou, X., Qiu, L., Chang, J. C., Bragg, J., & Zhang, A. X. (2025) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3715070.3757230 · Cited by 3
Co-designing Robot Dogs with and for Neurodivergent Individuals: Opportunities and Challenges
Kong, H., Xie, D., Chandra, A., Lowy, R., Maignan, A. F., Ha, S., Park, C. H., & Kim, J. G. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675603 · Cited by 6
Self-citationDesigning AI-Infused Interactive Systems for Online Communities: A Systematic Literature Review
Zhang, Y., Wang, X., Hu, J., Pan, Z., Peng, Z., & Ma, X. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816920 · Cited by 2
Exploring Mediation by an Embodied Virtual Agent in Immersive Triadic Collaborative Decision-Making
Han, B., Dong, Z., Zhang, J., Wen, R., Hirai, T., Clark, A., & Piumsomboon, T. (2026) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2026.3679103 · Cited by 3
Characterizing User-Reported Risks across LLM Chatbots
Li, L., Ma, R., Xue, Z., & Xiong, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791505 · Cited by 3
Potential Pitfalls of False Positives
Dey, I., Gnesdilow, D., Passonneau, R. J., & Puntambekar, S. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64315-6_45 · Cited by 4
A Framework for AI-Supported Mediation in Community-based Online CollaborationPreprint
Cho, S., Zachry, M. R., & McDonald, D. W. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.10015 · Cited by 2
Zhang, Z.-X., Wu, R.-L., Ren, F.-Y., Turaga, S., & Mendel, T. (2026) · arXiv.org · doi:10.48550/arxiv.2603.16008 · Cited by 3
Nixon, N., Choi, J., de Bastos, P. M., Samadi, M. A., Park, S., & Jaquay, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.27179 · Cited by 2
WikiCoach: Scaffolding Learning for Novices in Online Apprenticeship Communities
Zhou, M., Zhao, M., & Terveen, L. (2026) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3774935.3806185 · Cited by 1
How to Convey Resilience: Towards a Taxonomy for Conversational Agent Breakdown Recovery Strategies
Feng, S. (2025) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-80119-8_18 · Cited by 1
Enhancing Adaptive Human-Agent Teaming Systems with Functional Near-Infrared Spectroscopy
Eloy, L., Doherty, E. J., Spencer, C. A., & Hirshfield, L. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2524490 · Cited by 1
Politeness cannot make up for robots’ errors
Kumar, S., Itzhak, E., Tractinsky, N., Nimrod, G., Sarne‐Fleischmann, V., & Edan, Y. (2026) · Frontiers in Robotics and AI · doi:10.3389/frobt.2026.1760163
PolyTalk: A Configurable Prompt-Based, LLM-Powered Polyadic Chatbot
Yohannes, H. T., Kuhail, M. A., Yohanes, B. T., & Belay, M. H. (2025) · Student Research Conference on Applied Computing (SRC) · doi:10.1109/src65875.2025.11263704
The Postdiction as an approch to recommand commands based on interface state evolution
Bailly, G., Ali, N. A., Zhuang, C., Xia, A., & Gori, J. (2025) · Conference on l'Interaction Homme-Machine (IHM) · doi:10.1145/3765712.3765732
C-A2Meet: Malleable, Role-Aware AI Interfaces for Video Conferencing
Gunasekaran, T. S., Feng, Y., Bai, H., Pai, Y. S., & Billinghurst, M. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798532
Wan, E., Mougenot, C., & Sadek, M. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816292
Liang, H., Liu, Y., Yang, J., Wang, X., Wong, F. Y. M., Zhao, K., & Fan, M. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2706908
Bounds on user repair after context loss in AI handoffs
Rönnbäck, L. (2026) · Discover Informatics · doi:10.1007/s44564-026-00035-w
Metadata
- Authors
- Do, H. J., Kong, H.-K., Tetali, P., Lee, J., & Bailey, B. P.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Seattle University (US)
- University of Washington (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2023-04-14
- Volume
- 7
- Issue
- CSCW1
- Pages
- 1-23
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4366003147
- Cited by
- 32
- References
- 74
- Citations by year
- 2026: 12 · 2025: 12 · 2024: 8
- FWCI
- 9.47
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- chatbots, conversational repair, false positives, false negative errors, user acceptance, group participation, decision-making groups, precision-recall trade-off, trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.