CoChat

To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions

Open access
Do, Hyo Jin, Kong, Ha-Kyung, Tetali, Pooja, Lee, Jaewook, Bailey, Brian P. · 2023 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.

Abstract

Conversational agents (CAs) can analyze online conversations using natural language techniques and effectively facilitate group discussions by sending supervisory messages. However, if a CA makes imperfect interventions, users may stop trusting the CA and discontinue using it. In this study, we demonstrate how inaccurate interventions of a CA and a conversational repair strategy can influence user acceptance of the CA, members' participation in the discussion, perceived discussion experience between the members, and group performance. We built a CA that encourages the participation of members with low contributions in an online chat discussion in which a small group (3-6 members) performs a decision-making task. Two types of errors can occur when detecting under-contributing members: 1) false-positive (FP) errors happen when the CA falsely identifies a member as under-contributing and 2) false-negative (FN) errors occur when the CA misses detecting an under-contributing member. We designed a conversational repair strategy that gives users a chance to contest the detection results and the agent sends a correctional message if an error is detected. Through an online study with 175 participants, we found that participants who received FN error messages reported higher acceptance of the CA and better discussion experience, but participated less compared to those who received FP error messages. The conversational repair strategy moderated the effect of errors such as improving the perceived discussion experience of participants who received FP error messages. Based on our findings, we offer design implications for which model should be selected by practitioners between high precision (i.e., fewer FP errors) and high recall (i.e., fewer FN errors) models depending on the desired effects. When frequent FP errors are expected, we suggest using the conversational repair strategy to improve the perceived discussion experience.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3579532

Cites

76 of 76 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 76 references

Research works printed but not linked: 19

  1. Martin Adam, Michael Wessel, and Alexander Benlian. 2020. AI-based chatbots in customer service and their effects on user compliance. Electronic Markets (2020), 1-19.
  2. Saul Albert and Jan P de Ruiter. 2018. Repair: the interface between interaction and cognition. Topics in cognitive science 10, 2 (2018), 279-313.
  3. Dennis Benner, Edona Elshan, Sofia Schöbel, and Andreas Janson. 2021. What do you mean? a review on recovery strategies to overcome conversational breakdowns of conversational agents. In International Conference on Information Systems (ICIS).
  4. Anthony D Bergstrom. 2011. Social mirrors: visualization as conversation feedback. Ph.D. Dissertation. University of Illinois at Urbana-Champaign.
  5. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77-101.
  6. The Concord Consortium. 2017. Teaching Teamwork. Retrieved Jan 22, 2021 from https://learn.concord.org/resources/ 565/teaching-teamwork-adder
  7. Fred D Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly (1989), 319-340.
  8. Gerardine DeSanctis and R Brent Gallupe. 1987. A foundation for the study of group decision support systems. Management science 33, 5 (1987), 589-609.
  9. James Fishkin, Nikhil Garg, Lodewijk Gelauff, Ashish Goel, Kamesh Munagala, Sukolsak Sakshuwong, Alice Siu, and Sravya Yandamuri. 2018. Deliberative democracy with the Online Deliberation platform. In The 7th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2019). HCOMP.
  10. Asbjørn Følstad and Cameron Taylor. 2019. Conversational repair in chatbots for customer service: the effect of expressing uncertainty and suggesting alternatives. In International Workshop on Chatbot Research and Design. Springer, 201-214.
  11. Taemie Kim, Agnes Chang, Lindsey Holland, and Alex Sandy Pentland. 2008. Meeting mediator: enhancing group collaborationusing sociometric feedback. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. 457-466.
  12. Ben Lafreniere, Tanya R. Jonker, Stephanie Santosa, Mark Parent, Michael Glueck, Tovi Grossman, Hrvoje Benko, and Daniel Wigdor. 2021. False Positives vs. False Negatives: The Effects of Recovery Time and Cognitive Costs on Input Error Preference. In The 34th Annual ACM Symposium on User Interface Software and Technology. 54-68.
  13. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159-174.
  14. Christopher H Morrell. 1998. Likelihood ratio testing of variance components in the linear mixed-effects model using restricted maximum likelihood. Biometrics (1998), 1560-1568.
  15. Sherry L Piezon and William D Ferree. 2007. Perceptions of social loafing in online learning groups. In 23 rd Annual Conference on Distance Teaching & Learning, http://www. uwex. edu.
  16. Rachel Quednau. 2016. Top Strong Citizen Discussions this Week. Retrieved Jan 22, 2021 from https://www.strongtowns. org/journal/2016/4/13/top-strong-citizens-discussions?rq=slack
  17. Sandhya Saisubramanian, Shannon C Roberts, and Shlomo Zilberstein. 2021. Understanding User Attitudes Towards Negative Side Effects of AI Systems. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1-6.
  18. James A Shepperd. 1993. Productivity loss in performance groups: A motivation analysis. Psychological bulletin 113, 1 (1993), 67.
  19. Sam R Wilson, William C Barley, Luisa Ruge-Jones, and Marshall Scott Poole. 2020. Tacking Amid Tensions: Using Oscillation to Enable Creativity in Diverse Teams. The Journal of Applied Behavioral Science (2020), 0021886320960245.

Cited by

See all 37

Metadata

Authors
Do, H. J., Kong, H.-K., Tetali, P., Lee, J., & Bailey, B. P.
Affiliations
  • University of Illinois Urbana-Champaign (US)
  • Seattle University (US)
  • University of Washington (US)
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2023-04-14
Volume
7
Issue
CSCW1
Pages
1-23
ISSN
2573-0142 (ISSN-L)
OpenAlex
W4366003147
Cited by
32
References
74
Citations by year
2026: 12 · 2025: 12 · 2024: 8
FWCI
9.47
Type
journal article
Language
en
Open access
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
chatbots, conversational repair, false positives, false negative errors, user acceptance, group participation, decision-making groups, precision-recall trade-off, trust in AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.