CoChat

My Bad! Repairing Intelligent Voice Assistant Errors Improves Interaction

Cuadra, Andrea, Li, Shuran, Lee, Hansol, Cho, Jason, Ju, Wendy · 2021 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.

Abstract

One key technique people use in conversation and collaboration is conversational repair. Self-repair is the recognition and attempted correction of one's own mistakes. We investigate how the self-repair of errors by intelligent voice assistants affects user interaction. In a controlled human-participant study (N =101), participants asked Amazon Alexa to perform four tasks, and we manipulated whether Alexa would "make a mistake'' understanding the participant (for example, playing heavy metal in response to a request for relaxing music) and whether Alexa would perform a correction (for example, stating, "You don't seem pleased. Did I get that wrong?'') We measured the impact of self-repair on the participant's perception of the interaction in four conditions: correction (mistakes made and repair performed), undercorrection (mistakes made, no repair performed), overcorrection (no mistakes made, but repair performed), and control (no mistakes made, and no repair performed). Subsequently, we conducted free-response interviews with each participant about their interactions. This study finds that self-repair greatly improves people's assessment of an intelligent voice assistant if a mistake has been made, but can degrade assessment if no correction is needed. However, we find that the positive impact of self-repair in the wake of an error outweighs the negative impact of overcorrection. In addition, participants who recently experienced an error saw increased value in self-repair as a feature, regardless of whether they experienced a repair themselves.

Overview

Integrity signals

Registry

Crossref · 10.1145/3449101

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 61 references

Research works printed but not linked: 11

  1. Susan E Brennan et almbox. 2005. How conversation is shaped by visual and spoken evidence. Approaches to studying world-situated language use: Bridging the language-as-product and language-as-action traditions (2005), 95--129.
  2. 10.5555/371552.371555 · doi:10.5555/371552.371555
  3. The Economist. 2017. Terry Winograd: Where Humans still Beat Computers. The Economist (Jan 2017).
  4. Yuan Fan and Qiuchen Wang. 2013. Robot. US Patent App. 29/431,926.
  5. Petra Gieselmann. 2006. Comparing error-handling strategies in human-human and human-robot dialogues. In Proc. 8th Conf. Nat. Language Process.(KONVENS). Konstanz, Germany. 24--31.
  6. 10.5555/974305.974334 · doi:10.5555/974305.974334
  7. Harvey Sacks, Emanuel A Schegloff, and Gail Jefferson. 1978. A simplest systematics for the publisher of turn taking for conversation. In Studies in the publisher of conversational interaction. Elsevier, 7--55.
  8. Emanuel A Schegloff. 1997 a. Practices and actions: Boundary cases of other-initiated repair. Discourse processes, Vol. 23, 3 (1997), 499--545.
  9. Emanuel A Schegloff. 1997 b. Third turn repair. AMSTERDAM STUDIES IN THE THEORY AND HISTORY OF LINGUISTIC SCIENCE SERIES 4 (1997), 31--40.
  10. 10.1353/lan.1977.0041 · doi:10.1353/lan.1977.0041 · Paper page
  11. Bo Zhang, Qingsheng Cai, Jianfeng Mao, Eric Chang, and Baining Guo. 2001. Spoken dialogue management as planning and acting under uncertainty. In Seventh European conference on speech communication and technology.

Cited by

Metadata

Authors
Cuadra, A., Li, S., Lee, H., Cho, J., & Ju, W.
Affiliations
  • Cornell Tech (US)
  • Tongji University (CN)
  • Cornell University (US)
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2021-04-13
Volume
5
Issue
CSCW1
Pages
1-24
ISSN
2573-0142 (ISSN-L)
OpenAlex
W3159016082
Cited by
52
References
60
Citations by year
2026: 11 · 2025: 10 · 2024: 15 · 2023: 13 · 2022: 3
FWCI
2.74
Type
journal article
Language
en
Open access
closed
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
Funders
Mozilla Foundation
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
voice assistants, conversational repair, self-healing, error correction, overcorrection, undercorrection, Amazon Alexa, voice user interface design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.