The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index; 19 of the 68 sources its claims cite were checked for support.
A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures
Open accessAbstract
Despite huge gains in performance in natural language understanding via large language models in recent years, voice assistants still often fail to meet user expectations. In this study, we conducted a mixed-methods analysis of how voice assistant failures affect users’ trust in their voice assistants. To illustrate how users have experienced these failures, we contribute a crowdsourced dataset of 199 voice assistant failures, categorized across 12 failure sources. Relying on interview and survey data, we find that certain failures, such as those due to overcapturing users’ input, derail user trust more than others. We additionally examine how failures impact users’ willingness to rely on voice assistants for future tasks. Users often stop using their voice assistants for specific tasks that result in failures for a short period of time before resuming similar usage. We demonstrate the importance of low stakes tasks, such as playing music, towards building trust after failures.
Overview
CoChat Summary
## TL;DR
This mixed-methods study shows that not all voice assistant failures damage trust equally: response failures and some perception failures hurt trust more, while ambiguity- and spurious-trigger-related failures are less damaging. The authors also find that people often rebuild trust by returning first to low-stakes tasks like playing music, and they release a dataset of 199 real-world failures.
## Key findings
- The authors crowdsourced a dataset of 199 voice assistant failures from 107 unique crowd workers, spanning 12 failure categories.
- In the survey, failure type significantly affected trust in all three dimensions: ability (F(3, 2656.78) = 17.17, p < .001), benevolence (F(3, 2711.23) = 8.87, p < .001), and integrity (F(3, 2772.09) = 20.56, p < .001).
- Response failures were most damaging overall; compared with attention failures, they reduced trust in ability (m = 43.6), benevolence (m = 52.5), and integrity (m = 57.3). Within response failures, incorrect action was the reference and was described as the most damaging to integrity.
- Some failures were less harmful: ambiguity maintained the highest trust, with higher confidence in ability (m = 62.6) and benevolence (m = 67.9); spurious triggers were also relatively benign, with higher confidence in ability (m = 57.9) and benevolence (m = 64.2) than most other failure types.
- Users often paused use for the failed task, then resumed later; trust was often rebuilt through simple tasks such as playing music, and future-task trust was highest for playing a song (m = 3.29), lower for texting a coworker (m = 2.34), and lowest for transferring money (m = 1.56).
## Methods
- Mixed-methods design: the paper combined 12 remote interviews, a crowdsourced failure dataset, and a survey to study how voice assistant failures affect trust and future intended use.
- Dataset collection: 107 U.S.-based Mechanical Turk workers each submitted up to three failures; after cleaning and re-coding, the finalized dataset contained 199 failures across 12 categories. The authors adapted a taxonomy from Hong et al. and had two raters iteratively code 101 submitted failures with 70% interrater agreement.
- Survey and analysis: the survey used 268 Mechanical Turk participants and presented 12 failure scenarios drawn from the dataset. The main analysis used linear mixed models / mixed-linear regressions and ordinal mixed models, with participant ID as a random effect and general trust tendency as a control.
## Limitations
- The paper states that the study relied on retrospective recall for the dataset and interviews rather than in-situ observation, which introduces recall bias.
- The survey used hypothetical scenarios, not observed interactions; the authors note that future work could use ESM or diary studies to capture effects more directly.
- The sample consisted of frequent voice assistant users, who may forgive errors more easily than other populations; this may limit generalizability.
- The authors did not study embodied conversational agents or text-based conversational agents, so the findings may not generalize to those interfaces.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
7 Valid · 1 Not valid · 30 Not checked · 130 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“People generally expect to have functional conversations with voice assistants [11].”
What the sources say
- Supportedcites [11] [11] Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19) . Association for Computing Machinery, New York, NY, USA, 1-12. https://doi.org/10.1145/3290605.3300705 - source
The Discussion (p. 8) says participants conceptualised conversation with agents in almost purely transactional terms and described its qualities in highly functional terms. This supports the paraphrase that people generally expect functional conversations with voice assistants. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Standard NLP models are often evaluated on large datasets of coherent text-based questions and answers [48, 49] or paired written dialogue [68].”
What the sources say
- Supportedcites [48] [48] Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't know: Unanswerable questions for SQuAD. arXiv preprint arXiv:1806.03822 (2018). - source
The Introduction describes a large dataset of questions paired with text paragraphs, and §5.1 describes evaluating three model architectures on it. This citation is attached to the question-and-answer dataset portion of the sentence; the paired-dialogue portion has a separate citation. (LLM, conf=0.90)
- Supportedcites [49] [49] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016). - source
The citation is attached to the question-and-answer dataset portion of the sentence, not to paired written dialogue. The Introduction (p. 1) describes SQuAD as 107,785 question-answer pairs based on Wikipedia passages, and the Introduction (p. 2) reports model performance on it. (LLM, conf=0.90)
- Supportedcites [68] [68] Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243 (2018). source
Citation [68] is attached to the paired-written-dialogue portion of the sentence, not the questions-and-answers portion. The Introduction describes a dataset of 164,356 utterances from paired crowdworkers, and §3.4 Evaluation calls predicting the next utterance from dialogue history a standard dialogue task. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“We found that existing NLP datasets do not cover the wide breadth of possible conversational failure cases due to their emphasis on correct data for training.”
What the sources say
- Partly supportedcites [48] [48] Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't know: Unanswerable questions for SQuAD. arXiv preprint arXiv:1806.03822 (2018). - sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) reports that some reading-comprehension datasets exclude unanswerable examples and that other negative examples can be easy to identify. Section 3.3 (page unavailable) says multiple-choice datasets may be less suited to training user-facing systems. These passages support a narrower gap in question-answering datasets, but not the claim about the breadth of conversational failure cases across NLP datasets or its stated cause of emphasizing correct training data. (LLM, conf=0.82)
- Could not verifycites [49] [49] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016). - sourceExperimental — accuracy not yet measured
Section 2 (pp. 2–3) surveys limitations of existing reading-comprehension and question-answering datasets, but does not discuss conversational failure cases. Section 3 (p. 4) describes collecting answerable question–answer pairs; it does not say that an emphasis on correct training data causes existing NLP datasets to miss a broad range of conversational failures. Searches for conversational failures, incorrect answers, and training-data limitations found no passage making that claim. (LLM, conf=0.74)
- Partly supportedcites [50] [50] Siva Reddy, Danqi Chen, and Christopher D Manning. 2019. Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics 7 (2019), 249-266. - sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says existing reading-comprehension datasets lack conversational phenomena such as coreference and pragmatic reasoning, and the Introduction (p. 2) says large-scale datasets lack questions dependent on conversation history. These passages support a narrower gap in dataset coverage, but they do not establish a wide breadth of conversational failure cases or attribute the gap to an emphasis on correct training data. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [25] [25] Karthik Gopalakrishnan, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, and Dilek Hakkani-Tür. 2019. Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations. In Proc. Interspeech 2019 . 1891-1895. https://doi.org/10.21437/Interspeech.2019-3079 - source
- Could not verifycites [57] [57] Yueming Sun and Yi Zhang. 2018. Conversational Recommender System. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (Ann Arbor, MI, USA) (SIGIR '18) . Association for Computing Machinery, [New York, NY, USA, 235-244. https://doi.org/10.1145/3209978.3210002](https://doi.org/10.1145/3209978.3210002) - sourceExperimental — accuracy not yet measured
Our copy of Sun, Zhang & Chen's CRM paper never addresses the coverage of conversational failure cases in existing NLP datasets. The nearest passages are §4.4 (p. 6) — the Yelp dataset has no dialogue utterances and 'to the best of our knowledge, there is no such available dataset,' so they crowd-sourced utterances assuming 'users are cooperative and always inform the facet values of the target' — and §4.2 (p. 5), which notes that launching an untrained dialogue system to real users 'is likely to fail' and that simulated users are the common bootstrap. §2.1 (p. 2) only observes that prior work 'focus on NLP challenges instead of commercial success metrics.' Searched for: conversational failure cases, error/noisy utterance coverage, dataset limitations, related-work dataset coverage — no passage states the attributed claim, and none contradicts it. (LLM, conf=0.70)
- Partly supportedcites [68] [68] Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243 (2018). sourceExperimental — accuracy not yet measured
The Introduction and Related Work say existing dialogue datasets have limitations that contribute to problems such as inconsistent personality and weak long-term coherence. They do not say that NLP datasets fail to cover the wide breadth of possible conversational failure cases, or attribute such a gap to an emphasis on correct training data. Page numbers are unavailable in our copy. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Relying on interview and survey data, we find that certain failures, such as those due to overcapturing users' input, derail user trust more than others.”
- Also stated in: § 1 INTRODUCTION · plain
“We found that different types of voice assistant failures have a differential impact on trust.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 INTRODUCTION · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To illustrate how users have experienced these failures, we contribute a crowdsourced dataset of 199 voice assistant failures, categorized across 12 failure sources.”
- Also stated in: § 1 INTRODUCTION · plain
“C1: Iterating on the existing taxonomy of NLP failures, we crowdsource a dataset of 199 failures users have experienced across 12 different sources of failure.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 INTRODUCTION · plain
111 more claims
Create a free account to see all 168 claims and to verify the 13 references still open.
- Probably not a substantial claim. 30
- It may state two separate claims as one. 27
- The quote only partly states this claim. 5
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Qualitative Inquiry and Research Design: Choosing Among Five Approaches
Lewis, S. (2015) · Health Promotion Practice · doi:10.1177/1524839915580941 · Cited by 22,105
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 3×
“In the case of question answering, state-of-art models perform nearly at human-level for reading comprehension on standard benchmarks collected from Wikipedia [49].”
BackgroundClark, H. H. (1996) · doi:10.1017/cbo9780511620539 · Cited by 5,123
Research works printed but not linked: 2
- 10.18653/v1 · doi:10.18653/v1
- 10.5555/3524938.3525579 · doi:10.5555/3524938.3525579
Cited by
Trust in AI chatbots: A systematic review
Ng, S. W. T., & Zhang, R. (2025) · Telematics and Informatics · doi:10.1016/j.tele.2025.102240 · Cited by 142
The Trust Recovery Journey. The Effect of Timing of Errors on the Willingness to Follow AI Advice.
Kahr, P. K., Rooks, G., Snijders, C., & Willemsen, M. C. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645167 · Cited by 21
User acceptance of AI voice assistants in Jordan’s telecom industry
Al-kfairy, M., Mustafa, D., Al-Adaileh, A., Zriqat, S., & Sendaba, O. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100521 · Cited by 24
LLM-mediated domain-specific voice agents: the case of TextileBot
Zhong, S., Gatti, E., Hardwick, J., Ribul, M., Cho, Y., & Obrist, M. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2456667 · Cited by 22
“The human-computer interaction (HCI) community is seeing a rising emphasis on the development of voice-based conversational agents [95, 9, 94, 40].”
BackgroundNLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMs
Antoniak, M., Naik, A., Alvarado, C. S., Wang, L. L., & Chen, I. Y. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658982 · Cited by 51
Metadata
- Authors
- Baughan, A., Wang, X., Liu, A., Mercurio, A., Chen, J., & Ma, X.
- Affiliations
- University of Washington (US)
- Google (United States) (US)
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-16
- OpenAlex
- W4322831937
- Cited by
- 32
- References
- 67
- Citations by year
- 2026: 11 · 2025: 14 · 2024: 7
- FWCI
- 5.67
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- voice assistants, user trust, task abandonment, music playback, large language models, natural language understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures. https://app.cochat.ai/papers/10.1145/3544548.3581152
- The paper
- Baughan, A., Wang, X., Liu, A., Mercurio, A., Chen, J., & Ma, X. (2023). A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581152
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.