Designers Characterize Naturalness in Voice User Interfaces: Their Goals, Practices, and Challenges
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
This work investigates the practices and challenges of voice user interface (VUI) designers. Existing VUI design guidelines recommend that designers strive for natural human-agent conversation. However, the literature leaves a critical gap regarding how designers pursue naturalness in VUIs and what their struggles are in doing so. Bridging this gap is necessary for identifying designers’ needs and supporting them. Our interviews with 20 VUI designers identified 12 ways that designers characterize and approach naturalness in VUIs. We categorized these characteristics into three groupings based on the types of conversational context that each characteristic contributes to: Social, Transactional, and Core. Our results contribute new findings on designers’ challenges, such as a design dilemma in augmenting task-oriented VUIs with social conversations, difficulties in writing for spoken language, lack of proper tool support for imbuing synthesized voice with expressivity, and implications for developing design tools and guidelines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Machines and Mindlessness: Social Responses to Computers
Nass, C., & Moon, Y. (2000) · Journal of Social Issues · doi:10.1111/0022-4537.00153 · Cited by 4,022
Braun, V., Clarke, V., Hayfield, N., & Terry, G. (2019) · Handbook of Research Methods in Health Social Sciences · doi:10.1007/978-981-10-5251-4_103 · Cited by 2,910
Tacotron: towards end-to-end speech synthesis
Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1452 · Cited by 1,746 · cited 1×
“Hence, a demonstration-based prosody editing, leveraging our own voices [111], is a promising approach worth investigating.”
BackgroundEstablishing and maintaining long-term human-computer relationships
Bickmore, T. W., & Picard, R. W. (2005) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1067860.1067867 · Cited by 1,333 · cited 1×
“They investigate: a plethora of concerns such as ways to transfer human qualities onto machines [24, 80] and ways to maintain trust between users and computers [16, 97, 110], modeling human-computer relationships [1, 17], designing for diferent user groups such as older adults [98, 109], children [30, 87], understandi…”
BackgroundLuger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176 · cited 2×
“In these studies, various issues have been explored, including how VUIs fit into everyday settings [88], how users perceive social and functional roles in conversation [28], and the disparity between high user expectations and low system capability [63].”
BackgroundClarke, V., & Braun, V. (2014) · Encyclopedia of Critical Psychology · doi:10.1007/978-1-4614-5583-7_311 · Cited by 885
Voice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 5×
“Especially, given that modern voice assistants tend to be situated in complex and dynamic social settings [88], it is possible that conversational characteristics of a VUI considered natural in one setting are not perceived the same in another setting (e.g. an extremely human-like voice agent can…”
BackgroundInfluentialAre Machines Gender Neutral? Gender‐Stereotypic Responses to Computers With Voices
Nass, C., Moon, Y., & Green, N. (1997) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1997.tb00275.x · Cited by 670 · cited 1×
“They investigate: a plethora of concerns such as ways to transfer human qualities onto machines [24, 80] and ways to maintain trust between users and computers [16, 97, 110], modeling human-computer relationships [1, 17], designing for diferent user groups such as older adults [98, 109], children [30, 87], understandi…”
BackgroundRelational agents: a model and implementation of building user trust
Bickmore, T., & Cassell, J. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365304 · Cited by 479
Can computer personalities be human personalities?
Nass, C., Moon, Y., Fogg, B. J., Reeves, B., & Dryer, C. (1995) · Conference companion on Human factors in computing systems - CHI '95 · doi:10.1145/223355.223538 · Cited by 251
Designing SpeechActs: issues in speech user interfaces
Yankelovich, N., Levow, G.-A., & Marx, M. (1995) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '95 · doi:10.1145/223904.223952 · Cited by 250 · cited 2×
“With rapid advancements in Natural Language Understanding (NLU), VUI systems have transitioned from rudimentary speech-recognition based systems such as Audrey [32] and Harpy [62] in the 50s and 70s, to task-oriented systems like SpeechActs [117] in the 90s, and sophisticated conversational agents that are now pervasi…”
MethodSuede: a Wizard of Oz prototyping tool for speech user interfaces
Klemmer, S. R., Sinha, A. K., Chen, J., Landay, J. A., Aboobaker, N., & Wang, A. (2000) · Proceedings of the 13th annual ACM symposium on User interface software and technology - UIST '00 · doi:10.1145/354401.354406 · Cited by 230 · cited 2×
“’s SUEDE enabled Wizard-of-Oz style prototyping of VUIs [56].”
MethodDesigning Relational Agents as Long Term Social Companions for Older Adults
Vardoulakis, L. P., Ring, L., Barry, B., Sidner, C. L., & Bickmore, T. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33197-8_30 · Cited by 166
Conversational UX Design: A Practitioner's Guide to the Natural Conversation Framework
Research-Almaden, I., Moore, R. J., Arar, R., & Research-Almaden, I. (2019) · doi:10.1145/3304087 · Cited by 118 · cited 2×
“It seems that the strategies of our designers largely echo conversation repair and error recovery strategies suggested by the existing VUI design textbooks [29, 70].”
BackgroundMapping Perceptions of Humanness in Intelligent Personal Assistant Interaction
Doyle, P. R., Edwards, J., Dumbleton, O., Clark, L., & Cowan, B. R. (2019) · Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3338286.3340116 · Cited by 100 · cited 1×
“They investigate: a plethora of concerns such as ways to transfer human qualities onto machines [24, 80] and ways to maintain trust between users and computers [16, 97, 110], modeling human-computer relationships [1, 17], designing for diferent user groups such as older adults [98, 109], children [30, 87], understandi…”
BackgroundPyae, A., & Joelsson, T. N. (2018) · Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct · doi:10.1145/3236112.3236130 · Cited by 99
Voice assistants and older people: some open issues
Sayago, S., Neves, B. B., & Cowan, B. R. (2019) · Proceedings of the 1st International Conference on Conversational User Interfaces · doi:10.1145/3342775.3342803 · Cited by 97 · cited 1×
“…between users and computers [17, 97, 110], modeling human-computer relationships [3, 18], designing for different user groups such as older adults [98, 109], children [31, 87], understanding how users ‘view humanness in dialogue interaction’ [34], and examining stereotypes in this domain [27, 78].”
BackgroundTurtleTalk: An Educational Programming Game for Children with Voice User Interface
Jung, H., Kim, H. J., So, S., Kim, J., & Oh, C. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312773 · Cited by 33 · cited 1×
“For instance, a VUI in an interactive learning system can serve the role of an instructor [55], and not that of a task-oriented assistant nor a social companion.”
Background
Cited by
Pataranutaporn, P., Liu, R., Finn, E., & Maes, P. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00720-7 · Cited by 192
Biermann, O. C., Ma, N. F., & Yoon, D. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533506 · Cited by 116
“This implication poses design challenges in creating contextually adaptive AI agent, as foreshadowed in [50].”
BackgroundSelf-citationMetaphors in Voice User Interfaces: A Slippery Fish
Desai, S., & Twidale, M. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3609326 · Cited by 42
Chin, J., Desai, S., Lin, S. (.-H., & Mejia, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637365 · Cited by 27
“Moreover, practitioners and designers of voice interfaces consider the level of formality to be one of the most crucial design decisions [43].”
BackgroundWhat's The Talk on VUI Guidelines? A Meta-Analysis of Guidelines for Voice User Interface Design
Murad, C., Candello, H., & Munteanu, C. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3597129 · Cited by 36
Metadata
- Authors
- Kim, Y., Reza, M., McGrenere, J., & Yoon, D.
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-13
- OpenAlex
- W3015588689
- Cited by
- 42
- References
- 19
- FWCI
- 3.46
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.