Experiential Qualities of Whispering with Voice Assistants
Open accessThe DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
We present a Research through Design project that explores how whispering influences the ways people experience and interact with voice assistants. The research project includes a co-speculation workshop and the use of a design probe, which culminated in the production of a design fiction short film. Our design-led inquiry contributes with experiential qualities of whispering with voice assistants: creepiness, trust, and intimacy. Furthermore, we present how whispering opens up new dimensions of how and when voice interaction could be used. We propose that designers of whispering voice assistants should reflect on how they facilitate the experiential qualities of creepiness, trust, and intimacy, and reflect on the potential challenges whispering brings to the relation between a user and a voice assistant.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As voice assistants only respond to one person at a time, this can cause interpersonal tensions as well as difficulty operating the device if it is receiving conflicting commands [60].”
From Amazon Echo Show as a Multimodal Human-to-Human Care Support Tool within Self-Isolating Older UK Households (2022) · cites this paper as [60]
In the citing paper
“The Echo Show also encouraged a higher level of scrutiny and crosschecking, which has previously been observed with other smart home technology and VAs [60].”
From Amazon Echo Show as a Multimodal Human-to-Human Care Support Tool within Self-Isolating Older UK Households (2022) · cites this paper as [60]
Citations
Cites
Hall, E. T. (1973) · Leonardo · doi:10.2307/1572461 · Cited by 4,859 · cited 2×
“Research has focused on exploring systems that act based on user movements and proximity [16, 2], as well as systems that could identify emotions such as anxiety and respond to it accordingly [41].”
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 2×
“Voice assistants are also social actors [35] and contribute to enjoyment in the home, as people are having "fun" with their voice assistants; making jokes, dancing, impressing guests, and perceiving them as "best friends" and "helping hosts" [51].”
What should we expect from research through design?
Gaver, W. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208538 · Cited by 1,123 · cited 2×
“We follow a research through design methodology [12] to allow the research a holistic and experience-based approach based on design-led inquiry.”
BackgroundMethodFeeling robots and human zombies: Mind perception and the uncanny valley
Gray, K., & Wegner, D. M. (2012) · Cognition · doi:10.1016/j.cognition.2012.06.007 · Cited by 941 · cited 1×
“Interacting with a voice assistant through whispering thus prompts attributions of human-likeness [9], and as human bodies are seldom ascribed to robots [13], this mismatch causes a feeling of creepiness [14].”
BackgroundVoice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 2×
“In HCI, there is an increasing amount of research done on how people use and understand voice assistants [39, 48].”
BackgroundProxemic interaction: designing for a proximity and orientation-aware environment
Ballendat, T., Marquardt, N., & Greenberg, S. (2010) · ACM International Conference on Interactive Tabletops and Surfaces · doi:10.1145/1936652.1936676 · Cited by 413
Music, Search, and IoT: How People (Really) Use Voice Assistants
Ammari, T., Kaye, J., Tsai, J. Y., & Bentley, F. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3311956 · Cited by 356
"Hey Google is it OK if I eat you?": Initial Explorations in Child-Agent Interaction
Druga, S., Williams, R., Breazeal, C., & Resnick, M. (2017) · Proceedings of the 2017 Conference on Interaction Design and Children · doi:10.1145/3078072.3084330 · Cited by 330 · cited 1×
“The "fun" of voice assistants are also reflected in studies on children’s use of voice assistants [7].”
Background"Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage
Sciuto, A., Saini, A., Forlizzi, J., & Hong, J. I. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196772 · Cited by 327 · cited 5×
“Research has shown that the use of voice assistants is often initiated with excitement and exploration, after which the limitations slowly come across and a few use cases turn into routines [48].”
BackgroundInfluentialLeakiness and creepiness in app space: perceptions of privacy and mobile app use
Shklovski, I., Mainwaring, S. D., Skúladóttir, H. H., & Borgthorsson, H. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557421 · Cited by 284
Does computer-generated speech manifest personality? an experimental test of similarity-attraction
Nass, C., & Lee, K. M. (2000) · Proceedings of the SIGCHI conference on Human Factors in Computing Systems · doi:10.1145/332040.332452 · Cited by 242 · cited 1×
“The primary design materials of a voice assistant include the computer generated speech [34, 35, 52], the perceived persona or personality of the assistant [26], the algorithms executing the "conversation" and informational data accessed and stored through the Internet.”
BackgroundWallace, J., McCarthy, J., Wright, P. C., & Olivier, P. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466473 · Cited by 240 · cited 1×
“To explore how people’s relationship with a voice assistant would develop over time through whispering, we made a de-sign probe [55] (Figure 5) based on the insights from the workshops, such as concepts and sketches (Figure 3-4).”
MethodA Theory of Creepy: Technology, Privacy and Shifting Social Norms
Tene, O., & Polonetsky, J. (2013) · Yale Journal of Law and Technology · Cited by 209
Immodest Proposals: Research Through Design and Knowledge
Bardzell, J., Bardzell, S., & Hansen, L. K. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702400 · Cited by 196
The Design of Implicit Interactions: Making Interactive Systems Less Obnoxious
Ju, W., & Leifer, L. (2008) · Design Issues · doi:10.1162/desi.2008.24.3.72 · Cited by 177 · cited 2×
“Voice assistants are part of current conceptual visions of future technologies, such as faceless interaction [20] and implicit interaction [22].”
BackgroundStrengers, Y., Kennedy, J., Arcari, P., Nicholls, L., & Gregg, M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300875 · Cited by 164
Redström, J. (2017) · The MIT Press eBooks · doi:10.7551/mitpress/11160.001.0001 · Cited by 154
Design fictional interactions: why HCI should care about stories
Tanenbaum, T. J. (2014) · interactions · doi:10.1145/2648414 · Cited by 153
Living Without a Mobile Phone: An Autoethnography
Lucero, A. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196731 · Cited by 136
Intimate Futures: Staying with the Trouble of Digital Personal Assistants through Design Fiction
Søndergaard, M. L. J., & Hansen, L. K. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196766 · Cited by 128 · cited 6×
“We build on design fiction research with voice assistants focusing on social dimensions, such as work by Søndergaard and Hansen that explored how voice assistants are gendered [50], and Rogers et al’s work on a healthier voice-enabled Internet [45].”
BackgroundMethodInfluentialSelf-citationIntimate Heartbeats: Opportunities for Affective Communication Technology
Janssen, J. H., Bailenson, J. N., IJsselsteijn, W. A., & Westerink, J. H. D. M. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.13 · Cited by 115
Smart Home Security Cameras and Shifting Lines of Creepiness: A Design-Led Inquiry
Pierce, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300275 · Cited by 114
Research Fiction: Storytelling, Plot and Design
Blythe, M. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026023 · Cited by 108
Anxiety-based affective communication for implicit human–machine interaction
Rani, P., Sarkar, N., & Adams, J. A. (2007) · Advanced Engineering Informatics · doi:10.1016/j.aei.2006.11.009 · Cited by 93 · cited 1×
“Research has focused on exploring systems that act based on user movements and proximity [16, 2], as well as systems that could identify emotions such as anxiety and respond to it accordingly [41].”
BackgroundVoice as a Design Material: Sociophonetic Inspired Design Strategies in Human-Computer Interaction
Sutton, S. J., Foulkes, P., Kirk, D., & Lawson, S. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300833 · Cited by 92
Yip, J. C., Sobel, K., Gao, X., Hishikawa, A. M., Lim, A., Meng, L., Ofiana, R. F., Park, J., & Hiniker, A. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300303 · Cited by 88
Alternative Avenues for IoT: Designing with Non-Stereotypical Homes
Desjardins, A., Viny, J. E., Key, C., & Johnston, N. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300581 · Cited by 69
Intimacy and embodiment: implications for art and technology
Fels, S. (2000) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/357744.357749 · Cited by 68
Understanding Fidget Widgets: Exploring the Design Space of Embodied Self-Regulation
Karlesky, M., & Isbister, K. (2016) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2971485.2971557 · Cited by 66
Whispering: the murmur of power in a lo-fi world
Li, X. (2011) · Media Culture & Society · doi:10.1177/0163443710385498 · Cited by 56
Lee, S., Kim, S., & Lee, S. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312796 · Cited by 45 · cited 1×
“The primary design materials of a voice assistant include the computer generated speech [34, 35, 52], the perceived persona or personality of the assistant [26], the algorithms executing the "conversation" and informational data accessed and stored through the Internet.”
BackgroundKarlesky, M., & Isbister, K. (2014) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/2540930.2540978 · Cited by 41
Faceless Interaction—A Conceptual Examination of the Notion of Interface: Past, Present, and Future
Janlert, L.-E., & Stolterman, E. (2014) · Human-Computer Interaction · doi:10.1080/07370024.2014.944313 · Cited by 35
Videos of Things: Speculating on, Anticipating and Synthesizing Technological Mediations
Oogjes, D., & Wakkary, R. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025748 · Cited by 34
Digital Selves: Devices for intimate communications between homes
Grivas, K. N. (2006) · Personal and Ubiquitous Computing · doi:10.1007/s00779-005-0003-1 · Cited by 27 · cited 1×
“In HCI, intimacy has been concerned with physical together-ness [15].”
BackgroundConversation considered harmful?
Reeves, S. (2019) · Proceedings of the 1st International Conference on Conversational User Interfaces · doi:10.1145/3342775.3342796 · Cited by 26 · cited 2×
“Rather, instead of talking about "conversational design", Reeves proposes "conversation-sensitive design" pointing to how CUIs fit into and around conversation [44].”
BackgroundA need for trust in conversational interface research
Edwards, J., & Sanoubari, E. (2019) · Proceedings of the 1st International Conference on Conversational User Interfaces · doi:10.1145/3342775.3342809 · Cited by 22 · cited 3×
“Recently, it was highlighted that trust is a critical part of voice interaction, calling for measures on how researchers evaluate trust [8].”
BackgroundMurray‐Smith, R., Williamson, J. H., Hughes, S., Quaade, T., & Strachan, S. (2008) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1358628.1358683 · Cited by 12 · cited 1×
“The design of the probe has similarities to Stane by Murray-Smith et al. [33], a hand-held interaction device controlled by tactile input.”
MethodOur Friends Electric: Reflections on Advocacy and Design Research for the Voice Enabled Internet
Rogers, J., Clarke, L., Skelly, M., Taylor, N., Thomas, P., Thorne, M., Larsen, S., Odrozek, K., Kloiber, J., Bihr, P., et al. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300344 · Cited by 6
Fidget widgets: designing for the physical margins of digital workspaces
Karlesky, M., & Isbister, K. (2014) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/2540930.2559982 · Cited by 6 · cited 1×
“Previous studies show how engaging users’ interrelated bodily motions with, for example, fiddling properties can enhance focus and calm sensation [23, 24, 25].”
BackgroundAcoustical implicit communication in human-robot interaction
Lin, Y. H., Le, Z., Becker, E., & Makedon, F. S. (2010) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/1839294.1839300 · Cited by 6 · cited 1×
“Consequently, interaction with a voice assistant is processed through the literal meaning of the words while discarding factors such as voice modalities, body language and context of speech; factors that are vital for human-human interaction [28].”
Background
Research works printed but not linked: 12
- Herbert H Clark. 1996. Using language. Cambridge university press.
- Nicholas Epley, Adam Waytz, and John T Cacioppo. 2007. On seeing human: a three-factor theory of anthropomorphism. Psychological review 114, 4 (2007), 864.
- Ed Finn. What Algorithms Want: Imagination in the Age of Computing (reprint edition ed.). MIT Press.
- Heather M Gray, Kurt Gray, and Daniel M Wegner. 2007. Dimensions of mind perception. science 315, 5812 (2007), 619--619.
- Edward T Hall, Ray L Birdwhistell, Bernhard Bock, Paul Bohannan, A Richard Diebold Jr, Marshall Durbin, Munro S Edmonson, JL Fischer, Dell Hymes, Solon T Kimball, and others. 1968. Proxemics [and comments and replies]. Current anthropology 9, 2/3 (1968), 83--108.
- Mizuki Hisaka. 2018. Amazon Alexa Will Now Whisper Back To You, But Some People Find It ?Creepy'. (2018). Retrieved Aug 20, 2019 from https://www.inquisitr.com/5123302/amazon-alexa-will-nowwhisper-back-to-you-but-some-people-find-it-creepy/
- Jonas Löwgren. 2007. Pliability as an experiential quality: Exploring the aesthetics of interaction design. Artifact: Journal of Design Practice 1, 2 (2007), 85--95.
- Trieuvy Luu, Martijn van den Broeck, and Marie Louise Juul Sondergaard. 2018. Data economy: interweaving storytelling and world building in design fiction. In 10th Nordic Conference on Human-Computer Interaction (NORDICHI). ACM Digital Library, 771--786.
- Copenhangen Pride, Virtue, Equal AI, Koalition Interactive, and thirtysoundsgood. 2019. Meet Q The First Genderless Voice. (2019). Retrieved June 4, 2019 from http://www.genderlessvoice.com
- Erika Rawes. 2018. How to Enable Whisper Mode on Alexa. (2018). Retrieved March 13, 2019 from https://www.digitaltrends.com/home/how-to-enablewhisper-mode-on-alexa/
- R. Murray Schafer. Soundscape: Our Sonic Environment and the Tuning of the World (original ed. edition ed.). Destiny Books.
- Albrecht Schmidt. 2000. Implicit human computer interaction through context. Personal technologies 4, 2--3 (2000), 191--199.
Cited by
Voice in Human–Agent Interaction: A Survey
Seaborn, K., Miyake, N. P., Pennefather, P., & Otake-Matsuura, M. (2022) · ACM Computing Surveys · doi:10.1145/3386867 · Cited by 249
“Non-human animal voices and biological voices, as well as musical voices; whispering to but also the whispers of a vocal agent [142].”
BackgroundCreepy Technology:What Is It and How Do You Measure It?
Woźniak, P. W., Karolus, J., Lang, F., Eckerth, C., Schöning, J., Rogers, Y., & Niess, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445299 · Cited by 84
“Studies describing interactions with voice assistants [59, 69] and with autonomous cars [14, 49, 50] found that both were perceived as creepy.”
BackgroundStill Creepy After All These Years:The Normalization of Affective Discomfort in App Use
Seberger, J. S., Shklovski, I., Swiatek, E., & Patil, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502112 · Cited by 97
“empirical investigations seem to use creepiness as an umbrella term for a variety of feelings of discomfort that arise when people use privacy-invasive technologies or are faced with data-disclosure situations [48, 50, 61, 69, 73].”
BackgroundTolmeijer, S., Zierau, N., Janson, A., Wahdatehagh, J. S., Leimeister, J. M. M., & Bernstein, A. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451623 · Cited by 85
“, [15, 22, 23, 32]), there is a lack of empirical insights on the perceptual efects of gendered (and”
BackgroundEmpowering Resignation: There’s an App for That
Seberger, J. S., Llavore, M., Wyant, N. N., Shklovski, I., & Patil, S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445293 · Cited by 62
“…in technological interactions includes: characterizing the intimacy of whispers when conversing with intelligent personal assistants (IPAs) [77]; developing a taxonomy of technology experiences considered creepy by children [113]; and situating the perceptions of smart home technologies in…”
Background
Metadata
- Authors
- Parviainen, E., & Søndergaard, M. L. J.
- Affiliations
- KTH Royal Institute of Technology (SE)
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-13
- OpenAlex
- W3029832204
- Semantic Scholar
- 6accb263ab22b50cc4dd8332e0f5d1b365e4cf55
- Cited by
- 52
- References
- 41
- Citations by year
- 2026: 4 · 2025: 11 · 2024: 6 · 2023: 9 · 2022: 13 · 2021: 8 · 2020: 1
- FWCI
- 5.66
- Type
- conference paper
- Language
- en
- Open access
- green
- urn.kb.se
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- Funders
- Swedish Foundation for Strategic Research project (2017-05133)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- whispering, voice assistants, voice interaction, creepiness, trust, intimacy, design fiction, design probes, experiential quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.