Voice as a Design Material: Sociophonetic Inspired Design Strategies in Human-Computer Interaction
Open accessAbstract
While there is a renewed interest in voice user interfaces (VUI) in HCI, little attention has been paid to the design of VUI voice output beyond intelligibility and naturalness. We draw on the field of sociophonetics - the study of the social factors that influence the production and perception of speech - to highlight how current VUIs are based on a limited and homogenised set of voice outputs. We argue that current systems do not adequately consider the diversity of peoples' speech, how that diversity represents sociocultural identities, and how voices have the potential to shape user perceptions and experiences. Ultimately, as other technological developments have influenced the ideologies of language, the voice outputs of VUIs will influence the ideologies of speech. Based on our argument, we pose three design strategies for VUI voice output design - individualisation, context awareness, and diversification - to motivate new ways of conceptualising and designing these technologies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In VUI literature, the possibility of using voice as design material has been discussed.
“In VUI literature, the possibility of using voice as design material has been discussed [53].”
From Using Experience-Based Participatory Approach to Design Interactive Voice User Interfaces for Delivering Physical Activity Programs with Older Adults (2023) · cites this paper as [53]
This is consistent with the broader field of 'sociophonetics', which treats voice as an important design element.
“This is consistent with the broader feld of 'sociophonetics' [61], which treats voice as an important design element.”
From “A Painless Way to Learn:” Designing an Interactive Storytelling Voice User Interface to Engage Older Adults in Informal Health Information Learning (2023) · cites this paper as [61]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Bijker, W. E., Hughes, T. P., & Pinch, T. (1987) · Cited by 2,962 · see citing works
A Dictionary of Linguistics and Phonetics
Major, R. C., & Crystal, D. S. (1992) · Modern Language Journal · doi:10.2307/330198 · Cited by 2,766 · see citing works
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · see citing works
Variation and the indexical field <sup>1</sup>
Eckert, P. (2008) · Journal of Sociolinguistics · doi:10.1111/j.1467-9841.2008.00374.x · Cited by 2,046 · see citing works
Jocks and Burnouts: Social Categories and Identity in High School.
Useem, E. L., & Eckert, P. (1990) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2073258 · Cited by 1,572 · see citing works
McCarthy, J., & Wright, P. (2004) · The MIT Press eBooks · doi:10.7551/mitpress/6687.003.0011 · Cited by 1,371 · see citing works
Laver, J. (1994) · doi:10.1017/cbo9781139166621 · Cited by 1,298 · see citing works
(2021) · doi:10.26226/morressier.60c8d83cbea1445efd9a1905 · Cited by 1,230 · see citing works
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176 · see citing works
Voice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 1× · see citing works
“Indeed, there has been a growing interest in voice-based interaction at CHI (e.g. papers [61, 76, 82], courses [57–60], workshops [54, 56], and panels [55]).”
BackgroundEnglish with an accent: Language, ideology, and discrimination in the United States
Laurence, E. (2014) · Colombian Applied Linguistics Journal · doi:10.14483/udistrital.jour.calj.2013.2.a012 · Cited by 695 · see citing works
Can computer personalities be human personalities?
Nass, C., Moon, Y., Fogg, B. J., Reeves, B. B., & Dryer, D. C. (1995) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1995.1042 · Cited by 621 · see citing works
Nass, C., & Lee, K. M. (2001) · Journal of Experimental Psychology: Applied · doi:10.1037/1076-898x.7.3.171 · Cited by 522 · see citing works
(2005) · Cited by 505 · see citing works
Phonation type as a stylistic variable: The use of falsetto in constructing a persona1
Podesva, R. J. (2007) · Journal of Sociolinguistics · doi:10.1111/j.1467-9841.2007.00334.x · Cited by 484 · see citing works
Dürmüller, U. (1980) · Language · doi:10.2307/412686 · Cited by 414 · see citing works
ZHANG, Q. (2005) · Language in Society · doi:10.1017/s0047404505050153 · Cited by 370 · see citing works
Ideologised values for British accents1
Coupland, N., & Bishop, H. (2007) · Journal of Sociolinguistics · doi:10.1111/j.1467-9841.2007.00311.x · Cited by 355 · see citing works
"What can i help you with?": infrequent users' experiences of intelligent personal assistants
Cowan, B. R., Pantidi, N., Coyle, D., Morrissey, K., Clarke, P., Al-Shehri, S., Earley, D., & Bandeira, N. (2017) · Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3098279.3098539 · Cited by 351 · see citing works
Proceedings of the 18th International Congress of Phonetic Sciences
Skarabela, B., Ota, M., Fazekas, J., & Wihlborg, L. (2015) · Cited by 317 · see citing works
Munson, B., McDonald, E. C., DeBoe, N. L., et al. (2005) · Journal of Phonetics · doi:10.1016/j.wocn.2005.05.003 · Cited by 275 · see citing works
The role of beliefs in lexical alignment: evidence from dialogs with humans and computers.
Branigan, H., Pickering, M., Pearson, J., McLean, J., & Brown, A. (2011) · Cognition · doi:10.1016/j.cognition.2011.05.011 · Cited by 250 · see citing works
Does computer-generated speech manifest personality? an experimental test of similarity-attraction
Nass, C., & Lee, K. M. (2000) · Proceedings of the SIGCHI conference on Human Factors in Computing Systems · doi:10.1145/332040.332452 · Cited by 242 · see citing works
Men’s Identities and Sociolinguistic Variation: The Case of Fraternity Men
Kiesling, S. F. (1998) · Journal of Sociolinguistics · doi:10.1111/1467-9481.00031 · Cited by 233 · see citing works
DiSalvo, C. F. (2012) · Cited by 224 · see citing works
Acoustic Correlates of Perceived versus Actual Sexual Orientation in Men’s Speech
Linville, S. E. (1998) · Folia Phoniatrica et Logopaedica · doi:10.1159/000021447 · Cited by 221 · see citing works
Patterns for How Users Overcome Obstacles in Voice User Interfaces
Myers, C., Furqan, A., Nebolsky, J., Caro, K., & Zhu, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173580 · Cited by 213 · cited 2× · see citing works
“papers [61, 76, 82], courses [57–60], workshops [54, 56], and panels [55]).”
BackgroundCohen, M., Giangola, J. P., & Balogh, J. (2004) · Cited by 211 · see citing works
The State of Speech in HCI: Trends, Themes and Challenges
Clark, L., Doyle, P., Garaialde, D., Gilmartin, E., Schlögl, S., Edlund, J., Aylett, M., Cabral, J., Munteanu, C., Edwards, J., et al. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz016 · Cited by 193 · cited 1× · see citing works
“But thus far, there has been little guidance of how to do this [12] and what the repercussions of VUI voices might be.”
BackgroundAccents of English I: An Introduction
Wells, J. C. (1982) · Cited by 183 · see citing works
The influence of sexual orientation on vowel production (L)
Pierrehumbert, J. B., Bent, T., Munson, B., et al. (2004) · The Journal of the Acoustical Society of America · doi:10.1121/1.1788729 · Cited by 165 · see citing works
Speech interfaces from an evolutionary perspective
Nass, C., & Gong, L. (2000) · Communications of the ACM · doi:10.1145/348941.348976 · Cited by 142 · see citing works
Labov, W. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511613258.006 · Cited by 136 · see citing works
Categories, stereotypes, and the linguistic perception of sexuality
Levon, E. (2014) · Language in Society · doi:10.1017/s0047404514000554 · Cited by 136 · see citing works
Dixon, J., Mahoney, B., & Cocks, R. P. (2002) · Journal of Language and Social Psychology · doi:10.1177/02627x02021002004 · Cited by 135 · see citing works
Experience-centered design: designers, users, and communities in dialogue
(2011) · Choice Reviews Online · doi:10.5860/choice.48-2726 · Cited by 135 · see citing works
The Social Stratification of (r) in New York City Department Stores
Labov, W. (1997) · doi:10.1007/978-1-349-25582-5_14 · Cited by 126 · see citing works
Timming, A. R. (2016) · Work Employment and Society · doi:10.1177/0950017016630260 · Cited by 126 · see citing works
Sexuality in context: Variation and the sociolinguistic perception of identity
Levon, E. (2007) · Language in Society · doi:10.1017/s0047404507070431 · Cited by 118 · see citing works
Empirical evidence for gendered speech production: /s/ in Glaswegian
Stuart‐Smith, J. (2007) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · Cited by 109 · see citing works
The Effects of Synthesized Voice Accents on User Perceptions of Robots
Tamagawa, R., Watson, C. I., Kuo, I. H., MacDonald, B. A., & Broadbent, E. (2011) · International Journal of Social Robotics · doi:10.1007/s12369-011-0100-4 · Cited by 105 · see citing works
Mack, S., & Munson, B. (2011) · Journal of Phonetics · doi:10.1016/j.wocn.2011.10.002 · Cited by 100 · see citing works
Accent and Linguistic Prejudice within British Teacher Training
Baratta, A. (2017) · Journal of Language Identity & Education · doi:10.1080/15348458.2017.1359608 · Cited by 83 · see citing works
Is Spoken Language All-or-Nothing? Implications for Future Speech-Based Human-Machine Interaction
Moore, R. K. (2016) · Lecture notes in electrical engineering · doi:10.1007/978-981-10-2585-3_22 · Cited by 73 · see citing works
Similarity is more important than expertise: accent effects in speech interfaces
Dahlbäck, N., Wang, Q., Nass, C., & Alwin, J. (2007) · International Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240859 · Cited by 68 · see citing works
Wood, R. E. (1985) · Journal of Language and Social Psychology · doi:10.1177/0261927x8500400105 · Cited by 67 · see citing works
Schilling‐Estes, N. (1999) · Language · doi:10.2307/417748 · Cited by 66 · see citing works
None of a CHInd: relationship counselling for HCI and speech technology
Aylett, M., Kristensson, P., Whittaker, S., & Vazquez-Alvarez, Y. (2014) · CHI Extended Abstracts · doi:10.1145/2559206.2578868 · Cited by 61 · see citing works
Pitch range and women's sexual orientation
Waksler, R. (2001) · WORD · doi:10.1080/00437956.2001.11432508 · Cited by 57 · see citing works
Prof, J. J., & Jansen, J. (2011) · Cited by 56 · see citing works
Voice Interaction Design: Crafting the New Conversational Speech Systems
Harris, R. A. (2004) · Cited by 53 · see citing works
The Influence of Synthetic Voice on the Evaluation of a Virtual Character
Cabral, J. P., Cowan, B. R., Zibrek, K., & McDonnell, R. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-325 · Cited by 44 · see citing works
Lu, X., Li, S., & Fujimoto, M. (2019) · SpringerBriefs in computer science · doi:10.1007/978-981-15-0595-9_2 · Cited by 42 · see citing works
Dasgupta, R. (2018) · Apress eBooks · doi:10.1007/978-1-4842-4125-7 · Cited by 41 · see citing works
Relationship between Knowledge, Stereotyping, and Prejudice in Interethnic Communication
Matusitz, J. (2012) · PASOS Revista de Turismo y Patrimonio Cultural · doi:10.25145/j.pasos.2012.10.008 · Cited by 39 · see citing works
Nooz or nyooz?: The Complex Construction of Canadian Identity
Clarke, S. (2006) · The Canadian Journal of Linguistics / La revue canadienne de linguistique · doi:10.1353/cjl.2008.0013 · Cited by 36 · see citing works
Milroy, L., & Milroy, L. (2002) · Routledge eBooks · doi:10.4324/9780203026038 · Cited by 35 · see citing works
Lara, E. M. R., Martín‐Santana, J. D., & Molina, C. M. (2016) · Journal of Global Marketing · doi:10.1080/08911762.2015.1119919 · Cited by 34 · see citing works
Mills, M., & Stoneham, G. (2017) · Plymouth Marjon University Repository (Plymouth Marjon University) · Cited by 30 · see citing works
"Play PRBLMS": Identifying and Correcting Less Accessible Content in Voice Interfaces
Springer, A., & Cramer, H. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173870 · Cited by 27 · see citing works
Designing Speech, Acoustic and Multimodal Interactions
Munteanu, C., Irani, P., Oviatt, S., Aylett, M., Penn, G., Pan, S.-M., Sharma, N., Rudzicz, F., Gomez, R., Cowan, B. R., et al. (2017) · CHI Extended Abstracts · doi:10.1145/3027063.3027086 · Cited by 24 · see citing works
Towards Understanding How Speech Output Affects Navigation System Credibility
Cowan, B. R., Gannon, D., Walsh, J., Kinneen, J., O'Keefe, E., & Xie, L.-X. (2016) · CHI Extended Abstracts · doi:10.1145/2851581.2892469 · Cited by 23 · see citing works
Design of Speech-based Devices
Pitt, I. J., & Edwards, A. D. N. (2003) · Practitioner series · doi:10.1007/978-1-4471-0093-5 · Cited by 18 · see citing works
Stopping and Fronting in New Zealand Pasifika English
Bell, A., & Gibson, A. (2008) · Scholarly Commons (University of Pennsylvania) · Cited by 17 · see citing works
Spoken Interaction with Computers in a Native or Non-Native Language - Same of Different?
Dahlbäck, N., Swamy, S. M., Nass, C., Arvidsson, F., & Skågeby, J. (2001) · International Conference on Human-Computer Interaction · Cited by 13 · see citing works
Impact of English regional accents on user acceptance of voice user interfaces
Niculescu, A. I., White, G. M., Lan, S. S., et al. (2008) · doi:10.1145/1463160.1463235 · Cited by 13 · see citing works
Renals, S. J., & King, S. (2010) · doi:10.1002/9781444317251.ch22 · Cited by 12 · see citing works
In the Land of Invented Languages: A Celebration of Linguistic Creativity, Madness, and Genius
Okrent, A. (2010) · Medical Entomology and Zoology · Cited by 11 · see citing works
Designing Speech and Multimodal Interactions for Mobile, Wearable, and Pervasive Applications
Munteanu, C., Irani, P., Oviatt, S. L., et al. (2016) · doi:10.1145/2851581.2856506 · Cited by 11 · see citing works
They Know as Much as We Do: Knowledge Estimation and Partner Modelling of Artificial Partners.
Cowan, B. R., Branigan, H. P., Begum, H., et al. (2017) · Cognitive Science · Cited by 10 · see citing works
Munteanu, C., & Penn, G. (2014) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2559206.2567826 · Cited by 8 · see citing works
Children's Responses to Genuine Child Synthesized Speech in Child-Robot Interaction
Sandygulova, A., & O’Hare, G. M. P. (2015) · doi:10.1145/2701973.2702058 · Cited by 6 · see citing works
Munteanu, C., & Penn, G. (2016) · doi:10.1145/2851581.2856689 · Cited by 6 · see citing works
Munteanu, C., & Penn, G. (2017) · doi:10.1145/3027063.3027117 · Cited by 4 · see citing works
Munteanu, C., & Penn, G. (2015) · doi:10.1145/2702613.2706679 · Cited by 2 · see citing works
From Wood to Bits to Silicon Chips
Rowe, D. (2010) · IGI Global eBooks · doi:10.4018/978-1-61520-725-1.ch002 · Cited by 1 · see citing works
A reading list of recent advances in speech synthesis
King, S. (2015) · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- Hadumod Bussmann. 1998. 'speech'. In Routledge Dictionary of Lan- guage and Linguistics. Taylor & Francis, London, UK, 1106.
- Paul Foulkes, James M. Scobbie, and Dominic J. L. Watt. 2010. So- ciophonetics. In Handbook of Phonetic Sciences (2nd ed.), William J. Hardcastle, John Laver, and Fiona E. Gibbon (Eds.). Blackwell, Oxford, UK, 703ś754.
- Arthur Hughes, Peter Trudgill, and Dominic J. L. Watt. 2013. English accents & dialects : an introduction to social and regional varieties of English in the British Isles (5th ed.).
- James Milroy and Lesley Milroy. 1978. Belfast: change and variation in an urban vernacular. In Sociolinguistic patterns in British English, Peter Trudgill (Ed.). University Park Press, USA, 19ś36.
- Roger K. Moore. 2017. Appropriate Voices for Artefacts: Some Key Insights. In Proceedings of the 1st International Workshop on Vocal Interactivity in-and-between Humans, Animals and Robots (VIHAR) Skvöde, Sweden, 25-26 Aug 2017. 7ś11. http://vihar-2017.vihar.org/ assets/vihar2017_proceedings.pdf
- Cathy Pearl. 2016. Designing Voice User Interfaces : Principles of Con- versational Experiences. O'Reilly Media, Boston, USA.
- James Pfrehm. 2018. Technolingualism : the mind and the machine. Bloomsbury, Cambridge, Massachusetts. https://www.bloomsbury. com/uk/technolingualism-9781472578365/
- Marie Louise Juul Sùndergaard and Lone Koefoed Hansen. 2018. In- timate Futures. In Proceedings of the 2018 on Designing Interactive Systems Conference 2018 -DIS '18. ACM Press, New York, New York, USA, 869ś880. https://doi.org/10.1145/3196709.3196766 · DOI
- James Tompkinson. 2015. Accent evaluation and the perception of spoken threats. In Proceedings of the third Postgraduate and Academic Researchers in Linguistics at York conference (PARLAY 2015). University of York, UK, 115ś131. https://yorkpapersinlinguistics.files.wordpress. com/2016/06/james-tompkinson-parlay-proceedings-2015.pdf
- Jessica Wormald. 2016. Regional Variation in Punjabi-English. Ph.D. Dissertation. Universty of York, York, UK. http://etheses.whiterose.ac. uk/13188/
Cited by
Sorted by citations
Voice in Human–Agent Interaction: A Survey
Seaborn, K., Miyake, N. P., Pennefather, P., & Otake-Matsuura, M. (2022) · ACM Computing Surveys · doi:10.1145/3386867 · Cited by 249 · see citing works
Progressivity for voice interface design
Fischer, J. E., Reeves, S., Porcheron, M., & Sikveland, R. O. (2019) · Proceedings of the 1st International Conference on Conversational User Interfaces · doi:10.1145/3342775.3342788 · Cited by 88 · see citing works
You Complete Me: Human-AI Teams and Complementary Expertise
Zhang, Q., Lee, M. L., & Carter, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517791 · Cited by 85 · see citing works
Experiential Qualities of Whispering with Voice Assistants
Parviainen, E., & Søndergaard, M. L. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376187 · Cited by 52 · see citing works
Metaphors in Voice User Interfaces: A Slippery Fish
Desai, S., & Twidale, M. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3609326 · Cited by 42 · see citing works
Firefox Voice: An Open and Extensible Voice Assistant Built Upon the Web
Cambre, J., Williams, A. C., Razi, A., Bicking, I., Wallin, A., Tsai, J., Kulkarni, C., & Kaye, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445409 · Cited by 39 · see citing works
Desai, S., Lundy, M., & Chin, J. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3597141 · Cited by 25 · see citing works
Impact of Voice Fidelity on Decision Making: A Potential Dark Pattern?
Dubiel, M., Sergeeva, A., & Leiva, L. A. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645202 · Cited by 16 · see citing works
Desai, S., Hu, X., Lundy, M., & Chin, J. (2023) · International Conference on Human-Agent Interaction · doi:10.1145/3623809.3623827 · Cited by 13 · see citing works
- Authors
- Sutton, S. J., Foulkes, P., Kirk, D., & Lawson, S.
- Affiliations
- Northumbria University (GB)
- University of York (GB)
- Venue
- Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2019-05-02
- Pages
- 1-14
- OpenAlex
- W2931579440
- Cited by
- 92
- References
- 78
- Citations by year
- 2026: 7 · 2025: 17 · 2024: 13 · 2023: 19 · 2022: 12 · 2021: 9 · 2020: 13 · 2019: 2
- FWCI
- 3.44
- Type
- conference paper
- Language
- en
- Open access
- green
- orcid.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- Northumbria Research Link (Northumbria University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- voice user interface, sociophonetics, sociocultural identity, voice design, individualization, context awareness, user perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
