Measuring Voice UX Quantitatively: A Rapid Review
Open accessThe DOI matches its Crossref record; 52 of the 56 references it cites were found in a registry or an index.
Abstract
Computer voice is experiencing a renaissance through the growing popularity of voice-based interfaces, agents, and environments. Yet, how to measure the user experience (UX) of voice-based systems remains an open and urgent question, especially given that their form factors and interaction styles tend to be non-visual, intangible, and often considered disembodied or “body-less.” As a first step, we surveyed the ACM and IEEE literatures to determine which quantitative measures and measurements have been deemed important for voice UX. Our findings show that there is little consensus, even with similar situations and systems, as well as an overreliance on lab work and unvalidated scales. In response, we offer two high-level descriptive frameworks for guiding future research, developing standardized instruments, and informing ongoing review work. Our work highlights the current strengths and weaknesses of voice UX research and charts a path towards measuring voice UX in a more comprehensive way.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although popularly used in VUI literature [57], none of these scales were developed for VUIs, and their validity for VUIs remains limited.”
From “A Painless Way to Learn:” Designing an Interactive Storytelling Voice User Interface to Engage Older Adults in Informal Health Information Learning (2023) · cites this paper as [57]
Experimental — accuracy not yet measuredIn the citing paper
“which are widely employed in VUI re-search [71, 80]”
From Examining Humanness as a Metaphor to Design Voice User Interfaces (2024) · cites this paper as [71]
In the citing paper
“While this method is common practice in VUI research [71], it notably lacks the inclusion of direct interac-tion with the technology, offering insights that are observational rather than experiential.”
From Examining Humanness as a Metaphor to Design Voice User Interfaces (2024) · cites this paper as [71]
In the citing paper
“This questionnaire from robotics research has been successfully modifed to accommodate VUIs in previous studies (e.g., [24,57]).”
From “A Painless Way to Learn:” Designing an Interactive Storytelling Voice User Interface to Engage Older Adults in Informal Health Information Learning (2023) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · cited 1×
“Then, they conducted two inductive thematic analyses [5, 16] to create categorical frameworks for each of the IV and DV data.”
MethodLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Guest, G., MacQueen, K., & Namey, E. (2012) · doi:10.4135/9781483384436 · Cited by 7,924
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 1×
“human-robot interaction was developed [2], use the IV and DV frameworks as a guide based on consensus.”
Method(2006) · doi:10.1080/01449290500330331 · Cited by 3,026 · cited 1×
“Indeed, lack of definitional and operational consensus has been a running problem [1,18,26,28,29].”
BackgroundNass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 2×
“tend to react to humanlike features of computer voice as they would react to people, without realizing it [31, 40, 42].”
Evidence summaries: the evolution of a rapid review approach
Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-10 · Cited by 1,232 · cited 1×
“Rapid reviews are used to generate a quick yet systematic descriptive and narrative summary of a focused area of study [12, 14, 23].”
MethodCochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews
Garritty, C., Gartlehner, G., Nussbaumer-Streit, B., King, V. J., Hamel, C., Kamel, C., Affengruber, L., & Stevens, A. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.10.007 · Cited by 1,141 · cited 1×
“We conducted a rapid review of the literature following the Cochrane gold standard protocol [15].”
Understanding, scoping and defining user experience: a survey approach
Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P. O. S., & Kort, J. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518813 · Cited by 1,127
Artificial intelligence and communication: A Human–Machine Communication research agenda
Guzman, A. L., & Lewis, S. C. (2020) · New Media & Society · doi:10.1177/1461444819858691 · Cited by 951 · cited 1×
“Moreover, there has been a growing recognition that social and psychological factors, including afect and emotion, trust, credibility, rapport, and the relational context, need to be considered [17, 36].”
BackgroundExpediting systematic reviews: methods and implications of rapid reviews
Ganann, R., Ciliska, D., & Thomas, H. (2010) · Implementation Science · doi:10.1186/1748-5908-5-56 · Cited by 752 · cited 1×
“Rapid reviews are used to generate a quick yet systematic descriptive and narrative summary of a focused area of study [12, 14, 23].”
MethodBargas-Avila, J. A., & Hornbæk, K. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979336 · Cited by 660 · cited 1×
“Indeed, lack of definitional and operational consensus has been a running problem [1,18,26,28,29].”
Background(2005) · Cited by 505
(2008) · doi:10.1016/b978-0-12-373558-4.x0001-5 · Cited by 287
Designing social presence of social actors in human computer interaction
Lee, K. M., & Nass, C. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642662 · Cited by 257 · cited 3×
“hostile) [4] User Characteristics Comparing diferent groups of users or Gender Lab Male and female participants [30] manipulating afective state of the user Personality Emotion Lab Lab Comparing extrovert/introvert participants Driver emotion (happy/upset) [32] [41] Conversation Manipulating the voice system’s Style L…”
BackgroundImproving automotive safety by pairing driver emotion and car voice emotion
Nass, C., Jonsson, I.-M., Harris, H., Reaves, B., Endo, J., Brave, S., & Takayama, L. (2005) · CHI '05 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1056808.1057070 · Cited by 249 · cited 5×
“, lexical complexity, adaptation) [51]; Interaction time [27]; Driving performance [38]; Disclosure (user responses) [53]; Accidents [41] Subj Continued use [33]; Satisfaction [33]; System efectiveness [24]; Tone clarity [8]; Satisfaction [3]; System Usability Scale (SUS) [6, 44]; Speech User Interface Service”
BackgroundInfluentialVoice in Human–Agent Interaction: A Survey
Seaborn, K., Miyake, N. P., Pennefather, P., & Otake-Matsuura, M. (2022) · ACM Computing Surveys · doi:10.1145/3386867 · Cited by 249
Self-citationLallemand, C., Gronier, G., & Koenig, V. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2014.10.048 · Cited by 248 · cited 1×
“Indeed, lack of definitional and operational consensus has been a running problem [1,18,26,28,29].”
BackgroundThe State of Speech in HCI: Trends, Themes and Challenges
Clark, L., Doyle, P., Garaialde, D., Gilmartin, E., Schlögl, S., Edlund, J., Aylett, M., Cabral, J., Munteanu, C., Edwards, J., et al. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz016 · Cited by 193 · cited 2×
“This recent and rapid progress has triggered a renewed interest in research on “the voice of the machine” [9,45].”
BackgroundAt Your Service: Designing Voice Assistant Personalities to Improve Automotive User Interfaces
Braun, M., Mainz, A., Chadowitz, R., Pfleging, B., & Alt, F. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300270 · Cited by 182
Can computer-generated speech have gender?: an experimental test of gender stereotype
Lee, E. J., Nass, C., & Brave, S. (2000) · CHI '00 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/633292.633461 · Cited by 179 · cited 4×
“hostile) [4] User Characteristics Comparing diferent groups of users or Gender Lab Male and female participants [30] manipulating afective state of the user Personality Emotion Lab Lab Comparing extrovert/introvert participants Driver emotion (happy/upset) [32] [41] Conversation Manipulating the voice system’s Style L…”
BackgroundInfluentialFace Value? Exploring the Effects of Embodiment for a Group Facilitation Agent
Shamekhi, A., Liao, Q. V., Wang, D., Bellamy, R. K. E., & Erickson, T. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173965 · Cited by 124
Gendered voice and robot entities: Perceptions and reactions of male and female subjects
Crowelly, C. R., Villanoy, M., Scheutzz, M., & Schermerhornz, P. (2009) · 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2009.5354204 · Cited by 121 · cited 4×
“Engagement None Obj Interaction behavior (e.g., frequency and time) [37]; Emotional engagement (facial expressions) [47] Subj Cognitive engagement [47]; Time spent talking to virtual passenger [41]; Stimulation (UEQ modules) [4] Cognition Attention Obj Attentional allocation [27]; Situational awareness [38]; Driver at…”
BackgroundInfluentialMapping Perceptions of Humanness in Intelligent Personal Assistant Interaction
Doyle, P. R., Edwards, J., Dumbleton, O., Clark, L., & Cowan, B. R. (2019) · Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3338286.3340116 · Cited by 100 · cited 4×
“Subj Perceived Intelligence [8, 21, 46] Aggregate UX None Obj None Subj Human-likeness [13]; Attractiveness diferences [6]; Self-efciency [7]; Interaction questionnaire [19]; Godspeed questionnaire [19]”
BackgroundInfluentialEmpathy Is All You Need: How a Conversational Agent Should Respond to Verbal Abuse
Chin, H., Molefi, L. W., & Yi, M. Y. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376461 · Cited by 96 · cited 7×
“, lexical complexity, adaptation) [51]; Interaction time [27]; Driving performance [38]; Disclosure (user responses) [53]; Accidents [41] Subj Continued use [33]; Satisfaction [33]; System efectiveness [24]; Tone clarity [8]; Satisfaction [3]; System Usability Scale (SUS) [6, 44]; Speech User Interface Service”
BackgroundInfluentialLee, J.-E. R., & Nass, C. I. (2010) · Trust and Technology in a Ubiquitous Modern Environment · doi:10.4018/978-1-61520-901-9.ch001 · Cited by 91
Children asking questions: speech interface reformulations and personification preferences
Yarosh, S., Thompson, S., Watson, K., Chase, A., Senthilkumar, A., Yuan, Y., & Brush, A. J. B. (2018) · Proceedings of the 17th ACM Conference on Interaction Design and Children · doi:10.1145/3202185.3202207 · Cited by 84 · cited 3×
“Engagement None Obj Interaction behavior (e.g., frequency and time) [37]; Emotional engagement (facial expressions) [47] Subj Cognitive engagement [47]; Time spent talking to virtual passenger [41]; Stimulation (UEQ modules) [4] Cognition Attention Obj Attentional allocation [27]; Situational awareness [38]; Driver at…”
Background"Hey, Siri", "Ok, Google", "Alexa". Acceptance-Relevant Factors of Virtual Voice-Assistants
Burbach, L., Halbach, P., Plettenberg, N., Nakayama, J., Ziefle, M., & Calero Valdez, A. (2019) · 2019 IEEE International Professional Communication Conference (ProComm) · doi:10.1109/procomm.2019.00025 · Cited by 80
Trust in artificial voices: A "congruency effect" of first impressions and behavioural experience
Torre, I., Goslin, J., White, L., & Zanatto, D. (2018) · Proceedings of the Technology, Mind, and Society · doi:10.1145/3183654.3183691 · Cited by 77 · cited 2×
“regional accents [49]; [11] output American vs.”
BackgroundUnderstanding and Measuring User Experience in Conversational Interfaces
Kocaballi, A. B., Laranjo, L., & Coiera, E. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz015 · Cited by 68 · cited 4×
“Indeed, voice interaction is more than simple voice recognition; the field is starting to consider the context of use and importance of affective factors, trust, and sociality, as reflected in the recent panel [22] and related survey work [26].”
BackgroundInfluentialCurrent Challenges for UX Evaluation of Human-Robot Interaction
Lindblom, J., & Andreasson, R. (2016) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-41697-7_24 · Cited by 65
The Effects of Anthropomorphism and Non-verbal Social Behaviour in Virtual Assistants
Kontogiorgos, D., Pereira, A., Andersson, O., Koivisto, M., Gonzalez Rabal, E., Vartiainen, V., & Gustafson, J. (2019) · Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents · doi:10.1145/3308532.3329466 · Cited by 63 · cited 3×
“gaze frequency, conversation turns) [27] Subj Social presence [32, 33]; Intimacy [25, 33]; Closeness (Subjective closeness index) [53]; Perceived power [46]; Conformity in decision-making [30]; Helpfulness [25]; Self-validation [25] Attitudes Trust Obj Trust (investment game) [49] Subj Trust [7, 25, 33]; Trustworthine…”
BackgroundUser Experience with Smart Voice Assistants: The Accent Perspective
Pal, D., Arpnikanondt, C., Funilkul, S., & Varadarajan, V. (2019) · 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt45670.2019.8944754 · Cited by 60 · cited 1×
“, lexical complexity, adaptation) [51]; Interaction time [27]; Driving performance [38]; Disclosure (user responses) [53]; Accidents [41] Subj Continued use [33]; Satisfaction [33]; System efectiveness [24]; Tone clarity [8]; Satisfaction [3]; System Usability Scale (SUS) [6, 44]; Speech User Interface Service”
BackgroundSimilarity is more important than expertise: accent effects in speech interfaces
Dahlbäck, N., Wang, Q., Nass, C., & Alwin, J. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240859 · Cited by 53 · cited 5×
“Quality Questionnaire (SUISQ-R) [6]; Document feedback [38]; Voice understandability [10]; Perception of rapport [46]; Group decision performance [46]; Voice performance [11]; MeCue questionnaire [4] Engagement None Obj Interaction behavior (e.”
BackgroundInfluentialAn End-to-End Conversational Style Matching Agent
Hoegen, R., Aneja, D., McDuff, D., & Czerwinski, M. (2019) · Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents · doi:10.1145/3308532.3329473 · Cited by 51 · cited 4×
“high warning with details); [21]; [19] language style conversational fllers; conversation style matching Context Lab Environment (e.”
BackgroundInfluentialDesigning with Gaze: Tama -- a Gaze Activated Smart-Speaker
McMillan, D., Brown, B., Kawaguchi, I., Jaber, R., Solsona Belenguer, J., & Kuzuoka, H. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359278 · Cited by 44
Luo, Y., Lee, B., & Choe, E. K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376616 · Cited by 44 · cited 3×
“Form Factor Manipulation of the form, style, and None Lab, Comparison of diferent devices; voice-only [37]; [51]; type of voice system Field (disembodied systems) vs.”
BackgroundLee, S., Cho, M., & Lee, S. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3411840 · Cited by 38 · cited 5×
“, [33]; [13]; robots) [10]; [46] Voice Characteristics Manipulating sound as well as Gender Lab Masculine vs.”
BackgroundInfluentialAn Exploration of Speech-Based Productivity Support in the Car
Martelaro, N., Teevan, J. B., & Iqbal, S. T. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300494 · Cited by 33 · cited 7×
“, lexical complexity, adaptation) [51]; Interaction time [27]; Driving performance [38]; Disclosure (user responses) [53]; Accidents [41] Subj Continued use [33]; Satisfaction [33]; System efectiveness [24]; Tone clarity [8]; Satisfaction [3]; System Usability Scale (SUS) [6, 44]; Speech User Interface Service”
BackgroundInfluentialMental Workload and Language Production in Non-Native Speaker IPA Interaction
Wu, Y., Edwards, J., Cooney, O., Bleakley, A., Doyle, P. R., Clark, L., Rough, D. J., & Cowan, B. R. (2020) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3405755.3406118 · Cited by 33 · cited 3×
“Form Factor Manipulation of the form, style, and None Lab, Comparison of diferent devices; voice-only [37]; [51]; type of voice system Field (disembodied systems) vs.”
BackgroundExploring Effects of Conversational Fillers on User Perception of Conversational Agents
Jeong, Y., Lee, J., & Kang, Y. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312913 · Cited by 31 · cited 5×
“high warning with details); [21]; [19] language style conversational fllers; conversation style matching Context Lab Environment (e.”
BackgroundInfluential"I Almost Fell in Love with a Machine": Speaking with Computers Affects Self-disclosure
Yu, Q., Nguyen, T., Prakkamakul, S., & Salehi, N. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312918 · Cited by 30 · cited 3×
“gaze frequency, conversation turns) [27] Subj Social presence [32, 33]; Intimacy [25, 33]; Closeness (Subjective closeness index) [53]; Perceived power [46]; Conformity in decision-making [30]; Helpfulness [25]; Self-validation [25] Attitudes Trust Obj Trust (investment game) [49] Subj Trust [7, 25, 33]; Trustworthine…”
BackgroundShi, Y., Yan, X., Ma, X., Lou, Y., & Cao, N. (2018) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3170427.3188560 · Cited by 29 · cited 3×
“Swedish accents Emotions Lab Emotional expressions in graphical displays; [47]; [41] emotionality of voice (energetic vs.”
BackgroundImproving the Usability of Voice User Interfaces: A New Set of Ergonomic Criteria
Nowacki, C., Gordeeva, A., & Lizé, A.-H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49760-6_8 · Cited by 28
The Use of Voice Input to Induce Human Communication with Banking Chatbots
Kim, S., Goh, J., & Jun, S. (2018) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3173386.3176970 · Cited by 25 · cited 3×
“Engagement None Obj Interaction behavior (e.g., frequency and time) [37]; Emotional engagement (facial expressions) [47] Subj Cognitive engagement [47]; Time spent talking to virtual passenger [41]; Stimulation (UEQ modules) [4] Cognition Attention Obj Attentional allocation [27]; Situational awareness [38]; Driver at…”
BackgroundPanel: Voice Assistants, UX Design and Research
Kaye, J. '., Fischer, J., Hong, J., Bentley, F. R., Munteanu, C., Hiniker, A., Tsai, J. Y., & Ammari, T. (2018) · Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3170427.3186323 · Cited by 19 · cited 2×
“A round table at CHI 2018 highlighted the need for understanding current practice as well as what approaches and methods should be used [22].”
BackgroundConversational Interfaces for a Smart Campus: A Case Study
Bortoli, M., Furini, M., Mirri, S., Montangero, M., & Prandi, C. (2020) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/3399715.3399914 · Cited by 18 · cited 1×
“, lexical complexity, adaptation) [51]; Interaction time [27]; Driving performance [38]; Disclosure (user responses) [53]; Accidents [41] Subj Continued use [33]; Satisfaction [33]; System efectiveness [24]; Tone clarity [8]; Satisfaction [3]; System Usability Scale (SUS) [6, 44]; Speech User Interface Service”
BackgroundBrüggemeier, B., Breiter, M., Kurz, M., & Schiwy, J. (2020) · Proceedings of the 2nd Conference on Conversational User Interfaces · doi:10.1145/3405755.3406122 · Cited by 12 · cited 5×
“, lexical complexity, adaptation) [51]; Interaction time [27]; Driving performance [38]; Disclosure (user responses) [53]; Accidents [41] Subj Continued use [33]; Satisfaction [33]; System efectiveness [24]; Tone clarity [8]; Satisfaction [3]; System Usability Scale (SUS) [6, 44]; Speech User Interface Service”
BackgroundInfluentialVoice in computing: an overview of available technologies
Strathmeyer, C. R. (1990) · Computer · doi:10.1109/2.56866 · Cited by 10 · cited 1×
“interaction (HCI) and related disciplines [40, 48].”
Clarifying False Memories in Voice-based Search
Kiesel, J., Bahrami, A., Stein, B., Anand, A., & Hagen, M. (2019) · Proceedings of the 2019 Conference on Human Information Interaction and Retrieval · doi:10.1145/3295750.3298961 · Cited by 9 · cited 3×
“Engagement None Obj Interaction behavior (e.g., frequency and time) [37]; Emotional engagement (facial expressions) [47] Subj Cognitive engagement [47]; Time spent talking to virtual passenger [41]; Stimulation (UEQ modules) [4] Cognition Attention Obj Attentional allocation [27]; Situational awareness [38]; Driver at…”
BackgroundPatterns of gaze in speech agent interaction
Jaber, R., McMillan, D., Belenguer, J. S., & Brown, B. A. T. (2019) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3342775.3342791 · Cited by 9 · cited 1×
“query form) [39]; [20]; [53]; [47], [25]”
Advances in The Ergonomics in Manufacturing: Managing the Enterprise of the Future
(2020) · AHFE international · doi:10.54941/ahfe100426 · Cited by 4
Research works printed but not linked: 2
- Maureen Dobbins. 2017. Rapid Review Guidebook: Steps for Conducting a Rapid Review. National Collaborating Centre for Methods and Tools, Hamilton, ON. Retrieved from https://www.nccmt.ca/capacity-development/rapid-review-guidebook
- Yaniv Leviathan. 2018. Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone. Google AI Blog. Retrieved January 8, 2021 from http://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html
Cited by
Pataranutaporn, P., Liu, R., Finn, E., & Maes, P. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00720-7 · Cited by 192
Nonverbal Cues in Human–Robot Interaction: A Communication Studies Perspective
Urakami, J., & Seaborn, K. (2023) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3570169 · Cited by 185
Self-citationSchreibelmayr, S., & Mara, M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.787499 · Cited by 72
“They prompt the idea that a phone call from a bot, for example, could soon be hardly distinguishable from a real person (Oord et al., 2016; Seaborn and Urakami, 2021)—unless a different design decision is made by the creators of the voice.”
BackgroundExperimentation for Chatbot Usability Evaluation: A Secondary Study
Ren, R., Zapata, M., Castro, J. W., Dieste, Ó., & Acuña, S. T. (2022) · IEEE Access · doi:10.1109/access.2022.3145323 · Cited by 56
“…usability, including investigations on a specific types of chatbots [17], [18], [19], a qualitative study of critical factors that affect users’ perception of chatbots [20], a discussion on chatbot quality attributes [21], and a quantitative user experience research on voice-based chatbots [22].”
BackgroundUser Experience and Usability of Voice User Interfaces: A Systematic Literature Review
Deshmukh, A. M., & Chalmeta, R. (2024) · Information · doi:10.3390/info15090579 · Cited by 56
“…shows the Source Analysis through Bradford’s Law. understanding of the intellectual structure and impact dynamics within speci fi c research domains [35] This paper addresses the challenge of quantifying user experience (UX) in voice-based systems, identifying gaps, and proposing frameworks to…”
Influential
Metadata
- Authors
- Seaborn, K., & Urakami, J.
- Affiliations
- Tokyo Institute of Technology (JP)
- Venue
- Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-08
- Pages
- 1-8
- OpenAlex
- W3135939776
- Cited by
- 55
- References
- 56
- Citations by year
- 2026: 8 · 2025: 10 · 2024: 14 · 2023: 11 · 2022: 10 · 2021: 2
- FWCI
- 6.91
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- speech interfaces, voice assistants, quantitative measurement, embodied interaction, non-visual interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.