The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational Agents
The DOI matches its Crossref record; 73 of the 74 references it cites were found in a registry or an index.
Abstract
Conversational agents mimic natural conversation to interact with users. Since the effectiveness of interactions strongly depends on users’ perception of agents, it is crucial to design agents’ behaviors to provide the intended user perceptions. Research on human-agent and human-human communication suggests that speech specifics are associated with perceptions of communicating parties, but there is a lack of systematic understanding of how speech specifics of agents affect users’ perceptions. To address this gap, we present a framework outlining the relationships between elements of agents’ conversation architecture (dialog strategy, content affectiveness, content style and speech format) and aspects of users’ perception (interaction, ability, sociability and humanness). Synthesized based on literature reviewed from the domains of HCI, NLP and linguistics (n=57), this framework demonstrates both the identified relationships and the areas lacking empirical evidence. We discuss the implications of the framework for conversation design and highlight the inconsistencies with terminology and measurements.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“which are widely employed in VUI re-search [71, 80]”
From Examining Humanness as a Metaphor to Design Voice User Interfaces (2024) · cites this paper as [80]
In the citing paper
“This classification categorizes user perceptions of CUIs into two relevant categories: (1) perception of interaction with agents and (2) perception of agent’s characteristics.”
From Examining Humanness as a Metaphor to Design Voice User Interfaces (2024) · cites this paper as [80]
Citations
Cites
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307 · cited 1×
“Sometimes the Big-5 personality traits questionnaire [29] is used to map an agent’s disposition on various personality dimensions.”
MethodThe Uncanny Valley [From the Field]
Mori, M., MacDorman, K., & Kageki, N. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2192811 · Cited by 3,542 · cited 1×
“This may be a warning indicator to beware of the uncanny valley effect [67] when designing conversation architecture elements to elicit anthropomorphized perceptions from users.”
BackgroundBartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 4×
“A commonly used survey is adapted from the Godspeed questionnaire [3], which evaluates human-likeness based on users’ impression of the agent as fake / natural, machinelike / human-like, unconscious / conscious, and artificial / lifelike (used by [35], [41] and [70]).”
MethodInfluentialO’Brien, H. L., Cairns, P., & Hall, M. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.01.004 · Cited by 1,014 · cited 1×
“Some commonly used methods to evaluate engagement include the annoyance portion of the SASSI questionnaire [37], and the Use Engagement Scale (UES) [71].”
MethodRapp, A., Curti, L., & Boldi, A. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102630 · Cited by 643 · cited 2×
“Based on search terms used in previously published literature reviews [11, 73], the following keywords are used to search for publications related to conversational user interfaces:”
MethodA Taxonomy of Social Cues for Conversational Agents
Feine, J., Gnewuch, U., Morana, S., & Maedche, A. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.07.009 · Cited by 567 · cited 3×
“…elements of conversation architecture affect aspects of users’ perceptions or discussed the speech specifics of agents as part of a broader analysis of the state of research [11, 23], there is no comprehensive analysis of conversation architecture elements that are relevant to user perceptions.”
BackgroundTHE DEVELOPMENT OF A MEASURE OF PERCEIVED HOMOPHILY IN INTERPERSONAL COMMUNICATION
MCCROSKEY, J. C., RICHMOND, V. P., & DALY, J. A. (1975) · Human Communication Research · doi:10.1111/j.1468-2958.1975.tb00281.x · Cited by 496 · cited 1×
“Some commonly used methods to assess intimacy include the set of social attraction questions from the interpersonal attraction questionnaire [61] and the quality of relationship inventory (QRI) [72].”
MethodTHE MEASUREMENT OF TRUST AND ITS RELATIONSHIP TO SELF-DISCLOSURE
WHEELESS, L. R., & GROTZ, J. (1977) · Human Communication Research · doi:10.1111/j.1468-2958.1977.tb00523.x · Cited by 474 · cited 1×
“Some commonly used methods to evaluate trust include the Trust Propensity Scale [59] and the Individualized Trust Scale (ITS) [87].”
MethodWhat Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents
Clark, L., Pantidi, N., Cooney, O., Doyle, P., Garaialde, D., Edwards, J., Spillane, B., Gilmartin, E., Murad, C., Munteanu, C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300705 · Cited by 461 · cited 4×
“Currently, CAs are considered as a user-controlled tool rather than a social companion, with a focus on the utilitarian aspects of the conversation [12].”
BackgroundInfluentialHow perceptions of intelligence and anthropomorphism affect adoption of personal intelligent agents
Moussawi, S., Koufaris, M., & Benbunan-Fich, R. (2021) · Electronic Markets · doi:10.1007/s12525-020-00411-w · Cited by 406 · cited 2×
“Conversation architecture has been demonstrated to play an important role in users’ perceptions of agents [47, 68, 77].”
Background"I Hear You, I Feel You": Encouraging Deep Self-disclosure through a Chatbot
Lee, Y.-C., Yamashita, N., Huang, Y., & Fu, W. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376175 · Cited by 339
Towards a tool for the Subjective Assessment of Speech System Interfaces (SASSI)
HONE, K. S., & GRAHAM, R. (2000) · Natural Language Engineering · doi:10.1017/s1351324900002497 · Cited by 305 · cited 3×
“Some commonly used methods to evaluate engagement include the annoyance portion of the SASSI questionnaire [37], and the Use Engagement Scale (UES) [71].”
MethodHuman-like communication in conversational agents: a literature review and research agenda
Van Pinxteren, M. M. E., Pluymaekers, M., & Lemmink, J. G. A. M. (2020) · Journal of Service Management · doi:10.1108/josm-06-2019-0175 · Cited by 280 · cited 1×
“[81] examining the effects of agents’ human-like communicative behaviours on their relationships with users, and Kocaballi et al.”
BackgroundResilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224 · cited 6×
“For example, this element includes CAs designed to proactively initiate conversation repair [2, 14], or actively elicit feedback from users [89].”
BackgroundInfluentialAssessing the Quality of Personal Relationships
Pierce, G. R., Sarason, I. G., Sarason, B. R., Solky-Butzel, J. A., & Nagle, L. C. (1997) · Journal of Social and Personal Relationships · doi:10.1177/0265407597143004 · Cited by 195 · cited 1×
“Some commonly used methods to assess intimacy include the set of social attraction questions from the interpersonal attraction questionnaire [61] and the quality of relationship inventory (QRI) [72].”
MethodThe State of Speech in HCI: Trends, Themes and Challenges
Clark, L., Doyle, P., Garaialde, D., Gilmartin, E., Schlögl, S., Edlund, J., Aylett, M., Cabral, J., Munteanu, C., Edwards, J., et al. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz016 · Cited by 193 · cited 3×
“While various studies investigated how some elements of conversation architecture affect aspects of users’ perceptions or discussed the speech specifics of agents as part of a broader analysis of the state of research [11, 23], there is no comprehensive analysis of conversation architecture elements that are relevant…”
BackgroundMethodKim, S., Lee, J., & Gweon, G. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300316 · Cited by 190 · cited 9×
“Text Formatting capitalization, emoticons 6 [21][46][77][84][86][88]”
BackgroundMethodResultInfluentialDesigning for Workplace Reflection: A Chat and Voice-Based Conversational Agent
Kocielnik, R., Avrahami, D., Marlow, J., Lu, D., & Hsieh, G. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196784 · Cited by 119 · cited 1×
“Interestingly, studies explored more perceptions of sociability with agent related to voice-based CAs (n=40) compared to text-based CAs (n=24), potentially due to the assumption that interactions with text-based agents are perceived as less personal and more formal [49].”
BackgroundMapping Perceptions of Humanness in Intelligent Personal Assistant Interaction
Doyle, P. R., Edwards, J., Dumbleton, O., Clark, L., & Cowan, B. R. (2019) · Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3338286.3340116 · Cited by 100 · cited 2×
“Also,somestudiesfoundthatsmalltalkandhumour areconsideredunnecessaryoreveninauthenticinconversations withCAs,whiletheseelementsareimportantinscaffoldingcommu-nicationswithhumanpartners[12,17].”
UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital Library
Zheng, Q., Tang, Y., Liu, Y., Liu, W., & Huang, Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501855 · Cited by 100 · cited 2×
“In accordance with accepted practices, we defined the scope of our literature review by considering two factors: (1) time—specifically publications from 2010 to 2022—and (2) source—using the ACMDigital Library [60, 79, 93].”
MethodCho, E., Molina, M. D., & Wang, J. (2019) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2018.0571 · Cited by 87 · cited 1×
“Previous studies have found that modality may have an effect on the perception of humanness, as voice-based agents are perceived as more human-like as compared to text-based agents [9].”
BackgroundZargham, N., Reicherts, L., Bonfert, M., Voelkel, S. T., Schoening, J., Malaka, R., & Rogers, Y. (2022) · Proceedings of the 4th Conference on Conversational User Interfaces · doi:10.1145/3543829.3543834 · Cited by 84 · cited 1×
“Specifically for the element of agent initiated content, we found various research on conversation repairs [50, 74] and agents proactively sharing content with users [18, 92], but many of them did not not include assessments of users’ perceptions of CAs.”
BackgroundStigall, B., Waycott, J., Baker, S., & Caine, K. (2019) · Proceedings of the 31st Australian Conference on Human-Computer-Interaction · doi:10.1145/3369457.3369506 · Cited by 83 · cited 1×
“In accordance with accepted practices, we defined the scope of our literature review by considering two factors: (1) time—specifically publications from 2010 to 2022—and (2) source—using the ACMDigital Library [60, 79, 93].”
MethodI Believe in a Thing Called Bot: Perceptions of the Humanness of “Chatbots”
Westerman, D., Cross, A. C., & Lindmark, P. G. (2019) · Communication Studies · doi:10.1080/10510974.2018.1557233 · Cited by 76 · cited 10×
“Disfluency fillers, interjections, repetitions, typos 6 [5][38][41][85][86][90]”
BackgroundMethodInfluentialProceedings of the SIGDIAL 2011 Conference
Moore, J. D., Traum, D. R., Chai, J. Y., & Passonneau, R. J. (2011) · Cited by 73
Inferring Capabilities of Intelligent Agents from Their External Traits
Knijnenburg, B. P., & Willemsen, M. C. (2016) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2963106 · Cited by 71 · cited 2×
“Since these agents communicate in natural language, the effectiveness of interactions between users and conversational agents in achieving users’ goals depend heavily on the conversation architecture , which refers to the content and manner of communication utilized by the agents [47, 77].”
BackgroundEffects of Proactive Dialogue Strategies on Human-Computer Trust
Kraus, M., Wagner, N., & Minker, W. (2020) · Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3340631.3394840 · Cited by 67 · cited 1×
“For instance, Kraus et al. [51] found that the perception of competence of a proactive CA is dependent on task difficulty.”
BackgroundThe Effect of Emojis when interacting with Conversational Interface Assisted Health Coaching System
Fadhil, A., Schiavo, G., Wang, Y., & Yilma, B. A. (2018) · Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare · doi:10.1145/3240925.3240965 · Cited by 62 · cited 4×
“Credibility trustworthy, truthfulness, confidence 15 [1][7][19][21][33][35][40][42][51][54][55][58][77][80][88]”
BackgroundInfluentialKocaballi, A. B., Sezgin, E., Clark, L., Carroll, J. M., Huang, Y., Huh-Yoo, J., Kim, J., Kocielnik, R., Lee, Y.-C., Mamykina, L., et al. (2022) · Journal of Medical Internet Research · doi:10.2196/38525 · Cited by 62 · cited 1×
“[48] exploring the challenges and opportunities of CAs in healthcare.”
BackgroundExploring Language Style in Chatbots to Increase Perceived Product Value and User Engagement
Elsholz, E., Chamberlain, J., & Kruschwitz, U. (2019) · Proceedings of the 2019 Conference on Human Information Interaction and Retrieval · doi:10.1145/3295750.3298956 · Cited by 58 · cited 1×
“The perception aspect of user satisfaction is commonly used to evaluate conversational agents within the customer service domain answering transactional inquiries (e.g. [16, 20, 25]).”
BackgroundEffects of voice-adaptation and social dialogue on perceptions of a robotic learning companion
Lubold, N., Walker, E., & Pon-Barry, H. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451760 · Cited by 57 · cited 6×
“Social Presence connectedness, familiarity, psychological distance 18 [5][6][7][10][14][16][25][26][27][28][43][45][53][56][57] [58][69][86]”
BackgroundInfluentialPositive Emotion Elicitation in Chat-Based Dialogue Systems
Lubis, N., Sakti, S., Yoshino, K., & Nakamura, S. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2900910 · Cited by 55 · cited 4×
“Content Affectiveness Affective Language emotional expressions, sentimentadaptive responses 11 [15][16][33][39][53][56][66][77][84][91][94]”
BackgroundInfluentialChoi, D., Kwak, D., Cho, M., & Lee, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376569 · Cited by 54 · cited 9×
“Social Presence connectedness, familiarity, psychological distance 18 [5][6][7][10][14][16][25][26][27][28][43][45][53][56][57] [58][69][86]”
BackgroundMethodInfluentialUser Perceptions of Extraversion in Chatbots after Repeated Use
Völkel, S. T., Schoedel, R., Kaya, L., & Mayer, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502058 · Cited by 53 · cited 8×
“Text Formatting capitalization, emoticons 6 [21][46][77][84][86][88]”
BackgroundInfluentialMy Bad! Repairing Intelligent Voice Assistant Errors Improves Interaction
Cuadra, A., Li, S., Lee, H., Cho, J., & Ju, W. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449101 · Cited by 52 · cited 3×
“…of the agent contains Godspeed questionnaire [3]’s set of questions on likeability used by [55], Subjective Assessment of Speech System Interface (SASSI) questionnaire [37]’s set of questions on likeability used by [7, 10] and questionnaire items like “ this voice agent was likeable ” used by [14].”
BackgroundMethodAn End-to-End Conversational Style Matching Agent
Hoegen, R., Aneja, D., McDuff, D., & Czerwinski, M. (2019) · Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents · doi:10.1145/3308532.3329473 · Cited by 51 · cited 4×
“A commonly used survey is adapted from the Godspeed questionnaire [3], which evaluates human-likeness based on users’ impression of the agent as fake / natural, machinelike / human-like, unconscious / conscious, and artificial / lifelike (used by [35], [41] and [70]).”
BackgroundMethodInfluentialMeasuring the Effect of Mental Health Chatbot Personality on User Engagement
Moilanen, J., Visuri, A., Suryanarayana, S. A., Alorwu, A., Yatani, K., & Hosio, S. J. (2022) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3568444.3568464 · Cited by 51 · cited 6×
“[54][55][58][62][64][66][75][78][82][84][89][90][94]”
BackgroundInfluentialPerceived Emotional Intelligence in Virtual Agents
Yang, Y., Ma, X., & Fung, P. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3053163 · Cited by 50 · cited 7×
“Content Affectiveness Affective Language emotional expressions, sentimentadaptive responses 11 [15][16][33][39][53][56][66][77][84][91][94]”
BackgroundInfluentialTalk to Me on My Level – Linguistic Alignment for Chatbots
Spillner, L., & Wenig, N. (2021) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3447526.3472050 · Cited by 50 · cited 1×
“58 unique codes were created as part of this process, capturing features like sentiment-adaptive responses [16], lexical alignment [78], and typos [86].”
BackgroundEmpathic Chatbot Response for Medical Assistance
Daher, K., Casas, J., Khaled, O. A., & Mugellini, E. (2020) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3383652.3423864 · Cited by 47 · cited 4×
“Sociability with Agent Conversation Tone friendly, warm, empathetic, persuasive 20 [2][7][13][14][15][16][19][33][38][39][42][43][46][53][62] [66][80][90][91][94]”
BackgroundMethodInfluentialDoes it matter if a computer jokes
Khooshabeh, P., McCall, C., Gandhe, S., Gratch, J., & Blascovich, J. (2011) · CHI '11 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1979742.1979604 · Cited by 45 · cited 4×
“Sociability with Agent Conversation Tone friendly, warm, empathetic, persuasive 20 [2][7][13][14][15][16][19][33][38][39][42][43][46][53][62] [66][80][90][91][94]”
BackgroundInfluentialEffects of Smart Virtual Assistants' Gender and Language
Habler, F., Schwind, V., & Henze, N. (2019) · Mensch und Computer (MuC) · doi:10.1145/3340764.3344441 · Cited by 44 · cited 4×
“For voice-based agents specifically, the element of prosody encompasses vocal qualities like pitch [31, 42], speech rate [10], and spoken accent [22].”
BackgroundInfluentialCan a Humorous Conversational Agent Enhance Learning Experience and Outcomes?
Ceha, J., Lee, K. J., Nilsen, E., Goh, J., & Law, E. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445068 · Cited by 44 · cited 1×
“Several studies in our reviewed corpus have explored the effect of humour on various perceptions of agents (e.g. [ 6, 43]).”
BackgroundKinVoices: Using Voices of Friends and Family in Voice Interfaces
Chan, S., Gunasekaran, T. S., Pai, Y. S., Zhang, H., & Nanayakkara, S. C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479590 · Cited by 43 · cited 6×
“In the case of Chan et al’s study [7], participants rated the agent using kin voices as significantly more likeable compared to the generic voices, but it was perceived as eerie.”
BackgroundMethodInfluentialPolite or Direct? Conversation Design of a Smart Display for Older Adults Based on Politeness Theory
Hu, Y., Qu, Y., Maus, A., & Mutlu, B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517525 · Cited by 43 · cited 6×
“Perception of Agent’s Humanness Human-likeness human-like, natural, artificial, machine-like 20 [2][6][7][10][13][16][25][30][39][41][42][56][58][63][69] [70][77][85][86][94]”
BackgroundInfluentialThe RoPE Scale: a Measure of How Empathic a Robot is Perceived
Charrier, L., Rieger, A., Galdeano, A., Cordier, A., Lefort, M., & Hassas, S. (2019) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2019.8673082 · Cited by 41 · cited 2×
“In another study that also assesses the perception of empathy [15], the RoPE Scale [8] is used with questions like “ the robot cares about my feelings ” or “ the robot comforts me when I am upset.”
MethodDoes Chatbot Language Formality Affect Users’ Self-Disclosure?
Cox, S. R., & Ooi, W. T. (2022) · Proceedings of the 4th Conference on Conversational User Interfaces · doi:10.1145/3543829.3543831 · Cited by 41 · cited 9×
“Perception of Agent’s Humanness Human-likeness human-like, natural, artificial, machine-like 20 [2][6][7][10][13][16][25][30][39][41][42][56][58][63][69] [70][77][85][86][94]”
BackgroundResultInfluentialWhat's on Your Virtual Mind? Mind Perception in Human-Agent Negotiations
Lee, M., Lucas, G., Mell, J., Johnson, E., & Gratch, J. S. (2019) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3308532.3329465 · Cited by 38 · cited 5×
“Social Presence connectedness, familiarity, psychological distance 18 [5][6][7][10][14][16][25][26][27][28][43][45][53][56][57] [58][69][86]”
BackgroundInfluentialKeep it Short: A Comparison of Voice Assistants’ Response Behavior
Haas, G., Rietzler, M., Jones, M., & Rukzio, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517684 · Cited by 38 · cited 6×
“Perception of Agent’s Humanness Human-likeness human-like, natural, artificial, machine-like 20 [2][6][7][10][13][16][25][30][39][41][42][56][58][63][69] [70][77][85][86][94]”
BackgroundInfluentialSystem Personality and Persuasion in Human-Computer Dialogue
Andrews, P. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2209310.2209315 · Cited by 35 · cited 2×
“One of the studies showed that the CA’s perceived anthropomorphized personality trait of agreeableness has an influence on the users’ perception of credibility [1].”
Background"I Can Feel Your Empathic Voice": Effects of Nonverbal Vocal Cues in Voice User Interface
Kim, J., Kim, W., Nam, J., & Song, H. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3383075 · Cited by 32 · cited 7×
“Social Presence connectedness, familiarity, psychological distance 18 [5][6][7][10][14][16][25][26][27][28][43][45][53][56][57] [58][69][86]”
BackgroundMethodInfluentialExploring Effects of Conversational Fillers on User Perception of Conversational Agents
Jeong, Y., Lee, J., & Kang, Y. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312913 · Cited by 31 · cited 10×
“Disfluency fillers, interjections, repetitions, typos 6 [5][38][41][85][86][90]”
BackgroundMethodInfluentialExamining User Preference for Agreeableness in Chatbots
Völkel, S. T., & Kaya, L. (2021) · CUI 2021 - 3rd Conference on Conversational User Interfaces · doi:10.1145/3469595.3469633 · Cited by 31 · cited 3×
“[54][55][58][62][64][66][75][78][82][84][89][90][94]”
BackgroundEffects of Emotional Expressiveness on Voice Chatbot Interactions
Zhu, Q., Chau, A., Cohn, M., Liang, K.-H., Wang, H., Zellou, G., & Yu, Z. (2022) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3543829.3543840 · Cited by 31
Shen, H., & Wang, M. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1921367 · Cited by 29 · cited 7×
“Alignment content matching, lexical alignment, agreeableness 7 [33][35][40][55][78][82][83]”
BackgroundInfluentialFeijóo-García, P. G., Zalake, M. S., de Siqueira, A. G., Lok, B. C. L., & Hamza-Lup, F. G. (2021) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3472306.3478367 · Cited by 29
Persuasive Synthetic Speech: Voice Perception and User Behaviour
Dubiel, M., Halvey, M., Gallegos, P. O., & King, S. (2020) · Proceedings of the 2nd Conference on Conversational User Interfaces · doi:10.1145/3405755.3406120 · Cited by 28 · cited 8×
“Studies have demonstrated that conversation architecture elements can be used to design perceptions of agents tomake CAsmore persuasive [19].”
BackgroundMethodInfluentialLinnemann, G. A., & Jucks, R. (2018) · Interacting with Computers · doi:10.1093/iwc/iwy005 · Cited by 26 · cited 10×
“[54][55][58][62][64][66][75][78][82][84][89][90][94]”
BackgroundMethodInfluentialJestin, I., Fischer, J. E., Trigo, M. J. G., Large, D. R., & Burnett, G. (2022) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3543829.3543836 · Cited by 26 · cited 10×
“For voice-based agents specifically, the element of prosody encompasses vocal qualities like pitch [31, 42], speech rate [10], and spoken accent [22].”
BackgroundInfluentialUsers, Tasks, and Conversational Agents: A Personality Study
Roy, Q., Ghafurian, M., Li, W., & Hoey, J. (2021) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3472307.3484173 · Cited by 22 · cited 8×
“[54][55][58][62][64][66][75][78][82][84][89][90][94]”
BackgroundInfluentialExpressive Auditory Gestures in a Voice-Based Pedagogical Agent
Ceha, J., & Law, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517599 · Cited by 18 · cited 7×
“Disfluency fillers, interjections, repetitions, typos 6 [5][38][41][85][86][90]”
BackgroundInfluentialMiehle, J., Minker, W., & Ultes, S. (2018) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3213586.3226213 · Cited by 17 · cited 6×
“[54][55][58][62][64][66][75][78][82][84][89][90][94]”
BackgroundInfluentialEnhancing the Perceived Emotional Intelligence of Conversational Agents through Acoustic Cues
Hu, J., Huang, Y., Hu, X., & Xu, Y. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451660 · Cited by 17 · cited 6×
“Disfluency fillers, interjections, repetitions, typos 6 [5][38][41][85][86][90]”
BackgroundInfluentialVölkel, S. T., Meindl, S., & Hussmann, H. (2021) · CUI 2021 - 3rd Conference on Conversational User Interfaces · doi:10.1145/3469595.3469605 · Cited by 16
Artificial personality and disfluency
Wester, M., Aylett, M. P., Tomalin, M., & Dall, R. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-141 · Cited by 13
Improving Relationships Based on Positive Politeness Between Humans and Life-Like Agents
Miyamoto, T., Katagami, D., & Shigemitsu, Y. (2017) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3125739.3132585 · Cited by 12 · cited 2×
“[54][55][58][62][64][66][75][78][82][84][89][90][94]”
Inquisitive mind: a conversational news companion
Dubiel, M., Cervone, A., & Riccardi, G. (2019) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3342775.3342802 · Cited by 10 · cited 1×
“Specifically for the element of agent initiated content, we found various research on conversation repairs [50, 74] and agents proactively sharing content with users [18, 92], but many of them did not not include assessments of users’ perceptions of CAs.”
BackgroundShould Speech Dialogue System Use Honorific Expression?
Ouchi, S., Mizumaru, K., Sakamoto, D., & Ono, T. (2019) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3349537.3352775 · Cited by 9 · cited 2×
“The element of formality describes the linguistic style used by a conversational agent, such as using formal language like honorific expressions to address users [70].”
BackgroundMethodConversational Agents Replying with a Manzai-style Joke
Go, K., Onishi, T., Ogushi, A., & Miyata, A. (2021) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3520495.3520517 · Cited by 7 · cited 3×
“Engagement enjoyment, annoyance, desirable, intention to use 28 [1][5][6][7][13][14][20][21][27][28][33][40][42][45][46]”
BackgroundWilhelm, M., Otten, T., Schwaetzer, E., & Schumacher, K. (2022) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3543829.3544533 · Cited by 6 · cited 6×
“Text Formatting capitalization, emoticons 6 [21][46][77][84][86][88]”
BackgroundMethodInfluentialYang, J., Kikuchi, H., Uegaki, T., & Kikuchi, H. (2021) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3472307.3484651 · Cited by 4 · cited 5×
“Disfluency fillers, interjections, repetitions, typos 6 [5][38][41][85][86][90]”
BackgroundInfluentialRelating to speech evoked car personalities
Healey, J., & Szostak, D. (2013) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2468356.2468652 · Cited by 1 · cited 9×
“Alignment content matching, lexical alignment, agreeableness 7 [33][35][40][55][78][82][83]”
BackgroundInfluential
Research works printed but not linked: 16
- Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health 11, 4 (2019), 589–597.
- Sarah E Finch and Jinho D Choi. 2020. Towards unified dialogue system evaluation: A comprehensive analysis of current evaluation protocols. arXiv preprint arXiv:2006.06110 (2020). · arXiv
- Ulrich Gnewuch, Stefan Morana, Marc TP Adam, and Alexander Maedche. 2018. ‘The Chatbot is typing…’–The Role of Typing Indicators in Human-Chatbot Interaction. In Proceedings of the 17th Annual Pre-ICIS Workshop on HCI Research in MIS. 0–5.
- Ulrich Gnewuch, Stefan Morana, Marc TP Adam, and Alexander Maedche. 2022. Opposing Effects of Response Time in Human–Chatbot Interaction. Business & Information Systems Engineering (2022), 1–19.
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
- Jennifer Hill, W Randolph Ford, and Ingrid G Farreras. 2015. Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. Computers in human behavior 49 (2015), 245–250.
- TM Holtgraves, Stephen J Ross, CR Weywadt, and TL Han. 2007. Perceiving artificial social agents. Computers in human behavior 23, 5 (2007), 2163–2174.
- Hyeji Kim, Inchan Jung, and Youn-kyung Lim. 2022. Understanding the Negative Aspects of User Experience in Human-likeness of Voice-based Conversational Agents. In Designing Interactive Systems Conference. 1418–1427.
- Kazunori Komatani and Hiroshi G Okuno. 2010. Online error detection of barge-in utterances by using individual users’ utterance histories in spoken dialogue system. In Proceedings of the SIGDIAL 2010 Conference. 289–296.
- Nour Kteily, Emile Bruneau, Adam Waytz, and Sarah Cotterill. 2015. The ascent of man: Theoretical and empirical evidence for blatant dehumanization.Journal of personality and social psychology 109, 5 (2015), 901.
- Qianli Ma, Yaping Zhang, Wenti Xu, and Ronggang Zhou. 2022. Ask a Further Question or Give a List? How Should Conversational Agents Reply to Users’ Uncertain Queries. International Journal of Human–Computer Interaction (2022), 1–15.
- Roger C Mayer and James H Davis. 1999. The effect of the performance appraisal system on trust for management: A field quasi-experiment.Journal of applied psychology 84, 1 (1999), 123.
- Kelly McConvey, Shion Guha, and Anastasia Kuzminykh. 2023. A Human-Centered Review of Algorithms in Decision-Making in Higher Education. arXiv preprint arXiv:2302.05839 (2023). · arXiv
- Elayne Ruane, Abeba Birhane, and Anthony Ventresque. 2019. Conversational AI: Social and Ethical Considerations.. In AICS. 104–115.
- Suzanne Tolmeijer, Naim Zierau, Andreas Janson, Jalil Sebastian Wahdatehagh, Jan Marco Marco Leimeister, and Abraham Bernstein. 2021. Female by default?–exploring the effect of voice assistant gender and pitch on trait and trust attribution. In Extended abstracts of the 2021 CHI conference on human factors in computing systems. 1–7.
- Ziang Xiao, Sarah Mennicken, Bernd Huber, Adam Shonkoff, and Jennifer Thom. 2021. Let Me Ask You This: How Can a Voice Assistant Elicit Explicit User Feedback?Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–24.
Cited by
Examining Humanness as a Metaphor to Design Voice User Interfaces
Desai, S., Dubiel, M., & Leiva, L. A. (2024) · ACM Conversational User Interfaces 2024 · doi:10.1145/3640794.3665535 · Cited by 21
“You can also adjust the coverage amount based on your most critical financial needs to find a balance between sufficient coverage and affordability. two items on 7-point Likert scale: “If available, I intend to start using Z within the next month,” and “If available, in the next months, I plan to…”
BackgroundMethodToward Metaphor-Fluid Conversation Design for Voice User Interfaces
Desai, S., Chin, J., Wang, D., Cowan, B. R., & Twidale, M. B. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816223 · Cited by 16
Sun, X., Liu, Y., Bosch, J. A., & Li, Z. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711014 · Cited by 18
“For instance, the voice characteristics in voice assistants, such as tone and gender, play a critical role [63, 132, 161].”
Rahman, H., & Desai, S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790388 · Cited by 21
“To evaluate user perception of the CA, we use six measures covering Intelligence, Enjoyment, An-thropomorphism, Intention to Adopt, Trust, and Likeability based on Wei et al. [130]’s framework linking user perceptions to agent relationships.”
Tickling Proactivity: Exploring the Use of Humor in Proactive Voice Assistants
Zargham, N., Reicherts, L., Avanesi, V., Rogers, Y., & Malaka, R. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627777 · Cited by 12
“Wei et al. [92] found that users find humorous agents more friendly, intimate, and similar to themselves compared to their non-humorous counterparts.”
Background
Metadata
- Authors
- Wei, C. Z., Kim, Y.-H., & Kuzminykh, A.
- Affiliations
- University of Toronto (CA)
- NAVER Cloud (South Korea) (KR)
- Venue
- Proceedings of the 5th International Conference on Conversational User Interfaces
- Publisher
- ACM
- Published
- 2023-07-19
- Pages
- 1-16
- OpenAlex
- W4384520790
- Cited by
- 14
- References
- 74
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 4 · 2023: 1
- FWCI
- 2.48
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- chatbots, user perception, sociability, humanness, conversational design, speech synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.