The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational Agents
The DOI matches its Crossref record; 73 of the 74 references it cites were found in a registry or an index.
Abstract
Conversational agents mimic natural conversation to interact with users. Since the effectiveness of interactions strongly depends on users’ perception of agents, it is crucial to design agents’ behaviors to provide the intended user perceptions. Research on human-agent and human-human communication suggests that speech specifics are associated with perceptions of communicating parties, but there is a lack of systematic understanding of how speech specifics of agents affect users’ perceptions. To address this gap, we present a framework outlining the relationships between elements of agents’ conversation architecture (dialog strategy, content affectiveness, content style and speech format) and aspects of users’ perception (interaction, ability, sociability and humanness). Synthesized based on literature reviewed from the domains of HCI, NLP and linguistics (n=57), this framework demonstrates both the identified relationships and the areas lacking empirical evidence. We discuss the implications of the framework for conversation design and highlight the inconsistencies with terminology and measurements.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“which are widely employed in VUI re-search [71, 80]”
From Examining Humanness as a Metaphor to Design Voice User Interfaces (2024) · cites this paper as [80]
In the citing paper
“This classification categorizes user perceptions of CUIs into two relevant categories: (1) perception of interaction with agents and (2) perception of agent’s characteristics.”
From Examining Humanness as a Metaphor to Design Voice User Interfaces (2024) · cites this paper as [80]
Citations
Cites
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307 · cited 1×
“Sometimes the Big-5 personality traits questionnaire [29] is used to map an agent’s disposition on various personality dimensions.”
MethodThe Uncanny Valley [From the Field]
Mori, M., MacDorman, K., & Kageki, N. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2192811 · Cited by 3,542 · cited 1×
“This may be a warning indicator to beware of the uncanny valley effect [67] when designing conversation architecture elements to elicit anthropomorphized perceptions from users.”
BackgroundBartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 4×
“A commonly used survey is adapted from the Godspeed questionnaire [3], which evaluates human-likeness based on users’ impression of the agent as fake / natural, machinelike / human-like, unconscious / conscious, and artificial / lifelike (used by [35], [41] and [70]).”
MethodInfluentialO’Brien, H. L., Cairns, P., & Hall, M. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.01.004 · Cited by 1,014 · cited 1×
“Some commonly used methods to evaluate engagement include the annoyance portion of the SASSI questionnaire [37], and the Use Engagement Scale (UES) [71].”
MethodRapp, A., Curti, L., & Boldi, A. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102630 · Cited by 643 · cited 2×
“Based on search terms used in previously published literature reviews [11, 73], the following keywords are used to search for publications related to conversational user interfaces:”
MethodA Taxonomy of Social Cues for Conversational Agents
Feine, J., Gnewuch, U., Morana, S., & Maedche, A. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.07.009 · Cited by 567 · cited 3×
“…elements of conversation architecture affect aspects of users’ perceptions or discussed the speech specifics of agents as part of a broader analysis of the state of research [11, 23], there is no comprehensive analysis of conversation architecture elements that are relevant to user perceptions.”
BackgroundTHE DEVELOPMENT OF A MEASURE OF PERCEIVED HOMOPHILY IN INTERPERSONAL COMMUNICATION
MCCROSKEY, J. C., RICHMOND, V. P., & DALY, J. A. (1975) · Human Communication Research · doi:10.1111/j.1468-2958.1975.tb00281.x · Cited by 496 · cited 1×
“Some commonly used methods to assess intimacy include the set of social attraction questions from the interpersonal attraction questionnaire [61] and the quality of relationship inventory (QRI) [72].”
MethodTHE MEASUREMENT OF TRUST AND ITS RELATIONSHIP TO SELF-DISCLOSURE
WHEELESS, L. R., & GROTZ, J. (1977) · Human Communication Research · doi:10.1111/j.1468-2958.1977.tb00523.x · Cited by 474 · cited 1×
“Some commonly used methods to evaluate trust include the Trust Propensity Scale [59] and the Individualized Trust Scale (ITS) [87].”
MethodWhat Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents
Clark, L., Pantidi, N., Cooney, O., Doyle, P., Garaialde, D., Edwards, J., Spillane, B., Gilmartin, E., Murad, C., Munteanu, C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300705 · Cited by 461 · cited 4×
“Currently, CAs are considered as a user-controlled tool rather than a social companion, with a focus on the utilitarian aspects of the conversation [12].”
BackgroundInfluential
Research works printed but not linked: 16
- Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health 11, 4 (2019), 589–597.
- Sarah E Finch and Jinho D Choi. 2020. Towards unified dialogue system evaluation: A comprehensive analysis of current evaluation protocols. arXiv preprint arXiv:2006.06110 (2020). · arXiv
- Ulrich Gnewuch, Stefan Morana, Marc TP Adam, and Alexander Maedche. 2018. ‘The Chatbot is typing…’–The Role of Typing Indicators in Human-Chatbot Interaction. In Proceedings of the 17th Annual Pre-ICIS Workshop on HCI Research in MIS. 0–5.
- Ulrich Gnewuch, Stefan Morana, Marc TP Adam, and Alexander Maedche. 2022. Opposing Effects of Response Time in Human–Chatbot Interaction. Business & Information Systems Engineering (2022), 1–19.
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
- Jennifer Hill, W Randolph Ford, and Ingrid G Farreras. 2015. Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. Computers in human behavior 49 (2015), 245–250.
- TM Holtgraves, Stephen J Ross, CR Weywadt, and TL Han. 2007. Perceiving artificial social agents. Computers in human behavior 23, 5 (2007), 2163–2174.
- Hyeji Kim, Inchan Jung, and Youn-kyung Lim. 2022. Understanding the Negative Aspects of User Experience in Human-likeness of Voice-based Conversational Agents. In Designing Interactive Systems Conference. 1418–1427.
- Kazunori Komatani and Hiroshi G Okuno. 2010. Online error detection of barge-in utterances by using individual users’ utterance histories in spoken dialogue system. In Proceedings of the SIGDIAL 2010 Conference. 289–296.
- Nour Kteily, Emile Bruneau, Adam Waytz, and Sarah Cotterill. 2015. The ascent of man: Theoretical and empirical evidence for blatant dehumanization.Journal of personality and social psychology 109, 5 (2015), 901.
- Qianli Ma, Yaping Zhang, Wenti Xu, and Ronggang Zhou. 2022. Ask a Further Question or Give a List? How Should Conversational Agents Reply to Users’ Uncertain Queries. International Journal of Human–Computer Interaction (2022), 1–15.
- Roger C Mayer and James H Davis. 1999. The effect of the performance appraisal system on trust for management: A field quasi-experiment.Journal of applied psychology 84, 1 (1999), 123.
- Kelly McConvey, Shion Guha, and Anastasia Kuzminykh. 2023. A Human-Centered Review of Algorithms in Decision-Making in Higher Education. arXiv preprint arXiv:2302.05839 (2023). · arXiv
- Elayne Ruane, Abeba Birhane, and Anthony Ventresque. 2019. Conversational AI: Social and Ethical Considerations.. In AICS. 104–115.
- Suzanne Tolmeijer, Naim Zierau, Andreas Janson, Jalil Sebastian Wahdatehagh, Jan Marco Marco Leimeister, and Abraham Bernstein. 2021. Female by default?–exploring the effect of voice assistant gender and pitch on trait and trust attribution. In Extended abstracts of the 2021 CHI conference on human factors in computing systems. 1–7.
- Ziang Xiao, Sarah Mennicken, Bernd Huber, Adam Shonkoff, and Jennifer Thom. 2021. Let Me Ask You This: How Can a Voice Assistant Elicit Explicit User Feedback?Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–24.
Cited by
Examining Humanness as a Metaphor to Design Voice User Interfaces
Desai, S., Dubiel, M., & Leiva, L. A. (2024) · ACM Conversational User Interfaces 2024 · doi:10.1145/3640794.3665535 · Cited by 21
“You can also adjust the coverage amount based on your most critical financial needs to find a balance between sufficient coverage and affordability. two items on 7-point Likert scale: “If available, I intend to start using Z within the next month,” and “If available, in the next months, I plan to…”
BackgroundMethodToward Metaphor-Fluid Conversation Design for Voice User Interfaces
Desai, S., Chin, J., Wang, D., Cowan, B. R., & Twidale, M. B. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816223 · Cited by 16
Sun, X., Liu, Y., Bosch, J. A., & Li, Z. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711014 · Cited by 18
“For instance, the voice characteristics in voice assistants, such as tone and gender, play a critical role [63, 132, 161].”
Rahman, H., & Desai, S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790388 · Cited by 19
“To evaluate user perception of the CA, we use six measures covering Intelligence, Enjoyment, An-thropomorphism, Intention to Adopt, Trust, and Likeability based on Wei et al. [130]’s framework linking user perceptions to agent relationships.”
Tickling Proactivity: Exploring the Use of Humor in Proactive Voice Assistants
Zargham, N., Reicherts, L., Avanesi, V., Rogers, Y., & Malaka, R. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627777 · Cited by 12
“Wei et al. [92] found that users find humorous agents more friendly, intimate, and similar to themselves compared to their non-humorous counterparts.”
Background
Metadata
- Authors
- Wei, C. Z., Kim, Y.-H., & Kuzminykh, A.
- Affiliations
- University of Toronto (CA)
- NAVER Cloud (South Korea) (KR)
- Venue
- Proceedings of the 5th International Conference on Conversational User Interfaces
- Publisher
- ACM
- Published
- 2023-07-19
- Pages
- 1-16
- OpenAlex
- W4384520790
- Cited by
- 14
- References
- 74
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 4 · 2023: 1
- FWCI
- 2.48
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- chatbots, user perception, sociability, humanness, conversational design, speech synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational Agents. https://app.cochat.ai/papers/10.1145/3571884.3597139
- The paper
- Wei, C. Z., Kim, Y.-H., & Kuzminykh, A. (2023). The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational Agents. Proceedings of the 5th International Conference on Conversational User Interfaces. https://doi.org/10.1145/3571884.3597139
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.