CoChat

The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational Agents

Wei, Christina Ziying, Kim, Young-Ho, Kuzminykh, Anastasia · 2023 · Proceedings of the 5th International Conference on Conversational User Interfaces

The DOI matches its Crossref record; 73 of the 74 references it cites were found in a registry or an index.

Abstract

Conversational agents mimic natural conversation to interact with users. Since the effectiveness of interactions strongly depends on users’ perception of agents, it is crucial to design agents’ behaviors to provide the intended user perceptions. Research on human-agent and human-human communication suggests that speech specifics are associated with perceptions of communicating parties, but there is a lack of systematic understanding of how speech specifics of agents affect users’ perceptions. To address this gap, we present a framework outlining the relationships between elements of agents’ conversation architecture (dialog strategy, content affectiveness, content style and speech format) and aspects of users’ perception (interaction, ability, sociability and humanness). Synthesized based on literature reviewed from the domains of HCI, NLP and linguistics (n=57), this framework demonstrates both the identified relationships and the areas lacking empirical evidence. We discuss the implications of the framework for conversation design and highlight the inconsistencies with terminology and measurements.

Overview

Integrity signals

Registry

Crossref · 10.1145/3571884.3597139

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 73 references

Research works printed but not linked: 16

  1. Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health 11, 4 (2019), 589–597.
  2. Sarah E Finch and Jinho D Choi. 2020. Towards unified dialogue system evaluation: A comprehensive analysis of current evaluation protocols. arXiv preprint arXiv:2006.06110 (2020). · arXiv
  3. Ulrich Gnewuch, Stefan Morana, Marc TP Adam, and Alexander Maedche. 2018. ‘The Chatbot is typing…’–The Role of Typing Indicators in Human-Chatbot Interaction. In Proceedings of the 17th Annual Pre-ICIS Workshop on HCI Research in MIS. 0–5.
  4. Ulrich Gnewuch, Stefan Morana, Marc TP Adam, and Alexander Maedche. 2022. Opposing Effects of Response Time in Human–Chatbot Interaction. Business & Information Systems Engineering (2022), 1–19.
  5. Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
  6. Jennifer Hill, W Randolph Ford, and Ingrid G Farreras. 2015. Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. Computers in human behavior 49 (2015), 245–250.
  7. TM Holtgraves, Stephen J Ross, CR Weywadt, and TL Han. 2007. Perceiving artificial social agents. Computers in human behavior 23, 5 (2007), 2163–2174.
  8. Hyeji Kim, Inchan Jung, and Youn-kyung Lim. 2022. Understanding the Negative Aspects of User Experience in Human-likeness of Voice-based Conversational Agents. In Designing Interactive Systems Conference. 1418–1427.
  9. Kazunori Komatani and Hiroshi G Okuno. 2010. Online error detection of barge-in utterances by using individual users’ utterance histories in spoken dialogue system. In Proceedings of the SIGDIAL 2010 Conference. 289–296.
  10. Nour Kteily, Emile Bruneau, Adam Waytz, and Sarah Cotterill. 2015. The ascent of man: Theoretical and empirical evidence for blatant dehumanization.Journal of personality and social psychology 109, 5 (2015), 901.
  11. Qianli Ma, Yaping Zhang, Wenti Xu, and Ronggang Zhou. 2022. Ask a Further Question or Give a List? How Should Conversational Agents Reply to Users’ Uncertain Queries. International Journal of Human–Computer Interaction (2022), 1–15.
  12. Roger C Mayer and James H Davis. 1999. The effect of the performance appraisal system on trust for management: A field quasi-experiment.Journal of applied psychology 84, 1 (1999), 123.
  13. Kelly McConvey, Shion Guha, and Anastasia Kuzminykh. 2023. A Human-Centered Review of Algorithms in Decision-Making in Higher Education. arXiv preprint arXiv:2302.05839 (2023). · arXiv
  14. Elayne Ruane, Abeba Birhane, and Anthony Ventresque. 2019. Conversational AI: Social and Ethical Considerations.. In AICS. 104–115.
  15. Suzanne Tolmeijer, Naim Zierau, Andreas Janson, Jalil Sebastian Wahdatehagh, Jan Marco Marco Leimeister, and Abraham Bernstein. 2021. Female by default?–exploring the effect of voice assistant gender and pitch on trait and trust attribution. In Extended abstracts of the 2021 CHI conference on human factors in computing systems. 1–7.
  16. Ziang Xiao, Sarah Mennicken, Bernd Huber, Adam Shonkoff, and Jennifer Thom. 2021. Let Me Ask You This: How Can a Voice Assistant Elicit Explicit User Feedback?Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–24.

Cited by

See all 23

Metadata

Authors
Wei, C. Z., Kim, Y.-H., & Kuzminykh, A.
Affiliations
  • University of Toronto (CA)
  • NAVER Cloud (South Korea) (KR)
Venue
Proceedings of the 5th International Conference on Conversational User Interfaces
Publisher
ACM
Published
2023-07-19
Pages
1-16
OpenAlex
W4384520790
Cited by
14
References
74
Citations by year
2026: 4 · 2025: 5 · 2024: 4 · 2023: 1
FWCI
2.48
Type
conference paper
Language
en
Open access
closed
Locations
ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
chatbots, user perception, sociability, humanness, conversational design, speech synthesis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational Agents. https://app.cochat.ai/papers/10.1145/3571884.3597139
The paper
Wei, C. Z., Kim, Y.-H., & Kuzminykh, A. (2023). The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational Agents. Proceedings of the 5th International Conference on Conversational User Interfaces. https://doi.org/10.1145/3571884.3597139

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.