UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital Library
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Early conversational agents (CAs) focused on dyadic human-AI interaction between humans and the CAs, followed by the increasing popularity of polyadic human-AI interaction, in which CAs are designed to mediate human-human interactions. CAs for polyadic interactions are unique because they encompass hybrid social interactions, i.e., human-CA, human-to-human, and human-to-group behaviors. However, research on polyadic CAs is scattered across different fields, making it challenging to identify, compare, and accumulate existing knowledge. To promote the future design of CA systems, we conducted a literature review of ACM publications and identified a set of works that conducted UX (user experience) research. We qualitatively synthesized the effects of polyadic CAs into four aspects of human-human interactions, i.e., communication, engagement, connection, and relationship maintenance. Through a mixed-method analysis of the selected polyadic and dyadic CA studies, we developed a suite of evaluation measurements on the effects. Our findings show that designing with social boundaries, such as privacy, disclosure, and identification, is crucial for ethical polyadic CAs. Future research should also advance usability testing methods and trust-building guidelines for conversational AI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
The ripple effect: emotional contagion and its influence on group behavior
Barsade, S. G. (2002) · Administrative Science Quarterly · doi:10.2307/3094912 · Cited by 3,361
Enhancing the Effectiveness of Work Groups and Teams
Kozlowski, S. W. J., & Ilgen, D. R. (2006) · Psychological Science in the Public Interest · doi:10.1111/j.1529-1006.2006.00030.x · Cited by 2,623
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 2×
“, "the system language and behaviors do not reinforce undesirable and unfair stereotypes and biases given their social and cultural context") [6].”
BackgroundMethodNass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 1×
“as Social Actors paradigm [120], CAs are considered as "social actors".”
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388 · cited 6×
“reviews [44, 88, 139, 165] and was developed through several iterations and refnement by the research team, in line with prior HCI work.”
BackgroundInfluentialMcDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
AI-based chatbots in customer service and their effects on user compliance
Adam, M., Wessel, M., & Benlian, A. (2021) · Electronic Markets · doi:10.1007/s12525-020-00414-7 · Cited by 1,195 · cited 1×
“Most of the existing empirical studies focus on designing dyadic CAs [2, 42, 91, 148, 158], without addressing the unique design challenges of polyadic CAs.”
BackgroundChatbots: History, technology, and applications
Adamopoulou, E., & Moussiades, L. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100006 · Cited by 1,090 · cited 1×
“As promising Artifcial intelligence (AI) technologies, conversational agents are defned as "software that accepts natural language as input and generates natural language as output, engaging in a conversation with the user" [57]; chatbots, meanwhile, are computer programs designed to simulate conversation with human u…”
BackgroundMcTear, M., Callejas, Z., & Griol, D. (2016) · doi:10.1007/978-3-319-32967-3 · Cited by 594
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260288 · Cited by 579
Chaves, A. P., & Gerosa, M. A. (2021) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1841438 · Cited by 572 · cited 1×
“[39, 44, 88, 113, 153], they do not address how polyadic CAs are designed and evaluated, nor present the efects of using polyadic CAs”
BackgroundStephanidis, C., Salvendy, G., Antona, M., Chen, J. Y. C., Dong, J., Duffy, V. G., Fang, X., Fidopiastis, C., Fragomeni, G., Fu, L. P., et al. (2019) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2019.1619259 · Cited by 524
Evaluating and Informing the Design of Chatbots
Jain, M., Kumar, P., Kota, R., & Patel, S. N. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196735 · Cited by 376
Systematic Review: Trust-Building Factors and Implications for Conversational Agent Design
Rheu, M., Shin, J. Y., Peng, W., & Huh-Yoo, J. (2021) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1807710 · Cited by 360
"I Hear You, I Feel You": Encouraging Deep Self-disclosure through a Chatbot
Lee, Y.-C., Yamashita, N., Huang, Y., & Fu, W. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376175 · Cited by 339 · cited 2×
“, [33, 78, 93, 94, 137], perceived CA collecting metrics regarding user self-reported personal experience characteristics such as capability in handling requests [52], personresponses.”
BackgroundSelf-citationThe Personalization of Conversational Agents in Health Care: Systematic Review
Kocaballi, A. B., Berkovsky, S., Quiroz, J. C., Laranjo, L., Tong, H. L., Rezazadegan, D., Briatore, A., & Coiera, E. (2019) · Journal of Medical Internet Research · doi:10.2196/15360 · Cited by 332 · cited 1×
“Others are more specifc, such as studying CAs’ emotional intelligence [127], personalization [80], trust-building [142], and human-likeness [170].”
Background"Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage
Sciuto, A., Saini, A., Forlizzi, J., & Hong, J. I. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196772 · Cited by 327
Survey of conversational agents in health
Montenegro, J. L. Z., da Costa, C. A., & da Rosa Righi, R. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.03.054 · Cited by 322 · cited 1×
“[39, 44, 88, 113, 153], they do not address how polyadic CAs are designed and evaluated, nor present the efects of using polyadic CAs”
BackgroundTaking the time to care: empowering low health literacy hospital patients with virtual nurse agents
Bickmore, T. W., Pfeifer, L. M., & Jack, B. W. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518891 · Cited by 289
Designing mobile interfaces for novice and low-literacy users
Medhi, I., Patnaik, S., Brunskill, E., Gautama, S. N. N., Thies, W., & Toyama, K. (2011) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1959022.1959024 · Cited by 285
A Survey on Conversational Agents/Chatbots Classification and Design Techniques
Hussain, S., Ameri Sianaki, O., & Ababneh, N. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-15035-8_93 · Cited by 279
Self-Care Technologies in HCI: Trends, Tensions, and Opportunities
Nunes, F., Verdezoto, N., Fitzpatrick, G., Kyng, M., Grönvall, E., & Storni, C. (2015) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2803173 · Cited by 258
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247 · cited 1×
“popularity of CA applications exists in multiple domains, studies repeatedly reported CAs causing both pragmatic issues, in which chatbots failed to understand or to help users achieve their intended goals [50, 51, 93], and issues in which CAs failed to engage users”
BackgroundI Can Help You Change! An Empathic Virtual Agent Delivers Behavior Change Health Interventions
Lisetti, C., Amini, R., Yasavur, U., & Rishe, N. (2013) · ACM Transactions on Management Information Systems · doi:10.1145/2544103 · Cited by 242
Fox, J., & Gambino, A. (2021) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2020.0181 · Cited by 225 · cited 2×
“For example, prior work [55] identifed that human users’ expectations and perceptions towards machines are diferent from those towards human partners.”
BackgroundResilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224
On Technological Determinism: A Typology, Scope Conditions, and a Mechanism
Dafoe, A. (2015) · Science, Technology, & Human Values · doi:10.1177/0162243915579283 · Cited by 221
Response to a relational agent by hospital patients with depressive symptoms
Bickmore, T. W., Mitchell, S. E., Jack, B. W., Paasche-Orlow, M. K., Pfeifer, L. M., & O’Donnell, J. (2010) · Interacting with Computers · doi:10.1016/j.intcom.2009.12.001 · Cited by 218
AI as a moral crumple zone: The effects of AI-mediated communication on attribution and trust
Hohenstein, J., & Jung, M. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2019.106190 · Cited by 212 · cited 1×
“, 2018], they do not address how polyadic CAs are designed and evaluated, nor present the effects of using polyadic CAs on handling the challenges of human-human interaction [Hohenstein and Jung, 2020].”
BackgroundThe State of Speech in HCI: Trends, Themes and Challenges
Clark, L., Doyle, P., Garaialde, D., Gilmartin, E., Schlögl, S., Edlund, J., Aylett, M., Cabral, J., Munteanu, C., Edwards, J., et al. (2019) · Interacting with Computers · doi:10.1093/iwc/iwz016 · Cited by 193 · cited 1×
“This was also found by prior literature reviews of CAs [41, 139].”
ResultDesign Features of Embodied Conversational Agents in eHealth: a Literature Review
ter Stal, S., Kramer, L. L., Tabak, M., op den Akker, H., & Hermens, H. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102409 · Cited by 188
Conceptual Metaphors Impact Perceptions of Human-AI Collaboration
Khadpe, P., Krishna, R., Fei-Fei, L., Hancock, J. T., & Bernstein, M. S. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415234 · Cited by 183 · cited 1×
“Moreover, prior work also an overview of e what metrics uses usability questions from previous surv ys [75].”
BackgroundTrusting Virtual Agents: The Effect of Personality
Zhou, M. X., Mark, G., Li, J., & Yang, H. (2019) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3232077 · Cited by 176
Acceptance and usability of a relational agent interface by urban older adults
Bickmore, T. W., Caruso, L., & Clough-Gorr, K. (2005) · CHI '05 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1056808.1056879 · Cited by 142 · cited 4×
“, [15, 45, 97] and perceived ality [78, 137], and trust [89].”
BackgroundInfluentialUnderstanding Chatbot-mediated Task Management
Toxtli, C., Monroy-Hernández, A., & Cranshaw, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173632 · Cited by 127 · cited 5×
“, Google Allo) [66] and two papers [9, 168]), two to virtual meetings, e.”
BackgroundMethodInfluentialWang, Q., Saha, K., Gregori, E., Joyner, D., & Goel, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445645 · Cited by 120 · cited 4×
“, verbosity [Wang et al., 2021], readability [Wang et al.”
BackgroundMethodInfluentialLiao, Q. V., Mas-ud Hussain, M., Chandar, P., Davis, M., Khazaeni, Y., Crasso, M. P., Wang, D., Muller, M., Shami, N. S., & Geyer, W. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173577 · Cited by 116
Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop
Cranshaw, J., Elwany, E., Newman, T., Kocielnik, R., Yu, B., Soni, S., Teevan, J., & Monroy-Hernández, A. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025780 · Cited by 106
Beyond Dyadic Interactions: Considering Chatbots as Community Members
Seering, J., Luria, M., Kaufman, G., & Hammer, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300680 · Cited by 97
Effects of Persuasive Dialogues: Testing Bot Identities and Inquiry Strategies
Shi, W., Wang, X., Oh, Y. J., Zhang, J., Sahay, S., & Yu, Z. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376843 · Cited by 93 · cited 1×
“, 2020], and perceived naturalness [Shi et al., 2020].”
BackgroundTegos, S., Demetriadis, S., & Karakostas, A. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.07.014 · Cited by 90
Convey: Exploring the Use of a Context View for Chatbots
Jain, M., Kota, R., Kumar, P., & Patel, S. N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174042 · Cited by 74
AI-Supported Messaging: An Investigation of Human-Human Text Conversation with AI Support
Hohenstein, J., & Jung, M. (2018) · Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3170427.3188487 · Cited by 48
A Strategy for Information Presentation in Spoken Dialog Systems
Demberg, V., Winterboer, A., & Moore, J. D. (2011) · Computational Linguistics · doi:10.1162/coli_a_00064 · Cited by 34 · cited 2×
“, [15, 45, 97] and perceived ality [78, 137], and trust [89].”
BackgroundChats with Bots: Balancing Imitation and Engagement
Muresan, A., & Pohl, H. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3313084 · Cited by 34
Towards Academically Productive Talk Supported by Conversational Agents
Dyke, G., Howley, I., Adamson, D., Kumar, R., & Rosé, C. P. (2013) · Productive Multivocality in the Analysis of Group Interactions · doi:10.1007/978-1-4614-8960-3_25 · Cited by 21
Designing Chatbots as Community-Owned Agents
Luria, M., Seering, J., Forlizzi, J., & Zimmerman, J. (2020) · Proceedings of the 2nd Conference on Conversational User Interfaces · doi:10.1145/3405755.3406134 · Cited by 5 · cited 8×
“engages community members as a whole and opens opportunities for newcomers to participate in interactions as a way to welcome them on board [101, 154].”
BackgroundMethodInfluential
Cited by
A multifaceted vision of the human-AI collaboration: a comprehensive review
Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L. (2025) · IEEE Access · doi:10.1109/access.2025.3536095 · Cited by 73
“Those efforts aim to support the iterative design, development, and dissemination processes of learning systems oriented toward conversational AI agents [330] or to define a human-centered thinking approach to applying IML methods [82].”
IntroBot: Exploring the Use of Chatbot-assisted Familiarization in Online Collaborative Groups
Shin, D., Kim, S., Shang, R., Lee, J., & Hsieh, G. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580930 · Cited by 37
“The idea of using polyadic chatbot has also been explored for relationship-building among strangers [84].”
BackgroundInfluentialTang, Y., Zhang, N., Ciancia, M., & Wang, Z. (2024) · Companion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing · doi:10.1145/3678884.3681890 · Cited by 31
Self-citationWei, C. Z., Kim, Y.-H., & Kuzminykh, A. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3597139 · Cited by 14
“In accordance with accepted practices, we defined the scope of our literature review by considering two factors: (1) time—specifically publications from 2010 to 2022—and (2) source—using the ACMDigital Library [60, 79, 93].”
MethodSun, J., Zhang, Z., Wang, M., Li, N., Lu, Z., Xiang, Y., Zhang, L., Zhang, Y., Wang, Q., & Fan, M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713810 · Cited by 12
Metadata
- Authors
- Zheng, Q., Tang, Y., Liu, Y., Liu, W., & Huang, Y.
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-24
- OpenAlex
- W4225115595
- Cited by
- 100
- References
- 48
- FWCI
- 38.00
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.