"Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents
The DOI matches its Crossref record; 49 of the 51 references it cites were found in a registry or an index.
Abstract
The past four years have seen the rise of conversational agents (CAs) in everyday life. Apple, Microsoft, Amazon, Google and Facebook have all embedded proprietary CAs within their software and, increasingly, conversation is becoming a key mode of human-computer interaction. Whilst we have long been familiar with the notion of computers that speak, the investigative concern within HCI has been upon multimodality rather than dialogue alone, and there is no sense of how such interfaces are used in everyday life. This paper reports the findings of interviews with 14 users of CAs in an effort to understand the current interactional factors affecting everyday use. We find user expectations dramatically out of step with the operation of the systems, particularly in terms of known machine intelligence, system capability and goals. Using Norman's 'gulfs of execution and evaluation' [30] we consider the implications of these findings for the design of future systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Whereas, we found that the challenges our participants experienced were more elementary.”
From Exploring How Older Adults Use a Smart Speaker–Based Voice Assistant in Their First Interactions: Qualitative Study (2021) · cites this paper as [32]
In the citing paper
“In addition, it could be more important for novice users to have an appropriate mental model and understand the system rather than an efficient user experience [32].”
From Exploring How Older Adults Use a Smart Speaker–Based Voice Assistant in Their First Interactions: Qualitative Study (2021) · cites this paper as [32]
In the citing paper
“human-AI mixed-initiative presents a generally valuable and underexplored interface paradigm.”
From Mixed-Initiative Creative Interfaces (2017) · cites this paper as [20]
In the citing paper
“Not only do these breakdowns significantly diminish user satisfaction and damage trust, they also decrease the likelihood of continued use [18, 44].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [44]
In the citing paper
“This is because when voice assistants fail, such as by incorrectly answering a question, it derails user trust [27, 35].”
From A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures (2023) · cites this paper as [35]
17 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability
Guest, G., Bunce, A., & Johnson, L. (2006) · Field Methods · doi:10.1177/1525822x05279903 · Cited by 20,808
(2010) · Sociology for Pharmacists · doi:10.4324/9780203381175_chapter_9 · Cited by 14,152
Norman, D. (2016) · Vahlen eBooks · doi:10.15358/9783800648108 · Cited by 6,374
Thematic networks: an analytic tool for qualitative research
Attride-Stirling, J. (2001) · Qualitative Research · doi:10.1177/146879410100100307 · Cited by 5,506 · cited 2×
“The thematic network is a ‘web-like coding structure’ [2; 385] that allows for a rich understanding of the conceptual interplay occurring within the narrative.”
Background(1997) · Choice Reviews Online · doi:10.5860/choice.34-3702 · Cited by 4,887
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,738
Licklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938 · cited 1×
“When considering human-machine dialogue from an HCI perspective, some of the earliest work can be traced back to vision statements and position papers such as JCR Licklider (1960), ‘Man-machine symbiosis’ [25] and the early work of Richard Bolt [4].”
MethodEmbodied Conversational Agents
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/2697.001.0001 · Cited by 1,799
Interaction Ritual: Essays on Face-to-face Behaviour
Goffman, E. (1972) · Medical Entomology and Zoology · Cited by 1,449
Research works printed but not linked: 14
- Boden Margaret A, 2007, Proceedings of Artificial Companions in Society: Perspectives on the Present and Future. Oxford Internet Institute.
- Davis Martin, Engines of Logic: Mathematicians and the Origin of the Computer
- Glass James R., 1999, Challenges for Spoken Dialogue Systems. In the proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding. (ASRU), CO, USA
- 10.5555/1941863 · doi:10.5555/1941863
- Heuwinkel Kerstin, Your Virtual Butler: The Making-of. Robert Trappl (ed)
- Robnett Licklider Joseph Carl, 1960, HFE-1,(March
- Moore Roger, Spoken Language Processing: Where do we go from Here? In Your Virtual Butler: The Making-of. Robert Trappl (ed)
- Payr Sabine, Your Virtual Butler: The Making-of. Robert Trappl (ed)
- Preece Jenny, 2015, Interaction Design: Beyond Huma- Computer Interaction
- Pulman Stephen, Proceedings of the 2010 Workshop on Companionable Dialogue Systems. Assoc. for Computational Linguistics
- Trappl Robert, From Jeeves Jeannie to Siri, and Then? In Your Virtual Butler: The Making-of. Robert Trappl (ed)
- Vassallo Giorgio, Semantic Computing. Sheu, Yu, Ramamoorthy, Joshi & Zadeh.(eds)
- 10.5555/1870559.1870562 · doi:10.5555/1870559.1870562
- Wilks Yorick, 2010, Amsterdam: John Benjamins.
Cited by
Items 501–600 of 1,007
Overview of the Intelligent Personal Assistants
Lopatovska, I. (2019) · Ukrainian Journal on Library and Information Science · doi:10.31866/2616-7654.3.2019.169669 · Cited by 15
“…generally find that IPAs are valued and used in situations where users are engaged in other activities (e.g. driving, playing with children, cooking) thus gaining the most benefit from hands-free voice-based interactions with their digital assistants (Cowan et al., 2017; Luger and Sellen, 2016).”
BackgroundInfluentialYou've Got a Friend in Me: Children and Search Agents
Landoni, M., Murgia, E., Huibers, T. W. C., & Pera, M. S. (2020) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3386392.3397604 · Cited by 15
“Next step will be to investigate whether the difference in levels of competences has a direct impact on the willingness to explore and interact with new models and metaphors, as reported, for adults, in [11].”
BackgroundPerera, M., Ananthanarayan, S., Goncu, C., & Marriott, K. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713634 · Cited by 14
“Prior research has shown that virtual assistants such as Siri [14], Alexa [12] and Google Assistant [49] are beneficial for blind people [1, 2, 22, 94, 98], but are limited to lightweight tasks such as playing music, determining weather [71, 73], accessing emails, and setting reminders [2].”
Sun, X., Liu, Y., Bosch, J. A., & Li, Z. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711014 · Cited by 18
“Managing user expectations regarding the limitations of CUIs is essential to foster trust [100].”
Conversations with the News: Co-speculation into Conversational Interactions with News Content
Nordberg, O. E., & Guribye, F. (2023) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3571884.3597123 · Cited by 14
“With regards to conversational agents, spoken CUIs, such as Apple’s Siri and Amazon’s Alexa, there are still limitations in conversational abilities [8, 28, 43, 46], and there is an ongoing debate as to whether such interfaces can be considered truly conversational [8, 45, 46].”
BackgroundExploring Opportunities for Multimodality and Multiple Devices in Food Journaling
Silva, L. M., Ankrah, E. A., Huai, Y., & Epstein, D. A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3604256 · Cited by 15
“, desktop dining [40]), and voice assistants allow for multitasking while preparing food or doing other activities [61, 81].”
BackgroundHow May I Interrupt? Linguistic-Driven Design Guidelines for Proactive in-Car Voice Assistants
Meck, A.-M., Draxler, C., & Vogt, T. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2266251 · Cited by 23
“Various studies (Cowan et al., 2017; Luger & Sellen, 2016) have examined this theorem and found humanness in VAs to “ spark comparisons with human assistants ” (Cowan et al., 2017, p. 1), which is an explanation for the significant results on the Likert dimension intelligence.”
Tolzin, A., & Janson, A. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.042 · Cited by 22
“However, as communitive interactions are fragile, they can easily fail after misunderstandings during the dialogue (Benner et al., 2021) and impact the effectiveness between humans and computers (Luger & Sellen, 2016).”
BackgroundCo-Design Smart Disaster Management Systems with Indigenous Communities
Tsai, C.-H., Huang, C., Chen, Y., Zendejas, E., Krafka, S., & Zendejas, J. (2024) · Digital Government Research and Practice · doi:10.1145/3660643 · Cited by 18
“The decision to embrace a conversation-based design over traditional WIMP (Windows, Icons, Menus, and Pointers) interfaces was driven by the array of advantages it ofers [37].”
BackgroundIs being human-like beneficial? The effect of anthropomorphism on chatbot persuasion in e-commerce
Yang, M., Peng, X., Wang, Q., Zhao, Y., & Wang, X. (2025) · Internet Research · doi:10.1108/intr-10-2023-0866 · Cited by 25
Foody Talk: Exploring Opportunities for Conversational Food Journaling
Silva, L. M., Lu, X., Liang, E. X., & Epstein, D. A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713875 · Cited by 19
Speech acts redux: Beyond request-response interactions
Lee, M. (2020) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3405755.3406124 · Cited by 17
“Many users do not feel heard or understood, and often have to learn what CUIs are capable of over many interactions [26].”
BackgroundInfluentialDealing with Input Uncertainty in Automotive Voice Assistants
Tobisch, V., Funk, M., & Emfield, A. G. (2020) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3409120.3410660 · Cited by 17
“Research stresses the importance of adaptive feedback based on the user input within speech dialog systems in order to be perceived as human [12, 27].”
BackgroundHabscheid, S., Hector, T., Hrncal, C., & Waldecker, D. (2021) · Journal für Medienlinguistik · doi:10.21248/jfml.2021.44 · Cited by 24
A Conversational Approach for Modifying Service Mashups in IoT Environments
Kim, S.-H., & Ko, I. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517655 · Cited by 17
“• It has been observed that users are frustrated when a conversational agent cannot handle commands they thought were simple [37].”
BackgroundExploring Interactive Sound Design for Auditory Websites
Zhang, L., Shao, J., Liu, A. A., Jiang, L., Stangl, A. J., Fourney, A., Morris, M. R., & Findlater, L. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517695 · Cited by 15
Ahire, S., Rohs, M., & Simon, B. (2022) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3533406.3533420 · Cited by 14
Crafting Human-AI Collaborative Analysis for User Experience Evaluation
Kuang, E. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3577042 · Cited by 19
Identifying Personality Dimensions for Characters of Digital Agents
Braun, M., & Alt, F. (2020) · Human-computer interaction series · doi:10.1007/978-3-030-15954-2_8 · Cited by 14
“On the downside, their human appearance also leads users to expect unlimited versatility and the validity of given information (Bickmore et al. 2018; Luger and Sellen 2016).”
BackgroundStep-wise Recommendation for Complex Task Support
Nouri, E., Sim, R., Fourney, A., & White, R. W. (2020) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3343413.3377964 · Cited by 14
“There has been a steady growth in the popularity of smart digital assistance in recent years [1, 37].”
BackgroundRapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language ModelsPreprint
Lim, G., & Perrault, S. T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.12928 · Cited by 19
Niu, R., Hu, J., Peng, S., Cao, C. C., Liu, C., Han, S., & Guo, Y. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719762 · Cited by 14
“Humans often subconsciously perceive artificial intelligent (AI) agents as social actors during interactions [40], reflecting an expectation that AI agents exhibit “social smarts” [9, 37].”
Towards AI as Colleagues: Multi-Agent System Improves Structured Ideation Processes
Quan, K., Albassam, D., Wu, M., Ding, Z., & Chin, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790375 · Cited by 19
“Roundtable exchanges, hierarchical supervision, or progressive disclosure influence how people engage with multiple agents and how cognitive load is managed [72, 129].”
Mousa, M. M., Rashed, A. S., Mustafa, A., Zamil, A. M., AHMED, H. A. M., & Abdelghani, A. A. A. (2026) · Sustainability · doi:10.3390/su18062674 · Cited by 17
Garg, R. (2021) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3459990.3460732 · Cited by 13
“However, a conversational agent may even be abandoned, if it fails to fulfill expectations regarding privacy, to accurately recognize speech, to make it easy to perceive its functions, or to support multi-user interactions [8, 20, 32, 34, 34, 49, 50, 62].”
BackgroundWeber, J., Esau, M., Schiller, M. R. G., Thaden, E., Manstetten, D., & Stevens, G. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3595975 · Cited by 15
Joshi, R., Graefe, J., Kraus, M., & Bengler, K. J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60606-9_12 · Cited by 17
Unmaking-with AI: Tactics for Decentering through Design
Nicenboim, I. P., Søndergaard, M. L. J., Lindley, J., Reddy, A. V., Strengers, Y., Redström, J., & Giaccardi, E. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3685275 · Cited by 15
Limitations in speech recognition for young adults with down syndrome
Cibrian, F. L., Chen, Y. ‘., Anderson, K., Abrahamsson, C. M., & Motti, V. G. (2025) · Universal Access in the Information Society · doi:10.1007/s10209-025-01197-4 · Cited by 13
“During the 2010 era, mobile applications such as Siri, Google Assistant, and Cortana were introduced and considered “bad” personal assistants [52] thanks to the mismatch between users’ expectations of voice assistants and the limitations of these devices to answer simple questions.”
Introduction to Human-Computer Interaction
Hornbæk, K., Kristensson, P. O., & Oulasvirta, A. (2025) · doi:10.1093/oso/9780192864543.001.0001 · Cited by 20
Having an Animated Coffee with a Group of Chatbots from the 19 th Century
Candello, H. C. D. S. P., Pinhanez, C. S., Pichiliani, M. C., Guerra, M. A., & de Bayser, M. G. (2018) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3170427.3186519 · Cited by 13
“Due to the increased gain of the popularity of conversational agents [6], interacting with machines are becoming closer to natural language ways of communication.”
BackgroundDubiel, M., Halvey, M. J., Azzopardi, L. A., & Daronnat, S. (2018) · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · Cited by 23
Designing Conversational Dashboards for Effective Use in Crisis Response
Ruoff, M., Gnewuch, U., Maedche, A., & Scheibehenne, B. (2023) · Journal of the Association for Information Systems · doi:10.17705/1jais.00801 · Cited by 21
Albert, S., & Hamann, M. G. T. (2021) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3469595.3469608 · Cited by 15
Trust your guts: fostering embodied knowledge and sustainable practices through voice interaction
Esau, M., Lawo, D., Neifer, T., Stevens, G., & Boden, A. (2022) · Personal and Ubiquitous Computing · doi:10.1007/s00779-022-01695-9 · Cited by 15
Lin, G. E., Li, J. Y., Fazly, A., Pavlović, V., & Truong, K. N. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581006 · Cited by 13
“learnability of system limits and capabilities [10, 16, 43, 44, 50].”
Bot Personas as Off-The-Shelf Users
Vandenberghe, B. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3052767 · Cited by 13
“Luger and Sellen did the same with the ‘ gulf of excecution ’ and found that today, conversational agents seem to fail due to the poor mental models people have about these bots [16].”
BackgroundThe Importance of the Interface between Humans and Computers on the Effectiveness of eHRM
Johnson, R. D., Lukaszewski, K. M., Stone, D. L., & Tech, V. (2017) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00087 · Cited by 15
How Players Speak to an Intelligent Game Character Using Natural Language Messages
Allison, F., Luger, E. A., & Hofmann, K. (2018) · Transactions of the Digital Games Research Association · doi:10.26503/todigra.v4i2.88 · Cited by 13
Self-citationMiyamoto, T., Katagami, D., Shigemitsu, Y., Usami, M., Tanaka, T., Kanamori, H., Yoshihara, Y., & Fujikake, K. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.526942 · Cited by 13
“For example, the user rejects the agent if the agent’s utterance content falls far short of the user’s expectations (Luger and Sellen, 2016).”
BackgroundRana, K., Madaan, R., & Shukla, J. (2021) · IEEE RO-MAN · doi:10.1109/ro-man50785.2021.9515528 · Cited by 13
Enhancing Interactions for In-Car Voice User Interface with Gestural Input on the Steering Wheel
Cui, Z., Gong, H., Wang, Y., Shen, C., Zou, W., & Luo, S. (2021) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3409118.3475126 · Cited by 17
“However, the usability and control efficiency of today’s VUI remain to be enhanced through interaction design [6, 20, 23, 27, 30, 40].”
BackgroundLet Me Ask You This: How Can a Voice Assistant Elicit Explicit User Feedback?
Xiao, Z., Mennicken, S., Huber, B., Shonkoff, A., & Thom, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479532 · Cited by 23
Lakier, M., & Vogel, D. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512949 · Cited by 15
“Work by Luger and Sellen [53] demonstrates that people will often look for Easter eggs when first interacting with conversational agents, to gauge what the agent is capable of in a playful way.”
BackgroundAn Integrated Study of User Acceptance and Resistance on Voice Commerce
Chung, D., Kim, H., & Ahn, S. (2022) · International Journal of Innovation and Technology Management · doi:10.1142/s0219877022500250 · Cited by 20
CUI@CHI 2024: Building Trust in CUIs—From Design to Deployment
Desai, S., Wei, C. Z., Sin, J., Dubiel, M., Zargham, N., Ahire, S., Porcheron, M., Kuzminykh, A., Lee, M., Candello, H. C. D. S. P., et al. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3636287 · Cited by 16
“Specifcally, in critical domains such as fnance and health, overreliance on voice-based advice may lead to misguided fnancial decisions [19] or, in the case of health, the inadvertent disclosure of sensitive medical information without a complete understanding of the associated privacy risks,…”
Uddin, M. N., Low, W. T., Rahman, M. A., & Mokhtar, S. (2024) · The Proceedings of the International Conference on Business, Management and Leadership. · doi:10.33422/icbml.v1i1.374 · Cited by 13
“However, inappropriate chatbot responses can create a gap between user expectations and reality (Luger & Sellen, 2016).”
BackgroundRichter, F. (2024) · AI and Ethics · doi:10.1007/s43681-024-00626-z · Cited by 12
Do We Need Natural Language? Exploring Restricted Language Interfaces for Complex Domains
Mu, J., & Sarkar, A. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3312975 · Cited by 16
“Studies have assessed experiences and expectations with commercial products such as Alexa and Siri [3, 12, 13] and so are limited by the relatively simple services they currently provide.”
BackgroundDesigning Virtual Agents for Simulation-Based Learning in Virtual Reality
Ke, F., Dai, Z., Dai, C., Pachman, M., Chaulagain, R. S., & Yuan, X. (2020) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-7998-3250-8.ch008 · Cited by 16
Iniesto, F., Coughlan, T., & Lister, K. (2021) · Proceedings of the 18th International Web for All Conference · doi:10.1145/3430263.3452431 · Cited by 12
“Even though the functionality of the assistant was clear, we noticed the assistant needs to improve the user expectations of the actions that are possible to achieve through its use [20].”
BackgroundFarah, J. C., Sharma, V., Ingram, S., & Gillet, D. (2021) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3472307.3484677 · Cited by 14
“Nevertheless, significant decreases in the results of three other traits (rapport, common sense, and attentiveness)—though for each observed in only one condition/phrase combination—serve as a reminder that playful interactions such as the acknowledgment of humor can introduce an overall higher expectation of the chat…”
BackgroundAcer, U. G., den Broeck, M. V., Min, C., Dasari, M., & Kawsar, F. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3534573 · Cited by 14
Fank, J., Richardson, N. T., & Diermeyer, F. (2019) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-019-00296-w · Cited by 12
“If it is possible for an object to receive and respond in natural language or understand natural language, it is described as a ‘conversational agent’ [70].”
BackgroundGuo, J., Tao, D., & Yang, C. (2019) · Lecture notes in electrical engineering · doi:10.1007/978-981-13-8779-1_79 · Cited by 12
Candello, H. C. D. S. P., Pichiliani, M. C., Wessel, M. S., Pinhanez, C. S., & Muller, M. J. (2019) · Halfway to the Future Symposium · doi:10.1145/3363384.3363399 · Cited by 16
“We analyzed the data by applying a thematic-network analysis [1, 6, 21].”
BackgroundMethodInvestigating Expectations for Voice-based and Conversational Argument Search on the Web
Kiesel, J., Lang, K., Wachsmuth, H., Hornecker, E., & Stein, B. (2020) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3343413.3377978 · Cited by 13
Simonsen, L., Steinstø, T., Verne, G., & Bratteteig, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58141-1_11 · Cited by 16
Augmenting Conversational Agents with Ambient Acoustic Contexts
Park, C., Min, C., Bhattacharya, S., & Kawsar, F. (2020) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3379503.3403535 · Cited by 13
How Human Communication Influences Virtual Personal Assistants
Bylieva, D. S., Lobatyuk, V. V., Kuznetsov, D., & Anosova, N. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-65857-1_11 · Cited by 13
Informing the Design of a News Chatbot
Zhang, Z., Zhang, X., & Chen, L. (2021) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3472306.3478358 · Cited by 14
MOOC-O-Bot: Using Cognitive Technologies to Extend Knowledge Support in MOOCs
Mittal, A., Vigentini, L., Djatmiko, M., Prusty, B. G., Sharma, Y., & King, M. E. (2018) · IEEE International Conference on Teaching, Assessment, and Learning for Engineering · doi:10.1109/tale.2018.8615453 · Cited by 14
Challenges and Opportunities for the Design of Smart SpeakersPreprint
Long, T., & Chilton, L. B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.05741 · Cited by 14
“New users with little experience with VUIs tend to draw high expectations from their past experiences of human-human interaction [21, 60].”
BackgroundEkellem, E. A. F. (2024) · doi:10.36227/techrxiv.170593903.35081891/v1 · Cited by 12
Brachman, M., Pan, Q., Do, H. J., Dugan, C., Chaudhary, A., Johnson, J. M., Rai, P., Chakraborti, T., Gschwind, T., Laredo, J. A., et al. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584088 · Cited by 11
“Lack of understanding of a system can lead to gulfs between users’ expectations of systems and the reality of what the systems can do [57–59, 74, 83].”
BackgroundCooney, O., Doherty, K., Barry, M., Coyle, D., Doherty, G. J., & Cowan, B. R. (2024) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3640794.3665540 · Cited by 11
Member’s performance in human–AI hybrid teams: a perspective of adaptability theory
Chen, A., Lyu, A., & Lu, Y. (2024) · Information Technology and People · doi:10.1108/itp-05-2023-0513 · Cited by 14
A Survey on the Recent Advancements in Human-Centered Dialog Systems
Oruche, R., Goruganthu, S. K., Akula, R., Cheng, X., Goni, M. A., Shibo, B. W., Kee, K. F., Zampieri, M., & Calyam, P. P. (2025) · ACM Computing Surveys · doi:10.1145/3729220 · Cited by 18
Mpinganjira, M., Maduku, D. K., Rana, N. P., & Thusi, P. (2025) · Journal of Global Information Management · doi:10.4018/jgim.387836 · Cited by 12
Transparency in Language Generation: Levels of Automation
Edwards, J., Perrone, A., & Doyle, P. R. (2020) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3405755.3406136 · Cited by 12
“Subsequent failure to meet these expectations can lead to frustration, limited use and even abandonment of CUIs [8].”
BackgroundWhen Googling It Doesn’t Work: The Challenge of Finding Security Advice for Smart Home Devices
Turner, S., Nurse, J. R. C., & Li, S. (2021) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-81111-2_10 · Cited by 21
“It is widely recognised that this action, in part, arises from a wide-spread lack of accurate mental models about these devices [24], unless the user is already very technically minded [12].”
BackgroundAhire, S., Priegnitz, A., Önbas, O., Rohs, M., & Nejdl, W. (2021) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3472307.3484165 · Cited by 11
Motta, I., & Quaresma, M. (2023) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3571884.3604304 · Cited by 11
“, older adults, [10]) but also for the general public [3, 14].”
BackgroundRzepka, C. H., Berger, B., Koslow, A., & Heß, T. (2023) · ACM SIGMIS Database the DATABASE for Advances in Information Systems · doi:10.1145/3614178.3614185 · Cited by 12
“The major benefits that users expect of the use of VAs are gains in efficiency, convenience, and enjoyment (Luger & Sellen, 2016; Moussawi, 2018).”
BackgroundResultInfluentialJaddoh, A., Loizides, F., Lee, J., & RANA, O. F. (2023) · Universal Access in the Information Society · doi:10.1007/s10209-023-01037-3 · Cited by 11
“This is an issue for all users and not specifically for people with dysarthria [40].”
Musto, C., Martina, A. F. M., Iovine, A., Narducci, F., de Gemmis, M., & Semeraro, G. M. (2023) · Journal of Intelligent Information Systems · doi:10.1007/s10844-023-00835-8 · Cited by 11
“Finally, Section 5 contains the conclusions and outlines future work.”
Jung, H., Kim, H., & Ha, J.-W. (2020) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3405755.3406170 · Cited by 11
StockBabble: A Conversational Financial Agent to support Stock Market Investors
Sharma, S., Brennan, J., & Nurse, J. R. C. (2021) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3469595.3469620 · Cited by 11
“These could be accessed by the user to provide a more natural conversation flow and would seek to avoid known issues [9] related to poor agent responses and engagement.”
BackgroundWu, Y., Porcheron, M., Doyle, P. R., Edwards, J., Rough, D. J., Cooney, O., Bleakley, A., Clark, L., & Cowan, B. R. (2022) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3543829.3543839 · Cited by 14
“" [P13-L2] Reflecting on other work [43, 48], the purpose of this hyperarticulation was apparent to both sets of participants.”
BackgroundMethodInfluentialFoosherian, M., Freire, S. K., Niforatos, E., Hribernik, K., & Thoben, K. (2022) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3568444.3568454 · Cited by 12
“Conversational breakdown can lead to frustration, disappointment, mistrust, and dissatisfaction [4, 11, 26] if not addressed.”
BackgroundRecovering from Dialogue Failures Using Multiple Agents in Wealth Management Advice
Candello, H. C. D. S. P., & Pinhanez, C. S. (2018) · Human-computer interaction series · doi:10.1007/978-3-319-95579-7_7 · Cited by 14
Aladwan, A., Kelly, R., Baker, S., & Velloso, E. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300624 · Cited by 13
Avdic, M., & Vermeulen, J. (2020) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3441000.3441068 · Cited by 14
“Prior fndings point towards this misalignment between users’ expectations and the IPA’s capabilities [19, 39].”
BackgroundVoice Input Interface Failures and Frustration: Developer and User Perspectives
Goetsu, S., & Sakai, T. (2019) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3332167.3357103 · Cited by 14
“Most notably, it has been reported that the lack of system response and transparency will lead to lower satisfaction [2,5], which lead us to examine the gap between developer and user perspectives.”
BackgroundMultimodal Continuation-style Architectures for Human-Robot InteractionPreprint
Krishnaswamy, N., & Pustejovsky, J. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08161 · Cited by 14
“As robots take on more human-like appearances, this becomes even more important, as there exists a gulf between expectations that the robot will communicate and understand things in a human-like way and its actual multimodal capability (Luger & Sellen, 2016).”
Background“Try, Try, Try Again:” Sequence Analysis of User Interaction Data with a Voice User Interface
Myers, C. M., Laris Pardo, L. F., Acosta-Ruiz, A., Canossa, A., & Zhu, J. (2021) · CUI 2021 - 3rd Conference on Conversational User Interfaces · doi:10.1145/3469595.3469613 · Cited by 10
“HCI VUI research more commonly reviews and discusses the aggregated behavioral data of how users interact with a VUI [13, 31, 36] or observations of VUI interaction [10, 17, 33].”
BackgroundStelbotsky, B., & Park, S. (2024) · Public Management Review · doi:10.1080/14719037.2024.2443064 · Cited by 12
“Additionally, past negative experience with AI and chatbots contribute to this aversion, especially when chatbots fail to meet expectations for interpreting complex user inputs (Janssen et al. 2020; Luger and Sellen 2016).”
Mahmud, R., Wu, Y., Sawad, A. B., Berkovsky, S., Prasad, M., & Kocaballi, A. B. (2025) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3764687.3764714 · Cited by 13
Kruchten, P. B., & Gregory, P. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-48550-3 · Cited by 12
Supporting the Design of Smart Speakers to Foster a Sense of Ownership in Asian Indian Families
Garg, R. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517680 · Cited by 12
Investigating the Synonyms of Conversational Agents to Aid Cross-Disciplinary CA Research
Wang, L., Chen, C., & Huh, J. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585640 · Cited by 11
“Researchers call CA in many diferent ways including chatbot [3], dialogue system [12], conversational interface [12], intelligent agent [46], conversational assistant [9], virtual companion [69], avatar [69], etc.”
BackgroundDesign Principles for Interactive and Reflective Journaling with AI
Angenius, M., & Ghajargar, M. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-37963-5_6 · Cited by 12
Horstmann, A. C., Strathmann, C., Lambrich, L., & Krämer, N. C. (2023) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3570945.3607335 · Cited by 12
Exploring the influence of user characteristics on verbal aggression towards social chatbots
Chin, H., & Yi, M. Y. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2362957 · Cited by 12
“Similarly, a user’s technical knowledge was found to affect their patience when facing errors made by CAs, with less technically knowl-edgeable participants quitting their conversation with agents faster (Luger and Sellen 2016).”
Norval, C., & Singh, J. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3343802 · Cited by 12
“Firstly, the use of natural language has long been considered an intuitive way for people to interact with technology [11, 12].”
BackgroundIntelligent Personal Assistant in Business-Context: Key-feature Evaluation for User Acceptance
Hüsson, D., Holland, A., & Sánchez, R. A. (2020) · Business Systems Research Journal · doi:10.2478/bsrj-2020-0032 · Cited by 11
“Comparable study (Luger & Sellen, 2016) Comparable study (Lambeck et Comparable study (Lambeck, et al., 2014b)”
BackgroundCatania, F., Spitale, M., & Garzotto, F. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451666 · Cited by 12
Development and Evaluation of Usability Heuristics for Voice User Interfaces
Fulfagar, L., Gupta, A., Mathur, A., & Shrivastava, A. (2021) · Smart innovation, systems and technologies · doi:10.1007/978-981-16-0041-8_32 · Cited by 14
Voice and touch interaction: a user experience comparison of elderly people in smartphones
Tubin, C., Rodriguez, J. P. M., & Marchi, A. C. B. D. (2021) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3472301.3484366 · Cited by 13
“Luger e Sellen [35] destacam que as pessoas costumam ter expectativas altas em relação a sistemas baseado em conversação, principalmente as pessoas com mais baixo conhecimento técnico, como é o caso dos participantes deste estudo.”
BackgroundMiyamoto, T., Katagami, D., Tanaka, T., Kanamori, H., Yoshihara, Y., & Fujikake, K. (2021) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3472307.3484160 · Cited by 13
“A previous study [24] suggested the importance of designing an agent that supports a user’s”
Background
Metadata
- Authors
- Luger, E., & Sellen, A.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- Venue
- Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Pages
- 5286-5297
- OpenAlex
- W2405187948
- Semantic Scholar
- 814e62bb4f0ec6facd640fbd3e3fcbd189716d20
- Cited by
- 1,176
- References
- 51
- Citations by year
- 2026: 164 · 2025: 175 · 2024: 119 · 2023: 116 · 2022: 107 · 2021: 148 · 2020: 141 · 2019: 126 · 2018: 54 · 2017: 25 · 2016: 1
- FWCI
- 37.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- chatbots, voice assistants, everyday use, user expectations, human-computer interaction, multimodality, system capabilities, interface design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.