"Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents
The DOI matches its Crossref record; 49 of the 51 references it cites were found in a registry or an index.
Abstract
The past four years have seen the rise of conversational agents (CAs) in everyday life. Apple, Microsoft, Amazon, Google and Facebook have all embedded proprietary CAs within their software and, increasingly, conversation is becoming a key mode of human-computer interaction. Whilst we have long been familiar with the notion of computers that speak, the investigative concern within HCI has been upon multimodality rather than dialogue alone, and there is no sense of how such interfaces are used in everyday life. This paper reports the findings of interviews with 14 users of CAs in an effort to understand the current interactional factors affecting everyday use. We find user expectations dramatically out of step with the operation of the systems, particularly in terms of known machine intelligence, system capability and goals. Using Norman's 'gulfs of execution and evaluation' [30] we consider the implications of these findings for the design of future systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Whereas, we found that the challenges our participants experienced were more elementary.”
From Exploring How Older Adults Use a Smart Speaker–Based Voice Assistant in Their First Interactions: Qualitative Study (2021) · cites this paper as [32]
In the citing paper
“In addition, it could be more important for novice users to have an appropriate mental model and understand the system rather than an efficient user experience [32].”
From Exploring How Older Adults Use a Smart Speaker–Based Voice Assistant in Their First Interactions: Qualitative Study (2021) · cites this paper as [32]
In the citing paper
“human-AI mixed-initiative presents a generally valuable and underexplored interface paradigm.”
From Mixed-Initiative Creative Interfaces (2017) · cites this paper as [20]
In the citing paper
“Not only do these breakdowns significantly diminish user satisfaction and damage trust, they also decrease the likelihood of continued use [18, 44].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [44]
In the citing paper
“This is because when voice assistants fail, such as by incorrectly answering a question, it derails user trust [27, 35].”
From A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures (2023) · cites this paper as [35]
17 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability
Guest, G., Bunce, A., & Johnson, L. (2006) · Field Methods · doi:10.1177/1525822x05279903 · Cited by 20,808
(2010) · Sociology for Pharmacists · doi:10.4324/9780203381175_chapter_9 · Cited by 14,152
Norman, D. (2016) · Vahlen eBooks · doi:10.15358/9783800648108 · Cited by 6,374
Thematic networks: an analytic tool for qualitative research
Attride-Stirling, J. (2001) · Qualitative Research · doi:10.1177/146879410100100307 · Cited by 5,506 · cited 2×
“The thematic network is a ‘web-like coding structure’ [2; 385] that allows for a rich understanding of the conceptual interplay occurring within the narrative.”
Background(1997) · Choice Reviews Online · doi:10.5860/choice.34-3702 · Cited by 4,887
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,738
Licklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938 · cited 1×
“When considering human-machine dialogue from an HCI perspective, some of the earliest work can be traced back to vision statements and position papers such as JCR Licklider (1960), ‘Man-machine symbiosis’ [25] and the early work of Richard Bolt [4].”
MethodEmbodied Conversational Agents
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/2697.001.0001 · Cited by 1,799
Interaction Ritual: Essays on Face-to-face Behaviour
Goffman, E. (1972) · Medical Entomology and Zoology · Cited by 1,449
Research works printed but not linked: 14
- Boden Margaret A, 2007, Proceedings of Artificial Companions in Society: Perspectives on the Present and Future. Oxford Internet Institute.
- Davis Martin, Engines of Logic: Mathematicians and the Origin of the Computer
- Glass James R., 1999, Challenges for Spoken Dialogue Systems. In the proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding. (ASRU), CO, USA
- 10.5555/1941863 · doi:10.5555/1941863
- Heuwinkel Kerstin, Your Virtual Butler: The Making-of. Robert Trappl (ed)
- Robnett Licklider Joseph Carl, 1960, HFE-1,(March
- Moore Roger, Spoken Language Processing: Where do we go from Here? In Your Virtual Butler: The Making-of. Robert Trappl (ed)
- Payr Sabine, Your Virtual Butler: The Making-of. Robert Trappl (ed)
- Preece Jenny, 2015, Interaction Design: Beyond Huma- Computer Interaction
- Pulman Stephen, Proceedings of the 2010 Workshop on Companionable Dialogue Systems. Assoc. for Computational Linguistics
- Trappl Robert, From Jeeves Jeannie to Siri, and Then? In Your Virtual Butler: The Making-of. Robert Trappl (ed)
- Vassallo Giorgio, Semantic Computing. Sheu, Yu, Ramamoorthy, Joshi & Zadeh.(eds)
- 10.5555/1870559.1870562 · doi:10.5555/1870559.1870562
- Wilks Yorick, 2010, Amsterdam: John Benjamins.
Cited by
Items 601–700 of 1,007
Voice Commerce: Anthropomorphism Using Voice Assistants
Nguyen, H. S., Mladenow, A., Strauß, C., & Auer‐Srnka, K. J. (2021) · International Conference on Information Integration and Web-based Applications & Services (iiWAS) · doi:10.1145/3487664.3487724 · Cited by 12
“Nevertheless, many papers such as Luger and Schellen [41], Arnold, Hildebrandt and Schneider [3], McLean and Osei-Frimpong [44], Weber and Ludwig [70] and others point out some major problems concerning the adoption and acceptance of voice shopping that indirectly question the actual popularity of voice assistants for…”
BackgroundInfluentialEffects of Speaking Rate on Speech and Silent Speech Recognition
Pandey, L., & Arif, A. S. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519611 · Cited by 10
“They engage themselves in processes of repairing the errors by either reformulation, simplification, or hyperenunciation [32, 38, 40, 42, 50, 57].”
BackgroundFerreira, J. J., Fucs, A., & Segura, V. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3299052 · Cited by 13
“The investigation of AI-as (especially conversational agents or chatbots) interaction with users is presenting interesting results related to general tasks, like add an appointment to a calendar, call another person, shop online, and so on [4][8][12] using mainly voice commands.”
BackgroundOn the Pragmatics of the Turing Test
Jacquet, B., Jamet, F., & Baratgin, J. (2021) · International Conference on Information and Digital Technologies (IDT) · doi:10.1109/idt52577.2021.9497570 · Cited by 10
Fahn, V., & Riener, A. (2021) · Mensch und Computer (MuC) · doi:10.1145/3473856.3473872 · Cited by 14
Modeling Behavior Patterns with an Unfamiliar Voice User Interface
Myers, C. M., Grethlein, D., Furqan, A., Ontañón, S., & Zhu, J. (2019) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3320435.3320475 · Cited by 10
One Agent Too Many: User Perspectives on Approaches to Multi-agent Conversational AIPreprint
Clarke, C., Krishnamurthy, K., Talamonti, W. J., Kang, Y., Tang, L., & Mars, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.07123 · Cited by 10
“However, despite this widespread adoption the range of support still remains fairly limited and users spent a lot of time trying to get the expected response from conversational agents [29], [30].”
BackgroundClarifying False Memories in Voice-based Search
Kiesel, J., Bahrami, A., Stein, B., Anand, A., & Hagen, M. (2019) · Proceedings of the 2019 Conference on Human Information Interaction and Retrieval · doi:10.1145/3295750.3298961 · Cited by 9
“Voice-based search systems are characterized as mixed-initiative systems between users and memory-equipped agents [16, 17, 22],”
BackgroundReinders, S., Butler, M., & Marriott, K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713158 · Cited by 11
Lind, M. (2025) · AI & Society · doi:10.1007/s00146-025-02243-8 · Cited by 13
How Dynamic vs. Static Presentation Shapes User Perception and Emotional Connection to Text-Based AI
Bhat, M. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712131 · Cited by 12
Göldi, A., Rietsche, R., & Ungar, L. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713606 · Cited by 13
“While these early systems were groundbreaking, their rigidity and inability to handle complex user interactions often resulted in user dissatisfaction [45].”
Huang, E. J., Easterday, M. W., & Gerber, E. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757549 · Cited by 12
Understanding Human-AI Misalignment in LLM-Based Job-Seeking Support for Neurodivergent Users
Hall, K., Ma, M., Zhang, X., Swain, V. D., & Kim, J. G. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746361 · Cited by 10
Yun, B., Taranova, E., & Wang, A. Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791620 · Cited by 13
Mathur, N., Zubatiy, T., Rozga, A., & Mynatt, E. D. (2026) · ACM Transactions on Accessible Computing · doi:10.1145/3856982 · Cited by 9
Lemmer, S. J., Guo, A., & Corso, J. J. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584092 · Cited by 12
“Because the structure of deep neural networks—a single input produces a single output—solutions to such problems either evaluate error via independent inferences on a dataset [1, 31, 56] or consider team performance from perspectives such as trust and explainability [4, 14, 49, 67, 72], introducing novel interfaces [1…”
Background“Foggy sounds like nothing” — enriching the experience of voice assistants with sonic overlays
Esau, M., Marsh, A., Krauß, V., & Stevens, G. (2023) · Personal and Ubiquitous Computing · doi:10.1007/s00779-023-01722-3 · Cited by 9
Cross-Cultural Validation of Partner Models for Voice User Interfaces
Seaborn, K., Gessinger, I., Yoshida, S., Cowan, B. R., & Doyle, P. R. (2024) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3640794.3665537 · Cited by 11
“Partner models can affect UX, impacting trust in the system [48] and knowledge expectations [20].”
BackgroundPatterns of gaze in speech agent interaction
Jaber, R., McMillan, D., Belenguer, J. S., & Brown, B. A. T. (2019) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3342775.3342791 · Cited by 9
“Luger and Sellen [37] highlighted the limited functionality of existing commercially-available voice interfaces and how it causes a gulf between their capabilities and the users’ expectations.”
BackgroundLee, Y., & Malcein, L. A. (2020) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.212 · Cited by 13
Yingsoon, G. Y. (2020) · Advances in data mining and database management book series · doi:10.4018/978-1-7998-5876-8.ch010 · Cited by 15
Perceptions of Agent Loyalty with Ancillary Users
Reig, S., Carter, E. J., Tan, X. Z., Steinfeld, A. M., & Forlizzi, J. (2021) · International Journal of Social Robotics · doi:10.1007/s12369-020-00725-x · Cited by 9
“…with embodied conversational agents (ECA) and conversational user interfaces (CUI): people do not speak to conversational assistants in the same way that they speak to humans, but they do make certain social attributions to agents similarly to how they make social attributions to humans [43].”
BackgroundInterfacing participation in citizen science projects with conversational agents
Portela, M. (2021) · Human Computation · doi:10.15346/hc.v8i2.114 · Cited by 10
“Recent publications have shown novel findings in usability and user apprehension to these new interfaces and technologies (Luger & Sellen, 2016).”
BackgroundMethodReflectPal: Exploring Self-Reflection on Collaborative Activities Using Voice Assistants
Papachristos, E., Meldgaard, D. P., Thomsen, I. R., & Skov, M. B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85610-6_12 · Cited by 10
IntraVox: A Personalized Human Voice to Support Users with Complex Needs in Smart Homes
Salai, A.-M., Cook, G. A., & Holmquist, L. E. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85623-6_15 · Cited by 10
“Luger and Sellen [33] investigated users’ views on Conversational Agents (e.g., Siri, Cortana) and how this a↵ects everyday use.”
BackgroundInfluentialWeitz, K., Vanderlyn, L., Vu, N. T., & André, E. (2021) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.1 · Cited by 15
“Luger and Sellen (2016) came to a similar conclusion for speech assistants, like Cortana and Siri, which suggests that our findings are transferable to other conversational agents.”
ResultINTENT: Interactive Tensor Transformation Synthesis
Zhou, Z., Tang, M. T., Pan, Q., Tan, S., Wang, X., & Zhang, T. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545653 · Cited by 10
How can I help you? Design principles for task-oriented speech dialog systems in customer service
Oesterreich, T. D., Anton, E., Schuir, J., Brehm, A., & Teuteberg, F. (2022) · Information Systems and e-Business Management · doi:10.1007/s10257-022-00570-7 · Cited by 12
“To date, a wealth of terms is frequently used for different kinds of dialog systems, including digital assistants, chatbots, conversational agents, and machine conversation sys - tems (McTear et al. 2016, p. 39; Luger and Sellen 2016; Diederich et al. 2019a).”
BackgroundComputational research and the case for taking humor seriously
Cowie, R. (2023) · Humor - International Journal of Humor Research · doi:10.1515/humor-2023-0021 · Cited by 9
Jeromela, J., & Conlan, O. (2023) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3571884.3604317 · Cited by 10
The Human Likeness of Government Chatbots – An Empirical Study from Norwegian Municipalities
Følstad, A., Larsen, A. G., & Bjerkreim-Hanssen, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41138-0_8 · Cited by 11
Survey of AI Tool Usage in Programming Course: Early Observations
Saari, M., Rantanen, P., Nurminen, M., Kilamo, T., Systä, K., & Abrahamsson, P. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-48550-3_18 · Cited by 14
Tired of Wake Words? Moving Towards Seamless Conversations with Intelligent Personal Assistants
Ahire, S., & Rohs, M. (2020) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3405755.3406141 · Cited by 12
Understanding Task Differences to Leverage the Usability and Adoption of Voice Assistants (VAs)
Motta, I., & Quaresma, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-78227-6_35 · Cited by 10
gentiletti, M. G., Bourmaud, G., Fréjus, M., & Decortis, F. (2022) · Activites · doi:10.4000/activites.7428 · Cited by 12
Can direct address affect user engagement with chatbots embodied in physical spaces?
Candello, H. C. D. S. P., Pinhanez, C. S., Pichiliani, M. C., Vasconcelos, M., & Conde, H. (2019) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3342775.3342787 · Cited by 10
“A thematic network was applied to organize the patterns of reasoning across the analysis [24].”
MethodDifferent Types of Voice User Interface Failures May Cause Different Degrees of Frustration
Goetsu, S., & Sakai, T. (2020) · Institutional Repositories DataBase (IRDB) · doi:10.48550/arxiv.2002.03582 · Cited by 9
“For example, Luger and Sellen [3] interviewed n = 14 regular users of conversational agents such as Siri and concluded: users had poor mental models of how their CA worked and that these were reinforced through a lack of meaningful feedback regarding system capability and intelligence.”
BackgroundInfluentialAssessing the User Experience of an LLM-Based Conversational Assistant in Diabetes Mellitus Care
Furtado, V., Araujo, J., Furtado, E. S., Bomfim, R., Silva, V., Mororo, N. A. C., Vasconcelos, L., Carvalho, C., Severo, M., Torres, J., et al. (2025) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-025-00217-5 · Cited by 9
Engagement and Usability of Conversational Search – A Study of a Medical Resource Center Chatbot
Fergencs, T., & Meier, F. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-71292-1_26 · Cited by 10
“Luger and Sellen conducted a qualitative study using interviews and thematic network analysis to explore the mental models that users have about their voice assistants [19].”
BackgroundXian, C., & Fu, M. (2022) · IEEE International Conference on Power, Electronics and Computer Applications (ICPECA) · doi:10.1109/icpeca53709.2022.9718950 · Cited by 8
Enhancing Productivity Applications for People who are Blind using AI Assistants
Perera, M. U. S. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3638180 · Cited by 10
“These assistants are currently used for more lightweight and playful tasks such as playing music, determining weather [30, 31] and are limited to productivity related tasks such as accessing emails, scheduling meetings, and setting reminders [2].”
Elavarasi, T., Jayaprakash, M., Shivakumar., E., Savitha, N., Gayathri, K. C., & Agoramoorthy, M. (2025) · International Conference on Pervasive Computing and Social Networking (ICPCSN) · doi:10.1109/icpcsn65854.2025.11035776 · Cited by 8
Raise to Speak: An Accurate, Low-power Detector for Activating Voice Assistants on Smartwatches
Zhao, S., Westing, B., Scully, S. R., Nieto, H., Holenstein, R., Jeong, M., Sridhar, K., Newendorp, B., Bastian, M., Raman, S., et al. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330761 · Cited by 9
“Neither of these invocation methods are natural on smartwatches; the button invocation is a two-handed invocation while trigger phrases are unnatural when used frequently, especially in public environments [14].”
BackgroundIniesto, F., Coughlan, T., Lister, K., & Holmes, W. (2020) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3373625.3418017 · Cited by 8
“The VA needs to provide clear user expectations of the actions that are possible to achieve through its use [14].”
BackgroundSalai, A.-M., Cook, G. A., & Holmquist, L. E. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451828 · Cited by 9
“”) can become problematic for users with learning disabilities, speech impediments, or dementia [3, 4].”
BackgroundDesigning a Gender-Inclusive Conversational Agent For Pair Programming: An Empirical Investigation
Kuttal, S. K., Sedhain, A., & AuBuchon, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77772-2_4 · Cited by 10
“To increase participants’ trust [67, 79-81] the agent showed uncertainty about its work, asking for verification; for example, after adding code through the IDE, the agent would say “This might do the trick.”
BackgroundNaturally Together: A Systematic Approach for Multi-User Interaction With Natural Interfaces
Shavitt, C., Kuzminykh, A., Ridel, I., & Cauchard, J. R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476090 · Cited by 8
“Thus, the user’s sense of self-efficacy is affected by the recognizable/familiar aspects of interaction design and system’s feedback matching users’ mental model [73, 75].”
BackgroundMaharjan, R., Rohani, D. A., Doherty, K., Bækgaard, P., & Bardram, J. E. (2022) · IEEE Pervasive Computing · doi:10.1109/mprv.2022.3147374 · Cited by 9
“12 Therefore, although the current state of the technology may not be sophisticated enough to engage users in truly conversational interaction, our results suggest the value of this novel mode of interaction as means to support positive experiences of the self-report of mental health and wellbeing.”
BackgroundDesigning Supportive Conversational Agents With and For Teens
Lopatovska, I., & Davis, J. (2023) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3576840.3578283 · Cited by 8
“However, the current features that enable CAs to be potentially helpful to users also attribute to high user expectations and frequent disappointments in these agents [19–21].”
Background패션 디자인 주체에 따른 패션디자이너 역량 및 제품 품질 지각
정, 주., 장, 세., & Lee, Y. (2023) · Journal of the Korean Society of Clothing and Textiles · doi:10.5850/jksct.2023.47.4.743 · Cited by 8
Understanding User Preferences of Voice Assistant Answer Structures for Personal Health Data Queries
Rey, B., Sakamoto, Y., Sin, J., & Irani, P. (2024) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3640794.3665552 · Cited by 10
User-Aware Shared Perception for Embodied Agents
McNeely-White, D. G., Ruiz, J., Wang, I., Ortega, F. R., Beveridge, J. R., Draper, B. A., Bangar, R., Patil, D., Pustejovsky, J. D., Krishnaswamy, N., et al. (2019) · doi:10.1109/hcc46620.2019.00015 · Cited by 13
“Anecdotally, as in [1], users of our system have described frustration when learning what Diana can and can’t do, and especially when she doesn’t recognize their communication properly.”
BackgroundAddressing Hiccups in Conversations with Recommender Systems
Viswanathan, S., Guillot, F., Chang, M., Grasso, A. M., & Renders, J.-M. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533491 · Cited by 9
Revisiting Habitability in Conversational Systems
Kocaballi, A. B., Coiera, E. W., & Berkovsky, S. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3383014 · Cited by 8
“Since the current conversational systems’ capabilities are still limited, the habitability gap is persistent [20].”
BackgroundExploring Smart Agents for the Interaction with Multimodal Mediated Environments
Richer, R., Zhao, N., Eskofier, B. M., & Paradiso, J. A. (2020) · Multimodal Technologies and Interaction · doi:10.3390/mti4020027 · Cited by 8
“However, nowadays, conversational agents still suffer from a “gulf of evaluation” between users expectations and system operation [11].”
BackgroundShaikh, S. J., & Cruz, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12914 · Cited by 10
“This allows users to delegate assignments to them without entirely disengaging from their current task (Luger & Sellen, 2016; Goksel & Mutlu, 2016; López, Quesada, & Guerrero, 2018).”
BackgroundTeam dynamics in human-AI collaboration: effects on confidence, satisfaction, and accountability
Yousefi, M., Shahi, A., Sharifi, M., J Jorge Romera, A., Hoermann, S., & Piumsomboon, T. (2025) · Proceedings of the 27th International Conference on Multimodal Interaction · doi:10.1145/3716553.3750776 · Cited by 7
Smart Building Operations and Virtual Assistants Using LLM
Ly, R., Shojaei, A., & Gao, X. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728706 · Cited by 8
“In addition, while commercial solutions like Google Assistant, Alexa, and Bixby offer user-friendly interfaces for handling various commands with advanced NLP techniques, they typically struggle with implicit and complex instructions [11].”
Parental Responses to Aggressive Child Behavior towards Robots, Smart Speakers, and Tablets
Naggita, K., Athiley, E., Desta, B., & Sebo, S. S. (2022) · IEEE RO-MAN · doi:10.1109/ro-man53752.2022.9900841 · Cited by 9
“Lastly, Carlson et al. [22] has demonstrated that people view robots with more sympathy and as more emotionally capable than a computer and Luger and Sellen [17] has shown higher ratings of anthropomorphism and animacy for a Nao robot compared with a Google Home.”
BackgroundMethodInterdisciplinary perspectives on privacy awareness in lifelogging technology development
Wilkowska, W., Offermann, J., Colonna, L., Flórez‐Revuelta, F., Climent-Pérez, P., Mihailidis, A., Poli, A., Spinsante, S., & Ziefle, M. (2022) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-022-04486-5 · Cited by 8
“It allows for lively interactions and makes people feel like chatting with a real person (Luger and Sellen 2016; Nass et al. 1999).”
BackgroundCuciniello, M., Amorese, T., Greco, C., Callejas, Z., Vogel, C. M., Cordasco, G., & Esposito, A. A. (2023) · Human Behavior and Emerging Technologies · doi:10.1155/2023/8858268 · Cited by 7
Shin, J., Ko, H., Bae, S. W., & Kim, J. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2441530 · Cited by 9
“Through the growth of speech interfaces like Amazon’s Alexa, Apple’s Siri, Google’s Home Assistant, IBM’s Watson, and Microsoft’s Cortana, the importance of the communicative competence of VCAs as dialogue partners has increased (Doyle et al., 2021; Luger & Sellen, 2016).”
Ghosh, S., & Ghosh, S. (2021) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3406522.3446057 · Cited by 7
“perform well for simple tasks like setting reminders or alarms, calling or texting a contact, and getting weather or traffic updates, they are incapable of performing complex searches and engaging in multi-turn dialogues [27].”
BackgroundRaza, S. A., Vitale, J., Tonkin, M., Johnston, B., Billingsley, R., Herse, S., & Williams, M. (2022) · Intelligent Service Robotics · doi:10.1007/s11370-022-00411-z · Cited by 8
“[31] found via a user study that proprietary conversational agents frommany high tech companies failed to meet user expectations due to lack of transparency about the system’s capacity and intelligence.”
BackgroundUNDERSTANDING HOW CHATBOTS WORK: AN EXPLORATORY STUDY OF MENTAL MODELS IN CUSTOMER SERVICE CHATBOTS
Ordemann, S. (2021) · IADIS INTERNATIONAL JOURNAL ON WWW/INTERNET · doi:10.33965/ijwi_202119102 · Cited by 8
“Nevertheless, applying a mental model with human content when interacting with chatbots can lead to overly inflated expectations towards the chatbots (Go and Sundar, 2019; Luger and Sellen, 2016).”
BackgroundInfluentialExploring the Effects of Self-Mockery to Improve Task-Oriented Chatbot’s Social Intelligence
Liu, C., Zhou, S., Zhang, Y., Liu, D., Peng, Z., & Ma, X. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533461 · Cited by 9
“Moreover, based on our survey, applying self-oriented humor, a.k.a., self-mockery, to task-oriented chatbots is understudied in the HCI domain, although in the HHI it has proven to be helpful for adjusting intense atmosphere in communication, alleviating mental pressure, and amending participants’ relationships [45].”
InfluentialScrutability of Intelligent Personal Assistants
Jeromela, J. (2022) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3503252.3534355 · Cited by 8
Klüber, K., & Onnasch, L. (2022) · Robotics · doi:10.3390/robotics11050106 · Cited by 7
“However, this seems to be modulated by the technical savvy of users, as the study by Luger and Sellen showed that technically experienced users were more tolerant of communication failures and aborted interactions with voice assistants after a greater number of attempts compared to less technically savvy users [22].”
BackgroundInfluentialInteractive Journaling with AI: Probing into Words and Language as Interaction Design Materials
Angenius, M., & Ghajargar, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25581-6_10 · Cited by 10
Beyond Anthropomorphism: Unraveling the True Priorities of Chatbot Usage in SMEs
Makány, T., Roh, S., Hara, K., Hua, J. M., Ying, F. G. S., & Jie, W. T. Y. (2023) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3571884.3604315 · Cited by 7
Keskenler, M. F., Çelik, E., & Dal, D. (2024) · Journal of Electrical Engineering and Technology · doi:10.1007/s42835-023-01758-8 · Cited by 8
“AI-based studies are usually conducted using machine learning algorithms [17, 18].”
Towards Human-Like Conversational Search Systems
Dubiel, M. (2018) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3176349.3176360 · Cited by 8
“…awareness on how to communicate with them in order to obtain required results, which discourages frequent use of the system and limits its scope[13; 16] Moore et al. suggested that by making voice search to resemble human-human dialogue it can become a viable alternative of text-based…”
BackgroundCha, Y., Jang, J., Hong, Y., & Yi, M. Y. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3313045 · Cited by 7
“Previous research on conversational agents tried to understand user interaction with them by identifying the gulf of expectation on the conversational agents [4].”
BackgroundKassel, J.-F., & Rohs, M. (2019) · Designing Interactive Systems Conference · doi:10.1145/3322276.3322282 · Cited by 9
“Aligned with these findings, Luger and Sellen [28] showed that regular CI users focus rather on simple tasks than complex tasks.”
BackgroundMethodInfluentialAutomatic Speech Recognition: Do Emotions Matter?
Catania, F., Crovari, P., Spitale, M., & Garzotto, F. (2019) · 2019 IEEE International Conference on Conversational Data & Knowledge Engineering (CDKE) · doi:10.1109/cdke46621.2019.00009 · Cited by 7
“In both cases, the error committed by the ASR module could result in a big deal because it could bring to the user’s departure from the use of conversational technology [8].”
BackgroundConversational strategies : impact on search performance in a goal-oriented task
Dubiel, M., Halvey, M. J., Azzopardi, L. A., Anderson, D., & Daronnat, S. (2020) · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · Cited by 38
Bührke, J., Brendel, B., Lichtenberg, S., Diederich, S., & Morana, S. (2021) · Lecture notes in information systems and organisation · doi:10.1007/978-3-030-86797-3_19 · Cited by 7
Sayago, S., & Ribera, M. (2020) · International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion (DSAI) · doi:10.1145/3439231.3440603 · Cited by 8
Tool Support for Improving Software Quality in Machine Learning Programs
Cheng, K. S., Huang, P.-C., Ahn, T.-H., & Song, M. (2023) · Information · doi:10.3390/info14010053 · Cited by 8
A Framework and Content Analysis of Social Cues in the Introductions of Customer Service Chatbots
van Hooijdonk, C. M. J., Martijn, G., & Liebrecht, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25581-6_8 · Cited by 9
Designing Context-Aware Chatbots for Product Configuration
Niederer, T., Schloss, D., & Christensen, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25581-6_12 · Cited by 7
From Design Assistants to Design Peers: Turning Daphne into an AI Companion for Mission Designers
Martin, A. V., & Selva, D. (2019) · AIAA SciTech Forum · doi:10.2514/6.2019-0402 · Cited by 10
AssistantGraph: An Approach for Reusable and Composable Data-Driven Assistant Components
Meurisch, C., Bayrak, B., & Mühlhäuser, M. (2019) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2019.00079 · Cited by 11
“Recent years have seen the rise of digital assistants such as Apple’s Siri (2011), Google Assistant (former: Google Now, 2012), Microsoft’s Cortana (2015), or Amazon’s Alexa (2015) in everyday life [1].”
BackgroundHCI Research Challenges for the Next Generation of Conversational Systems
Pinhanez, C. S. (2020) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3405755.3406153 · Cited by 7
“It has been reported that users of conversational systems often learn, in a short amount of time, that they can get answers from those systems by using a simplified form of natural language [12].”
BackgroundThe Rise of Interactional Multimodality in Human-Computer Interaction
Klowait, N. O., & Ерофеева, M. (2021) · Monitoring obŝestvennogo mneniâ: èkonomičeskie i socialʹnye peremeny · doi:10.14515/monitoring.2021.1.1793 · Cited by 8
Sun, L., Li, Z., Zhang, Y., Liu, Y., Lou, S., & Zhou, Z. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451819 · Cited by 12
“However, breakdown repair strategies such as confrmation and repeat do not work efectively enough, due to the lack of suf-fcient transparency of agent status [8], the cause of errors [55], or afordance [58].”
BackgroundMahmood, A., Wang, J., & Huang, C.-M. (2026) · ACM Transactions on Accessible Computing · doi:10.1145/3796236 · Cited by 6
Sugan, M., Nadkarni, S., Sanaeipoor, S., Patel, S., Natarajamoorthy, S. Y., Rashidi, E., & Miller, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3721270 · Cited by 6
The perceived outcomes of generative artificial intelligence in users’ everyday life
Han, Z., Zhao, H., Chen, J., & Yin, S. (2025) · Aslib Journal of Information Management · doi:10.1108/ajim-02-2025-0073 · Cited by 6
Persuasive Conversational Agents for Environmental Sustainability: A Survey
Giudici, M., Crovari, P., & Garzotto, F. (2025) · ACM Computing Surveys · doi:10.1145/3774751 · Cited by 7
Privacy Control in Conversational LLM Platforms: A Walkthrough Study
Li, Z., Wu, Y., Li, Y., Gui, X., & Luo, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791054 · Cited by 9
“Users might feel they have more control over their interactions when, in reality, the inherent ambiguity of NL control can affect their ability to foresee, verify, and review how their data is being handled [68].”
Teachable Conversational Agents for Crowdwork: Effects on Performance and Trust
Chhibber, N., Goh, J., & Law, E. (2022) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3555223 · Cited by 5
“Consequently, agent-based interactions has attracted a lot of attention in HCI research [12, 45, 47, 50, 68].”
BackgroundYouth Expectations and Perceptions of Generative Artificial Intelligence in Higher Education.
Cotino-Arbelo, A. E., González, C. S. G., & Molina‐Gil, J. (2025) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2025.02.004 · Cited by 7
Han, H., Park, B., & Seo, K. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720286 · Cited by 10
Markl, N., & Lai, C. (2021) · Cited by 27
Public Perceptions Towards Synthetic Voice Technology
Noah, B., Sethumadhavan, A., Lovejoy, J., & Mondello, D. (2021) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181321651128 · Cited by 6
“These concerns, echoed in other work (Luger & Sellen, 2016), suggest that dynamics currently governing interpersonal communication may simply require the use of human voices— regardless of the perceived voice quality in computerized options.”
BackgroundMagic and Reality: what children’s drawings tell us about their perception of technology
Valguarnera, S., Sylla, C., & Landoni, M. (2022) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3501712.3535295 · Cited by 7
“Does looking at technology as it can do magic have an impact on design with and for children? We need to further explore whether children behave like adults, and according to [6] if when their expectations - for magic - are not fulfilled, lose their trust in technology and refrain from using it for complex tasks or if…”
BackgroundToward Connecting Speech Acts and Search Actions in Conversational Search Tasks
Ghosh, S., Ghosh, S., & Shah, C. (2023) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl57899.2023.00027 · Cited by 9
“However, the use cases of personal assistants are quite limited currently, and they are incapable of comprehending complex queries [56].”
BackgroundEnd-Users Know Best: Identifying Undesired Behavior of Alexa Skills Through User Review Analysis
Aldeen, M. S., Young, J. A., Liao, S., Chang, T.-Y., Cheng, L., Cai, H., Luo, X., & Hu, H. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3678517 · Cited by 8
“There is a gap between user expectations and the current capabilities of VPA [45].”
Metadata
- Authors
- Luger, E., & Sellen, A.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- Venue
- Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Pages
- 5286-5297
- OpenAlex
- W2405187948
- Semantic Scholar
- 814e62bb4f0ec6facd640fbd3e3fcbd189716d20
- Cited by
- 1,176
- References
- 51
- Citations by year
- 2026: 164 · 2025: 175 · 2024: 119 · 2023: 116 · 2022: 107 · 2021: 148 · 2020: 141 · 2019: 126 · 2018: 54 · 2017: 25 · 2016: 1
- FWCI
- 37.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- chatbots, voice assistants, everyday use, user expectations, human-computer interaction, multimodality, system capabilities, interface design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.