CoChat

"Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents

Luger, Ewa, Sellen, Abigail · 2016 · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

The DOI matches its Crossref record; 49 of the 51 references it cites were found in a registry or an index.

Abstract

The past four years have seen the rise of conversational agents (CAs) in everyday life. Apple, Microsoft, Amazon, Google and Facebook have all embedded proprietary CAs within their software and, increasingly, conversation is becoming a key mode of human-computer interaction. Whilst we have long been familiar with the notion of computers that speak, the investigative concern within HCI has been upon multimodality rather than dialogue alone, and there is no sense of how such interfaces are used in everyday life. This paper reports the findings of interviews with 14 users of CAs in an effort to understand the current interactional factors affecting everyday use. We find user expectations dramatically out of step with the operation of the systems, particularly in terms of known machine intelligence, system capability and goals. Using Norman's 'gulfs of execution and evaluation' [30] we consider the implications of these findings for the design of future systems.

Overview

Integrity signals

Registry

Crossref · 10.1145/2858036.2858288

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

17 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 49 references

Research works printed but not linked: 14

  1. Boden Margaret A, 2007, Proceedings of Artificial Companions in Society: Perspectives on the Present and Future. Oxford Internet Institute.
  2. Davis Martin, Engines of Logic: Mathematicians and the Origin of the Computer
  3. Glass James R., 1999, Challenges for Spoken Dialogue Systems. In the proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding. (ASRU), CO, USA
  4. 10.5555/1941863 · doi:10.5555/1941863
  5. Heuwinkel Kerstin, Your Virtual Butler: The Making-of. Robert Trappl (ed)
  6. Robnett Licklider Joseph Carl, 1960, HFE-1,(March
  7. Moore Roger, Spoken Language Processing: Where do we go from Here? In Your Virtual Butler: The Making-of. Robert Trappl (ed)
  8. Payr Sabine, Your Virtual Butler: The Making-of. Robert Trappl (ed)
  9. Preece Jenny, 2015, Interaction Design: Beyond Huma- Computer Interaction
  10. Pulman Stephen, Proceedings of the 2010 Workshop on Companionable Dialogue Systems. Assoc. for Computational Linguistics
  11. Trappl Robert, From Jeeves Jeannie to Siri, and Then? In Your Virtual Butler: The Making-of. Robert Trappl (ed)
  12. Vassallo Giorgio, Semantic Computing. Sheu, Yu, Ramamoorthy, Joshi & Zadeh.(eds)
  13. 10.5555/1870559.1870562 · doi:10.5555/1870559.1870562
  14. Wilks Yorick, 2010, Amsterdam: John Benjamins.

Cited by

Items 101–200 of 1,007

Metadata

Authors
Luger, E., & Sellen, A.
Affiliations
Microsoft Research (United Kingdom) (GB)
Venue
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2016-05-07
Pages
5286-5297
OpenAlex
W2405187948
Cited by
1,176
References
51
Citations by year
2026: 164 · 2025: 175 · 2024: 119 · 2023: 116 · 2022: 107 · 2021: 148 · 2020: 141 · 2019: 126 · 2018: 54 · 2017: 25 · 2016: 1
FWCI
37.00
Type
conference paper
Language
en
Open access
closed
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
chatbots, voice assistants, everyday use, user expectations, human-computer interaction, multimodality, system capabilities, interface design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.