The DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
All Work and No Play?
Abstract
Many conversational agents (CAs) are developed to answer users' questions in a specialized domain. In everyday use of CAs, user experience may extend beyond satisfying information needs to the enjoyment of conversations with CAs, some of which represent playful interactions. By studying a field deployment of a Human Resource chatbot, we report on users' interest areas in conversational interactions to inform the development of CAs. Through the lens of statistical modeling, we also highlight rich signals in conversational interactions for inferring user satisfaction with the instrumental usage and playful interactions with the agent. These signals can be utilized to develop agents that adapt functionality and interaction styles. By contrasting these signals, we shed light on the varying functions of conversational interactions. We discuss design implications for CAs, and directions for developing adaptive agents based on users' conversational behaviors.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In consequence, knowledge on chatbot use has been gathered for a range of contexts—in the private sphere [87], at work [74], and in public spaces [17].”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [74]
Citations
Cites
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452
(1985) · Cambridge University Press eBooks · doi:10.1017/cbo9780511665868 · Cited by 4,199
On seeing human: A three-factor theory of anthropomorphism
Epley, N., Waytz, A., & Cacioppo, J. T. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.864 · Cited by 4,069 · cited 1×
“According to the theory of anthropomorphism [16], these behaviors are not only to satisfy sociality needs through a sense of social contact, but also to reduce uncertainty about the system to interact more effectively.”
BackgroundA classification of illocutionary acts
Searle, J. R. (1976) · Language in Society · doi:10.1017/s0047404500006837 · Cited by 3,118
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,807
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
Microcomputer Playfulness: Development of a Measure With Workplace Implications
Webster, J., & Martocchio, J. J. (1992) · MIS Quarterly · doi:10.2307/249576 · Cited by 1,153
How to Do Things with Words: The William James Lectures Delivered at Harvard University in 1955
Austin, J. L. (1975) · Medical Entomology and Zoology · Cited by 1,038
Personalizing search via automated analysis of interests and activities
Teevan, J. B., Dumais, S., & Horvitz, E. (2005) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1076034.1076111 · Cited by 803
Implicit feedback for inferring user preference
Kelly, D., & Teevan, J. B. (2003) · ACM SIGIR Forum · doi:10.1145/959258.959260 · Cited by 746
The impact of animated interface agents: a review of empirical research
Dehn, D. M., & van Mulken, S. (2000) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0325 · Cited by 567
Walker, M. A., Litman, D. J., Kamm, C., & Abella, A. (1997) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/976909.979652 · Cited by 563
Speech Act Theory and Pragmatics
Searle, J. R., Kiefer, F., & Bierwisch, M. (1980) · Springer eBooks · doi:10.1007/978-94-009-8964-1 · Cited by 559
Direct manipulation vs. interface agents
Shneiderman, B., & Maes, P. (1997) · Interactions · doi:10.1145/267505.267514 · Cited by 518
A Theoretical Framework for Conversational Search
Radlinski, F., & Craswell, N. (2017) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3020165.3020183 · Cited by 473
To err is human(-like): effects of robot gesture on perceived anthropomorphism and likability
Salem, M., Eyssel, F., Rohlfing, K., Kopp, S., & Joublin, F. (2013) · International Journal of Social Robotics · doi:10.1007/s12369-013-0196-9 · Cited by 407
A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application
Kopp, S., Gesellensetter, L., Krämer, N. C., & Wachsmuth, I. (2005) · Lecture notes in computer science · doi:10.1007/11550617_28 · Cited by 400
Pay attention!: designing adaptive agents that monitor and improve user engagement
Szafir, D., & Mutlu, B. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207679 · Cited by 304
Taking the time to care: empowering low health literacy hospital patients with virtual nurse agents
Bickmore, T. W., Pfeifer, L. M., & Jack, B. W. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518891 · Cited by 289
Embodied conversational agents: representation and intelligence in user interfaces
Cassell, J. (2001) · AI Magazine · doi:10.1609/aimag.v22i4.1593 · Cited by 286
The language that gets people to give
Mitra, T., & Gilbert, É. (2014) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2531602.2531656 · Cited by 259
Negotiated Collusion: Modeling Social Language and its Relationship Effects in Intelligent Agents
Cassell, J., & Bickmore, T. (2003) · User Modeling and User-Adapted Interaction · doi:10.1023/a:1024026532471 · Cited by 253
Automatic analysis of affective postures and body motion to detect engagement with a game companion,
Sanghvi, J., Castellano, G., Leite, I., Pereira, A., McOwan, P. W., & Paiva, A. (2011) · Proceedings of the 6th international conference on Human-robot interaction · doi:10.1145/1957656.1957781 · Cited by 250
Ravenclaw: dialog management using hierarchical task decomposition and an expectation agenda
Bohus, D., & Rudnicky, A. I. (2003) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.2003-255 · Cited by 199
Wahlster, W., & Kobsa, A. (1989) · Springer eBooks · doi:10.1007/978-3-642-83230-7_1 · Cited by 174
Automatic Online Evaluation of Intelligent Assistants
Jiang, J., Awadallah, A. H., Jones, R., Özertem, U., Zitouni, I., Kulkarni, R. G., & Khan, O. Z. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741669 · Cited by 154
A risk minimization framework for information retrieval
Zhai, C., & Lafferty, J. (2005) · Information Processing & Management · doi:10.1016/j.ipm.2004.11.003 · Cited by 150
Receptionist or information kiosk
Lee, M. K., Kiesler, S. B., & Forlizzi, J. (2010) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1718918.1718927 · Cited by 145
Carroll, J. M., & Thomas, J. C. (1988) · ACM SIGCHI Bulletin · doi:10.1145/49108.1045604 · Cited by 143
Media inequality in conversation
Shechtman, N., & Horowitz, L. M. (2003) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642661 · Cited by 139
(2017) · doi:10.1145/2998181.2998298 · Cited by 131
Phrases that signal workplace hierarchy
Gilbert, E. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145359 · Cited by 98
Liao, Q. V., Davis, M. J., Geyer, W., Muller, M., & Shami, N. S. (2016) · Designing Interactive Systems Conference · doi:10.1145/2901790.2901842 · Cited by 93
Self-citationA model of user adoption of interface agents for email notification
Serenko, A. (2008) · Interacting with Computers · doi:10.1016/j.intcom.2008.04.004 · Cited by 92
You, robot: on the linguistic construction of artificial others
Coeckelbergh, M. (2010) · AI & Society · doi:10.1007/s00146-010-0289-z · Cited by 85
Optimal rare query suggestion with implicit user feedback
Song, Y., & He, L.-W. (2010) · International World Wide Web Conference (WWW) · doi:10.1145/1772690.1772782 · Cited by 84
Relational Agents Improve Engagement and Learning in Science Museum Visitors
Bickmore, T., Pfeifer, L. M., & Schulman, D. J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23974-8_7 · Cited by 78
What would you Ask a conversational Agent? Observations of Human-Agent Dialogues in a Museum Setting
Robinson, S. M., Traum, D. R., Ittycheriah, M., & Henderer, J. (2008) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3bs9odrfj3mf · Cited by 58
Ogan, A., Finkelstein, S. L., Mayfield, E., D'Adamo, C., Matsuda, N., & Cassell, J. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207684 · Cited by 57
Conversational agent use in a café
Porcheron, M. (2016) · Nottingham ePrints (University of Nottingham) · doi:10.17639/nott.61 · Cited by 56
Rudeness and Rapport: Insults and Learning Gains in Peer Tutoring
Ogan, A., Finkelstein, S. L., Walker, E., Carlson, R., & Cassell, J. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-30950-2_2 · Cited by 54
Investigating shared attention with a virtual agent using a gaze-based interface
Peters, C., Asteriadis, S., & Karpouzis, K. C. (2009) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-009-0029-1 · Cited by 51
Speech technology on trial: Experiences from the August system
Gustafson, J., & Bell, L. (2000) · Natural Language Engineering · doi:10.1017/s1351324900002485 · Cited by 37
The role of choice and customization on users' interaction with embodied conversational agents
Xiao, J., Stasko, J., & Catrambone, R. (2007) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240820 · Cited by 37
Towards Conversational Agents That Attend to and Adapt to Communicative User Feedback
Buschmeier, H., & Kopp, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23974-8_19 · Cited by 34
Automatic Recognition of Conversational Strategies in the Service of a Socially-Aware Dialog System
Zhao, R., Sinha, T., Black, A. W., & Cassell, J. (2016) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w16-3647 · Cited by 33
An Empirical Study of the Effect of Agent Competence on User Performance and Perception
Xiao, J., Stasko, J., & Catrambone, R. (2004) · doi:10.1109/aamas.2004.61 · Cited by 32
Bian, Y., Yang, C., Guan, D., Xiao, S., Gao, F., Shen, C., & Meng, X. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858351 · Cited by 31
Utilitarian vs. hedonic robots
Lee, N., Shin, H.-C., & Sundar, S. S. (2011) · doi:10.1145/1957656.1957722 · Cited by 30
Leveraging Conversational Systems to Assists New Hires During Onboarding
Chandar, P., Khazaeni, Y., Davis, M. J., Muller, M., Crasso, M., Liao, Q. V., Shami, N. S., & Geyer, W. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67684-5_23 · Cited by 30
Self-citationEmbodied conversational agents
CassellJustine (2001) · AI Magazine · doi:10.5555/567363.567368 · Cited by 23
Domain-Oriented Conversation with H.C. Andersen
Bernsen, N. O., & Dybkjær, L. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24842-2_14 · Cited by 22
Making computer tasks at work more playful: Implications for systems analysts and designers
Webster, J. (1988) · ACM SIGCPR Conference on Computer Personnel Research (SIGCPR) · doi:10.1145/57216.57227 · Cited by 21
Nakano, Y., Walker, M. A., Paiva, A., & Neff, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33197-8 · Cited by 21
Wang, W. Y., Finkelstein, S. L., Ogan, A., Black, A. W., & Cassell, J. (2018) · Figshare · doi:10.1184/r1/6469757 · Cited by 19
Your Virtual Butler: the making-of
Trappl, R. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37346-6 · Cited by 10
Dialogues in Context: An Objective User-Oriented Evaluation Approach for Virtual Human Dialogue
Robinson, S. M., Roque, A., & Traum, D. R. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/44fxd5anv8uv · Cited by 7
23. Speech act theory and pragmatics
(2003) · John Benjamins Publishing Company eBooks · doi:10.1075/z.120.27spe · Cited by 5
Flagged first, then by citations
1 of the 59 references couldn’t be matched to a known paper.
Research works printed but not linked: 12
- John Langshaw Austin. 1975. How to do things with words. Oxford university press.
- Niels Bernsen and Laila Dybkjær. 2004. Domain-oriented conversation with HC Andersen. Affective Dialogue Systems (2004), 142--153.
- 10.5555/2041666.2041675 · DOI
- 10.5555/2041666.2041689 · DOI
- Mark G Core and James Allen. 1997. Coding dialogs with the DAMSL annotation scheme. In AAAI fall symposium on communicative action in humans and machines, Vol. 56. Boston, MA.
- Dan Fletcher. 2010. The 50 Worst Inventions: Microsoft Bob. TIME. (27 May 2010).
- 10.5555/79061 · DOI
- Chris Matyszczyk. 2012. Apple's Siri wrong 38 percent of the time in test. CNET. (30 June 2012).
- Robert J. Moore, Rafah A. Hosn, and Ashima Arora. 2016. The Machinery of Natural Conversation and the Design of Conversational Machines. In American Sociological Association annual meeting.
- Luke Swartz. 2003. Why people hate the paperclip: Labels appearance behavior and social responses to user interface agents. Ph.D. Dissertation. Stanford University Palo Alto CA.
- 10.5555/2554494 · DOI
- 10.5555/1018409.1018745 · DOI
Cited by
Sorted by citations
Rapp, A., Curti, L., & Boldi, A. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102630 · Cited by 733
Entertainment industry economics: a guide for financial analysis
Vogel, H. L. (2007) · Choice Reviews Online · doi:10.5860/choice.45-0982 · Cited by 562
Evaluating and Informing the Design of Chatbots
Jain, M., Kumar, P., Kota, R., & Patel, S. N. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196735 · Cited by 376
Chatbots, Humbots, and the Quest for Artificial General Intelligence
Grudin, J., & Jacques, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300439 · Cited by 275
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247
Metadata
- Authors
- Liao, Q. V., Mas-ud Hussain, M., Chandar, P., Davis, M., Khazaeni, Y., Crasso, M. P., Wang, D., Muller, M., Shami, N. S., & Geyer, W.
- Affiliations
- IBM (United States) (US)
- Northwestern University, Evanston, IL, USA
- Spotify Research, Cambridge, MA, USA
- IBM Research - Cambridge (United States) (US)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-19
- Pages
- 1-13
- OpenAlex
- W2487652696
- Cited by
- 116
- References
- 59
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 13 · 2023: 10 · 2022: 18 · 2021: 27 · 2020: 14 · 2019: 13 · 2018: 5
- FWCI
- 6.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- chatbots, playful interaction, user satisfaction, field deployment, conversational behavior, adaptive agents, user experience
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review