Exploring the Role of Conversational Cues in Guided Task Support with Virtual Assistants
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Voice-based conversational assistants are growing in popularity on ubiquitous mobile and stationary devices. Cortana, as well as Google Home, Amazon Echo, and others, can provide support for various tasks from managing reminders to booking a hotel. However, with few exceptions, user input is limited to explicit queries or commands. In this work, we explore the role of implicit conversational cues in guided task completion scenarios. In a Wizard of Oz study, we found that, for the task of cooking a recipe, nearly one-quarter of all user-assistant exchanges were initiated from implicit conversational cues rather than from plain questions. Given that these implicit cues occur in such high frequency, we conclude by presenting a set of design implications for the design of guided task experiences in contemporary conversational assistants.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similar to previous findings, issues due to device timing out [61,83] errors due to speech recognition/mistranscription, and lack of conversational nature [59] and the need to remember specific keywords [61] is challenging for older adults.”
From Use of Intelligent Voice Assistants by Older Adults with Low Technology Use (2020) · cites this paper as [83]
In the citing paper
“there are also a number of concerns including deception, infantilization, and privacy [63,70].”
From "Phantom Friend" or "Just a Box with Information": Personification and Ontological Categorization of Smart Speaker-based Voice Assistants by Older Adults (2019) · cites this paper as [70]
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
A simplest systematics for the organization of turn-taking for conversation
Sacks, H., Schegloff, E. A., & Jefferson, G. (1974) · Language · doi:10.2307/412243 · Cited by 11,350 · cited 1×
“Sacks et al. [27] analyze and detail rules according to which the turn-taking behaviour occurs.”
BackgroundLevels of processing: A framework for memory research
Craik, F. I. M., & Lockhart, R. S. (1972) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(72)80001-x · Cited by 9,566 · cited 1×
“With this sort of “memory rehearsal” behavior people refresh and maintain items in their short-term memory [10], which is believed to rely on the same pathways as language and speech.”
BackgroundAttention, intentions, and the structure of discourse
Grosz, B. J., & Sidner, C. L. (1986) · doi:10.5555/12457.12458 · Cited by 2,580
Clark, H. H., & Schaefer, E. F. (1989) · Cognitive Science · doi:10.1207/s15516709cog1302_7 · Cited by 1,699 · cited 5×
“These behaviors have been called “acknowl-edgements”, “demonstration” and “display” [9].”
BackgroundInfluentialLuger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176 · cited 2×
“…using voice-based assistants on their phones, personal computers (PCs), and internet of things (IoT) devices to complete short tasks, such as setting up an alarm clock or checking weather forecasts [17, 21], as well as for multi-step tasks such as making a restaurant reservation or booking a taxi.”
BackgroundDialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech
Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Ess-Dykema, C. V., & Meteer, M. (2000) · Computational Linguistics · doi:10.1162/089120100561737 · Cited by 1,108 · cited 2×
“These features have already proven to be valuable in improving the detection of dialogue acts [29, 16], which can be seen as a similar – but coarser-grained – taxonomy of spoken intents.”
Achieving Human Parity in Conversational Speech Recognition
Xiong, W., Droppo, J., Huang, X., Seide, F. T. B., Seltzer, M., Stolcke, A., Yu, D., & Zweig, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.05256 · Cited by 451 · cited 1×
“Considering the latest improvements in speech recognition [31], the tooling now exists to capture and transcribe user utterances in high fidelity.”
BackgroundDeep Learning for Answer Sentence Selection
Yu, L., Hermann, K. M., Blunsom, P., & Pulman, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1632 · Cited by 340 · cited 1×
“Furthermore, we argue that a system that is able to recognize and correctly act on these cues can achieve high levels of user satisfaction, even when constrained to a simple response model (e.g., limited to sentence selection for question answering [32]).”
BackgroundToward Human Parity in Conversational Speech Recognition
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M. L., Stolcke, A., Yu, D., & Zweig, G. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2756440 · Cited by 245
CONVERSATION ACTS IN TASK‐ORIENTED SPOKEN DIALOGUE
Traum, D. R., & Hinkelman, E. A. (1992) · Computational Intelligence · doi:10.1111/j.1467-8640.1992.tb00380.x · Cited by 229
Ravenclaw: dialog management using hierarchical task decomposition and an expectation agenda
Bohus, D., & Rudnicky, A. I. (2003) · 8th European Conference on Speech Communication and Technology (Eurospeech 2003) · doi:10.21437/eurospeech.2003-255 · Cited by 195 · cited 1×
“[7] describe a dialogue management tool – RavenClaw – that was used to build multiple tasksupport oriented dialogue systems [4, 6, 26].”
Method(2017) · doi:10.1145/2998181.2998298 · Cited by 131 · cited 1×
“[25] characterize the changes that happen when an automatic conversational assistant is introduced as one of the discourse participants.”
Background‘Realness’ in Chatbots: Establishing Quantifiable Criteria
Morrissey, K., & Kirakowski, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39330-3_10 · Cited by 110 · cited 1×
“These types of interactions are very natural and effortless, and people may exhibit them even when interacting with automatic systems that are incapable of perceiving or reciprocating these signals [17, 20].”
BackgroundPredicting User Satisfaction with Intelligent Assistants
Kiseleva, J., Williams, K., Hassan Awadallah, A., Crook, A. C., Zitouni, I., & Anastasakos, T. (2016) · Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval · doi:10.1145/2911451.2911521 · Cited by 108 · cited 1×
“These features have already proven to be valuable in improving the detection of dialogue acts [29, 16], which can be seen as a similar – but coarser-grained – taxonomy of spoken intents.”
BackgroundLET's GO: improving spoken dialog systems for the elderly and non-natives
Raux, A., Langner, B., Black, A. W., & Eskénazi, M. (2003) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.2003-313 · Cited by 86 · cited 1×
“[7] describe a dialogue management tool – RavenClaw – that was used to build multiple tasksupport oriented dialogue systems [4, 6, 26].”
MethodAffirmative Cue Words in Task-Oriented Dialogue
Gravano, A., Hirschberg, J., & Beňuš, Š. (2012) · Computational Linguistics · doi:10.1162/coli_a_00083 · Cited by 77 · cited 3×
“work has studied the importance of conversational cues in human-human task-oriented dialogue over a range of tasks [12, 13].”
BackgroundManaging Human-Robot Engagement with Forecasts and... um ... Hesitations
Bohus, D., & Horvitz, E. (2014) · Proceedings of the 16th International Conference on Multimodal Interaction · doi:10.1145/2663204.2663241 · Cited by 73 · cited 1×
“Human-to-human conversation is an immensely complex process, filled with implicit cues – both verbal and nonverbal [19, 14, 9, 30, 5].”
BackgroundConversational agent use in a café
Porcheron, M. (2016) · Nottingham ePrints (University of Nottingham) · doi:10.17639/nott.61 · Cited by 56
LARRI: A Language-Based Maintenance and Repair Assistant
Bohus, D., & Rudnicky, A. I. (2005) · Text, speech and language technology · doi:10.1007/1-4020-3075-4_12 · Cited by 43 · cited 2×
“While these guided task completion scenarios can be quite rich and complex [4, 6, 18], the protocol for communicating with contemporary voice assistants is comparatively simple and constrained, and usually follows the <trigger word, question, answer> paradigm.”
BackgroundMethodNguyêñ, D. T. K., & Fussell, S. R. (2015) · Journal of Language and Social Psychology · doi:10.1177/0261927x15571538 · Cited by 43 · cited 1×
“These conversational cues allow people to better communicate agreement or disagreement, emotional state, and whether they successfully perceived and processed the information [22].”
BackgroundAnalyzing and Predicting Task Reminders
Graus, D., Bennett, P. N., White, R. W., & Horvitz, E. (2016) · Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization · doi:10.1145/2930238.2930239 · Cited by 40 · cited 2×
“What’s next? work has studied the importance of conversational cues in human-human task-oriented dialogue over a range of tasks [12, 13].”
BackgroundTalking through procedures: an intelligent space station procedure assistant
Aist, G., Early, S., Gorrell, G., Phan, S., Dowding, J., Hockey, B. A., Rayner, M., Hieronymus, J. L., Bohus, D., Boven, B., et al. (2003) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/1067737.1067781 · Cited by 18 · cited 2×
“While these guided task completion scenarios can be quite rich and complex [4, 6, 18], the protocol for communicating with contemporary voice assistants is comparatively simple and constrained, and usually follows the <trigger word, question, answer> paradigm.”
BackgroundMethodAnalyzing the Effect of Entrainment on Dialogue Acts
Mizukami, M., Yoshino, K., Neubig, G., Traum, D. R., & Nakamura, S. (2016) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w16-3640 · Cited by 16 · cited 1×
“Human-to-human conversation is an immensely complex process, filled with implicit cues – both verbal and nonverbal [19, 14, 9, 30, 5].”
BackgroundStarting to cook a tutoring dialogue system
Martins, F., Pardal, J. P., Franqueira, L., Arez, P., & Mamede, N. (2008) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2008.4777861 · Cited by 15 · cited 1×
“While these guided task completion scenarios can be quite rich and complex [4, 6, 18], the protocol for communicating with contemporary voice assistants is comparatively simple and constrained, and usually follows the <trigger word, question, answer> paradigm.”
BackgroundThe acoustics of eye contact: detecting visual attention from conversational audio cues
Eyben, F., Weninger, F. J., Paletta, L., & Schuller, B. W. (2013) · doi:10.1145/2535948.2535949 · Cited by 11
Research works printed but not linked: 3
- 2017. Alexa, Say What?! Voice-Enabled Speaker Usage to Grow Nearly 130% This Year. https://www.emarketer.com/Article/Alexa-Say-WhatVoice-Enabled-Speaker-Usage-Grow-Nearly-130-ThisYear/1015812. (2017). {Online; retrieved 5-Jan-2018}.
- John Brooke and others. 1996. SUS -- A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.
- Sheryl Ong and Aaron Suplizio. 2016. Unpacking the Breakout Success of the Amazon Echo. https://www.experian.com/innovation/thoughtleadership/amazon-echo-consumer-survey.jsp. (2016). {Online; retrieved 20-October-2016}.
Cited by
Use of Intelligent Voice Assistants by Older Adults with Low Technology Use
Pradhan, A., Lazar, A., & Findlater, L. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3373759 · Cited by 285
“Similar to previous findings, issues due to device timing out [61, 83] errors due to speech recognition/mistranscription, and lack of conversational nature [59] and the need to remember specific keywords [61] is challenging for older adults.”
ResultPradhan, A., Findlater, L., & Lazar, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359316 · Cited by 223
“To date, designers and researchers have largely focused on the conversational aspects of voice assistants, such as designing conversational cues and conversation dynamics of these collaborative technologies (e.g., [53,54,69]).”
BackgroundKim, S., & Choudhury, A. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106914 · Cited by 200
“…(Druga et al., 2017; Garg & Sengupta, 2020), low-income populations (Robinson et al., 2018), privacy concerns (Lau et al., 2018; Liao et al., 2019), conversational aspects (Purington et al., 2017; Vtyurina & Fourney, 2018), and personification (Lopatovska & Williams, 2018; Pradhan et al., 2019).”
Kim, S. (2021) · JMIR mHealth and uHealth · doi:10.2196/20427 · Cited by 153
“Among several, one stream of research has focused on exploring the use of voice assistants in various contexts, such as different locations [15,16] or tasks [17,18].”
Stangl, A., Morris, M. R., & Gurari, D. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376404 · Cited by 139
“Finally, others have inquired into how people who are BLV want to interact with image descriptions, and how different delivery structures impact their experience [28, 35, 40].”
Background
Metadata
- Authors
- Vtyurina, A., & Fourney, A.
- Affiliations
- University of Waterloo (CA)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-21
- Pages
- 1-7
- OpenAlex
- W2791006121
- Cited by
- 83
- References
- 25
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 13 · 2023: 7 · 2022: 11 · 2021: 11 · 2020: 17 · 2019: 13 · 2018: 2
- FWCI
- 4.37
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- chatbots, virtual assistants, task support, voice interaction, Wizard-of-Oz study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.