Using gaze patterns to predict task intent in collaboration
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
In everyday interactions, humans naturally exhibit behavioral cues, such as gaze and head movements, that signal their intentions while interpreting the behavioral cues of others to predict their intentions. Such intention prediction enables each partner to adapt their behaviors to the intent of others, serving a critical role in joint action where parties work together to achieve a common goal. Among behavioral cues, eye gaze is particularly important in understanding a person's attention and intention. In this work, we seek to quantify how gaze patterns may indicate a person's intention. Our investigation was contextualized in a dyadic sandwich-making scenario in which a "worker" prepared a sandwich by adding ingredients requested by a "customer." In this context, we investigated the extent to which the customers' gaze cues serve as predictors of which ingredients they intend to request. Predictive features were derived to represent characteristics of the customers' gaze patterns. We developed a support vector machine-based (SVM-based) model that achieved 76% accuracy in predicting the customers' intended requests based solely on gaze features. Moreover, the predictor made correct predictions approximately 1.8 s before the spoken request from the customer. We further analyzed several episodes of interactions from our data to develop a deeper understanding of the scenarios where our predictor succeeded and failed in making correct predictions. These analyses revealed additional gaze patterns that may be leveraged to improve intention prediction. This work highlights gaze cues as a significant resource for understanding human intentions and informs the design of real-time recognizers of user intention for intelligent systems, such as assistive robots and ubiquitous devices, that may enable more complex capabilities and improved user experience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Rizzolatti, G., & Craighero, L. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144230 · Cited by 7,433 · cited 1×
“…evidence from neuroscience and developmental psychology has shown that action monitoring allows people to use their behavior repertoire and motor system to predict and understand others’ actions and intentions (Blakemore and Decety, 2001; Buccino et al., 2001; Rizzolatti and Craighero, 2004).”
BackgroundClark, H. H. (1996) · doi:10.1017/cbo9780511620539 · Cited by 5,123
Baron‐Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001) · Journal of Child Psychology and Psychiatry · doi:10.1111/1469-7610.00715 · Cited by 4,555 · cited 3×
“Humans employ a number of behavioral cues, such as gaze and gestures, when working with others on a task (Morris and Desebrock, 1977; White, 1989; Clark and Brennan, 1991; Shibata et al., 1995; Baron-Cohen et al., 2001; Meltzoff and Brooks, 2001; Bangerter, 2004).”
BackgroundClark, H. H., & Brennan, S. E. (1991) · Perspectives on socially shared cognition. · doi:10.1037/10096-006 · Cited by 4,555
Pretense and representation: The origins of "theory of mind."
Leslie, A. M. (1987) · Psychological Review · doi:10.1037/0033-295x.94.4.412 · Cited by 3,379
Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study
Buccino, G., Binkofski, F., Fink, G. R., Fadiga, L., Fogassi, L., Gallese, V., Seitz, R. J., Zilles, K., Rizzolatti, G., & Freund, H. ‐. (2001) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2001.01385.x · Cited by 2,488
Joint action: bodies and minds moving together
SEBANZ, N., BEKKERING, H., & KNOBLICH, G. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.12.009 · Cited by 1,950
Gaze cueing of attention: visual attention, social cognition, and individual differences
Frischen, A., Bayliss, A. P., & Tipper, S. P. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.4.694 · Cited by 1,473
Cross-language analysis of phonetic units in language addressed to infants
Kuhl, P. K., Andruski, J. E., Chistovich, I. A., Chistovich, L. A., Kozhevnikova, E. V., Ryskina, V. L., Stolyarova, E. I., Sundberg, U., & Lacerda, F. (1997) · Science · doi:10.1126/science.277.5326.684 · Cited by 1,041
Cited by
Anticipatory robot control for efficient human-robot collaboration
Huang, C.-M., & Mutlu, B. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451737 · Cited by 227
Self-citationMy Bad! Repairing Intelligent Voice Assistant Errors Improves Interaction
Cuadra, A., Li, S., Lee, H., Cho, J., & Ju, W. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449101 · Cited by 52
Crowdsourcing Thumbnail Captions: Data Collection and Validation
Aguirre, C., Cao, S., Mahmood, A., & Huang, C.-M. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3589346 · Cited by 6
Self-citationLook at Me When I Talk to You: A Video Dataset to Enable Voice Assistants to Recognize Errors
Cuadra, A., Lee, H., Cho, J., & Ju, W. (2021) · arXiv · doi:10.48550/arxiv.2104.07153 · Cited by 5
Metadata
- Authors
- Huang, C.-M., Andrist, S., Sauppé, A., & Mutlu, B.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2015-07-24
- Volume
- 6
- Pages
- 1049-1049
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W1496918241
- Cited by
- 167
- References
- 11
- FWCI
- 18.89
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.