The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Anticipatory robot control for efficient human-robot collaboration
Abstract
Efficient collaboration requires collaborators to monitor the behaviors of their partners, make inferences about their task intent, and plan their own actions accordingly. To work seamlessly and efficiently with their human counterparts, robots must similarly rely on predictions of their users' intent in planning their actions. In this paper, we present an anticipatory control method that enables robots to proactively perform task actions based on anticipated actions of their human partners. We implemented this method into a robot system that monitored its user's gaze, predicted his or her task intent based on observed gaze patterns, and performed anticipatory task actions according to its predictions. Results from a human-robot interaction experiment showed that anticipatory control enabled the robot to respond to user requests and complete the task faster-2.5 seconds on average and up to 3.4 seconds-compared to a robot using a reactive control method that did not anticipate user intent. Our findings highlight the promise of performing anticipatory actions for achieving efficient human-robot teamwork.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Prior research in the domain of human-robot collaboration has demonstrated how a robot can anticipate its human collaborator's task needs based on the observation of the collaborator's eye gaze and provide anticipatory robot assistance [29].”
From Understanding User Reliance on AI in Assisted Decision-Making (2022) · cites this paper as [29]
Citations
Cites
RRT-connect: An efficient approach to single-query path planning
Kuffner, J. J., & LaValle, S. M. (2000) · Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065) · doi:10.1109/robot.2000.844730 · Cited by 3,555
Mirror neurons and the simulation theory of mind-reading
Gallese, V (1998) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01262-5 · Cited by 3,201
Joint action: bodies and minds moving together
SEBANZ, N., BEKKERING, H., & KNOBLICH, G. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.12.009 · Cited by 1,950
Tomasello, M. (2009) · doi:10.7551/mitpress/8470.001.0001 · Cited by 1,896
The Roles of Vision and Eye Movements in the Control of Activities of Daily Living
Land, M. F., Mennie, N. R., & Rusted, J. M. (1999) · Perception · doi:10.1068/p2935 · Cited by 1,128
Joint attention as social cognition.
Tomasello, M. (1995) · Medical Entomology and Zoology · Cited by 1,004
Eye–Hand Coordination in Object Manipulation
Johansson, R. S., Westling, G., Bäckström, A., & Flanagan, J. R. (2001) · Journal of Neuroscience · doi:10.1523/jneurosci.21-17-06917.2001 · Cited by 873
(2013) · doi:10.1109/hri.2013.6483603 · Cited by 595
Prediction in Joint Action: What, When, and Where
Sebanz, N., & Knoblich, G. (2009) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2009.01024.x · Cited by 571
Perception of Another Person's Looking Behavior
Gibson, J. J., & Pick, A. D. (1963) · The American Journal of Psychology · doi:10.2307/1419779 · Cited by 400
Research works printed but not linked: 1
- seltman, 2012, Experimental Design for Behavioral and Social Sciences
Cited by
Sorted by citations
Social Eye Gaze in Human-Robot Interaction: A Review
Admoni, H., & Scassellati, B. M. (2017) · Journal of Human-Robot Interaction · doi:10.5898/jhri.6.1.admoni · Cited by 623
“Interviews can also be used to elicit anecdotal evidence that supports or explains the study’s findings (Huang & Mutlu, 2016; Mutlu, Yamaoka, Kanda, Ishiguro, & Hagita, 2009).”
BackgroundMethodArtificial Intelligence in Advanced Manufacturing: Current Status and Future Outlook
Arinez, J. F., Chang, Q., Gao, R. X., Xu, C., & Zhang, J. (2020) · Journal of Manufacturing Science and Engineering · doi:10.1115/1.4047855 · Cited by 584
“In other work [82], an “ anticipatory control ” method is developed to proactively plan robot actions based on anticipation of the human operator ’ s intention during task performance.”
Cobot programming for collaborative industrial tasks: An overview
Zaatari, S. E., Marei, M., Li, W., & Usman, Z. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2019.03.003 · Cited by 470
Hentout, A., Mustapha, A., Maoudj, A., & Akli, I. (2019) · Advanced Robotics · doi:10.1080/01691864.2019.1636714 · Cited by 463
Embodiment in Socially Interactive Robots
Deng, E., Mutlu, B., & Matarić, M. J. (2019) · Foundations and Trends in Robotics · doi:10.1561/2300000056 · Cited by 261
“approaches to human-robot collaboration, such as using cross-training to improve task sharing between human and robot workers [220], planning shared work plans taking human ergonomics into consideration [236], and adapting robot actions to human motion, availability, adaptability, and intent [127, 160, 129, 221].”
BackgroundMethodSelf-citationAppearance-Based Gaze Estimation Using Dilated-Convolutions
Chen, Z., & Shi, B. E. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20876-9_20 · Cited by 185
“Gaze tracking has long been considered as an important research topic, as it has various promising real-world applications, such as gaze-based interfaces [5, 20], foveated rendering in virtual reality [19], behavioral analysis [11] and human-robot interaction [12].”
BackgroundGombolay, M., Bair, A., Huang, C., & Shah, J. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364916688255 · Cited by 160
“…that humans typically prefer the robot to have greater autonomy (Baraglia et al., 2016; Gombolay et al., 2015; Hoffman and Breazeal, 2007; Huang and Mutlu, 2016; Liu et al., 2016), roboticists must balance the desire for increased automation and the performance improvements it yields…”
BackgroundSelf-citationLiu, H., & Wang, L. (2019) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2019.11.001 · Cited by 149
A Review of Human–Machine Cooperation in the Robotics Domain
Yang, C., Zhu, Y., & Chen, Y. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3131684 · Cited by 141
Probabilistic Human Intent Recognition for Shared Autonomy in Assistive Robotics
Jain, S., & Argall, B. (2019) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3359614 · Cited by 125
“For example, user gaze patterns are studied to understand human control behavior for shared manipulation [3] and to perform anticipatory actions [14].”
BackgroundMethodOptimal Scheduling of Human–Robot Collaborative Assembly Operations With Time Petri Nets
Casalino, A., Zanchettin, A. M., Piroddi, L., & Rocco, P. (2019) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2019.2932150 · Cited by 118
“The development of anticipatory behaviors has been extensively investigated in the literature, and several results have highlighted the corresponding benefits [22], [23].”
ResultControlling Object Hand-Over in Human–Robot Collaboration Via Natural Wearable Sensing
Wang, W., Li, R., Diekel, Z. M., Chen, Y., Zhang, Z., & Jia, Y. (2018) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2018.2883176 · Cited by 106
“Notably, a robot-assisted system working with a human has many advantages in assembly tasks compared with conventional automatic machines [5].”
Chadalavada, R. T., Andreasson, H., Schindler, M., Palm, R., & Lilienthal, A. J. (2019) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2019.101830 · Cited by 99
Human-Centered Collaborative Robots With Deep Reinforcement Learning
Ghadirzadeh, A., Chen, X., Yin, W.-J., Yi, Z., Björkman, M., & Kragic, D. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3047730 · Cited by 88
Andrist, S., Gleicher, M. L., & Mutlu, B. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026033 · Cited by 87
“Recent work has explored how a collaborative robot in the worker role might react to user gaze in order to increase efficiency and naturalness of the interaction [28].”
BackgroundSelf-citationSaleh, K., Hossny, M., & Nahavandi, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793991 · Cited by 85
“Thus, the understanding of human behaviors and intentions was considered as one of the most crucial capabilities that AGVs need to acquire [4], [5].”
BackgroundTelepresence Social Robotics towards Co-Presence: A Review
Almeida, L., Menezes, P. J., & Dias, J. M. M. (2022) · Applied Sciences · doi:10.3390/app12115557 · Cited by 84
Schydlo, P., Raković, M., Jamone, L., & Santos-Victor, J. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460924 · Cited by 71
“This ability finds applications in areas such as cooperative robotics [1], [2], auto-mobile safety [3], elderly care [4], among many others [5].”
BackgroundRobot-Assisted Disassembly Sequence Planning With Real-Time Human Motion Prediction
Lee, M.-L., Liu, W., Behdad, S., Liang, X. P., & Zheng, M. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2022.3185889 · Cited by 69
Understanding User Reliance on AI in Assisted Decision-Making
Cao, S., & Huang, C.-M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555572 · Cited by 71
“Prior research in the domain of human-robot collaboration has demonstrated how a robot can anticipate its human collaborator’s task needs based on the observation of the collaborator’s eye gaze and provide anticipatory robot assistance [29].”
BackgroundSelf-citationDesigning Human-Agent Collaborations: Commitment, responsiveness, and support
Cila, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517500 · Cited by 65
“perform anticipatory actions [71]; utilizing gestures for object inference in interactions [156]; and using motion patterns for planning”
MethodHuman Gaze Following for Human-Robot Interaction
Saran, A., Majumdar, S., Short, E. S., Thomaz, A. L., & Niekum, S. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593580 · Cited by 64
“Gaze information enables the establishment of joint attention between the human and robot partner, the recognition of human behavior [19] and the execution of anticipatory actions [20].”
BackgroundGaze-Based Intention Estimation: Principles, Methodologies, and Applications in HRI
Belardinelli, A. (2024) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3656376 · Cited by 63
“…localize the point of regard (POR) in 3D, and specific methods have been proposed for that, often relying on fiducial markers [e.g., Haji Fathaliyan et al. 2018; Huang and Mutlu 2016; Takahashi et al. 2018] but new solutions are trying to do without such aides [Liu et al. 2020; Weber et al. 2018].”
BackgroundMethodInfluentialLiterature Review on Recent Trends and Perspectives of Collaborative Robotics in Work 4.0
Weidemann, C., Mandischer, N., van Kerkom, F., Corves, B. J., Hüsing, M., Kraus, T., & Garus, C. (2023) · Robotics · doi:10.3390/robotics12030084 · Cited by 64
User-Adaptive Interaction in Social Robots: A Survey Focusing on Non-physical Interaction
Martins, G. S., Santos, L., & Dias, J. M. M. (2018) · International Journal of Social Robotics · doi:10.1007/s12369-018-0485-4 · Cited by 61
Combining gaze and AI planning for online human intention recognition
Singh, R., Miller, T., Newn, J., Velloso, E., Vetere, F., & Sonenberg, L. (2020) · Artificial Intelligence · doi:10.1016/j.artint.2020.103275 · Cited by 63
Deep learning-based human action recognition to leverage context awareness in collaborative assembly
Moutinho, D., Rocha, L. F., Costa, C. M., Teixeira, L. F., & Veiga, G. M. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102449 · Cited by 59
Recursive Bayesian Human Intent Recognition in Shared-Control Robotics
Jain, S., & Argall, B. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593766 · Cited by 57
“For example, gaze patterns are studied to understand shared manipulation [13] and anticipatory actions [14].”
Lin, C. J., Wang, K., Tadesse, A. A., & Woldegiorgis, B. H. (2021) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2021.12.001 · Cited by 53
Human Gaze Assisted Artificial Intelligence: A Review
Zhang, R., Saran, A., Liu, B., Zhu, Y., Guo, S., Niekum, S., Ballard, D., & Hayhoe, M. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/689 · Cited by 48
“Intention-revealing gaze enhances collaboration in object referring [Fang et al. , 2015], teleoperation [Yu et al. , 2014], shared autonomy [Aronson et al. , 2018], collaborative manipulation [Huang and Mutlu, 2016], and assisted reaching and grasping [ Shafti et al. , 2019 ] .”
BackgroundRossi, S., Rossi, A., & Dautenhahn, K. (2020) · International Journal of Social Robotics · doi:10.1007/s12369-020-00650-z · Cited by 47
“patterns are often used to achieve a collaborative human– robot interaction [55], these also play a fundamental role during non-interactive tasks.”
MethodMoGaze: A Dataset of Full-Body Motions that Includes Workspace Geometry and Eye-Gaze
Kratzer, P., Bihlmaier, S., Midlagajni, N. B., Prakash, R., Toussaint, M., & Mainprice, J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3043167 · Cited by 47
“Using data from eye tracking, their system uses early prediction to make the robot proactively support the human [17].”
MethodObject Transfer Point Estimation for Fluent Human-Robot Handovers
Nemlekar, H., Dutia, D., & Li, Z. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794008 · Cited by 45
“, gaze or body gestures [16]), inferring the implicit intent encoded in human motion can be more efficient and less intrusive, as it minimizes end user effort for maintaining explicit or even exaggerated communication.”
BackgroundHuman Factors Considerations for Enabling Functional Use of Exosystems in Operational Environments
Stirling, L., Siu, H. C., Jones, E. M., & Duda, K. (2018) · IEEE Systems Journal · doi:10.1109/jsyst.2018.2821689 · Cited by 44
“In human–robot teams, studies find that the ability to anticipate human behavior can improve the safety and fluency between a human and robot (e.g., [61]–[63]).”
BackgroundTowards High Performance Low Complexity Calibration in Appearance Based Gaze Estimation
Chen, Z., & Shi, B. E. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3148386 · Cited by 43
“EYE gaze has been used successfully in many promising applications, such as human-computer interfaces [1], [2], human-robot interaction [3], health care [4], virtual reality [5], [6] and social behavioral analysis [7], [8].”
MethodExploring tasks and challenges in human-robot collaborative systems: A review
Asif, S., Callari, T., Khan, F., Eimontaite, I., Hubbard, E.-M., Bahraini, M. S., Webb, P., & Lohse, N. (2026) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103102 · Cited by 39
“Moreover, advanced control strategies and AI integration are critical enablers that enhance the adaptability and efficiency of HRCSs systems [146].”
A network analytic approach to gaze coordination during a collaborative task
Andrist, S., Ruis, A. R., & Shaffer, D. W. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.07.017 · Cited by 39
“In particular, such models will aid in the development of anticipatory rather than reactive control mechanisms for agent behavior, improving coordination in collaborative contexts (Hayashi, 2016; Huang & Mutlu, 2016; Olsen et al., 2016).”
BackgroundThe relevance of signal timing in human-robot collaborative manipulation
Cini, F., Banfi, T., Ciuti, G., Craighero, L., & Controzzi, M. (2021) · Science Robotics · doi:10.1126/scirobotics.abg1308 · Cited by 39
Toward Active Physical Human–Robot Interaction: Quantifying the Human State During Interactions
Hu, Y. H., Abe, N., Benallegue, M., Yamanobe, N., Venture, G., & Yoshida, E. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3138684 · Cited by 39
Gaze-Based Intention Estimation for Shared Autonomy in Pick-and-Place Tasks
Fuchs, S., & Belardinelli, A. (2021) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2021.647930 · Cited by 39
“…makes it difficult to compare our system to other approaches presented in the literature, since often either natural eye-hand coordination (Haji Fathaliyan et al., 2018; Wang et al., 2020) or no eye-hand coordination at all (Huang and Mutlu, 2016) is used for intention recognition (see section 2).”
BackgroundInfluentialCooney, M. D., & Menezes, M. L. R. (2018) · Multimodal Technologies and Interaction · doi:10.3390/mti2030052 · Cited by 39
Predicting Intention Through Eye Gaze Patterns
Koochaki, F., & Najafizadeh, L. (2018) · IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas.2018.8584665 · Cited by 40
Mediating Human-Robot Interactions with Virtual, Augmented, and Mixed Reality
Szafir, D. J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-21565-1_9 · Cited by 38
Zhang, Z., Peng, G., Wang, W., Chen, Y., Jia, Y., & Liu, S. (2022) · Sensors · doi:10.3390/s22114279 · Cited by 38
“[30] devised an anticipatory control method to assist human partners based on observed gaze patterns.”
MethodHayes, B., & Shah, J. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989778 · Cited by 37
“This type of anticipatory information can improve task completion time, idle time, concurrent motion, and human-robot separation distance during human-robot collaboration [1], [2], [3], [4], [5], [6], [7].”
BackgroundDini, A., Murko, C., Yahyanejad, S., Augsdörfer, U., Hofbaur, M., & Paletta, L. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206301 · Cited by 36
“To work seamlessly and efficiently with their human counterparts, robots must similarly rely on predictions of the human worker’s behavior, emotions, task specific actions and intent to plan their actions, such as, in anticipatory control with humanintheloop architecture [3] to enable robots to proactively perform…”
BackgroundBarz, M., & Sonntag, D. (2021) · Sensors · doi:10.3390/s21124143 · Cited by 38
“This can help in analyzing complex interaction behavior of humans: it bears the potential to facilitate novel real-time adaptive humancomputer interaction [7,17], and to boost the efficiency in research based on eye tracking by automating the time-consuming and expensive data annotation process [12].”
BackgroundPreference-Based Assistance Prediction for Human-Robot Collaboration Tasks
Grigore, E. C., Roncone, A., Mangin, O., & Scassellati, B. M. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593716 · Cited by 33
A scoping review of gaze and eye tracking-based control methods for assistive robotic arms
Fischer-Janzen, A., Wendt, T. M., & Laerhoven, K. V. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1326670 · Cited by 36
Construction Robot Teleoperation Safeguard Based on Real-Time Human Hand Motion Prediction
Zhou, T., Zhu, Q., Shi, Y., & Du, J. (2022) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0002289 · Cited by 32
Saleh, K., Hossny, M., & Nahavandi, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.12.091 · Cited by 30
Gaze Tracking Using an Unmodified Web Camera and Convolutional Neural Network
Ansari, M. F., Kasprowski, P., & Obetkal, M. (2021) · Applied Sciences · doi:10.3390/app11199068 · Cited by 30
“Gaze estimation has long been recognized as an important research topic since it has strong real-world applications, for instance, in human–computer interfaces [1,2], gazebased interfaces [3,4], virtual reality [5,6], health care [7], behavioral analysis [8], and communication skills [9].”
BackgroundGaze-based Intention Anticipation over Driving Manoeuvres in Semi-Autonomous Vehicles
Wu, M., Louw, T., Lahijanian, M., Ruan, W., Huang, X., Merat, N., & Kwiatkowska, M. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967779 · Cited by 31
Singh, J., & Modi, N. (2019) · Heliyon · doi:10.1016/j.heliyon.2019.e03033 · Cited by 29
“Robots can be assigned various task, the sequence of which can be decided upon the eye gaze of the user [122, 123].”
BackgroundOffset Calibration for Appearance-Based Gaze Estimation via Gaze Decomposition
Chen, Z., & Shi, B. E. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093419 · Cited by 30
“As an important cue about people’s intent, eye gaze has been used in many applications, such as human-computer interfaces [4, 23, 30], human-robot interaction [16], virtual reality [24, 29], social behavioral analysis [15] and health care [12].”
BackgroundZhou, T., Wang, Y., Zhu, Q., & Du, J. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104232 · Cited by 27
Effects of Integrated Intent Recognition and Communication on Human-Robot Collaboration
Chang, M. L., Gutierrez, R. A., Khante, P., Short, E. S., & Thomaz, A. L. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593359 · Cited by 27
A neural integrator model for planning and value-based decision making of a robotics assistant
Wojtak, W., Ferreira, F., Vicente, P., Louro, L., Bicho, E. G., & Erlhagen, W. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05224-8 · Cited by 26
“hold out the object that the operator needs next has a major impact on user satisfaction [30, 32, 46].”
BackgroundHuman-Robot Interaction in a Shared Augmented Reality Workspace
Qiu, S., Liu, H., Zhang, Z., Zhu, Y., & Zhu, S. C. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340781 · Cited by 28
“From a robot’s angle, it models humans by inferring goals [36], [37], tracking mental states [38], [39], predicting actions [40], and recognizing intention and attention [41], [42].”
BackgroundVisual Search Target Inference in Natural Interaction Settings with Machine Learning
Barz, M., Stauden, S., & Sonntag, D. (2020) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3379155.3391314 · Cited by 24
Mahadevan, K., Sousa, M., Tang, A., & Grossman, T. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445355 · Cited by 23
“The technique is a re-implementation of an existing technique that prior work has proposed to request assistance from a robot [2] and to command it [18], serving as a baseline for comparisons.”
BackgroundMethodVargas, A. M., Cominelli, L., Dell’Orletta⋄, F., & Scilingo, E. P. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2020.591164 · Cited by 25
“…robots have obtained encouraging results in several research fields such as educational robotics (Budiharto et al., 2017), collaborative robotics (Huang and Mutlu, 2016; Gustavsson et al., 2017), surgical robotics (Zinchenko et al., 2017), assistive robotics (Wu et al., 2014; Zhou et al., 2018),…”
BackgroundEye Gaze for Assistive Manipulation
Aronson, R. M., & Admoni, H. (2020) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3371382.3377434 · Cited by 21
“Moreover, these observations have been used to build collaborative systems that monitor gaze behavior to determine people’s intentions during food serving [10] or handover [9] tasks.”
BackgroundOptimizing the Timing of Intelligent Suggestion in Virtual Reality
Yu, D., Desai, R., Zhang, T., Benko, H., Jonker, T. R., & Gupta, A. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545632 · Cited by 22
“Much existing research focuses onbuildingmodels that appropriate gaze traces or scanpaths to predict selection intentions [21, 37, 39, 59, 61, 72].”
BackgroundIVO Robot: A New Social Robot for Human-Robot Collaboration
Laplaza, J., Rodríguez-Linares, N., Domínguez-Vidal, J. E., Herrero, F., Hernandez, S., Lopez, A., Sanfeliu, A., & Garrell, A. (2022) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889458 · Cited by 22
“5 s in advance in line with the work done by Huang et al. [10].”
ResultFostering short-term human anticipatory behavior in human-robot collaboration
Psarakis, L., Nathanael, D., & Marmaras, N. (2021) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2021.103241 · Cited by 27
What is Proactive Human-Robot Interaction? - A Review of a Progressive Field and Its Definitions
broek, M. K. V. D., & Moeslund, T. B. (2024) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3650117 · Cited by 21
Learning Semantics of Gestural Instructions for Human-Robot Collaboration
Shukla, D., Erkent, Ö., & Piater, J. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00007 · Cited by 21
“Huang and Mutlu (2016) presented an anticipatory control method that enables the robot to proactively perform a pick-and-place task based on anticipated actions of their human partners.”
BackgroundDetecting errors in pick and place procedures
Bovo, R., Binetti, N., Brumby, D. P., & Julier, S. (2020) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3377325.3377497 · Cited by 21
“They can be used to identify a user’s mental states (hesitation, confusion, clarity) [11] and intention (i.”
BackgroundERA: Expert Retrieval and Assembly for Early Action Prediction
Foo, L. G., Li, T., Rahmani, H., Ke, Q., & Liu, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_38 · Cited by 20
Multi-modal Intention Prediction with Probabilistic Movement Primitives
Dermy, O., Charpillet, F., & Ivaldi, S. (2018) · Springer proceedings in advanced robotics · doi:10.1007/978-3-319-89327-3_14 · Cited by 20
“Others to improve the robot behavior by ensuring the safety of the interaction [24]; by anticipating the action of their partner [13]; or by adapting robot actions according to the intention of their partner [17].”
BackgroundSpiking Neural Networks for early prediction in human–robot collaboration
Zhou, T., & Wachs, J. P. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919872252 · Cited by 20
“Gaze was also used to predict human intentions for anticipatory motion planning in a robotic servant (Huang and Mutlu, 2016).”
MethodZhu, Z., Zhang, D., Chi, C., Li, M., & Lee, D.-J. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3210219 · Cited by 19
Preliminary Work on a Virtual Reality Interface for the Guidance of Underwater Robots
de la Cruz, M., Casañ, G. A., Sanz, P. J., & Marín, R. (2020) · Robotics · doi:10.3390/robotics9040081 · Cited by 18
“Although there are systems which try to use AI solutions, in which the robot is partially autonomous from the human operator’s instructions which can reduce the impact of his/her decisions [13], or systems in which there is anticipation of the user actions [14], we consider that most problems can be solved using bette…”
MethodBalancing Efficiency and Comfort in Robot-Assisted Bite Transfer
Belkhale, S., Gordon, E. K., Chen, Y., Srinivasa, S. S., Bhattacharjee, T., & Sadigh, D. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812332 · Cited by 17
Inferring Goals with Gaze during Teleoperated Manipulation
Aronson, R. M., Almutlak, N., & Admoni, H. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636551 · Cited by 17
“such as teleoperating a robot [6], generating robot demonstrations [24], or requesting objects from a robot server [8].”
BackgroundHierarchical Intention Tracking for Robust Human-Robot Collaboration in Industrial Assembly Tasks
Huang, Z., Mun, Y.-J., Li, X., Xie, Y., Zhong, N., Liang, W., Geng, J., Chen, T., & Driggs-Campbell, K. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160515 · Cited by 17
“To estimate human intention, various types of observation on human behavior are used as input, including human trajectories [1], [25], [26], gesture [16], [27], gaze [28], speech [29], facial expressions [30], and force-torque measurements [8], [24].”
BackgroundScenario-Based Programming of Voice-Controlled Medical Robotic Systems
Rogowski, A. (2022) · Sensors · doi:10.3390/s22239520 · Cited by 17
“There are also works dealing with the combination of voice and haptic control [22] or voice and operator’s gaze monitoring [23].”
BackgroundTransfer Learning of Human Preferences for Proactive Robot Assistance in Assembly Tasks
Nemlekar, H., Dhanaraj, N., Guan, A., Gupta, S. K., & Nikolaidis, S. (2023) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3568162.3576965 · Cited by 17
Domínguez-Vidal, J. E., Rodríguez, N., & Sanfeliu, A. (2024) · International Journal of Social Robotics · doi:10.1007/s12369-024-01103-7 · Cited by 17
Cui, Y., Poon, J., Miró, J. V., Yamazaki, K., Sugimoto, K., & Matsubara, T. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9798-2 · Cited by 17
“Examples of partner anticipation include the work by Huang and Mutlu (2016), using Support Vector Machines to predict eye gaze and treat it as the human partner’s focus of attention, which formed the basis for subsequent robot activity.”
MethodPerson-Specific Gaze Estimation from Low-Quality Webcam Images
Ansari, M. F., Kasprowski, P., & Peer, P. (2023) · Sensors · doi:10.3390/s23084138 · Cited by 16
“Gaze estimation is a significant research problem due to its various applications in real life, including human–computer interactions [1], behavioral analysis [2], virtual reality [3,4], and health care [5].”
BackgroundMondellini, M., Prajod, P., Nicora, M. L., Chiappini, M., Micheletti, E., Storm, F. A., Vertechy, R., André, E., & Malosio, M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1245857 · Cited by 16
Physical symbol grounding and instance learning through demonstration and eye tracking
Penkov, S., Bordallo, A., & Ramamoorthy, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989697 · Cited by 16
“The predominant use of eye tracking in the field of robotics is in human-robot interaction settings, where gaze information enables the recognition of human behaviour [19] and the execution of anticipatory actions [20].”
BackgroundSeeming autonomy, technology and the uncanny valley
Gahrn-Andersen, R. (2020) · AI & Society · doi:10.1007/s00146-020-01040-9 · Cited by 16
Gaze-Speech Coordination Influences the Persuasiveness of Human-Robot Dialog in the Wild
Fischer, K., Langedijk, R. M., Nissen, L. D., Ramírez, E. R., & Palinko, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62056-1_14 · Cited by 16
“[5] find that a robot that responds quickly to the user’s need for assistance and then moves quickly to fulfill its task is perceived as more polite than a robot that takes more time to respond.”
BackgroundPredicting short-term next-active-object through visual attention and hand position
Jiang, J., Nan, Z., Chen, H., Chen, S., & Zheng, N. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.12.069 · Cited by 15
Preemptive Motion Planning for Human-to-Robot Indirect Placement Handovers
Choi, A., Jawed, M. K., & Joo, J. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811558 · Cited by 15
“Similar to [33], we use a set of weights where (cid:15) is a decay factor.”
BackgroundMethodSocial Moments: A Perspective on Interaction for Social Robotics
Durantin, G., Heath, S., & Wiles, J. (2017) · Frontiers in Robotics and AI · doi:10.3389/frobt.2017.00024 · Cited by 15
Correia, F., Melo, F. S., & Paiva, A. (2022) · Topics in Cognitive Science · doi:10.1111/tops.12634 · Cited by 20
Visual Search Target Inference Using Bag of Deep Visual Words
Stauden, S., Barz, M., & Sonntag, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00111-7_25 · Cited by 14
“Inferring visual search targets helps to construct and improve intelligent user interfaces in many fields, e.g., robotics [9] or similar to examples in [18].”
BackgroundInvoking Principles of Groupware to Develop and Evaluate Present and Future Human-Agent Teams
Flathmann, C., Schelble, B. G., Tubre, B., McNeese, N. J., & Rodeghero, P. (2020) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3406499.3415072 · Cited by 14
“can be seen as a type of HAT, their autonomous agents exist in a cyber-physcial domain allowing for different physical interactions not directly comparable to purely cyber agents [31].”
BackgroundHuman Movement Direction Classification using Virtual Reality and Eye Tracking
Pettersson, J., & Falkman, P. (2020) · Procedia Manufacturing · doi:10.1016/j.promfg.2020.10.015 · Cited by 15
“Other ways are to monitor the gaze o predict an upcoming decision [13] or analyze bioelectric signals, such as electromyography, to predict human motion [5].”
BackgroundGaze-based Attention Recognition for Human-Robot Collaboration
Prajod, P., Nicora, M. L., Malosio, M., & André, E. (2023) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3594806.3594819 · Cited by 14
Pettersson, J., & Falkman, P. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1157957 · Cited by 15
Turn-Taking Prediction for Human–Robot Collaborative Assembly Considering Human Uncertainty
Xu, W., Feng, S., Yao, B., Ji, Z., & Liu, Z. (2023) · Journal of Manufacturing Science and Engineering · doi:10.1115/1.4063231 · Cited by 13
Rogowski, A., Bieliszczuk, K., & Rapcewicz, J. (2020) · Sensors · doi:10.3390/s20247287 · Cited by 13
“In order to increase effectiveness of speech-based human-robot teamwork, Huang and Mutlu developed anticipatory robot control system based on user’s gaze monitoring [25].”
BackgroundStargazer: An Interactive Camera Robot for Capturing How-To Videos Based on Subtle Instructor Cues
Li, J., Sousa, M., Mahadevan, K., Wang, B. K., Aoyagui, P. A., Yu, N., Yang, A., Balakrishnan, R., Tang, A., & Grossman, T. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580896 · Cited by 17
“Tracking the human’s gaze enables anticipatory robot planning and execution of actions [2, 4, 5, 25, 40], or can be used to prompt robot takeover when it detects user hesitation [47].”
BackgroundUnsupervised Outlier Detection in Appearance-Based Gaze Estimation
Chen, Z., Deng, D., Pi, J., & Shi, B. E. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00139 · Cited by 14
“Human gaze has been recognized as an important cue for inferring people’s intent in many applications, such as human-computer interfaces [4, 25, 29], human-robot interaction [18], virtual reality [26, 28], social behavioral analysis [16], long-range tracking [36] and health care [12].”
BackgroundTwo-Stage Clustering of Human Preferences for Action Prediction in Assembly Tasks
Nemlekar, H., Modi, J., Gupta, S. K., & Nikolaidis, S. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561649 · Cited by 14
“Human preferences for robot actions can also be measured from EEG signals [15], or from a short window of human arm motion [16]–[18] or gaze pattern [19].”
Background
Metadata
- Authors
- Huang, C.-M., & Mutlu, B.
- Affiliations
- Yale University (US)
- University of Wisconsin–Madison (US)
- Venue
- 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI)
- Publisher
- IEEE
- Published
- 2016-03
- Pages
- 83-90
- OpenAlex
- W4229726131
- Cited by
- 227
- References
- 31
- Citations by year
- 2026: 17 · 2025: 28 · 2024: 31 · 2023: 27 · 2022: 28 · 2021: 21 · 2020: 24 · 2019: 22 · 2018: 20 · 2017: 8 · 2016: 1
- FWCI
- 28.71
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Human-Robot Interaction (HRI) (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- anticipatory control, human-robot collaboration, gaze prediction, human-robot interaction, task completion time
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
