LazyDAgger: Reducing Context Switching in Interactive Imitation Learning
Abstract
Corrective interventions while a robot is learning to automate a task provide an intuitive method for a human supervisor to assist the robot and convey information about desired behavior. However, these interventions can impose significant burden on a human supervisor, as each intervention interrupts other work the human is doing, incurs latency with each context switch between supervisor and autonomous control, and requires time to perform. We present LazyDAgger, which extends the interactive imitation learning (IL) algorithm SafeDAgger to reduce context switches between supervisor and autonomous control. We find that LazyDAgger improves the performance and robustness of the learned policy during both learning and execution while limiting burden on the supervisor. Simulation experiments suggest that LazyDAgger can reduce context switches by an average of 60% over SafeDAgger on 3 continuous control tasks while maintaining state-of-the-art policy performance. In physical fabric manipulation experiments with an ABB YuMi robot, LazyDAgger reduces context switches by 60% while achieving a 60% higher success rate than SafeDAgger at execution time.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior work has attempted to develop systems that can selectively query humans for demonstrations when they are needed, in order to reduce human operator time and burden.
“Prior work has also attempted to develop systems that can selectively query humans for demonstrations when they are needed, in order to reduce human operator time and burden [67-70].”
From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [68]
Cites
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759 · see citing works
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653 · see citing works
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037 · cited 1× · see citing works
“We model the environment as a discrete-time Markov decision process (MDP) M with states s 2 S , actions a 2 A , and time horizon T [37].”
MethodProceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,865 · see citing works
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 3× · see citing works
“Experimental Setup: For all MuJoCo environments, we use a reinforcement learning agent trained with TD3 [16] as an algorithmic supervisor.”
MethodAddressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650 · cited 1× · see citing works
“Experimental Setup: For all MuJoCo environments, we use a reinforcement learning agent trained with TD3 [16] as an algorithmic supervisor.”
MethodA survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227 · see citing works
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015430 · Cited by 2,855 · cited 1× · see citing works
“Many methods use demonstrations directly for policy learning [20, 35, 36, 49], while others use reinforcement learning to train a policy using a reward function inferred from demonstrations [1, 9, 15, 17, 54].”
MethodInternational Conference on Machine Learning (ICML)-2005
津田, 宏. (2006) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 2,558 · see citing works
Generative adversarial imitation learning
Ho, J., & Ermon, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03476 · Cited by 1,942 · cited 1× · see citing works
“Many methods use demonstrations directly for policy learning [20, 35, 36, 49], while others use reinforcement learning to train a policy using a reward function inferred from demonstrations [1, 9, 15, 17, 54].”
MethodA Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941 · see citing works
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A., & Puterman, M. L. (1995) · Technometrics · doi:10.2307/1269932 · Cited by 1,889 · see citing works
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,868 · see citing works
Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors
Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pástor, P., & Schaal, S. (2012) · Neural Computation · doi:10.1162/neco_a_00393 · Cited by 1,586 · see citing works
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · cited 1× · see citing works
“There has also been significant recent interest in active preference queries for learning reward functions from pairwise preferences over demonstrations [7, 10, 13, 19, 34, 41].”
BackgroundEfficient Training of Artificial Neural Networks for Autonomous Navigation
Pomerleau, D. (1991) · Neural Computation · doi:10.1162/neco.1991.3.1.88 · Cited by 693 · see citing works
A survey of inverse reinforcement learning: Challenges, methods and progress
Arora, S., & Doshi, P. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103500 · Cited by 619 · cited 1× · see citing works
“Learning from Offline Demonstrations: Learning from demonstrations [3, 4, 33] is a popular imitation learning approach, as it requires minimal supervisor burden: the”
BackgroundBehavioral Cloning from Observation
Torabi, F., Warnell, G., & Stone, P. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/687 · Cited by 574 · cited 2× · see citing works
“Cloning [49], DAgger [40], and SafeDAgger [53] in terms of the total supervisor burden and task performance.”
BackgroundMethodBose, B. K. (1990) · IEEE Transactions on Industrial Electronics · doi:10.1109/41.103436 · Cited by 536 · see citing works
Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues
Chen, J. Y. C., & Barnes, M. (2014) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2013.2293535 · Cited by 505 · cited 1× · see citing works
“However, the above algorithms assume that the human is continuously monitoring the system to determine when to intervene, which may not be practical in large-scale systems or continuous learning settings [12, 14, 25, 46].”
Background(2013) · Lincoln Repository (University of Lincoln) · Cited by 503 · see citing works
Learning Robust Rewards with Adversarial Inverse Reinforcement LearningPreprint
Fu, J., Luo, K. Z., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.11248 · Cited by 452 · cited 1× · see citing works
“Many methods use demonstrations directly for policy learning [20, 35, 36, 49], while others use reinforcement learning to train a policy using a reward function inferred from demonstrations [1, 9, 15, 17, 54].”
MethodAn Algorithmic Perspective on Imitation Learning
Osa, T., Pajarinen, J., Neumann, G. M., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680834116 · Cited by 387 · see citing works
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · UvA-DARE (University of Amsterdam) · Cited by 385 · see citing works
A natural semantics for lazy evaluation
Launchbury, J. (1993) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/158511.158618 · Cited by 375 · cited 1× · see citing works
“The name LazyDAgger is inspired by the concept of lazy evaluation in programming language theory [29], where”
MethodProceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems
Chung, J. J., Miklić, D., Sabattini, L., Tumer, K., & Siegwart, R. (2019) · Adaptive Agents and Multi-Agents Systems · Cited by 330 · see citing works
Validating Human–Robot Interaction Schemes in Multitasking Environments
Crandall, J. W., Goodrich, M. A., Olsen, D. R., et al. (2005) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2005.850587 · Cited by 270 · cited 1× · see citing works
“However, the above algorithms assume that the human is continuously monitoring the system to determine when to intervene, which may not be practical in large-scale systems or continuous learning settings [12, 14, 25, 46].”
BackgroundActive Preference-Based Learning of Reward Functions
Sadigh, D., Dragan, A. D., Sastry, S., & Seshia, S. A. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.053 · Cited by 260 · cited 1× · see citing works
“There has also been significant recent interest in active preference queries for learning reward functions from pairwise preferences over demonstrations [7, 10, 13, 19, 34, 41].”
BackgroundRecovery RL: Safe Reinforcement Learning With Learned Recovery Zones
Thananjeyan, B., Balakrishna, A., Nair, S., Luo, M., Srinivasan, K., Hwang, M., Gonzalez, J. E., Ibarz, J., Finn, C., & Goldberg, K. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3070252 · Cited by 218 · see citing works
Self-citationThe 11th International Conference on Autonomous Agents and Multiagent Systems
Conitzer, V., Winkoff, M., Pedgham, L., et al. (2012) · RiuNet (Universitat Politècnica de València) · Cited by 206 · see citing works
HG-DAgger: Interactive Imitation Learning with Human Experts
Kelly, M., Sidrane, C., Driggs-Campbell, K., & Kochenderfer, M. J. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793698 · Cited by 189 · cited 3× · see citing works
“Amir et al. [2], Kahn et al. [23], Kelly et al. [24], Mandlekar et al. [30], Spencer et al. [45], and Wang et al. [51] instead consider learning from human supervisors and present learning algorithms which utilize the timing and nature of human interventions to update the learned policy.”
BackgroundShared Autonomy via Deep Reinforcement Learning
Reddy, S., Dragan, A., & Levine, S. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.005 · Cited by 175 · cited 1× · see citing works
“of human advice can be unintuitive, since the learner may visit states that are significantly far from those the human supervisor would visit, making it difficult for humans to judge what correct behavior looks like without interacting with the environment themselves [38, 45].”
BackgroundExtrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from ObservationsPreprint
Brown, D. S., Goo, W., Nagarajan, P., & Niekum, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06387 · Cited by 171 · see citing works
Self-citationAsking Easy Questions: A User-Friendly Approach to Active Reward LearningPreprint
Biyik, E., Palan, M., Landolfi, N. C., Losey, D. P., & Sadigh, D. (2019) · Conference on Robot Learning · doi:10.48550/arxiv.1910.04365 · Cited by 151 · cited 1× · see citing works
“There has also been significant recent interest in active preference queries for learning reward functions from pairwise preferences over demonstrations [7, 10, 13, 19, 34, 41].”
BackgroundDART: Noise Injection for Robust Imitation LearningPreprint
Laskey, M., Lee, J., Fox, R., Dragan, A. D., & Goldberg, K. Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09327 · Cited by 131 · cited 2× · see citing works
“While DAgger has a number of desirable theoretical properties, labeling every state is costly in human time and can be a non-intuitive form of human feedback [27].”
BackgroundSelf-citationReward learning from human preferences and demonstrations in Atari
Ibarz, B., Leike, J., Pohlen, T., Irving, G., Legg, S., & Amodei, D. (2018) · arXiv · doi:10.48550/arxiv.1811.06521 · Cited by 117 · cited 1× · see citing works
“There has also been significant recent interest in active preference queries for learning reward functions from pairwise preferences over demonstrations [7, 10, 13, 19, 34, 41].”
BackgroundDeep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisor
Seita, D., Ganapathi, A., Hoque, R., Hwang, M., Cen, E., Tanwani, A. K., Balakrishna, A., Thananjeyan, B., Ichnowski, J., Jamali, N., et al. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341608 · Cited by 113 · cited 6× · see citing works
“Environment: We evaluate LazyDAgger on the fabric smoothing task from [44] (shown in Figure 4) using the simulation environment from [44].”
BackgroundMethodInfluentialSelf-citationQuery-Efficient Imitation Learning for End-to-End Simulated Driving
Zhang, J., & Cho, K. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10857 · Cited by 111 · see citing works
Interactive teaching strategies for agent training
Amir, O., Kamar, E. S., Kolobov, A., & Grosz, B. J. (2016) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 108 · see citing works
Learning Reward Functions by Integrating Human Demonstrations and Preferences
Palan, M., Shevchuk, G., Landolfi, N. C., & Sadigh, D. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.023 · Cited by 106 · cited 1× · see citing works
“There has also been significant recent interest in active preference queries for learning reward functions from pairwise preferences over demonstrations [7, 10, 13, 19, 34, 41].”
BackgroundEnsembleDAgger: A Bayesian Approach to Safe Imitation Learning
Menda, K., Driggs-Campbell, K., & Kochenderfer, M. J. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968287 · Cited by 100 · see citing works
Xie, L., Wang, S., Rosa, S., Markham, A., & Trigoni, N. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461203 · Cited by 83 · cited 1× · see citing works
“[52] leverage interventions from suboptimal supervisors to accelerate policy learning, but assume that the supervisors are algorithmic and thus can be queried cheaply.”
BackgroundVisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation
Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A., Jamali, N., Yamane, K., Iba, S., & Goldberg, K. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.034 · Cited by 74 · see citing works
Self-citationLaND: Learning to Navigate From Disengagements
Kahn, G., Abbeel, P., & Levine, S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3060404 · Cited by 53 · cited 1× · see citing works
“Amir et al. [2], Kahn et al. [23], Kelly et al. [24], Mandlekar et al. [30], Spencer et al. [45], and Wang et al. [51] instead consider learning from human supervisors and present learning algorithms which utilize the timing and nature of human interventions to update the learned policy.”
BackgroundLaskey, M., Staszak, S., Hsieh, W. Y.-S., Mahler, J., Pokorny, F. T., Dragan, A. D., & Goldberg, K. Y. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487167 · Cited by 52 · see citing works
Self-citationSafe Imitation Learning via Fast Bayesian Reward Inference from PreferencesPreprint
Brown, D. S., Coleman, R., Srinivasan, R., & Niekum, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09089 · Cited by 51 · cited 2× · see citing works
“There has also been significant recent interest in active preference queries for learning reward functions from pairwise preferences over demonstrations [7, 10, 13, 19, 34, 41].”
BackgroundSelf-citationLearning from Interventions: Human-robot interaction as both explicit and implicit feedback
Spencer, J. C., Choudhury, S., Barnes, M., Schmittle, M., Chiang, M., Ramadge, P. J., & Srinivasa, S. S. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.055 · Cited by 44 · see citing works
Where Do You Think You're Going?: Inferring Beliefs about Dynamics from BehaviorPreprint
Reddy, S., Dragan, A. D., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08010 · Cited by 37 · see citing works
Human-in-the-Loop Imitation Learning using Remote Teleoperation
Mandlekar, A., Xu, D., Martín-Martín, R., Zhu, Y., Fei-Fei, L., & Savarese, S. (2020) · arXiv · doi:10.48550/arxiv.2012.06733 · Cited by 35 · cited 1× · see citing works
“Amir et al. [2], Kahn et al. [23], Kelly et al. [24], Mandlekar et al. [30], Spencer et al. [45], and Wang et al. [51] instead consider learning from human supervisors and present learning algorithms which utilize the timing and nature of human interventions to update the learned policy.”
BackgroundSafe Reinforcement Learning Using Advantage-Based InterventionPreprint
Wagener, N., Boots, B., & Cheng, C.-A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09110 · Cited by 33 · see citing works
Scaled Autonomy: Enabling Human Operators to Control Robot Fleets
Swamy, G., Reddy, S., Levine, S., & Dragan, A. D. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196792 · Cited by 28 · cited 2× · see citing works
“However, the above algorithms assume that the human is continuously monitoring the system to determine when to intervene, which may not be practical in large-scale systems or continuous learning settings [12, 14, 25, 46].”
BackgroundAC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal TeachersPreprint
Kurenkov, A., Mandlekar, A., Martín-Martín, R., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04121 · Cited by 24 · cited 1× · see citing works
“Kurenkov et al. [26] and Xie et al. [52] leverage interventions from suboptimal supervisors to accelerate policy learning, but assume that the supervisors are algorithmic and thus can be queried cheaply.”
BackgroundNumerical Verification of Affine Systems with up to a Billion DimensionsPreprint
Bak, S., Tran, H.-D., & Johnson, T. T. (2019) · EasyChair preprint · doi:10.29007/b8zr · Cited by 20 · see citing works
Reward Learning from Narrated Demonstrations
Tung, H.-Y. F., Harley, A. W., Huang, L.-K., & Fragkiadaki, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00732 · Cited by 19 · cited 1× · see citing works
“Recent work has augmented demonstrations with additional offline information such as pairwise preferences [10, 11], human gaze [42], and natural language descriptions [50].”
BackgroundIntervention Aided Reinforcement Learning for Safe and Practical Policy Optimization in NavigationPreprint
Wang, F., Zhou, B., Chen, K., Fan, T., Zhang, X., Li, J., Tian, H., & Pan, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.06187 · Cited by 19 · cited 1× · see citing works
“[51] instead consider learning from human supervisors and present learning algorithms which utilize the timing and nature of human interventions to update the learned policy.”
BackgroundOn-Policy Robot Imitation Learning from a Converging SupervisorPreprint
Balakrishna, A., Thananjeyan, B., Lee, J., Li, F., Zahed, A., Gonzalez, J. E., & Goldberg, K. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03423 · Cited by 18 · cited 1× · see citing works
“More generally, the idea of learning from action advice has been widely explored in imitation learning algorithms [5, 21, 22, 28].”
BackgroundSelf-citationUnderstanding Teacher Gaze Patterns for Robot LearningPreprint
Saran, A., Short, E. S., Thomaz, A. L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.07202 · Cited by 13 · cited 1× · see citing works
“with additional offline information such as pairwise preferences [10, 11], human gaze [42], and natural language descriptions [50].”
BackgroundActive Attention-Modified Policy Shaping: Socially Interactive Agents Track
Faulkner, T. K., Gutierrez, R. A., Short, E. S., Hoffman, G., & Thomaz, A. L. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/qzlb8105 · Cited by 8 · see citing works
Interactive Imitation Learning in State-SpacePreprint
Jauhri, S., Celemin, C. E., & Kober, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.00524 · Cited by 7 · cited 1× · see citing works
“More generally, the idea of learning from action advice has been widely explored in imitation learning algorithms [5, 21, 22, 28].”
BackgroundPerformance Evaluation of Imitation Learning Algorithms with Human Experts
Båvenstrand, E., & Berggren, J. (2019) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 5 · see citing works
Flagged first, then by citations
4 of the 64 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Brown, 2019, Better-than-demonstrator imitation learning via automaticaly-ranked demonstrations
- Judah, Active imitation learning via state queries
- Saunders, 2018, Trial without error: Towards safe rl with human intervention
Cited by
Sorted by citations
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges
Zare, M., Kebria, P. M., Khosravi, A., & Nahavandi, S. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3395626 · Cited by 230 · see citing works
“LazyDAgger [24] reduces the number of context switches between the expert and autonomous control, while ThriftyDAgger [20] intervenes only when states are sufficiently novel or risky, thus reducing the intervention burden by limiting the total number of interventions to an user-specified budget.”
Imitation Learning: Progress, Taxonomies and Challenges
Zheng, B., Verma, S., Zhou, J. J., Tsang, I. W.-H., & Chen, F. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3213246 · Cited by 137 · see citing works
“) 8: (LazyDAGGer [67]: Determine the need of query by whether the state is safe through pre-designed threshold.”
BackgroundMethodRT-H: Action Hierarchies Using Language
Belkhale, S., Ding, T., Xiao, T., Sermanet, P., Vuong, Q., Tompson, J., Chebotar, Y., Dwibedi, D., & Sadigh, D. (2024) · arXiv · doi:10.48550/arxiv.2403.01823 · Cited by 71
A survey of demonstration learning
Correia, A., & Alexandre, L. A. (2024) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2024.104812 · Cited by 34 · see citing works
Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning
Du, M., Lee, O. Y., Nair, S., & Finn, C. (2022) · Robotics: Science and Systems · doi:10.15607/rss.2022.xviii.009 · Cited by 24 · see citing works
Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Research
Hoque, R., Shivakumar, K., Aeron, S., Deza, G., Ganapathi, A., Wong, A., Lee, J., Zeng, A., Vanhoucke, V., & Goldberg, K. Y. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981253 · Cited by 20 · see citing works
Self-citationPATO: Policy Assisted TeleOperation for Scalable Robot Data Collection
Dass, S., Pertsch, K., Zhang, H., Lee, Y., Lim, J. J., & Nikolaidis, S. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.013 · Cited by 16 · see citing works
Knowledge- and ambiguity-aware robot learning from corrective and evaluative feedback
Celemin, C. E., & Kober, J. (2023) · Neural Computing and Applications · doi:10.1007/s00521-022-08118-z · Cited by 15 · see citing works
“It has motivated the development of a family of methods SafeDAgger [26], EnsembleDAgger [27], LazyDAgger [28], or ThriftyDAgger [29], which deal in a different way the teacher interventions in order to improve feedback efficiency or safety.”
MethodNguyễn, T. P., Nguyen, H. T., & Ngo, H. Q. T. (2023) · Textile Research Journal · doi:10.1177/00405175231199257 · Cited by 13 · see citing works
“29 To provide a concise summary of the state-of-the-art background works, an overview has been compiled in Table 1.”
BackgroundA survey on imitation learning for contact-rich tasks in robotics
Tsuji, T., Kato, Y., Solak, G., Zhang, H., Petrič, T., Nori, F., & Ajoudani, A. (2026) · The International Journal of Robotics Research · doi:10.1177/02783649261417694 · Cited by 11 · see citing works
IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning
Hoque, R., Mandlekar, A., Garrett, C. R., Goldberg, K. Y., & Fox, D. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801523 · Cited by 10 · see citing works
“I-Gen employs a similar mechanism for synthesizing trajectories but has several key differences.”
Self-citationOn Optimizing Interventions in Shared Autonomy
Tan, W., Koleczek, D., Pradhan, S., Perello, N., Chettiar, V., Rohra, V., Rajaram, A., Srinivasan, S., Hossain, H. M. S., & Chandak, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20471 · Cited by 10 · see citing works
“(Hoque et al. 2021b) and (Hoque et al. 2021a) used imitation learning and supervised learning to predict action discrepancy or the novelty and the risk of the state to decide when to intervene, which have shown promising results on robotic tasks.”
Learning to Explore Informative Trajectories and Samples for Embodied Perception
Jing, Y., & Kong, T. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160951 · Cited by 11 · see citing works
“The active interactive imitation learning [20]–[22] are also related to our work in disagreement and uncertainty measuring to decide whether to request a label from the human.”
BackgroundWelte, E., & Rayyes, R. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1682437 · Cited by 8 · see citing works
“For example, LazyDAgger [45], where an additional learned meta-controller in the agent decides whether an expert should be consulted, thus reducing the number of human interactions.”
InfluentialMEGA-DAgger: Imitation Learning with Multiple Imperfect Experts
Sun, X., Yang, S., Zhou, M., et al. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.11414064 · Cited by 8 · see citing works
Active Reward Learning from Online Preferences
Myers, V., Bıyık, E., & Sadigh, D. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160439 · Cited by 7 · see citing works
Human-Robot Deformation Manipulation Skill Transfer: Sequential Fabric Unfolding Method For Robots
Fu, T., Bai, Y., Li, C., et al. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3329768 · Cited by 7 · see citing works
“Furthermore, Hoque et al. proposed LazyDAgger [18], [19], an extension of interactive imitation learning for robot folding of fabric.”
BackgroundInteractive Imitation Learning in Robotics: A Survey
Celemin, C. E., Pérez‐Dattari, R., Chisari, E., Franzese, G., Rosa, L. D. S., Prakash, R., Ajanović, Z., Ferraz, M., Valada, A., & Kober, J. (2022) · doi:10.1561/9781638281276 · Cited by 7 · see citing works
Greedy-DAgger - A Student Rollout Efficient Imitation Learning Algorithm
Torok, M., Deghat, M., & Song, Y. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3536297 · Cited by 6 · see citing works
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
Mandlekar, A., Nasiriany, S., Wen, B., Akinola, I., Narang, Y., Fan, L., Zhu, Y., & Fox, D. (2023) · arXiv · doi:10.48550/arxiv.2310.17596 · Cited by 5 · see citing works
Oh, H., & Matsubara, T. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3366755 · Cited by 5 · see citing works
“Thus,asinpriorworks[4],[5],[6],[7],thedesigned IILaimstooptimizepolicywhileminimizing C .”
BackgroundMethodInfluentialBrandfonbrener, D., Tu, S., Singh, A., Welker, S., Boodoo, C., Matni, N., & Varley, J. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161096 · Cited by 4 · see citing works
Online Imitation Learning for Manipulation via Decaying Relative Correction through Teleoperation
Pan, C., Cheng, H. H., & Hughes, J. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246547 · Cited by 3 · see citing works
“Providing correction or feedback in online human-interactive imitation learning requires significant attention and effort from human experts [5].”
Offline Preference-Based Apprenticeship LearningPreprint
Shin, D., Brown, D. S., & Dragan, A. (2021) · arXiv.org · doi:10.48550/arxiv.2107.09251 · Cited by 20 · see citing works
Self-citationParametrization of Compliant, Object-Level Controllers from Human Demonstrations
Galbally, E., Piedra, A., Brosque, C., et al. (2022) · Springer proceedings in advanced robotics · doi:10.1007/978-3-031-08140-8_42 · Cited by 1 · see citing works
Uncertainties based queries for Interactive policy learning with evaluations and corrections
Celemin, C. E., & Kober, J. (2021) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3461615.3485404
“Users could also directly teach how to perform with corrective demonstrations [4, 5] or relative corrections [7].”
BackgroundSibona, F., Luijkx, J., van der Heijden, B., Ferranti, L., & Indri, M. (2023) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin51400.2023.10218251
“A family of robot-gated IIL methods, stemming from DAgger [25], allows the agent to query a teacher intervention according to estimations of quantities related to task performance and uncertainty [26], [27].”
MethodFeldmeier, P., & Fraser, G. (2025) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3747189
“This approach is frequently applied to robot locomotion [Hoque et al., 2021; Jaegle et al., 2021] or autonomous driving [Kelly et al., 2019; Zhang and Cho, 2017] tasks but also to learn playing games [De Haan et al., 2019].”
Towards a Dynamic Shapley Value-Based Evaluations for Autonomous Robotic Learning from Videos
Chang, X., Chao, F., Copner, N. J., Shang, C., & Shen, Q. (2026) · Advances in intelligent systems and computing · doi:10.1007/978-3-032-07938-1_33
Oracle-Guided Soft Shielding for Safe Move Prediction in Chess
Rajendran, P. T., Arnéz, F., Espinoza, H., et al. (2025) · International Conference on Machine Learning and Applications · doi:10.1109/icmla66185.2025.00130
“Variants like En-semble DAgger [26], Lazy DAgger [27], and Human-Gated DAgger [28] optimize expert queries, while MEGA-DAgger [29] and SafeDAgger [9] introduce multi-expert frameworks or safety policies for robustness.”
Ying, S., He, S., & Gu, J. (2026) · Transactions in GIS · doi:10.1111/tgis.70403
“To further reduce the workload of human experts and optimize the control-switching process, mechanical gating algorithms such as SafeDAgger (Zhang and Cho 2017), LazyDAgger (Hoque, Balakrishna, Putterman, et al. 2021), and ThriftyDAgger (Hoque, Balakrishna, et al. 2021) were subsequently developed.”
Deng, X., Zhu, H., Zhang, L., et al. (2026) · AI · doi:10.3390/ai7090373
ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning
Hoque, R., Balakrishna, A., Novoseller, E., Wilcox, A., Brown, D. S., & Goldberg, K. (2021) · arXiv · doi:10.48550/arxiv.2109.08273 · Cited by 31 · see citing works
Self-citation
- Authors
- Hoque, R., Balakrishna, A., Putterman, C., Luo, M., Brown, D. S., Seita, D., Thananjeyan, B., Novoseller, E., & Goldberg, K.
- Affiliations
- University of California, Berkeley (US)
- ORCID
- Venue
- 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE)
- Publisher
- IEEE
- Published
- 2021-08-23
- Pages
- 502-509
- ISSN
- 2161-8070 (ISSN-L)
- OpenAlex
- W3202598378
- Semantic Scholar
- 7d937358c6c7c2ce29720aab7c4ed2c630e9730d
- Cited by
- 31
- References
- 64
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 5 · 2023: 8 · 2022: 6 · 2021: 3
- FWCI
- 2.54
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE International Conference on Automation Science and Engineering (CASE) (publishedVersion)
- Funders
- Siemens
- Toyota Research Institute
- Autodesk
- Intel
- Nvidia
- Intuitive Surgical
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- context switching, corrective actions, human-robot interaction, robot learning from demonstration, fabric manipulation, policy robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
