HG-DAgger: Interactive Imitation Learning with Human Experts
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Imitation learning has proven to be useful for many real-world problems, but approaches such as behavioral cloning suffer from data mismatch and compounding error issues. One attempt to address these limitations is the DAgger algorithm, which uses the state distribution induced by the novice to sample corrective actions from the expert. Such sampling schemes, however, require the expert to provide action labels without being fully in control of the system. This can decrease safety and, when using humans as experts, is likely to degrade the quality of the collected labels due to perceived actuator lag. In this work, we propose HG-DAgger, a variant of DAgger that is more suitable for interactive imitation learning from human experts in real-world systems. In addition to training a novice policy, HG-DAgger also learns a safety threshold for a model-uncertainty-based risk metric that can be used to predict the performance of the fully trained novice in different regions of the state space. We evaluate our method on both a simulated and real-world autonomous driving task, and demonstrate improved performance over both DAgger and behavioral cloning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
ALVINN: An autonomous land vehicle in a neural network
Pomerleau, D. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6603146.v1 · Cited by 1,340
Semiautonomous Vehicular Control Using Driver Modeling
Shia, V. A., Gao, Y., Vasudevan, R., Campbell, K. D., Lin, T., Borrelli, F., & Bajcsy, R. (2014) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2014.2325776 · Cited by 114
Cited by
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Zhao, T. Z., Kumar, V., Levine, S., & Finn, C. (2023) · arXiv · doi:10.48550/arxiv.2304.13705 · Cited by 673
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Zhao, T., Kumar, V., Levine, S., & Finn, C. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.016 · Cited by 674
Zhan, W., Sun, L., Wang, D., Shi, H., Clausse, A., Naumann, M., Kümmerle, J., Königshof, H., Stiller, C., Fortelle, A. D. L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03088 · Cited by 328
BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., & Finn, C. (2022) · arXiv · doi:10.48550/arxiv.2202.02005 · Cited by 167
Interactive Language: Talking to Robots in Real Time
Lynch, C., Wahid, A., Tompson, J., Ding, T., Betker, J., Baruch, R., Armstrong, T., & Florence, P. (2022) · arXiv · doi:10.48550/arxiv.2210.06407 · Cited by 113
Agentic AI: A Comprehensive Survey of Technologies, Applications, and Societal Implications
Pati, A. K. (2025) · IEEE Access · doi:10.1109/access.2025.3585609 · Cited by 100
Huang, Z., Sheng, Z., Ma, C., & Chen, S. (2024) · Communications in Transportation Research · doi:10.1016/j.commtr.2024.100127 · Cited by 88
RT-H: Action Hierarchies Using Language
Belkhale, S., Ding, T., Xiao, T., Sermanet, P., Vuong, Q., Tompson, J., Chebotar, Y., Dwibedi, D., & Sadigh, D. (2024) · arXiv · doi:10.48550/arxiv.2403.01823 · Cited by 71
“As in prior interactive IL methods [5, 40], the Fig.”
MethodHuman-in-the-Loop Imitation Learning using Remote Teleoperation
Mandlekar, A., Xu, D., Martín-Martín, R., Zhu, Y., Fei-Fei, L., & Savarese, S. (2020) · arXiv · doi:10.48550/arxiv.2012.06733 · Cited by 35
“…trajectories until the number of intervention data samples reached roughly 33% of the initial dataset samples to ensure that all methods would be able to receive the same number of human-annotated samples at each round, regardless of base policy quality, and to be consistent with prior work [18].”
BackgroundMethodResultInfluentialLazyDAgger: Reducing Context Switching in Interactive Imitation Learning
Hoque, R., Balakrishna, A., Putterman, C., Luo, M., Brown, D. S., Seita, D., Thananjeyan, B., Novoseller, E., & Goldberg, K. (2021) · 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case49439.2021.9551469 · Cited by 31
“Amir et al. [2], Kahn et al. [23], Kelly et al. [24], Mandlekar et al. [30], Spencer et al. [45], and Wang et al. [51] instead consider learning from human supervisors and present learning algorithms which utilize the timing and nature of human interventions to update the learned policy.”
BackgroundThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning
Hoque, R., Balakrishna, A., Novoseller, E., Wilcox, A., Brown, D. S., & Goldberg, K. (2021) · arXiv · doi:10.48550/arxiv.2109.08273 · Cited by 31
“We consider the interactive imitation learning (IL) setting [15], where the robot does not have access to a shaped reward function or to the MDP’s transition dynamics but can temporarily cede control to a supervisor who uses policy π h : S → A .”
BackgroundMethodInfluentialBehavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets
Du, M., Nair, S., Sadigh, D., & Finn, C. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.011 · Cited by 20
PATO: Policy Assisted TeleOperation for Scalable Robot Data Collection
Dass, S., Pertsch, K., Zhang, H., Lee, Y., Lim, J. J., & Nikolaidis, S. (2022) · arXiv · doi:10.48550/arxiv.2212.04708 · Cited by 16
“Based on the DAgger algorithm [27], works have investigated having the human themselves decide when to intervene [28], using ensemble-based support estimates [29], using discrepancies between model output and human inputs [30], [31] or risk estimates based on predicted future returns [32].”
Method
Metadata
- Authors
- Kelly, M., Sidrane, C., Driggs-Campbell, K., & Kochenderfer, M. J.
- Venue
- 2019 International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 8077-8083
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2895144567
- Cited by
- 172
- References
- 4
- FWCI
- 5.11
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.