CoChat
Sign Up
PreprintCited by 167Robot Manipulation and LearningReinforcement Learning in RoboticsDomain Adaptation and Few-Shot Learning

BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning

Open access
Jang, Eric, Irpan, Alex, Khansari, Mohi, Kappler, Daniel, Ebert, Frederik, Lynch, Corey, Levine, Sergey, Finn, Chelsea·2022·arXiv
DOI 10.48550/arxiv.2202.02005arXiv 2202.02005
Open this paper in CoChat — free accountOpen PDF

Abstract

In this paper, we study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks, a long-standing challenge in robot learning. We approach the challenge from an imitation learning perspective, aiming to study how scaling and broadening the data collected can facilitate such generalization. To that end, we develop an interactive and flexible imitation learning system that can learn from both demonstrations and interventions and can be conditioned on different forms of information that convey the task, including pre-trained embeddings of natural language or videos of humans performing the task. When scaling data collection on a real robot to more than 100 distinct tasks, we find that this system can perform 24 unseen manipulation tasks with an average success rate of 44%, without any robot demonstrations for those tasks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2202.02005

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The tasks MimicGen was demonstrated on have pose variation well beyond typical prior works using BC from human demos.

    “The tasks also have pose variation well beyond typical prior works using BC from human demos [1,3-7,30,33,55,56].”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [3]

    Section 4 (p. 3) describes BC-Z as a demonstration-based manipulation system with objects in randomized poses, so it is a relevant prior-work example. Appendix B.2 (p. 14) says objects were only occasionally shuffled between demonstrations. The paper does not compare its pose variation with MimicGen’s or establish that MimicGen’s is well beyond it. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • A dataset of over 20,000 trajectories enables generalization to tasks with modest changes in objects and goals.

    “For example, [3] showed that a dataset of over 20,000 trajectories enables generalization to tasks with modest changes in objects and goals.”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [3]

  • A large number of robot learning works and benchmarks use quasi-static tasks, making the system broadly applicable.

    “However, a large number of robot learning works and benchmarks use quasi-static tasks [1,3-7,14,18,19,22,30,33,51, 55, 56, 64-66], making the system broadly applicable.”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [3]

  • Several related efforts also collect large datasets (e.g. 1000s of demonstrations) by using a large number of human operators over extended periods of time.

    “Several related efforts [3-6, 27] also collect large datasets (e.g. 1000s of demonstrations) by using a large number of human operators over extended periods of time.”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [3]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 30 · Cited by 167

Research works printed but not linked: 47

  1. [1][1] C. Finn, T. Yu, T. Zhang, P. Abbeel, and S. Levine. One-shot visual imitation learning via meta-learning. In Conference on Robot Learning , pages 357-368. PMLR, 2017. -
  2. [2][2] S. James, M. Bloesch, and A. J. Davison. Task-embedded control networks for few-shot imitation learning. In Conference on Robot Learning , pages 783-795. PMLR, 2018. -
  3. [4][4] A. Bonardi, S. James, and A. J. Davison. Learning one-shot imitation from humans without humans. IEEE Robotics and Automation Letters , 5(2):3533-3539, 2020. -
  4. [5][5] S. Young, D. Gandhi, S. Tulsiani, A. Gupta, P. Abbeel, and L. Pinto. Visual imitation made easy. arXiv e-prints , pages arXiv-2008, 2020. -
  5. [6][6] Y. Duan, M. Andrychowicz, B. C. Stadie, J. Ho, J. Schneider, I. Sutskever, P. Abbeel, and W. Zaremba. One-shot imitation learning. arXiv preprint arXiv:1703.07326 , 2017. - · arXiv
  6. [7][7] S. Dasari and A. Gupta. Transformers for one-shot visual imitation. Conference on Robot Learning (CoRL) , 2020. -
  7. [8][8] R. Rahmatizadeh, P. Abolghasemi, L. B¨ ol¨ oni, and S. Levine. Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration. In 2018 IEEE international conference on robotics and automation (ICRA) , pages 3758-3765. IEEE, 2018. -
  8. [10][10] S. Khansari-Zadeh and A. Billard. Learning stable non-linear dynamical systems with gaussian mixture models. IEEE Transaction on Robotics , 27(5):943-957, 2011. -
  9. [11][11] A. Billard, Y. Epars, S. Calinon, S. Schaal, and G. Cheng. Discovering optimal imitation strategies. Robotics and autonomous systems , 47(2-3):69-77, 2004. -
  10. [12][12] S. Schaal, J. Peters, J. Nakanishi, and A. Ijspeert. Learning movement primitives. In Robotics research. the eleventh international symposium , pages 561-572. Springer, 2005. -
  11. [13][13] R. Chalodhorn, D. B. Grimes, K. Grochow, and R. P. Rao. Learning to walk through imitation. In IJCAI , volume 7, pages 2084-2090, 2007. -
  12. [15][15] K. M¨ ulling, J. Kober, O. Kroemer, and J. Peters. Learning to select and generalize striking movements in robot table tennis. The International Journal of Robotics Research , 32(3):263279, 2013. -
  13. [17][17] T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel. Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In 2018 IEEE International Conference on Robotics and Automation (ICRA) , pages 5628-5635. IEEE, 2018. -
  14. [18][18] A. Zhou, E. Jang, D. Kappler, A. Herzog, M. Khansari, P. Wohlhart, Y. Bai, M. Kalakrishnan, S. Levine, and C. Finn. Watch, try, learn: Meta-learning from demonstrations and reward. International Conference on Learning Representations (ICLR) , 2020. -
  15. [19][19] T. L. Paine, S. G. Colmenarejo, Z. Wang, S. Reed, Y. Aytar, T. Pfaff, M. W. Hoffman, G. BarthMaron, S. Cabi, D. Budden, et al. One-shot high-fidelity imitation: Training large-scale deep nets with rl. arXiv preprint arXiv:1810.05017 , 2018. - · arXiv
  16. [20][20] D.-A. Huang, S. Nair, D. Xu, Y. Zhu, A. Garg, L. Fei-Fei, S. Savarese, and J. C. Niebles. Neural task graphs: Generalizing to unseen tasks from a single video demonstration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8565-8574, 2019. -
  17. [21][21] D. Pathak, P. Mahmoudieh, G. Luo, P. Agrawal, D. Chen, Y. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell. Zero-shot visual imitation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops , pages 2050-2053, 2018. -
  18. [22][22] P. Goyal, R. J. Mooney, and S. Niekum. Zero-shot task adaptation using natural language. arXiv preprint arXiv:2106.02972 , 2021. - · arXiv
  19. [23][23] S. Stepputtis, J. Campbell, M. Phielipp, S. Lee, C. Baral, and H. B. Amor. Languageconditioned imitation learning for robot manipulation tasks. arXiv preprint arXiv:2010.12083 , 2020. - · arXiv
  20. [24][24] C. Lynch and P. Sermanet. Grounding language in play. arXiv preprint arXiv:2005.07648 , 2020. - · arXiv
  21. [25][25] S. Calinon, P. Evrard, E. Gribovskaya, A. Billard, and A. Kheddar. Learning collaborative manipulation tasks by demonstration using a haptic interface. In 2009 International Conference on Advanced Robotics , pages 1-6. IEEE, 2009. -
  22. [26][26] S. Ross, G. Gordon, and D. Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics , pages 627-635. JMLR Workshop and Conference Proceedings, 2011. -
  23. [27][27] S. Ross and J. A. Bagnell. Reinforcement and imitation learning via interactive no-regret learning. arXiv preprint arXiv:1406.5979 , 2014. - · arXiv
  24. [28][28] M. Laskey, J. Lee, R. Fox, A. Dragan, and K. Goldberg. Dart: Noise injection for robust imitation learning. In Conference on robot learning , pages 143-156. PMLR, 2017. -
  25. [29][29] M. Kelly, C. Sidrane, K. Driggs-Campbell, and M. J. Kochenderfer. Hg-dagger: Interactive imitation learning with human experts. In 2019 International Conference on Robotics and Automation (ICRA) , pages 8077-8083. IEEE, 2019. -
  26. [30][30] J. Spencer, S. Choudhury, M. Barnes, M. Schmittle, M. Chiang, P. Ramadge, and S. Srinivasa. Learning from interventions: Human-robot interaction as both explicit and implicit feedback. 07 2020. doi:10.15607/RSS.2020.XVI.055. - · DOI
  27. [31][31] M. Khansari, D. Kappler, J. Luo, J. Bingham, and M. Kalakrishnan. Action image representation: Learning scalable deep grasping policies with zero real world data. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 3597-3603, 2020. -
  28. [32][32] L. Pinto and A. Gupta. Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours. In 2016 IEEE international conference on robotics and automation (ICRA) , pages 3406-3413. IEEE, 2016. -
  29. [33][33] J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea, and K. Goldberg. Dexnet 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv preprint arXiv:1703.09312 , 2017. - · arXiv
  30. [34][34] D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, et al. Scalable deep reinforcement learning for vision-based robotic manipulation. In Conference on Robot Learning , pages 651-673. PMLR, 2018. -
  31. [35][35] J. Hatori, Y. Kikuchi, S. Kobayashi, K. Takahashi, Y. Tsuboi, Y. Unno, W. Ko, and J. Tan. Interactively picking real-world objects with unconstrained spoken language instructions. In IEEE International Conference on Robotics and Automation (ICRA) , 2018. -
  32. [36][36] A. Gupta, A. Murali, D. Gandhi, and L. Pinto. Robot learning in homes: Improving generalization and reducing dataset bias. arXiv preprint arXiv:1807.07049 , 2018. - · arXiv
  33. [37][37] F. Sadeghi, A. Toshev, E. Jang, and S. Levine. Sim2real viewpoint invariant visual servoing by recurrent control. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4691-4699, 2018. -
  34. [38][38] J. Tobin, L. Biewald, R. Duan, M. Andrychowicz, A. Handa, V. Kumar, B. McGrew, A. Ray, J. Schneider, P. Welinder, et al. Domain randomization and generative models for robotic grasping. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 3482-3489. IEEE, 2018. -
  35. [39][39] B. Mehta, M. Diaz, F. Golemo, C. J. Pal, and L. Paull. Active domain randomization. In Conference on Robot Learning , pages 1162-1176. PMLR, 2020. -
  36. [42][42] C. Finn and S. Levine. Deep visual foresight for planning robot motion. In 2017 IEEE International Conference on Robotics and Automation (ICRA) , pages 2786-2793. IEEE, 2017. -
  37. [43][43] S. Dasari, F. Ebert, S. Tian, S. Nair, B. Bucher, K. Schmeckpeper, S. Singh, S. Levine, and C. Finn. Robonet: Large-scale multi-robot learning. arXiv preprint arXiv:1910.11215 , 2019. - · arXiv
  38. [44][44] Y. Chebotar, K. Hausman, Y. Lu, T. Xiao, D. Kalashnikov, J. Varley, A. Irpan, B. Eysenbach, R. Julian, C. Finn, et al. Actionable models: Unsupervised offline reinforcement learning of robotic skills. arXiv preprint arXiv:2104.07749 , 2021. - · arXiv
  39. [45][45] D. Kalashnikov, J. Varley, Y. Chebotar, B. Swanson, R. Jonschkowski, C. Finn, S. Levine, and K. Hausman. Mt-opt: Continuous multi-task robotic reinforcement learning at scale. arXiv preprint arXiv:2104.08212 , 2021. - · arXiv
  40. [46][46] Y. Yang, D. Cer, A. Ahmad, M. Guo, J. Law, N. Constant, G. H. Abrego, S. Yuan, C. Tar, Y.-H. Sung, et al. Multilingual universal sentence encoder for semantic retrieval. arXiv preprint arXiv:1907.04307 , 2019. - · arXiv
  41. [48][48] K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In European conference on computer vision , pages 630-645. Springer, 2016. -
  42. [49][49] P. J. Huber. Robust estimation of a location parameter. In Breakthroughs in statistics , pages 492-518. Springer, 1992. -
  43. [50][50] A. v. d. Oord, Y . Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 , 2018. - · arXiv
  44. [51][51] D. Ho, K. Rao, Z. Xu, E. Jang, M. Khansari, and Y. Bai. Retinagan: An object-aware approach to sim-to-real transfer. In 2021 International Conference on Robotics and Automation (ICRA) , 2021. -
  45. [52][52] P. de Haan, D. Jayaraman, and S. Levine. Causal confusion in imitation learning. arXiv preprint arXiv:1905.11979 , 2019. - · arXiv
  46. [53][53] H. Zhang, M. Ciss´ e, Y. N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk minimization. In ICLR (Poster) , 2018. URL https://openreview.net/forum?id=r1Ddp1-Rb . -
  47. [54][54] A. Miech, J.-B. Alayrac, L. Smaira, I. Laptev, J. Sivic, and A. Zisserman. End-to-end learning of visual representations from uncurated instructional videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9879-9889, 2020.

Authors
Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., & Finn, C.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2202.02005
arXiv
2202.02005
OpenAlex
W3205423339
Cited by
167
References
30
Citations by year
2026: 1 · 2025: 22 · 2024: 76 · 2023: 53 · 2022: 14 · 2021: 1
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
video embedding

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review