CoChat
Sign Up
PreprintCited by 12Robot Manipulation and LearningReinforcement Learning in RoboticsDomain Adaptation and Few-Shot Learning

Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations

Open access
Liang, Junchi, Wen, Bowen, Bekris, Kostas, Boularias, Abdeslam·2022·arXiv
DOI 10.48550/arxiv.2203.03797arXiv 2203.03797
Open this paper in CoChat — free accountOpen PDF

Abstract

This work aims to learn how to perform complex robot manipulation tasks that are composed of several, consecutively executed low-level sub-tasks, given as input a few visual demonstrations of the tasks performed by a person. The sub-tasks consist of moving the robot's end-effector until it reaches a sub-goal region in the task space, performing an action, and triggering the next sub-task when a pre-condition is met. Most prior work in this domain has been concerned with learning only low-level tasks, such as hitting a ball or reaching an object and grasping it. This paper describes a new neural network-based framework for learning simultaneously low-level policies as well as high-level policies, such as deciding which object to pick next or where to place it relative to other objects in the scene. A key feature of the proposed approach is that the policies are learned directly from raw videos of task demonstrations, without any manual annotation or post-processing of the data. Empirical results on object manipulation tasks with a robotic arm show that the proposed network can efficiently learn from real visual demonstrations to perform the tasks, and outperforms popular imitation learning algorithms.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2203.03797

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Replay-based imitation learning methods are promising for learning manipulation tasks using a handful of demonstrations.

    “Replay-based imitation learning methods are promising for learning manipulation tasks using a handful of demonstrations [8-11, 46-48]”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [48]

    The Abstract says the framework learns robot manipulation tasks from a few visual demonstrations, and the Evaluation reports data-efficient learning from demonstration trajectories. However, the Learning Approach describes policy training and inference from demonstrations, not a replay-based imitation method; that characterization is unsupported by this paper. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Several recent works learn policies using only a handful of demonstrations by replaying demonstrations in new scenes.

    “Several recent works learn policies using only a handful of demonstrations by replaying the demonstrations in new scenes [8-11, 46-48].”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [48]

    Algorithm 1 describes learning policies from demonstration trajectories, and Section D (Evaluation; page unavailable) tests them with random object layouts. But Section D reports using 70 training trajectories for painting and 40 for each other task—not clearly a handful—and the learning approach trains policy networks rather than replaying demonstrations in new scenes. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 27

  1. S. Calinon, Robot Programming by Demonstration, 1st ed. USA: CRC Press, Inc., 2009.
  2. O. Kroemer and G. S. Sukhatme, "Learning spatial preconditions of manipulation skills using random forests," in Proceedings of the IEEE-RAS International Conference on Humanoid Robotics, 2016. [Online]. Available: http://robotics.usc.edu/publications/954/
  3. A. S. Wang and O. Kroemer, "Learning robust manipulation strategies with multimodal state transition models and recovery heuristics," in Proceedings of (ICRA) International Conference on Robotics and Automation, May 2019, pp. 1309 -1315.
  4. Z. Wang, C. R. Garrett, L. P. Kaelbling, and T. Lozano-Pérez, "Learning compositional models of robot skills for task and motion planning," Int. J. Robotics Res., vol. 40, no. 6-7, 2021. [Online]. Available: https://doi.org/10.1177/02783649211004615 · DOI
  5. Z. Su, O. Kroemer, G. E. Loeb, G. S. Sukhatme, and S. Schaal, "Learn- ing to switch between sensorimotor primitives using multimodal haptic signals," in Proceedings of International Conference on Simulation of Adaptive Behavior (SAB '16): From Animals to Animats 14, August 2016, pp. 170 -182.
  6. H. Le, N. Jiang, A. Agarwal, M. Dudík, Y. Yue, and H. Daumé, "Hierarchical imitation and reinforcement learning," in International conference on machine learning. PMLR, 2018, pp. 2917-2926.
  7. K. Shiarlis, M. Wulfmeier, S. Salter, S. Whiteson, and I. Posner, "Taco: Learning task decomposition via temporal alignment for control," in International Conference on Machine Learning. PMLR, 2018, pp. 4654-4663.
  8. G. Konidaris and A. Barto, "Skill discovery in continuous rein- forcement learning domains using skill chaining," in Advances in Neural Information Processing Systems, Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta, Eds., vol. 22. Curran Associates, Inc., 2009.
  9. M. Toussaint, K. R. Allen, K. A. Smith, and J. B. Tenenbaum, "Differentiable physics and stable modes for tool-use and manipulation planning," in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, S. Kraus, Ed. ijcai.org, 2019, pp. 6231-6235. [Online]. Available: https://doi.org/10.24963/ijcai.2019/869 · DOI
  10. L. P. Kaelbling, "Learning to achieve goals," in IN PROC. OF IJCAI- 93. Morgan Kaufmann, 1993, pp. 1094-1098.
  11. L. P. Kaelbling and T. Lozano-Pérez, "Hierarchical task and motion planning in the now," in Proceedings of the 1st AAAI Conference on Bridging the Gap Between Task and Motion Planning, ser. AAAIWS'10-01. AAAI Press, 2010, pp. 33-42.
  12. R. T. Icarte, T. Klassen, R. Valenzano, and S. McIlraith, "Using reward machines for high-level task specification and decomposition in reinforcement learning," ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80. Stockholmsm ÃCssan, Stockholm Sweden: PMLR, 10-15 Jul 2018, pp. 2107-2116.
  13. R. Toro Icarte, E. Waldie, T. Klassen, R. Valenzano, M. Castro, and S. McIlraith, "Learning reward machines for partially observable reinforcement learning," Advances in Neural Information Processing Systems, vol. 32, pp. 15 523-15 534, 2019.
  14. B. Wen, C. Mitash, B. Ren, and K. E. Bekris, "se (3)-tracknet: Data- driven 6d pose tracking by calibrating image residuals in synthetic domains," in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 10 367-10 373.
  15. D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, and S. Levine, "Scalable deep reinforcement learning for vision-based robotic manip- ulation," ser. Proceedings of Machine Learning Research, A. Billard, A. Dragan, J. Peters, and J. Morimoto, Eds., vol. 87. PMLR, 29-31 Oct 2018, pp. 651-673.
  16. R. Fox, R. Shin, S. Krishnan, K. Goldberg, D. Song, and I. Stoica, "Parametrized hierarchical procedures for neural programming," in International Conference on Learning Representations, 2018. [Online]. Available: https://openreview.net/forum?id=rJl63fZRb
  17. D. Xu, S. Nair, Y. Zhu, J. Gao, A. Garg, L. Fei-Fei, and S. Savarese, "Neural task programming: Learning to generalize across hierarchical tasks." CoRR, vol. abs/1710.01813, 2017. [Online]. Available: http: //dblp.uni-trier.de/db/journals/corr/corr1710.html#abs-1710-01813 · arXiv
  18. D. Huang, S. Nair, D. Xu, Y. Zhu, A. Garg, L. Fei-Fei, S. Savarese, and J. C. Niebles, "Neural task graphs: Generalizing to unseen tasks from a single video demonstration," CoRR, vol. abs/1807.03480, 2018. [Online]. Available: http://arxiv.org/abs/1807.03480 · arXiv
  19. S. Nair, M. Babaeizadeh, C. Finn, S. Levine, and V. Kumar, "TRASS: time reversal as self-supervision," in 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, May 31 -August 31, 2020. IEEE, 2020, pp. 115-121. [Online]. Available: https://doi.org/10.1109/ICRA40945.2020.9196862 · DOI
  20. M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welin- der, B. McGrew, J. Tobin, P. Abbeel, and W. Zaremba, "Hindsight ex- perience replay," in Proceedings of the 31st International Conference on Neural Information Processing Systems, ser. NIPS'17. Curran Associates Inc., 2017, pp. 5055-5065.
  21. S. Nair and C. Finn, "Hierarchical foresight: Self-supervised learning of long-horizon tasks via visual subgoal generation," CoRR, vol. abs/1909.05829, 2019. [Online]. Available: http://arxiv.org/abs/1909. 05829 · arXiv
  22. P.-L. Bacon, J. Harb, and D. Precup, "The option-critic architecture," ser. AAAI'17. AAAI Press, 2017, pp. 1726-1734.
  23. O. Nachum, S. Gu, H. Lee, and S. Levine, "Data-efficient hierarchical reinforcement learning," in Proceedings of the 32nd International Conference on Neural Information Processing Systems, ser. NIPS'18, 2018, pp. 3307-3317.
  24. B. Eysenbach, A. Gupta, J. Ibarz, and S. Levine, "Diversity is all you need: Learning skills without a reward function," in International Conference on Learning Representations, 2019. [Online]. Available: https://openreview.net/forum?id=SJx63jRqFm
  25. C. Mitash, B. Wen, K. Bekris, and A. Boularias, "Scene-level pose estimation for multiple instances of densely packed objects," in Con- ference on Robot Learning. PMLR, 2020, pp. 1133-1145.
  26. J. Ho and S. Ermon, "Generative adversarial imitation learning," Advances in neural information processing systems, vol. 29, pp. 4565- 4573, 2016.
  27. B. Wen and K. Bekris, "Bundletrack: 6d pose tracking for novel objects without instance or category-level 3d models," IROS, 2021.

Cited by

Sorted by citations

Authors
Liang, J., Wen, B., Bekris, K., & Boularias, A.
Affiliations
Rutgers, The State University of New Jersey (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
Pages
8591-8597
ISSN
1050-4729 (ISSN-L)
DOI
10.48550/arxiv.2203.03797
arXiv
2203.03797
OpenAlex
W4221153200
Cited by
12
References
0
Citations by year
2026: 2 · 2025: 5 · 2024: 1 · 2023: 3 · 2022: 1
FWCI
3.82
Type
conference paper
Language
en
Open access
Licence
Locations
  • Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
National Science Foundation (1734492, 1846043)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
visual demonstration, imitation learning, robotic manipulation, robot manipulators, object manipulation, video-based learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review