Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration
Open accessAbstract
We introduce a simple new method for visual imitation learning, which allows a novel robot manipulation task to be learned from a single human demonstration, without requiring any prior knowledge of the object being interacted with. Our method models imitation learning as a state estimation problem, with the state defined as the end-effector’s pose at the point where object interaction begins, as observed from the demonstration. By modelling a manipulation task as a coarse, approach trajectory followed by a fine, interaction trajectory, this state estimator can be trained in a self-supervised manner, by automatically moving the end-effector’s camera around the object. At test time, the end-effector is moved to the estimated state through a linear path, at which point the demonstration’s end-effector velocities are simply repeated, enabling convenient acquisition of a complex interaction trajectory without actually needing to explicitly learn a policy. Real-world experiments on 8 everyday tasks show that our method can learn a diverse range of skills from just a single human demonstration, whilst also yielding a stable and interpretable controller. Videos at: www.robot-learning.uk/coarse-to-fine-imitation-learning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Several recent works learn policies using only a handful of demonstrations by replaying demonstrations in new scenes.
“Several recent works learn policies using only a handful of demonstrations by replaying the demonstrations in new scenes [8-11, 46-48].”
From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [9]
The Introduction describes replaying a single demonstration’s end-effector velocities after reaching an estimated pose, and the Results report testing each task at 20 object poses. This supports learning from very few demonstrations and using the demonstrated motion in changed configurations. But the Introduction and Conclusions explicitly say the method does not need to learn a policy, so that part of the attribution is inaccurate. (LLM, conf=0.88)
Experimental — accuracy not yet measuredSeveral recent works propose imitation learning methods that replay previous human demonstrations.
“In fact, several recent works build on this intuition and propose imitation learning methods that replay previous human demonstrations [8-11].”
From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [9]
Replay-based imitation learning methods are promising for learning manipulation tasks using a handful of demonstrations.
“Replay-based imitation learning methods are promising for learning manipulation tasks using a handful of demonstrations [8-11, 46-48]”
From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · see citing works
Generative adversarial imitation learning
Ho, J., & Ermon, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03476 · Cited by 1,942 · see citing works
Overcoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712 · see citing works
Learning and generalization of motor skills by learning from demonstration
Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152385 · Cited by 702 · see citing works
Silver, T., Allen, K., Tenenbaum, J., & Kaelbling, L. (2018) · arXiv · doi:10.48550/arxiv.1812.06298 · Cited by 113 · see citing works
Flagged first, then by citations
Research works printed but not linked: 29
- T. Zhang, Z. McCarthy, O. Jowl, D. Lee, X. Chen, K. Goldberg, and P. Abbeel, "Deep imitation learning for complex manipulation tasks from virtual reality teleoperation," in IEEE International Conference on Robotics and Automation (ICRA), 2018.
- M. Vecerik, T. Hester, J. Scholz, F. Wang, O. Pietquin, B. Piot, N. Heess, T. Rothörl, T. Lampe, and M. Riedmiller, "Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards," arXiv preprint arXiv:1707.08817, 2017. · arXiv
- C. Finn, T. Yu, T. Zhang, P. Abbeel, and S. Levine, "One-shot visual imitation learning via meta-learning," in Conference on Robot Learning (CoRL), 2017.
- P. Englert and M. Toussaint, "Learning manipulation skills from a single demonstration," The International Journal of Robotics Research, 2018.
- Y. Y. Tsai, H. Xu, Z. Ding, C. Zhang, E. Johns, and B. Huang, "DROID: Minimizing the reality gap using single-shot human demon- stration," IEEE Robotics and Automation Letters (RA-L), 2021.
- R. Rahmatizadeh, P. Abolghasemi, L. Bölöni, and S. Levine, "Vision- based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration," in IEEE International Conference on Robotics and Automation (ICRA), 2018.
- P. Florence, L. Manuelli, and R. Tedrake, "Self-supervised correspon- dence in visuomotor policy learning," IEEE Robotics and Automation Letters (RA-L), 2019.
- S. Young, D. Gandhi, S. Tulsiani, A. Gupta, P. Abbeel, and L. Pinto, "Visual imitation made easy," in Conference on Robot Learning (CoRL), 2020.
- S. James, A. J. Davison, and E. Johns, "Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task," in Conference on Robot Learning (CoRL), 2017.
- N. Di Palo and E. Johns, "SAFARI: Safe and active robot imitation learning with imagination," arXiv preprint arXiv:2011.09586, 2020. · arXiv
- S. Paradis, M. Hwang, B. Thananjeyan, J. Ichnowski, D. Seita, D. Fer, T. Low, J. E. Gonzalez, and K. Goldberg, "Intermittent visual servoing: Efficiently learning policies robust to instrument changes for high- precision surgical manipulation," arXiv preprint arXiv:2011.06163, 2020. · arXiv
- T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine, "Residual reinforcement learning for robot control," in IEEE International Conference on Robotics and Automation (ICRA), 2019.
- B. Thananjeyan, A. Balakrishna, U. Rosolia, F. Li, R. McAllister, J. E. Gonzalez, S. Levine, F. Borrelli, and K. Goldberg, "Safety augmented value estimation from demonstrations (SAVED): Safe deep model- based RL for sparse cost robotic tasks," IEEE Robotics and Automation Letters (RA-L), 2020.
- Y.-Y. Tsai, B. Xiao, E. Johns, and G.-Z. Yang, "Constrained space optimization and reinforcement learning for complex tasks," IEEE Robotics and Automation Letters (RA-L), 2020.
- G. Schoettler, A. Nair, J. Luo, S. Bahl, J. A. Ojea, E. Solowjow, and S. Levine, "Deep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.
- C. Finn, S. Levine, and P. Abbeel, "Guided cost learning: Deep inverse optimal control via policy optimization," in International Conference on Machine Learning (ICML), 2016.
- P. Sermanet, C. Lynch, Y. Chebotar, J. Hsu, E. Jang, S. Schaal, S. Levine, and G. Brain, "Time-contrastive networks: Self-supervised learning from video," in IEEE International Conference on Robotics and Automation (ICRA), 2018.
- Y. Schroecker, M. Vecerik, and J. Scholz, "Generative predecessor models for sample-efficient imitation learning," in International Con- ference on Learning Representations (ICLR), 2019.
- A. Nair, D. Chen, P. Agrawal, P. Isola, P. Abbeel, J. Malik, and S. Levine, "Combining self-supervised learning and imitation for vision-based rope manipulation," in IEEE International Conference on Robotics and Automation (ICRA), 2017.
- M. Sieb, Z. Xian, A. Huang, O. Kroemer, and K. Fragkiadaki, "Graph- structured visual imitation," in Conference on Robot Learning (CoRL), 2020.
- Y. Duan, M. Andrychowicz, B. C. Stadie, J. Ho, J. Schneider, I. Sutskever, P. Abbeel, and W. Zaremba, "One-shot imitation learning," in Advances in Neural Information Processing Systems (NeurIPS), 2017.
- S. James, M. Bloesch, and A. J. Davison, "Task-embedded control networks for few-shot imitation learning," in Conference on Robot Learning (CoRL), 2018.
- C. Finn, P. Abbeel, and S. Levine, "Model-agnostic meta-learning for fast adaptation of deep networks," in International Conference on Machine Learning (ICML), 2017.
- C. Yu, Z. Cai, H. Pham, and Q.-C. Pham, "Siamese convolutional neural network for sub-millimeter-accurate camera pose estimation and visual servoing," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019.
- E. Y. Puang, K. Peng Tee, and W. Jing, "KOVIS: Keypoint-based visual servoing with zero-shot sim-to-real transfer for robotics manip- ulation," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.
- M. Argus, L. Hermann, J. Long, and T. Brox, "Flowcontrol: Optical flow based visual servoing," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.
- J. Hartung, G. Knapp, and B. K. Sinha, Statistical meta-analysis with applications. John Wiley & Sons, 2011.
- S. Thrun, "Probabilistic robotics," Communications of the ACM, 2002.
- A. Kendall, M. Grimes, and R. Cipolla, "PoseNet: A convolutional network for real-time 6-DOF camera relocalization," in IEEE Interna- tional Conference on Computer Vision (ICCV), 2015.
Cited by
Sorted by citations
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Zhao, T., Kumar, V., Levine, S., & Finn, C. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.016 · Cited by 674 · see citing works
RT-H: Action Hierarchies Using Language
Belkhale, S., Ding, T., Xiao, T., Sermanet, P., Vuong, Q., Tompson, J., Chebotar, Y., Dwibedi, D., & Sadigh, D. (2024) · arXiv · doi:10.48550/arxiv.2403.01823 · Cited by 71
“Another line of work shows the benefits of separating coarse and fine action abstractions in IL, but requiring both coarse and fine action annotations [29, 30].”
On the Effectiveness of Retrieval, Alignment, and Replay in Manipulation
Palo, N. D., & Johns, E. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3349832 · Cited by 11 · see citing works
“As shown in [9], this enables a task to be learned from just a single demonstration.”
BackgroundMethodInfluentialSelf-citationLearning Multi-Stage Tasks with One Demonstration via Self-Replay
Di Palo, N., & Johns, E. (2021) · arXiv · doi:10.48550/arxiv.2111.07447 · Cited by 8 · see citing works
“Following this framework, introduced in [3] as Figure 2: An example of the training steps for a two-stage task: picking a cup and placing it on a plate.”
BackgroundMethodInfluentialSelf-citationMimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
Mandlekar, A., Nasiriany, S., Wen, B., Akinola, I., Narang, Y., Fan, L., Zhu, Y., & Fox, D. (2023) · arXiv · doi:10.48550/arxiv.2310.17596 · Cited by 5 · see citing works
Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration
Chenu, A., Serris, O., Sigaud, O., & Perrin-Gilbert, N. (2022) · arXiv · doi:10.48550/arxiv.2211.04786
“Thus, when considering a sequence of goals corresponding to low-dimensional projections of high-dimensional states, it is essential to ensure that the agent can achieve each goal by reaching states compatible with the achievement of the next goal.”
- Authors
- Johns, E.
- Affiliations
- Imperial College London (GB)
- Corresponding author
- Johns, Edward
- ORCID
- Johns, Edward
- Venue
- 2021 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2021-05-30
- Pages
- 4613-4619
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W3163183849
- Cited by
- 119
- References
- 5
- Citations by year
- 2026: 20 · 2025: 33 · 2024: 26 · 2023: 20 · 2022: 10 · 2021: 10
- FWCI
- 24.62
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Royal Academy of Engineering
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
- Keywords
- visual imitation learning, robotic manipulation, state estimation, self-supervised learning, end-effector pose estimation, object interaction, human-robot interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
