The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning
Open accessAbstract
Humans and animals are capable of learning a new behavior by observing others perform the skill just once.We consider the problem of allowing a robot to do the same -learning from a video of a human, even when there is domain shift in the perspective, environment, and embodiment between the robot and the observed human.Prior approaches to this problem have hand-specified how human and robot actions correspond and often relied on explicit human pose detection systems.In this work, we present an approach for one-shot learning from a video of a human by using human and robot demonstration data from a variety of previous tasks to build up prior knowledge through meta-learning.Then, combining this prior knowledge and only a single video demonstration from a human, the robot can perform the task that the human demonstrated.We show experiments on both a PR2 arm and a Sawyer arm, demonstrating that after meta-learning, the robot can learn to place, push, and pick-andplace new objects using just one video of a human performing the manipulation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Self-citationA survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227
Deep domain confusion Maximizing for domain invariance
Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., & Darrell, T. (2014) · arXiv · doi:10.48550/arxiv.1412.3474 · Cited by 2,285
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.019 · Cited by 2,139
Learning from Simulated and Unsupervised Images through Adversarial Training
Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., & Webb, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.241 · Cited by 1,814
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.18 · Cited by 1,603
ALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. (2018) · doi:10.1184/r1/6603146 · Cited by 1,338
Unsupervised Visual Domain Adaptation Using Subspace Alignment
Fernando, B., Habrard, A., Sebban, M., & Tuytelaars, T. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.368 · Cited by 1,314
Learning to Generalize: Meta-Learning for Domain Generalization
Li, D., Yang, Y., Song, Y.-Z., & Hospedales, T. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11596 · Cited by 1,251
Learning and generalization of motor skills by learning from demonstration
Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152385 · Cited by 702
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618
Self-citationDeep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461249 · Cited by 609
Self-citationComputational approaches to motor learning by imitation
Schaal, S., Ijspeert, A., & Billard, A. (2003) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.2002.1258 · Cited by 587
Imitation: Is cognitive neuroscience solving the correspondence problem?
Brass, M., & Heyes, C. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.08.007 · Cited by 580
Gong, B., Grauman, K. L., & Sha, F. (2013) · Cited by 459
Learning to select and generalize striking movements in robot table tennis
Mülling, K., Kober, J., Kroemer, O., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364912472380 · Cited by 396
Recasting Gradient-Based Meta-Learning as Hierarchical BayesPreprint
Grant, E., Finn, C., Levine, S., Darrell, T., & Griffiths, T. L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.08930 · Cited by 390
Self-citationOne-Shot Visual Imitation Learning via Meta-Learning
Finn, C., Yu, T., Zhang, T., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1709.04905 · Cited by 341
Self-citationTabula rasa: Model transfer for object category detection
Aytar, Y., & Zisserman, A. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126504 · Cited by 334
Teaching and learning of robot tasks via observation of human performance
Dillmann, R. (2004) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2004.03.005 · Cited by 303
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
Liu, Y., Gupta, A., Abbeel, P., & Levine, S. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8462901 · Cited by 278
Self-citationOnline movement adaptation based on previous sensor experiences
Pastor, P., Righetti, L., Kalakrishnan, M., & Schaal, S. (2011) · 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2011.6095059 · Cited by 219
Tai, L., Zhang, J., Liu, M., & Burgard, W. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460968 · Cited by 192
Learning a visuomotor controller for real world robotic grasping using simulated depth imagesPreprint
Viereck, U., ten Pas, A., Saenko, K., & Platt, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04652 · Cited by 178
Robot Learning Manipulation Action Plans by "Watching" Unconstrained Videos from the World Wide Web
Yang, Y., Li, Y., Fermuller, C., & Aloimonos, Y. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9671 · Cited by 177
Ramírez-Amaro, K., Beetz, M., & Cheng, G. (2015) · Artificial Intelligence · doi:10.1016/j.artint.2015.08.009 · Cited by 135
Interactive grasp learning based on human demonstration
Ekvall, S., & Kragić, D. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2004.1308798 · Cited by 123
First-Person Activity Forecasting with Online Inverse Reinforcement Learning
Rhinehart, N., & Kitani, K. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.399 · Cited by 122
Unsupervised Perceptual Rewards for Imitation Learning
Sermanet, P., Xu, K., & Levine, S. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.050 · Cited by 121
Self-citationOne-Shot Learning with a Hierarchical Nonparametric Bayesian Model
Salakhutdinov, R., Tenenbaum, J. B., & Torralba, A. B. (2011) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 100
Learning collaborative manipulation tasks by demonstration using a haptic interface
Calinon, S., Evrard, P., Gribovskaya, E., Billard, A., & Kheddar, A. (2009) · HAL · Cited by 97
Satellite image-based localization via learned embeddings
Kim, D.-K., & Walter, M. R. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989239 · Cited by 81
Towards Robust Skill Generalization: Unifying Learning from Demonstration and Motion Planning.
Rana, M. A., Mukadam, M., Ahmadzadeh, S. R., Chernova, S., & Boots, B. (2017) · Conference on Robot Learning · Cited by 70
A syntactic approach to robot imitation learning using probabilistic activity grammars
Lee, K., Su, Y., Kim, T., & Demiris, Y. (2013) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2013.08.003 · Cited by 61
Zeng, A., Song, S., Yu, K., Donlon, E., Hogan, F. R., Bauzá, M., Ma, D., Taylor, O., Liu, M., Romo, E., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.01330 · Cited by 45
Visual recognition of grasps for human-to-robot mapping
Kjellström, H., Romero, J., & Kragić, D. (2008) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2008.4650917 · Cited by 37
Teaching a Humanoid Robot to Recognize and Reproduce Social Cues
Calinon, S., & Billard, A. G. (2006) · IEEE RO-MAN · doi:10.1109/roman.2006.314458 · Cited by 37
Equivalence of regularization and truncated iteration for general ill-posed problems
Santos, R. J. (1996) · Linear Algebra and its Applications · doi:10.1016/0024-3795(94)00114-6 · Cited by 32
Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective
Akgun, Cakmak, Yoo, & Thomaz (2012) · Cited by 29
Learning real manipulation tasks from virtual demonstrations using LSTMPreprint
Rahmatizadeh, R., Abolghasemi, P., Behal, A., & Bölöni, L. L. (2016) · arXiv (Cornell University) · Cited by 27
Autonomy Infused Teleoperation with Application to BCI ManipulationPreprint
Muelling, K., Venkatraman, A., Valois, J., Downey, J. E., Weiss, J. M., Javdani, S., Hebert, M., Schwartz, A. B., Collinger, J. L., & Bagnell, J. A. (2015) · arXiv (Cornell University) · doi:10.1184/r1/6552149 · Cited by 19
Cited by
RLBench: The Robot Learning Benchmark & Learning Environment
James, S., Ma, Z., Arrojo, D. R., & Davison, A. J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974707 · Cited by 405
Advances and Challenges in Meta-Learning: A Technical Review
Vettoruzzo, A., Bouguelia, M.-R., Vanschoren, J., Rögnvaldsson, T., & Santosh, K. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357847 · Cited by 314
Multi-Scale Metric Learning for Few-Shot Learning
Jiang, W., Huang, K., Geng, J., & Deng, X. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2995754 · Cited by 298
Deep reinforcement learning: a survey
Wang, H.-N., Liu, N., Zhang, Y.-Y., Feng, D.-W., Huang, F., Li, D.-S., & Zhang, Y.-M. (2020) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900533 · Cited by 274
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15818 · Cited by 276
“…how robots can generalize to novel object instances [31, 34, 32, 35, 36], to tasks involving novel combinations of objects and skills [37, 38, 39, 40, 41], to new goals or language instructions [42, 43, 41, 44, 45, 46, 47], to tasks with novel semantic object categories [48, 49], and to…”
BackgroundSelf-citation
Metadata
- Authors
- Yu, T., Finn, C., Dasari, S., Xie, A., Zhang, T., Abbeel, P., & Levine, S.
- Venue
- Robotics: Science and Systems XIV
- Published
- 2018-06-26
- OpenAlex
- W2942459156
- Cited by
- 275
- References
- 42
- FWCI
- 62.87
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.