The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration
Open accessAbstract
We propose a technique for multi-task learning from demonstration that trains the controller of a low-cost robotic arm to accomplish several complex picking and placing tasks, as well as non-prehensile manipulation. The controller is a recurrent neural network using raw images as input and generating robot arm trajectories, with the parameters shared across the tasks. The controller also combines VAE-GAN-based reconstruction with autoregressive multimodal action prediction. Our results demonstrate that it is possible to learn complex manipulation tasks, such as picking up a towel, wiping an object, and depositing the towel to its previous position, entirely from raw images with direct behavior cloning. We show that weight sharing and reconstruction-based regularization substantially improve generalization and robustness, and training on multiple tasks simultaneously increases the success rate on all tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
A survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
Self-citationOn Learning, Representing, and Generalizing a Task in a Humanoid Robot
Calinon, S., Guenter, F., & Billard, A. (2007) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2006.886952 · Cited by 1,090
Learning and generalization of motor skills by learning from demonstration
Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152385 · Cited by 702
Calinon, S., D'halluin, F., Caldwell, D. G., & Billard, A. G. (2009) · 2009 9th IEEE-RAS International Conference on Humanoid Robots · doi:10.1109/ichr.2009.5379592 · Cited by 73
Research works printed but not linked: 36
- graves, 2013, Generating Sequences with Recurrent Neural Networks
- mclachlan, 1988, Mixture models. inference and applications to clustering, Statistics Textbooks and Monographs, 1
- ba, 2016, Layer normalization
- goodfellow, 2014, Generative adversarial nets, Advances in neural information processing systems, 2672
- kingma, 2013, Auto-encoding variational bayes
- srivastava, 2014, Dropout: A simple way to prevent neural networks from overfitting, The Journal of Machine Learning Research, 15, 1929
- kingma, 2014, Semi-supervised learning with deep generative models, Advances in neural information processing systems, 3581
- larochelle, 2011, The neural autoregressive distribution estimator, AISTATS, 1, 2
- bishop, 1994, Mixture density networks, Technical Report
- forbes, 2014, Robot programming by demonstration with crowdsourced action fixes, Second AAAI Conference on Human Computation and Crowdsourcing
- sutskever, 2014, Sequence to sequence learning with neural networks, Advances in Neural Information Processing Systems (NIPS), 3104
- 10.1145/1957656.1957788 · DOI
- fang, 2016, Learning models for constraint-based motion parameterization from interactive physics-based simulation, IEEE Int Conf Intell Robots Syst (IROS), 4005
- 10.1016/j.neunet.2008.02.003 · DOI · Paper page
- 10.2174/1573399812666160613113556 · DOI · Paper page
- pinto, 2016, Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours, IEEE International Conference on Robotics and Automation (ICRA), 763
- mayer, 2006, A system for robotic heart surgery that learns to tie knots using recurrent neural networks, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 543
- ioffe, 2015, Batch Normalization Accelerating Deep Network Training by Reducing Internal Covariate Shift
- sung, 2015, Robobarista: Object part-based transfer of manipulation trajectories from crowd-sourcing in 3d point-clouds, International Symposium on Robotics Research (ISRR)
- rahmatizadeh, 2016, Learning real manipulation tasks from virtual demonstrations using lstm
- 10.1109/TRO.2015.2495003 · DOI
- larsen, 2016, Autoencoding beyond pixels using a learned similarity metric, International Conference on Machine Learning, 1558
- 10.1177/0278364915594244 · DOI
- lioutikov, 2016, Learning manipulation by sequencing motor primitives with a two-armed robot, Intelligent Autonomous Systems, 1601, 10.1007/978-3-319-08338-4_115 · DOI
- 10.1109/MRA.2010.936947 · DOI · Paper page
- kehoe, 2013, Cloud-based robot grasping with the Google object recognition engine, IEEE International Conference on Robotics and Automation (ICRA), 4263
- 10.1177/0278364910371999 · DOI · Paper page
- agrawal, 2016, Learning to poke by poking Experiential learning of intuitive physics
- bengio, 1999, Modeling high-dimensional discrete data with multi-layer neural networks, Advances in Neural Information Processing Systems (NIPS), 99, 400
- tieleman, 2012, Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude, COURSERA Neural Networks for Machine Learning, 4
- deisenroth, 2014, Multi-task policy search for robotics, IEEE International Conference on Robotics and Automation (ICRA), 3876, 10.1109/ICRA.2014.6907421 · DOI
- caruana, 1995, Learning many related tasks at the same time with backpropagation, Advances in neural information processing systems, 657
- 10.1016/0004-3702(92)90065-6 · DOI · Paper page
- pinto, 2016, Learning to push by grasping Using multiple tasks for effective learning
- oord, 2016, Pixel recurrent neural networks, International Conference on Machine Learning, 1747
- devin, 2016, Learning modular neural network policies for multi-task and multi-robot transfer
Cited by
Recent Advances in Robot Learning from Demonstration
Ravichandar, H., Polydoros, A. S., Chernova, S., & Billard, A. (2020) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-100819-063206 · Cited by 752
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Zhao, T. Z., Kumar, V., Levine, S., & Finn, C. (2023) · arXiv · doi:10.48550/arxiv.2304.13705 · Cited by 673
Self-citationLearning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Zhao, T., Kumar, V., Levine, S., & Finn, C. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.016 · Cited by 674
Self-citationDeep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461249 · Cited by 609
Diffusion policy: Visuomotor policy learning via action diffusion
Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., & Song, S. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649241273668 · Cited by 613
“The simplest form of explicit policies maps from world state or observation directly to action (Pomerleau, 1988; Zhang et al., 2018; Florence et al., 2020; Ross et al., 2011; Toyer et al., 2020; Rahmatizadeh et al., 2018); Bojarski et al., 2016).”
Metadata
- Authors
- Rahmatizadeh, R., Abolghasemi, P., Boloni, L., & Levine, S.
- Venue
- 2018 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 3758-3765
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2735668228
- Cited by
- 205
- References
- 9
- FWCI
- 42.79
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration. https://app.cochat.ai/papers/10.1109/icra.2018.8461076
- The paper
- Rahmatizadeh, R., Abolghasemi, P., Boloni, L., & Levine, S. (2018). Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration. 2018 IEEE International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/icra.2018.8461076
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.