The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Deep spatial autoencoders for visuomotor learning
Open accessAbstract
Reinforcement learning provides a powerful and flexible framework for automated acquisition of robotic motion skills. However, applying reinforcement learning requires a sufficiently detailed representation of the state, including the configuration of task-relevant objects. We present an approach that automates state-space construction by learning a state representation directly from camera images. Our method uses a deep spatial autoencoder to acquire a set of feature points that describe the environment for the current task, such as the positions of objects, and then learns a motion skill with these feature points using an efficient reinforcement learning method based on local linear models. The resulting controller reacts continuously to the learned feature points, allowing the robot to dynamically manipulate objects in the world with closed-loop control. We demonstrate our method with a PR2 robot on tasks that include pushing a free-standing toy block, picking up a bag of rice using a spatula, and hanging a loop of rope on a hook at various positions. In each task, our method automatically learns to track task-relevant objects and manipulate their configuration with the robot's arm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
A New Approach to Visual Servoing in Robotics
Espiau, B., Chaumette, F., & Rives, P. (1992) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.143350 · Cited by 1,384
A new approach to visual servoing in robotics
Espiau, B., Chaumette, F., & Rives, P. (1993) · Lecture Notes in Computer Science · doi:10.1007/3-540-57132-9_8 · Cited by 1,346
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Self-citationResidual Algorithms: Reinforcement Learning with Function Approximation
Baird, L. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 918
(2007) · Cited by 849
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831
A Survey on Policy Search for Robotics
Deisenroth, M. P. (2011) · Foundations and Trends in Robotics · doi:10.1561/2300000021 · Cited by 798
Learning and generalization of motor skills by learning from demonstration
Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152385 · Cited by 702
Relative end-effector control using Cartesian position based visual servoing
Wilson, W. J., Williams Hulls, C. C., & Bell, G. S. (1996) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.538974 · Cited by 623
Policy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 564
ITERATIVE LINEAR QUADRATIC REGULATOR DESIGN FOR NONLINEAR BIOLOGICAL MOVEMENT SYSTEMS
(2004) · Proceedings of the First International Conference on Informatics in Control, Automation and Robotics · doi:10.5220/0001143902220229 · Cited by 543
Relative Entropy Policy Search
Peters, J., Mulling, K., & Altun, Y. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7727 · Cited by 455
Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics
Levine, S., & Abbeel, P. (2014) · Cited by 389
Self-citationLearning contact-rich manipulation skills with guided policy search
Levine, S., Wagener, N., & Abbeel, P. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7138994 · Cited by 267
Self-citationStochastic policy gradient reinforcement learning on a simple 3D biped
Tedrake, R., Zhang, T. W., & Seung, H. S. (2005) · 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566) · doi:10.1109/iros.2004.1389841 · Cited by 264
Experimental evaluation of uncalibrated visual servoing for precision manipulation
Jagersand, M., Fuentes, O., & Nelson, R. (2002) · Proceedings of International Conference on Robotics and Automation · doi:10.1109/robot.1997.606723 · Cited by 224
Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot
Endo, G., Morimoto, J., Matsubara, T., Nakanishi, J., & Cheng, G. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364907084980 · Cited by 207
Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning
Deisenroth, M., Rasmussen, C., & Fox, D. (2011) · Robotics: Science and Systems VII · doi:10.15607/rss.2011.vii.008 · Cited by 205
Autonomous reinforcement learning on raw visual input data in a real world application
Lange, S., Riedmiller, M., & Voigtlander, A. (2012) · The 2012 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252823 · Cited by 202
Reinforcement Learning to adjust Robot Movements to New Situations
Kober, J., Öztop, E., & Peters, J. (2010) · Robotics: Science and Systems · doi:10.15607/rss.2010.vi.005 · Cited by 160
Closing the learning-planning loop with predictive state representations
Boots, B., Siddiqi, S. M., & Gordon, G. J. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911404092 · Cited by 159
Active, uncalibrated visual servoing
Yoshimi, B. H., & Allen, P. K. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1994.350995 · Cited by 149
Bagnell, J. A., & Schneider, J. (2018) · Figshare · doi:10.1184/r1/6552458 · Cited by 148
Reinforcement Learning with Factored States and Actions
Sallans, B., & Hinton, G. E. (2004) · Journal of Machine Learning Research · doi:10.5555/1005332.1016794 · Cited by 139
Learning Contact-Rich Manipulation Skills with Guided Policy SearchPreprint
Levine, S., Wagener, N., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.05611 · Cited by 71
Self-citationAcquiring visual servoing reaching and grasping skills using neural reinforcement learning
Lampe, T., & Riedmiller, M. (2013) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2013.6707053 · Cited by 66
State Representation Learning in Robotics: Using Prior Knowledge about Physical Interaction
Jonschkowski, R., & Brock, O. (2014) · Robotics: Science and Systems · doi:10.15607/rss.2014.x.019 · Cited by 47
Cited by
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationEnd-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Self-citation
Metadata
- Authors
- Finn, C., Tan, X. Y., Duan, Y., Darrell, T., Levine, S., & Abbeel, P.
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 512-519
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2210483910
- Cited by
- 430
- References
- 33
- FWCI
- 30.13
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.