The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Deep spatial autoencoders for visuomotor learning
Open accessAbstract
Reinforcement learning provides a powerful and flexible framework for automated acquisition of robotic motion skills. However, applying reinforcement learning requires a sufficiently detailed representation of the state, including the configuration of task-relevant objects. We present an approach that automates state-space construction by learning a state representation directly from camera images. Our method uses a deep spatial autoencoder to acquire a set of feature points that describe the environment for the current task, such as the positions of objects, and then learns a motion skill with these feature points using an efficient reinforcement learning method based on local linear models. The resulting controller reacts continuously to the learned feature points, allowing the robot to dynamically manipulate objects in the world with closed-loop control. We demonstrate our method with a PR2 robot on tasks that include pushing a free-standing toy block, picking up a bag of rice using a spatula, and hanging a loop of rope on a hook at various positions. In each task, our method automatically learns to track task-relevant objects and manipulate their configuration with the robot's arm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
A New Approach to Visual Servoing in Robotics
Espiau, B., Chaumette, F., & Rives, P. (1992) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.143350 · Cited by 1,384
A new approach to visual servoing in robotics
Espiau, B., Chaumette, F., & Rives, P. (1993) · Lecture Notes in Computer Science · doi:10.1007/3-540-57132-9_8 · Cited by 1,346
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Self-citationResidual Algorithms: Reinforcement Learning with Function Approximation
Baird, L. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 918
Cited by
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
“Although deep neural networks can potentially produce very complex and rich models [81, 112, 27], sometimes simpler, more data-efficient methods are preferable [34].”
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationEnd-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Self-citationTaskonomy: Disentangling Task Transfer Learning
Zamir, A. R., Sax, A., Shen, W., Guibas, L., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00391 · Cited by 1,128
An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution
Liu, R., Lehman, J., Molino, P., Such, F. P., Frank, E., Sergeev, A., & Yosinski, J. (2018) · arXiv · doi:10.48550/arxiv.1807.03247 · Cited by 784
“Moreover, several previous works have found it necessary or useful to inject geometric information to networks, for example, in prior networks to enhance spatial smoothness [32], in segmentation networks [2, 20], and in robotics control through a spatial softmax layer and an expected coordinate layer that map scenes t…”
BackgroundGibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763
Deep visual foresight for planning robot motion
Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599
Self-citationHow to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570
Self-citationDeep learning, reinforcement learning, and world models
Matsuo, Y., LeCun, Y., Sahani, M., Precup, D., Silver, D., Sugiyama, M., Uchibe, E., & Morimoto, J. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.03.037 · Cited by 543
Deep reinforcement learning in a handful of trials using probabilistic dynamics models
Chua, K., Calandra, R., McAllister, R., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.12114 · Cited by 525
Self-citationLesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.004 · Cited by 506
Model-Based Reinforcement Learning for Atari
Kaiser, L., Babaeizadeh, M., Milos, P., Osinski, B., Campbell, R. H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1903.00374 · Cited by 467
“Though most of such works do not use image observations, several recent works have incorporated images into real-world (Finn et al., 2016; Finn & Levine, 2017; Babaeizadeh et al., 2017a; Ebert et al., 2017; Piergiovanni et al., 2018; Paxton et al., 2019; Rybkin et al., 2018; Ebert et al., 2018) and…”
BackgroundSelf-citationReinforcement learning for control: Performance, stability, and deep approximators
Buşoniu, L., de Bruin, T., Tolić, D., Kober, J., & Palunko, I. (2018) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2018.09.005 · Cited by 463
One-Shot Visual Imitation Learning via Meta-Learning
Finn, C., Yu, T., Zhang, T., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1709.04905 · Cited by 341
“The final convolutional layer is transformed into spatial feature points using a spatial soft-argmax [28, 29] and concatenated with the robot’s configuration.”
MethodSelf-citationDeep learning in robotics: A review of recent research
Pierson, H. A., & Gashler, M. S. (2017) · Advanced Robotics · doi:10.1080/01691864.2017.1365009 · Cited by 331
Explainability in deep reinforcement learning
Heuillet, A., Couthouis, F., & Díaz-Rodríguez, N. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106685 · Cited by 319
State representation learning for control: An overview
Lesort, T., Díaz-Rodríguez, N., Goudou, J.-F., & Filliat, D. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.006 · Cited by 301
Visual Reinforcement Learning with Imagined Goals
Nair, A., Pong, V. H., Dalal, M., Bahl, S., Lin, S., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.04742 · Cited by 288
Self-citationAdvancements in Humanoid Robots: A Comprehensive Review and Future Prospects
Tong, Y., Liu, H., & Zhang, Z. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.124140 · Cited by 288
Deep reinforcement learning with successor features for navigation across similar environments
Zhang, J., Springenberg, J. T., Boedecker, J., & Burgard, W. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206049 · Cited by 278
Deep reinforcement learning: a survey
Wang, H.-N., Liu, N., Zhang, Y.-Y., Feng, D.-W., Huang, F., Li, D.-S., & Zhang, Y.-M. (2020) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900533 · Cited by 274
Deep reinforcement learning in computer vision: a comprehensive survey
Le, N., Rathour, V. S., Yamazaki, K., Luu, K., & Savvides, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10061-9 · Cited by 261
Artificial intelligence (AI) in augmented reality (AR)-assisted manufacturing applications: a review
Sahu, C. K., Young, C., & Rai, R. (2021) · International Journal of Production Research · doi:10.1080/00207543.2020.1859636 · Cited by 256
Learning Dexterous In-Hand Manipulation
OpenAI, Andrychowicz, M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., et al. (2018) · arXiv · doi:10.48550/arxiv.1808.00177 · Cited by 248
Darla Improving zero-shot transfer in reinforcement learning
Higgins, I., Pal, A., Rusu, A. A., Matthey, L., Burgess, C. P., Pritzel, A., Botvinick, M., Blundell, C., & Lerchner, A. (2017) · arXiv · doi:10.48550/arxiv.1707.08475 · Cited by 219
Stochastic latent actor–critic: Deep reinforcement learning with a latent variable model
Lee, A. X., Nagabandi, A., Abbeel, P., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1907.00953 · Cited by 159
Self-citationDecoupling Representation Learning from Reinforcement Learning
Stooke, A., Lee, K., Abbeel, P., & Laskin, M. (2020) · arXiv · doi:10.48550/arxiv.2009.08319 · Cited by 123
Self-citationDeep predictive policy training using reinforcement learning
Ghadirzadeh, A., Maki, A., Kragic, D., & Bjorkman, M. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206046 · Cited by 113
Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Ebert, F., Yang, Y., Schmeckpeper, K., Bucher, B., Georgakis, G., Daniilidis, K., Finn, C., & Levine, S. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.063 · Cited by 94
Self-citationBridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Ebert, F., Yang, Y., Schmeckpeper, K., Bucher, B., Georgakis, G., Daniilidis, K., Finn, C., & Levine, S. (2021) · arXiv · doi:10.48550/arxiv.2109.13396 · Cited by 94
Self-citationGTI: Learning to Generalize across Long-Horizon Tasks from Human Demonstrations
Mandlekar, A., Xu, D., Martín-Martín, R., Savarese, S., & Fei-Fei, L. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.061 · Cited by 90
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
Mandlekar, A., Xu, D., Wong, J., Nasiriany, S., Wang, C., Kulkarni, R., Fei-Fei, L., Savarese, S., Zhu, Y., & Martín-Martín, R. (2021) · arXiv · doi:10.48550/arxiv.2108.03298 · Cited by 66
Training Robots Without Robots: Deep Imitation Learning for Master-to-Robot Policy Transfer
Kim, H., Ohmura, Y., Nagakubo, A., & Kuniyoshi, Y. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3262423 · Cited by 28
“The visualized example of two-dimensional visual features extracted by SpatialSoftmax on the global vision processing network demonstrated the difference in the feature coordinates between the demonstration data collected by the master (validation set) and the robot data collected by DA-force during the test (Fig.”
MethodInfluentialLearning to Generalize Across Long-Horizon Tasks from Human Demonstrations
Mandlekar, A., Xu, D., Martín-Martín, R., Savarese, S., & Fei-Fei, L. (2020) · arXiv · doi:10.48550/arxiv.2003.06085 · Cited by 17
“These feature extractors terminate with a spatial-softmax layer [13] that infer low dimensional keypoint representations from spatial feature representations.”
BackgroundMethodInfluentialCoarse-to-Fine for Sim-to-Real: Sub-Millimetre Precision Across Wide Task Spaces
Valassakis, E., Di Palo, N., & Johns, E. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636388 · Cited by 17
Learning Multi-Stage Tasks with One Demonstration via Self-Replay
Di Palo, N., & Johns, E. (2021) · arXiv · doi:10.48550/arxiv.2111.07447 · Cited by 8
“67 Behavioural Cloning methods [13, 4, 14, 5, 7, 6, 1, 15, 16] learn a policy network directly from 68 human demonstrations.”
Method
Metadata
- Authors
- Finn, C., Tan, X. Y., Duan, Y., Darrell, T., Levine, S., & Abbeel, P.
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 512-519
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2210483910
- Cited by
- 430
- References
- 33
- FWCI
- 30.13
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.