The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Asymmetric Actor Critic for Image-Based Robot Learning
Open accessAbstract
Deep reinforcement learning (RL) has proven a powerful technique in many sequential decision making domains.However, robotics poses many challenges for RL, most notably training on a physical system can be expensive and dangerous, which has sparked significant interest in learning control policies using a physics simulator.While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator.In this work, we propose the Asymmetric Actor Critic, which learns a vision-based control policy while taking advantage of access to the underlying state to significantly speed up training.Concretely, our algorithm employs an actor-critic training algorithm in which the critic is trained on full states while the actor (or policy) is trained on images.We show that using these asymmetric inputs improves performance on a range of simulated tasks.Finally, we combine this method with domain randomization and show real robot experiments for several tasks like picking, pushing, and moving a block.We achieve this simulation to real-world transfer without training on any real-world data.Videos of these experiments can be found in www.goo.gl/b57WTs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Self-citationMulti-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017) · arXiv · doi:10.48550/arxiv.1706.02275 · Cited by 3,377
Self-citationDeterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
A survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 6×
“Another option is to train entirely on rendered partial observations (camera images) of the robot [11], [17].”
BackgroundMethodResultInfluentialSelf-citationLevine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Counterfactual Multi-Agent Policy Gradients
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11794 · Cited by 1,873
Acceleration of Stochastic Approximation by Averaging
Polyak, B. T., & Juditsky, A. B. (1992) · SIAM Journal on Control and Optimization · doi:10.1137/0330046 · Cited by 1,827
Playing for Data: Ground Truth from Computer Games
Richter, S. R., Vineet, V., Roth, S., & Koltun, V. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_7 · Cited by 1,815
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355
Self-citationEnd-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Self-citation(2015) · Journal of Machine Learning Research · Cited by 1,119
Robotic grasping and contact: a review
Bicchi, A., & Kumar, V. (2002) · Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065) · doi:10.1109/robot.2000.844081 · Cited by 1,069
Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours
Pinto, L., & Gupta, A. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487517 · Cited by 1,047
Self-citationUniversal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 701
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618
Robust adversarial reinforcement learning
Pinto, L., Davidson, J., Sukthankar, R., & Gupta, A. (2017) · arXiv · doi:10.48550/arxiv.1703.02702 · Cited by 507
Self-citationSim-to-Real Robot Learning from Pixels with Progressive Nets
Rusu, A. A., Vecerik, M., Rothörl, T., Heess, N., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1610.04286 · Cited by 385
EPOpt: Learning Robust Neural Network Policies Using Model EnsemblesPreprint
Rajeswaran, A., Ghotra, S., Ravindran, B., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.01283 · Cited by 277
Off-Policy Temporal Difference Learning with Function Approximation
Precup, D., Sutton, R. S., & Dasgupta, S. (2001) · Cited by 243
Preparing for the unknown: Learning a universal policy with online system identification
Yu, W., Tan, J., Karen Liu, C., & Turk, G. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.048 · Cited by 237
Preparing for the Unknown: Learning a Universal Policy with Online System Identification
Yu, W., Tan, J., Liu, C. K., & Turk, G. (2017) · arXiv · doi:10.48550/arxiv.1702.02453 · Cited by 237
Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
James, S., Davison, A. J., & Johns, E. (2017) · arXiv · doi:10.48550/arxiv.1707.02267 · Cited by 208
A Framework for Push-Grasping in Clutter
Dogar, M., & Srinivasa, S. (2011) · Robotics: Science and Systems VII · doi:10.15607/rss.2011.vii.009 · Cited by 201
Oh, K.-J., Yea, S., & Ho, Y.-S. (2009) · 2009 Picture Coding Symposium · doi:10.1109/pcs.2009.5167450 · Cited by 183
Learning a visuomotor controller for real world robotic grasping using simulated depth imagesPreprint
Viereck, U., ten Pas, A., Saenko, K., & Platt, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04652 · Cited by 178
The Curious Robot: Learning Visual Representations via Physical Interactions
Pinto, L., Gandhi, D., Han, Y., Park, Y., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_1 · Cited by 162
Self-citationTransfer from Simulation to Real World through Learning Deep Inverse Dynamics ModelPreprint
Christiano, P. F., Shah, Z., Mordatch, I., Schneider, J., Blackwell, T., Tobin, J., Abbeel, P., & Zaremba, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03518 · Cited by 159
Self-citationLearning to push by grasping: Using multiple tasks for effective learning
Pinto, L., & Gupta, A. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989249 · Cited by 117
Self-citation3D Simulation for Robot Arm Control with Deep Q-LearningPreprint
James, S., & Johns, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.03759 · Cited by 86
Efficient reinforcement learning for robots using informative simulated priors
Cutler, M., & How, J. P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139550 · Cited by 68
Supervision via competition: Robot adversaries for learning tasks
Pinto, L., Davidson, J., & Gupta, A. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989190 · Cited by 68
Self-citationDepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition
Planche, B., Wu, Z., Ma, K., Sun, S., Kluckner, S., Lehmann, O., Chen, T., Hutter, A., Zakharov, S., Kosch, H., et al. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00011 · Cited by 67
Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor PoliciesPreprint
Zhang, F., Leitner, J., Milford, M. J., & Corke, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.06781 · Cited by 47
Improved Learning of Dynamics Models for Control
Venkatraman, A., Capobianco, R., Pinto, L., Hebert, M., Nardi, D., & Bagnell, J. A. (2017) · Springer proceedings in advanced robotics · doi:10.1007/978-3-319-50115-4_61 · Cited by 35
Self-citationReducing uncertainty of objects by robot pushing
Balorda, Z. (2002) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.1990.126132 · Cited by 34
Learning omnidirectional path following using dimensionality reduction
Kolter, J. Z., & Ng, A. Y. (2007) · Robotics: Science and Systems · doi:10.15607/rss.2007.iii.033 · Cited by 31
Deep Predictive Policy Training using Reinforcement LearningPreprint
Ghadirzadeh, A., Maki, A., Kragić, D., & Björkman, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00727 · Cited by 21
Probabilistically safe policy transfer
Held, D., McCarthy, Z., Zhang, M., Shentu, F., & Abbeel, P. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989680 · Cited by 18
Self-citation
Cited by
Solving Rubik's Cube with a Robot Hand
OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.07113 · Cited by 602
Self-citationHow to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570
Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
Makoviychuk, V., Wawrzyniak, L., Guo, Y., Lu, M., Storey, K., Macklin, M., Hoeller, D., Rudin, N., Allshire, A., Handa, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.10470 · Cited by 509
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., & Fox, D. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793789 · Cited by 457
Emergent Tool Use From Multi-Agent Autocurricula
Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I. (2019) · arXiv · doi:10.48550/arxiv.1909.07528 · Cited by 424
“[59] Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, and Pieter Abbeel.”
Background
Metadata
- Authors
- Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., & Abbeel, P.
- Venue
- Robotics: Science and Systems XIV
- Published
- 2018-06-26
- OpenAlex
- W2766614170
- Cited by
- 343
- References
- 48
- FWCI
- 14.46
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.