The DOI matches its Crossref record; 184 of the 193 references it cites were found in a registry or an index.
How to train your robot with deep reinforcement learning: lessons we have learned
Open accessAbstract
Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“POMDPs more accurately describe real-world applications of robotic systems [7] and are a major tool for training modern DRL and DIL end-to-end policy controllers.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, DRL is data-hungry and trained through trial and error which requires human involvement for constantly resetting the environment and preventing unsafe actions [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“(4) high-dimensional/continuous state settings exacerbate the challenge of exploration and further affect data efficiency.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We can also utilise demonstration data to further reduce the distribution shift, as it causes the model to learn the important part of the state space [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In recent years, data-driven approaches with deep learning (DL) [6], such as deep reinforcement learning (DRL) [7] and deep imitation learning (DIL) [8], have demonstrated many advantages in performing task-level and skill-level control problems (Section 3.3).”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“(3) DRL algorithms often come with a set of hyper-parameters, and the algorithms are often sensitive to these hyper-parameters and the application domain”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“(2) data efficiency (both sampling and learning efficiency) is a major concern for such complex settings”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“MBRL is generally more data-efficient than MFRL algorithms and it generalises better with large and diverse data [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“(1) theoretically, we have no guarantee of convergence of the DRL algorithms”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,043
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881
(2017) · Cited by 11,625
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Advances in Neural Information Processing Systems 14
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 9,187
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Self-citationSoft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
Self-citationTrust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Self-citationAddressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 2×
“This can include various dynamics parameters (Peng et al. 2018b; Tan et al. 2018) of the robot and the environment, as well as visual and rendering parameters such as textures and lighting (Sadeghi and Levine 2017; Tobin et al. 2017).”
BackgroundA Tutorial on the Cross-Entropy Method
de Boer, P.-T., Kroese, D. P., Mannor, S., & Rubinstein, R. Y. (2005) · Annals of Operations Research · doi:10.1007/s10479-005-5724-z · Cited by 3,110
2009 International Joint Conference on Artificial Intelligence
(2009) · doi:10.1109/jcai15434.2009 · Cited by 2,846
Raibert, M. H., & Tello, E. R. (1986) · IEEE Expert · doi:10.1109/mex.1986.4307016 · Cited by 2,751
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01316-z · Cited by 2,683
End-to-End Training of Deep Visuomotor PoliciesPreprint
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 2,641
Self-citationSelf-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393
Constrained Markov Decision Processes
Altman, E. (1999) · Routledge eBooks · doi:10.1201/9781315140223 · Cited by 2,265
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00982 · Cited by 2,194
Soft Actor-Critic Algorithms and Applications
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994
Self-citationLevine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
Self-citationA Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Learning from Simulated and Unsupervised Images through Adversarial Training
Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., & Webb, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.241 · Cited by 1,888
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,868
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.18 · Cited by 1,603
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989385 · Cited by 1,582
Self-citationLearning agile and dynamic motor skills for legged robots
Hwangbo, J., Lee, J., Dosovitskiy, A., Bellicoso, D., Tsounis, V., Koltun, V., & Hutter, M. (2019) · Science Robotics · doi:10.1126/scirobotics.aau5872 · Cited by 1,482
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460528 · Cited by 1,309 · cited 4×
“In fact, most of the prior work used simulation (Brockman et al. 2016; Coumans and Bai 2016) to evaluate and benchmark the learning algorithms (H ¨ am ¨ al ¨ ainen et al. 2015; Heess et al. 2017; Yu et al. 2018; Peng et al. 2018a).”
BackgroundMethodInfluentialExploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_12 · Cited by 1,224
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Science Robotics · doi:10.1126/scirobotics.abc5986 · Cited by 1,156
PILCO: A Model-Based and Data-Efficient Approach to Policy Search
Deisenroth, M. P., & Rasmussen, C. E. (2011) · Cambridge University Engineering Department Publications Database · Cited by 1,136
Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours
Pinto, L., & Gupta, A. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487517 · Cited by 1,099
Robotics: Science and Systems (RSS)
Trinkle, J., & Matsuoka, Y. (2010) · AI Magazine · doi:10.1609/aimag.v31i1.2237 · Cited by 1,082
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058
(2013) · The MIT Press eBooks · doi:10.7551/mitpress/9816.001.0001 · Cited by 1,021
Peng, X. B., Abbeel, P., Levine, S., & van de Panne, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201311 · Cited by 946
Self-citationGradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask NetworksPreprint
Chen, Z., Badrinarayanan, V., Lee, C., & Rabinovich, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02257 · Cited by 898
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889
Movement imitation with nonlinear dynamical systems in humanoid robots
Ijspeert, A. J., Nakanishi, J., & Stefan Schaal, A. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2002.1014739 · Cited by 862
Multi-Task Learning as Multi-Objective Optimization
Sener, O., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1810.04650 · Cited by 852
A Survey on Policy Search for Robotics
Deisenroth, M. P. (2011) · now publishers, Inc. eBooks · doi:10.1561/2300000021 · Cited by 845
Deep Exploration via Bootstrapped DQNPreprint
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 837
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831
Constrained Policy Optimization
Achiam, J., Held, D., Tamar, A., & Abbeel, P. (2017) · arXiv · doi:10.48550/arxiv.1705.10528 · Cited by 804
Reinforcement Learning with Deep Energy-Based PoliciesPreprint
Haarnoja, T., Tang, H., Abbeel, P., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08165 · Cited by 776
Self-citationContinuous Deep Q-Learning with Model-based AccelerationPreprint
Gu, S., Lillicrap, T., Sutskever, I., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00748 · Cited by 770
Self-citationLearning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000448343 · Cited by 752
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.010 · Cited by 735
Self-citation(2016) · neural information processing systems · Cited by 735
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · arXiv · doi:10.48550/arxiv.1804.10332 · Cited by 734
Self-citationOvercoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.10293 · Cited by 697
Self-citationKakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 691
Guided Cost Learning: Deep Inverse Optimal Control via Policy OptimizationPreprint
Finn, C., Levine, S., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00448 · Cited by 683
Self-citationClosing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach
Morrison, D., Leitner, J., & Corke, P. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.021 · Cited by 662
Emergence of Locomotion Behaviours in Rich Environments
Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Eslami, S. M. A., et al. (2017) · arXiv · doi:10.48550/arxiv.1707.02286 · Cited by 657
Grasp Pose Detection in Point Clouds
ten Pas, A., Gualtieri, M., Saenko, K., & Platt, R. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917735594 · Cited by 646
Off-Policy Deep Reinforcement Learning without ExplorationPreprint
Fujimoto, S., Meger, D., & Precup, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02900 · Cited by 646
Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping
Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K., et al. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460875 · Cited by 636
Self-citationDynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics
Schaal, S. (2006) · Springer eBooks · doi:10.1007/4-431-31381-8_23 · Cited by 620
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618
Self-citationTime-Contrastive Networks: Self-Supervised Learning from Video
Sermanet, P., Lynch, C., Chebotar, Y., Hsu, J., Jang, E., Schaal, S., Levine, S., & Brain, G. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462891 · Cited by 617
Self-citationLearning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., & Funkhouser, T. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593986 · Cited by 606
Deep visual foresight for planning robot motion
Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599
Self-citationPolicy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2004.1307456 · Cited by 586
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582
Learning to Adapt: Organisational Adaptation to Climate Change Impacts
Berkhout, F., Hertin, J., & Gann, D. (2006) · Climatic Change · doi:10.1007/s10584-006-9089-3 · Cited by 555
The limits and potentials of deep learning for robotics
Sünderhauf, N., Brock, O., Scheirer, W. J., Hadsell, R. T., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M. J., et al. (2018) · QUT ePrints (Queensland University of Technology) · Cited by 519
Policy Gradient Methods for Robotics
Peters, J., & Stefan Schaal, A. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.282564 · Cited by 511
Stabilizing Off-Policy Q-Learning via Bootstrapping Error ReductionPreprint
Kumar, A., Fu, J., Tucker, G., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00949 · Cited by 508
Self-citationRobust adversarial reinforcement learning
Pinto, L., Davidson, J., Sukthankar, R., & Gupta, A. (2017) · arXiv · doi:10.48550/arxiv.1703.02702 · Cited by 507 · cited 1×
“We use domain randomization, which samples different physical parameters, or add perturbation forces, either randomly or adversarially (Pinto et al. 2017), to the robot during training, to force it to learn to react under a wide variety of observations.”
MethodRelative Entropy Policy Search
Peters, J., Mülling, K., & Altün, Y. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7727 · Cited by 506
Asymmetric Actor Critic for Image-Based Robot Learning
Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., & Abbeel, P. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.008 · Cited by 506
Actor-Mimic: Deep Multitask and Transfer Reinforcement LearningPreprint
Parisotto, E., Ba, J. L., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06342 · Cited by 504
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1611.02779 · Cited by 503
Deep spatial autoencoders for visuomotor learning
Finn, C., Tan, X., Duan, Y., Darrell, T., Levine, S., & Abbeel, P. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487173 · Cited by 488
Self-citationLevine, S., & Koltun, V. (2013) · Cited by 481
Self-citationVisual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic ControlPreprint
Ebert, F., Finn, C., Dasari, S., Xie, A., Lee, A. X., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00568 · Cited by 480
Self-citationResidual Reinforcement Learning for Robot Control
Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J. A., Solowjow, E., & Levine, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794127 · Cited by 480
Self-citationLeveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
Vecerik, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Rothörl, T., Lampe, T., & Riedmiller, M. (2017) · arXiv · doi:10.48550/arxiv.1707.08817 · Cited by 478
One-Shot Imitation LearningPreprint
Duan, Y., Andrychowicz, M., Stadie, B. C., Ho, O. J., Schneider, J., Sutskever, I., Abbeel, P., & Zaremba, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07326 · Cited by 473
Learning to Walk Via Deep Reinforcement Learning
Haarnoja, T., Ha, S., Zhou, A., Tan, J., Tucker, G., & Levine, S. (2019) · Robotics: Science and Systems XV · doi:10.15607/rss.2019.xv.011 · Cited by 430
Self-citationJames, S., Wohlhart, P., Kalakrishnan, M., Kalashnikov, D., Irpan, A., Ibarz, J., Levine, S., Hadsell, R., & Bousmalis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01291 · Cited by 420
Self-citationModel-Ensemble Trust-Region Policy OptimizationPreprint
Kurutach, T., Clavera, I., Duan, Y., Tamar, A., & Abbeel, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10592 · Cited by 405
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context VariablesPreprint
Rakelly, K., Zhou, A., Quillen, D., Finn, C., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08254 · Cited by 403
Self-citationLearning Neural Network Policies with Guided Policy Search under Unknown Dynamics
Levine, S., & Abbeel, P. (2014) · Cited by 389
Self-citationVariational Intrinsic ControlPreprint
Gregor, K., Rezende, D. J., & Wierstra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07507 · Cited by 386
NATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE
Hilton, W. (1983) · Cited by 376
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning
Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv · doi:10.48550/arxiv.1803.11347 · Cited by 370
Self-citationDeepMPC: Learning Deep Latent Features for Model Predictive Control
Lenz, I., Knepper, R. A., & Saxena, A. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.012 · Cited by 349
Lee, M. A., Zhu, Y., Srinivasan, K., Shah, P., Savarese, S., Fei-Fei, L., Garg, A., & Bohg, J. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793485 · Cited by 349
Robot trajectory optimization using approximate inference
Toussaint, M. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553508 · Cited by 345
One-Shot Visual Imitation Learning via Meta-Learning
Finn, C., Yu, T., Zhang, T., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1709.04905 · Cited by 341
Self-citationLearning by Playing - Solving Sparse Reward Tasks from ScratchPreprint
Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Van de Wiele, T., Mnih, V., Heess, N., & Springenberg, J. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10567 · Cited by 334
Mahler, J., Matl, M., Liu, X., Li, A., Gealy, D. V., & Goldberg, K. Y. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460887 · Cited by 326
Overlapping experiment infrastructure
Tang, D., Agarwal, A., O'Brien, D., & Meyer, M. (2010) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1835804.1835810 · Cited by 314
Robot learning from demonstration by constructing skill trees
Konidaris, G., Kuindersma, S., Grupen, R. A., & Barto, A. G. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911428653 · Cited by 287
Learning contact-rich manipulation skills with guided policy search
Levine, S., Wagener, N., & Abbeel, P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7138994 · Cited by 282
Self-citationGradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
Zhao, C., Badrinarayanan, V., Lee, C., & Rabinovich, A. (2018) · International Conference on Machine Learning · Cited by 275
Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., & Hadsell, R. (2015) · arXiv · doi:10.48550/arxiv.1511.06295 · Cited by 270
Off-Policy Deep Reinforcement Learning without Exploration
Fujimoto, S. (2024) · TIB Data Manager · doi:10.57702/vkcnqiqb · Cited by 269
RoboNet: Large-Scale Multi-Robot Learning
Dasari, S., Ebert, F., Tian, S., Nair, S., Bucher, B., Schmeckpeper, K., Singh, S., Levine, S., & Finn, C. (2019) · Digital Eprints Services at ISB (DESI) (Indian School of Business) · doi:10.48550/arxiv.1910.11215 · Cited by 269
Self-citationActive Preference-Based Learning of Reward Functions
Sadigh, D., Dragan, A. D., Sastry, S., & Seshia, S. A. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.053 · Cited by 260
Deep Dynamics Models for Learning Dexterous ManipulationPreprint
Nagabandi, A., Konoglie, K., Levine, S., & Kumar, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11652 · Cited by 245
Self-citationPreparing for the unknown: Learning a universal policy with online system identification
Yu, W., Tan, J., Karen Liu, C., & Turk, G. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.048 · Cited by 237
Self-citationModel predictive control using neural networks
Draeger, A., Engell, S., & Ranke, H. (1995) · IEEE Control Systems · doi:10.1109/37.466261 · Cited by 236
Learning Navigation Behaviors End-to-End With AutoRL
Chiang, H.-T. L., Faust, A., Fišer, M., & Francis, A. G., Jr. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2899918 · Cited by 226
Learning symmetric and low-energy locomotion
Yu, W., Turk, G., & Liu, C. K. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201397 · Cited by 224
(2021) · Journal of Machine Learning Research · Cited by 224
Recovery RL: Safe Reinforcement Learning With Learned Recovery Zones
Thananjeyan, B., Balakrishna, A., Nair, S., Luo, M., Srinivasan, K., Hwang, M., Gonzalez, J. E., Ibarz, J., Finn, C., & Goldberg, K. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3070252 · Cited by 218
Self-citationAn Optimistic Perspective on Offline Reinforcement LearningPreprint
Agarwal, R., Schuurmans, D., & Norouzi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04543 · Cited by 217
Reinforcement Learning in Robotics: A Survey
Kober, J., & Peters, J. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-03194-1_2 · Cited by 213
Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
James, S., Davison, A. J., & Johns, E. (2017) · arXiv · doi:10.48550/arxiv.1707.02267 · Cited by 208
End-To-End Robotic Reinforcement Learning without Reward Engineering
Singh, A., Yang, L., Finn, C., & Levine, S. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.073 · Cited by 208
Self-citationComposable Deep Reinforcement Learning for Robotic Manipulation
Haarnoja, T., Pong, V. H., Zhou, A., Dalal, M., Abbeel, P., & Levine, S. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460756 · Cited by 193
Self-citationDeep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards
Schoettler, G., Nair, A., Luo, J., Bahl, S., Ojea, J. A., Solowjow, E., & Levine, S. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341714 · Cited by 191
Self-citationLearning to be Safe: Deep RL with a Safety CriticPreprint
Srinivasan, K., Eysenbach, B., Ha, S., Tan, J., & Finn, C. (2020) · arXiv.org · doi:10.48550/arxiv.2010.14603 · Cited by 185
Self-citationDexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost
Zhu, H., Gupta, A., Rajeswaran, A., Levine, S., & Kumar, V. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794102 · Cited by 184
Self-citationDense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic ManipulationPreprint
Florence, P., Manuelli, L., & Tedrake, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08756 · Cited by 181
Learning a visuomotor controller for real world robotic grasping using simulated depth imagesPreprint
Viereck, U., ten Pas, A., Saenko, K., & Platt, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04652 · Cited by 178
RL-CycleGAN: Reinforcement Learning Aware Simulation-to-Real
Rao, K., Harris, C., Irpan, A., Levine, S., Ibarz, J., & Khansari, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01117 · Cited by 178
Self-citationLong-Range Indoor Navigation With PRM-RL
Francis, A. G., Jr., Faust, A., Chiang, H.-T. L., Hsu, J., Kew, J. C., Fišer, M., & Lee, T.-W. E. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.2975428 · Cited by 172
Simple random search of static linear policies is competitive for reinforcement learning
Mania, H., Guy, A., & Recht, B. (2018) · neural information processing systems · Cited by 163
Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement LearningPreprint
Peng, X. B., Kumar, A., Zhang, G., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00177 · Cited by 158
Self-citationCartman: The Low-Cost Cartesian Manipulator that Won the Amazon Robotics Challenge
Morrison, D., Tow, A. W., McTaggart, M., Smith, R. S., Kelly-Boxall, N., Wade-McCue, S., Erskine, J., Grinover, R., Gurman, A., Hunn, T., et al. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463191 · Cited by 145
Path integral guided policy search
Chebotar, Y., Kalakrishnan, M., Yahya, A., Li, A., Schaal, S., & Levine, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989384 · Cited by 131
Self-citationCombining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement LearningPreprint
Chebotar, Y., Hausman, K., Zhang, M., Sukhatme, G. S., Schaal, S., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.03078 · Cited by 131
Self-citationDeep predictive policy training using reinforcement learning
Ghadirzadeh, A., Maki, A., Kragic, D., & Bjorkman, M. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206046 · Cited by 113
Silver, T., Allen, K., Tenenbaum, J., & Kaelbling, L. (2018) · arXiv · doi:10.48550/arxiv.1812.06298 · Cited by 113
Data Efficient Reinforcement Learning for Legged RobotsPreprint
Yang, Y., Caluwaerts, K., Işçen, A., Zhang, T., Tan, J., & Sindhwani, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03613 · Cited by 113
Self-citationSelf-Supervised Correspondence in Visuomotor Policy Learning
Florence, P., Manuelli, L., & Tedrake, R. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2956365 · Cited by 111
Policies Modulating Trajectory GeneratorsPreprint
Işçen, A., Caluwaerts, K., Tan, J., Zhang, T., Coumans, E., Sindhwani, V., & Vanhoucke, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02812 · Cited by 96
Self-citationEX2: Exploration with Exemplar Models for Deep Reinforcement Learning
Fu, J., Co-Reyes, J. D., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01260 · Cited by 95
Self-citationBıyık, E., Losey, D. P., Palan, M., Landolfi, N. C., Shevchuk, G., & Sadigh, D. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211041652 · Cited by 95
Learning Fast Adaptation With Meta Strategy Optimization
Yu, W., Tan, J., Bai, Y., Coumans, E., & Ha, S. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974685 · Cited by 92
Self-citationOnline control of simulated humanoids using particle belief propagation
Hämäläinen, P., Rajamäki, J., & Liu, C. K. (2015) · ACM Transactions on Graphics · doi:10.1145/2767002 · Cited by 90
On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference
Rawlik, K., Toussaint, M., & Vijayakumar, S. (2012) · Robotics: Science and Systems · doi:10.15607/rss.2012.viii.045 · Cited by 90
Learning to dress: Synthesizing human dressing motion via deep reinforcement learning
Clegg, A., Yu, W., Tan, J., Liu, C. K., & Turk, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275048 · Cited by 89
Self-citationImitation Learning from Imperfect DemonstrationPreprint
Wu, Y.-H., Charoenphakdee, N., Bao, H., Tangkaratt, V., & Sugiyama, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09387 · Cited by 89
Task-Embedded Control Networks for Few-Shot Imitation LearningPreprint
James, S., Bloesch, M., & Davison, A. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03237 · Cited by 88
Improvisation through Physical Understanding: Using Novel Objects As Tools with Visual Foresight
Xie, A., Ebert, F., Levine, S., & Finn, C. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.001 · Cited by 88
Self-citationConference on artificial intelligence
(1987) · Computers in Biology and Medicine · doi:10.1016/0010-4825(87)90050-3 · Cited by 88
Collaborative Evolutionary Reinforcement LearningPreprint
Khadka, S., Majumdar, S., Nassar, T., Dwiel, Z., Tumer, E. C., Miret, S., Liu, Y., & Tumer, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00976 · Cited by 84
Guided Policy Search via Approximate Mirror DescentPreprint
Montgomery, W. H., & Levine, S. (2016) · arXiv (Cornell University) · Cited by 82
Self-citationAn Optimistic Perspective on Offline Deep Reinforcement Learning
Agarwal, R., Schuurmans, D., & Norouzi, M. (2020) · International Conference on Machine Learning · Cited by 82
Tan, J., Gu, Y., Liu, C. K., & Turk, G. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601121 · Cited by 76
Self-citationLeave no Trace: Learning to Reset for Safe and Autonomous Reinforcement LearningPreprint
Eysenbach, B., Gu, S., Ibarz, J., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06782 · Cited by 76
Self-citationLearning to Walk via Deep Reinforcement Learning
Haarnoja, T., Ha, S., Zhou, A., Tan, J., Tucker, G., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1812.11103 · Cited by 53
Self-citationLearning Deep Visuomotor Policies for Dexterous Hand Manipulation
Jain, D., Li, A. R., Singhal, S., Rajeswaran, A., Kumar, V., & Todorov, E. V. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794033 · Cited by 49
Learning to sequence movement primitives from demonstrations
Manschitz, S., Kober, J., Gienger, M., & Peters, J. (2014) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2014.6943187 · Cited by 47
On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference
Rawlik, K., Toussaint, M., & Vijayakumar, S. (2013) · The MIT Press eBooks · doi:10.7551/mitpress/9816.003.0050 · Cited by 45
Unsupervised Curricula for Visual Meta-Reinforcement LearningPreprint
Jabri, A., Hsu, K., Eysenbach, B., Gupta, A., Levine, S., & Finn, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.04226 · Cited by 43
Self-citationAutomated Deep Reinforcement Learning Environment for Hardware of a Modular Legged Robot
Ha, S., Kim, J., & Yamane, K. (2018) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/urai.2018.8442201 · Cited by 42
Few-Shot Goal Inference for Visuomotor Learning and PlanningPreprint
Xie, A., Singh, A., Levine, S., & Finn, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00482 · Cited by 42
Self-citationSE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Control
Byravan, A., Leeb, F., Meier, F., & Fox, D. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461184 · Cited by 40
Learning sequential motor tasks
Daniel, C., Neumann, G. M., Kroemer, O., & Peters, J. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630937 · Cited by 38
The limits and potentials of deep learning for robotics
Sünderhauf, N., Brock, O., Scheirer, W., Hadsell, R., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M., et al. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364918770733 · Cited by 38
Learning Image-Conditioned Dynamics Models for Control of Underactuated Legged Millirobots
Nagabandi, A., Yang, G., Asmar, T., Pandya, R., Kahn, G., Levine, S., & Fearing, R. S. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594193 · Cited by 34
Self-citationRay Interference: a Source of Plateaus in Deep Reinforcement LearningPreprint
Schaul, T., Borsa, D., Modayil, J., & Pascanu, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.11455 · Cited by 33
Realtime State Estimation with Tactile and Visual Sensing. Application to Planar Manipulation
Yu, K., & Rodríguez, A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463183 · Cited by 33
End-to-End Robotic Reinforcement Learning without Reward EngineeringPreprint
Singh, A., Yang, L., Hartikainen, K., Finn, C., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.07854 · Cited by 31
Self-citationValue constrained model-free continuous controlPreprint
Bohez, S., Abdolmaleki, A., Neunert, M., Buchli, J., Heess, N., & Hadsell, R. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04623 · Cited by 29
NoRML: No-Reward Meta Learning
Yang, Y., Caluwaerts, K., Işçen, A., Tan, J., & Finn, C. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/elvr9404 · Cited by 28
Self-citationMontgomery, W. H., Ajay, A., Finn, C., Abbeel, P., & Levine, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989383 · Cited by 26
Self-citationThinking While Moving: Deep Reinforcement Learning with Concurrent Control
Xiao, T., Jang, E., Kalashnikov, D., Levine, S., Ibarz, J., Hausman, K., & Herzog, A. (2020) · arXiv · doi:10.48550/arxiv.2004.06089 · Cited by 25
Self-citationExperience-Embedded Visual ForesightPreprint
Lin, Y.-C., Bauzá, M., & Isola, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.05071 · Cited by 22
Recovery RL: Safe Reinforcement Learning with Learned Recovery ZonesPreprint
Thananjeyan, B., Balakrishna, A., Nair, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15920 · Cited by 20
Simultaneously Learning Vision and Feature-Based Control Policies for Real-World Ball-In-A-Cup
Schwab, D., Springenberg, J. T., Martins, M. F., Neunert, M., Lampe, T., Abdolmaleki, A., Hertweck, T., Hafner, R., Nori, F., & Riedmiller, M. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.027 · Cited by 18
Modular Hopping and Running via Parallel Composition
De, A. (2017) · ScholarlyCommons (University of Pennsylvania) · Cited by 14
Learning Fast Adaptation with Meta Strategy OptimizationPreprint
Yu, W., Tan, J., Bai, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12995 · Cited by 12
Self-citationA Framework for Data-Driven RoboticsPreprint
Cabi, S., Colmenarejo, S. G., Novikov, A., Konyushkova, K., Reed, S., Jeong, R., Żołna, K., Aytar, Y., Budden, D. M., Vecerík, M., et al. (2019) · arXiv (Cornell University) · Cited by 10
Regrasping using Tactile Perception and Supervised Policy Learning
Hausman, K., Chebotar, Y., Kroemer, O., Sukhatme, G. S., & Stefan Schaal, A. (2017) · National Conference on Artificial Intelligence · Cited by 6
Research works printed but not linked: 61
- Achiam J, 2017, International Conference on Machine Learning
- Agarwal R, 2020, International Conference on Machine Learning
- Andrychowicz M, 2017, Advances in Neural Information Processing Systems, 5048
- Burda Y, 2019, International Conference on Learning Representations
- Cabi S, 2019, arXiv preprint arXiv:1909.12200 · arXiv
- Chebotar Y, 2017, International Conference on Machine Learning, 703
- Chen Z, 2018, International Conference on Machine Learning
- Clavera I, 2019, International Conference on Learning Representations
- Deisenroth M, 2011, International Conference on Machine Learning, 465
- Deisenroth MP, 2013, Foundations and Trends in Robotics, 2, 1
- Devlin J, 2019, NAACL-HLT
- Duan Y, 2017, Advances in Neural Information Processing Systems, 1087
- Eysenbach B, 2018, International Conference on Learning Representations
- Finn C, 2017, International Conference on Machine Learning
- Finn C, 2016, International Conference on Machine Learning, 49
- Finn C, 2017, Proceedings of Machine Learning Research, 78, 357
- Fox R, 2016, Conference on Uncertainty in Artificial Intelligence
- Fu J, 2017, Advances in Neural Information Processing Systems, 2577
- Fujimoto S, 2019, International Conference on Machine Learning
- Fujimoto S, 2018, International Conference on Machine Learning
- Garofolo JS, 1993, NASA STI/Recon Technical Report, 93
- Goodfellow I, 2014, Advances in Neural Information Processing Systems, 2672
- Gregor K, 2017, International Conference on Learning Representations, Workshop Track Proceedings
- Gu S, 2016, International Conference on Machine Learning, 2829
- Haarnoja T, 2017, International Conference on Machine Learning, 1352
- Haarnoja T, 2018, International Conference on Machine Learning
- Hausman K, 2017, AAAI Symposium on Interactive Multi-Sensory Object Perception for Embodied Agents
- Iscen A, 2018, Conference on Robot Learning
- Jabri A, 2019, Advances in Neural Information Processing Systems, 10519
- James S, 2018, Conference on Robot Learning
- James S, 2017, Conference on Robot Learning
- Kalashnikov D, 2018, Conference on Robot Learning, Proceedings of Machine Learning Research
- Khadka S, 2019, International Conference on Machine Learning
- Kroemer O, 2019, CoRR abs/1907.03146 · arXiv
- Kumar A, 2019, Advances in Neural Information Processing Systems, 32
- Kurutach T, 2018, International Conference on Learning Representations
- Levine S, 2014, Advances in Neural Information Processing Systems, 1071
- Levine S, 2016, The Journal of Machine Learning Research, 17, 1334
- Levine S, 2013, International Conference on Machine Learning
- Mania H, 2018, Advances in Neural Information Processing Systems
- Montgomery WH, 2016, Advances in Neural Information Processing Systems, 4008
- Nagabandi A, 2020, Conference on Robot Learning
- Osband I, 2016, Advances in Neural Information Processing Systems, 4026
- Parisotto E, 2016, CoRR
- Paszke A, 2017, Advances in Neural Information Processing Systems Workshop on Autodiff
- Pinto L, 2017, International Conference on Machine Learning
- Rakelly K, 2019, International Conference on Machine Learning
- Riedmiller M, 2018, International Conference on Machine Learning
- Schaul T, 2019, Multidisciplinary Conference on Reinforcement Learning and Decision Making
- Schoettler G, 2019, International Conference on Intelligent Robots and Systems
- Schulman J, 2015, International Conference on Machine Learning
- Sener O, 2018, Advances in Neural Information Processing Systems, 527
- Tedrake R, 2015, Workshop on Adaptive and Learning Systems
- Viereck U, 2017, Conference on Robot Learning
- Wu YH, 2019, International Conference on Machine Learning, Proceedings of Machine Learning Research
- Xiao T, 2020, International Conference on Learning Representations
- Xie A, 2019, arXiv preprint arXiv:1904.05538 · arXiv
- Xie A, 2018, Proceedings of Machine Learning Research, 87, 40
- Yang Y, 2019, AAMAS
- Yang Y, 2020, Conference on Robot Learning
- Yen-Chen L, 2020, Conference on Robot Learning
Cited by
Sorted by citations
Manakitsa, N., Maraslidis, G. S., Moysis, L., & Fragulis, G. F. (2024) · Technologies · doi:10.3390/technologies12020015 · Cited by 355
Explainable Reinforcement Learning: A Survey and Comparative Review
Milani, S., Topin, N., Veloso, M., & Fang, F. (2023) · ACM Computing Surveys · doi:10.1145/3616864 · Cited by 347
Robot learning towards smart robotic manufacturing: A review
Liu, Z., Liu, Q., Xu, W., Wang, L., & Zhou, Z. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102360 · Cited by 251
Recent advancements of robotics in construction
Xiao, B., Chen, C., & Yin, X. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104591 · Cited by 204
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
Tang, C., Abbatematteo, B., Hu, J., Chandra, R., Martín-Martín, R., & Stone, P. (2024) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-030323-022510 · Cited by 276
Metadata
- Authors
- Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S.
- Affiliations
- Google (United States) (US)
- Everyday Robots, X, The Moonshot Factory, Mountain View, CA, USA
- Corresponding author
- Ibarz, Julian
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2021-01-31
- Volume
- 40
- Issue
- 4-5
- Pages
- 698-721
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W3126321819
- Cited by
- 570
- References
- 193
- Citations by year
- 2026: 63 · 2025: 123 · 2024: 145 · 2023: 112 · 2022: 89 · 2021: 38
- FWCI
- 33.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- The International Journal of Robotics Research (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, robot learning, embodied agents, physical robots, robot skill learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
