How to train your robot with deep reinforcement learning: Lessons we have learned
Open accessAbstract
Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
POMDPs more accurately describe real-world applications of robotic systems and are a major tool for training modern DRL and DIL end-to-end policy controllers.
“POMDPs more accurately describe real-world applications of robotic systems [7] and are a major tool for training modern DRL and DIL end-to-end policy controllers.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Section 4.8 (p. 13) says real robotic systems have latency and unobservable intervening states that violate an assumption of the MDP formulation. This supports the cited clause’s concern about modeling real-world robots, but the paper does not say that POMDPs describe them more accurately. The citation is attached to that clause, not to the subsequent claim about training DRL and DIL controllers. (LLM, conf=0.78)
Experimental — accuracy not yet measuredDeep reinforcement learning is data-hungry and trained through trial and error, requiring human involvement for constantly resetting the environment and preventing unsafe actions.
“First, DRL is data-hungry and trained through trial and error which requires human involvement for constantly resetting the environment and preventing unsafe actions [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Section 4.7 (p. 13) says deep RL requires substantial data and that, in most robot-learning experiments to date, a human operator has reset the scene and stopped unsafe operation. The claim overstates this as a constant requirement: the same section says robots need to run without human supervision to scale data collection. (LLM, conf=0.88)
Experimental — accuracy not yet measuredHigh-dimensional or continuous state settings exacerbate the challenge of exploration and further affect data efficiency.
“(4) high-dimensional/continuous state settings exacerbate the challenge of exploration and further affect data efficiency.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Section 4.2 (p. 8) says high-dimensional observations increase training and sample complexity, supporting the data-efficiency concern. Section 4.4 (p. 10) says exploration is particularly difficult with sparse rewards, but does not say that high-dimensional or continuous states exacerbate exploration or that this is what reduces data efficiency. (LLM, conf=0.84)
Experimental — accuracy not yet measuredUtilising demonstration data can further reduce distribution shift because it causes the model to learn the important part of the state space.
“We can also utilise demonstration data to further reduce the distribution shift, as it causes the model to learn the important part of the state space [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Section 4.4.4 (p. 11) says that adding demonstrations to model-based RL training can help a learned model capture correct dynamics in important parts of the state space. It does not say that this reduces distribution shift, and its explanation is specific to model-based RL rather than models generally. (LLM, conf=0.85)
Experimental — accuracy not yet measuredData-driven approaches with deep learning, such as deep reinforcement learning and deep imitation learning, have demonstrated advantages in performing task-level and skill-level control problems.
“In recent years, data-driven approaches with deep learning (DL) [6], such as deep reinforcement learning (DRL) [7] and deep imitation learning (DIL) [8], have demonstrated many advantages in performing task-level and skill-level control problems (Section 3.3).”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Deep reinforcement learning algorithms are often sensitive to hyper-parameters and the application domain.
“(3) DRL algorithms often come with a set of hyper-parameters, and the algorithms are often sensitive to these hyper-parameters and the application domain”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Data efficiency, both sampling and learning efficiency, is a major concern for complex deep reinforcement learning settings.
“(2) data efficiency (both sampling and learning efficiency) is a major concern for such complex settings”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Model-based reinforcement learning is generally more data-efficient than model-free reinforcement learning and generalises better with large and diverse data.
“MBRL is generally more data-efficient than MFRL algorithms and it generalises better with large and diverse data [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
There is no theoretical guarantee of convergence for deep reinforcement learning algorithms.
“(1) theoretically, we have no guarantee of convergence of the DRL algorithms”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Section 4.1 (p. 7) describes unstable learning, sensitivity to hyperparameters, and different outcomes across runs. It does not address whether deep reinforcement learning algorithms have theoretical convergence guarantees. Searches for convergence and theoretical guarantees found no passage establishing the claim. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,100 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
(2017) · Cited by 11,625 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · see citing works
Advances in Neural Information Processing Systems 14
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 9,184 · see citing works
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512 · see citing works
Self-citationSoft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · see citing works
Self-citationTrust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578 · see citing works
Self-citationEnd-to-End Training of Deep Visuomotor PoliciesPreprint
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2015) · Journal of machine learning research · doi:10.48550/arxiv.1504.00702 · Cited by 4,011 · see citing works
Self-citationAddressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650 · see citing works
A Tutorial on the Cross-Entropy Method
Boer, P., Kroese, D. P., Mannor, S., & Rubinstein, R. (2005) · Annals of Operations Research · doi:10.1007/s10479-005-5724-z · Cited by 3,348 · see citing works
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198 · see citing works
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · see citing works
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 2× · see citing works
“This can include various dynamics parameters (Peng et al. 2018b; Tan et al. 2018) of the robot and the environment, as well as visual and rendering parameters such as textures and lighting (Sadeghi and Levine 2017; Tobin et al. 2017).”
Background2009 International Joint Conference on Artificial Intelligence
(2009) · doi:10.1109/jcai15434.2009 · Cited by 2,846 · see citing works
Raibert, M. H., & Tello, E. R. (1986) · IEEE Expert · doi:10.1109/mex.1986.4307016 · Cited by 2,752 · see citing works
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01316-z · Cited by 2,460 · see citing works
Self-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393 · see citing works
Constrained Markov Decision Processes
Altman, E. (1999) · Routledge eBooks · doi:10.1201/9781315140223 · Cited by 2,265 · see citing works
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00982 · Cited by 2,194 · see citing works
End-to-end training of deep visuomotor policies
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016) · Journal of Machine Learning Research · Cited by 2,192 · see citing works
Self-citationSoft Actor-Critic Algorithms and Applications
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994 · see citing works
Self-citationLevine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971 · see citing works
Self-citationA Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941 · see citing works
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,868 · see citing works
Learning from Simulated and Unsupervised Images through Adversarial Training
Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., & Webb, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.241 · Cited by 1,831 · see citing works
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · see citing works
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.18 · Cited by 1,603 · see citing works
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989385 · Cited by 1,485 · see citing works
Self-citationLearning agile and dynamic motor skills for legged robots
Hwangbo, J., Lee, J., Dosovitskiy, A., Bellicoso, D., Tsounis, V., Koltun, V., & Hutter, M. (2019) · Science Robotics · doi:10.1126/scirobotics.aau5872 · Cited by 1,482 · see citing works
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355 · see citing works
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460528 · Cited by 1,309 · cited 4× · see citing works
“In fact, most of the prior work used simulation (Brockman et al. 2016; Coumans and Bai 2016) to evaluate and benchmark the learning algorithms (H ¨ am ¨ al ¨ ainen et al. 2015; Heess et al. 2017; Yu et al. 2018; Peng et al. 2018a).”
BackgroundMethodInfluentialExploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_12 · Cited by 1,224 · see citing works
Learning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Science Robotics · doi:10.1126/scirobotics.abc5986 · Cited by 1,156 · see citing works
PILCO: A Model-Based and Data-Efficient Approach to Policy Search
Deisenroth, M. P., & Rasmussen, C. E. (2011) · Cambridge University Engineering Department Publications Database · Cited by 1,136 · see citing works
Robotics: Science and Systems (RSS)
Trinkle, J., & Matsuoka, Y. (2010) · AI Magazine · doi:10.1609/aimag.v31i1.2237 · Cited by 1,082 · see citing works
Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours
Pinto, L., & Gupta, A. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487517 · Cited by 1,060 · see citing works
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · see citing works
(2013) · The MIT Press eBooks · doi:10.7551/mitpress/9816.001.0001 · Cited by 1,019 · see citing works
Peng, X. B., Abbeel, P., Levine, S., & van de Panne, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201311 · Cited by 946 · see citing works
Self-citationDeep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889 · see citing works
Movement imitation with nonlinear dynamical systems in humanoid robots
Ijspeert, A. J., Nakanishi, J., & Stefan Schaal, A. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2002.1014739 · Cited by 862 · see citing works
Multi-Task Learning as Multi-Objective Optimization
Sener, O., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1810.04650 · Cited by 852 · see citing works
Deep Exploration via Bootstrapped DQNPreprint
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 837 · see citing works
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831 · see citing works
A Survey on Policy Search for Robotics
Deisenroth, M. P. (2011) · Foundations and Trends in Robotics · doi:10.1561/2300000021 · Cited by 815 · see citing works
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask NetworksPreprint
Chen, Z., Badrinarayanan, V., Lee, C., & Rabinovich, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02257 · Cited by 797 · see citing works
Continuous Deep Q-Learning with Model-based AccelerationPreprint
Gu, S., Lillicrap, T., Sutskever, I., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00748 · Cited by 770 · see citing works
Self-citationLearning quadrupedal locomotion over challenging terrain
Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., & Hutter, M. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000448343 · Cited by 752 · see citing works
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.010 · Cited by 735 · see citing works
Self-citation(2016) · neural information processing systems · Cited by 735 · see citing works
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · arXiv · doi:10.48550/arxiv.1804.10332 · Cited by 734 · see citing works
Self-citationOvercoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712 · see citing works
Constrained Policy OptimizationPreprint
Achiam, J., Held, D., Tamar, A., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10528 · Cited by 706 · see citing works
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.10293 · Cited by 697 · see citing works
Self-citationKakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 691 · see citing works
Policy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 676 · see citing works
Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach
Morrison, D., Leitner, J., & Corke, P. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.021 · Cited by 662 · see citing works
Emergence of Locomotion Behaviours in Rich Environments
Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Eslami, S. M. A., et al. (2017) · arXiv · doi:10.48550/arxiv.1707.02286 · Cited by 657 · see citing works
Reinforcement Learning with Deep Energy-Based PoliciesPreprint
Haarnoja, T., Tang, H., Abbeel, P., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08165 · Cited by 638 · see citing works
Self-citationUsing Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping
Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K., et al. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460875 · Cited by 636 · see citing works
Self-citationDynamic Movement Primitives -A Framework for Motor Control in Humans and Humanoid Robotics
Schaal, S. (2006) · Springer eBooks · doi:10.1007/4-431-31381-8_23 · Cited by 620 · see citing works
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618 · see citing works
Self-citationGrasp Pose Detection in Point Clouds
ten Pas, A., Gualtieri, M., Saenko, K., & Platt, R. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917735594 · Cited by 610 · see citing works
Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., & Funkhouser, T. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593986 · Cited by 606 · see citing works
Deep visual foresight for planning robot motion
Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599 · see citing works
Self-citationGuided Cost Learning: Deep Inverse Optimal Control via Policy OptimizationPreprint
Finn, C., Levine, S., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00448 · Cited by 589 · see citing works
Self-citationBrockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582 · see citing works
Time-Contrastive Networks: Self-Supervised Learning from Video
Sermanet, P., Lynch, C., Chebotar, Y., Hsu, J., Jang, E., Stefan Schaal, A., Levine, S., & Brain, G. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462891 · Cited by 579 · see citing works
Self-citationLearning to Adapt: Organisational Adaptation to Climate Change Impacts
Berkhout, F., Hertin, J., & Gann, D. (2006) · Climatic Change · doi:10.1007/s10584-006-9089-3 · Cited by 555 · see citing works
Off-Policy Deep Reinforcement Learning without ExplorationPreprint
Fujimoto, S., Meger, D., & Precup, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02900 · Cited by 533 · see citing works
The limits and potentials of deep learning for robotics
Sünderhauf, N., Brock, O., Scheirer, W. J., Hadsell, R. T., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M. J., et al. (2018) · QUT ePrints (Queensland University of Technology) · Cited by 519 · see citing works
Policy Gradient Methods for Robotics
Peters, J., & Stefan Schaal, A. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.282564 · Cited by 511 · see citing works
Robust adversarial reinforcement learning
Pinto, L., Davidson, J., Sukthankar, R., & Gupta, A. (2017) · arXiv · doi:10.48550/arxiv.1703.02702 · Cited by 507 · cited 1× · see citing works
“We use domain randomization, which samples different physical parameters, or add perturbation forces, either randomly or adversarially (Pinto et al. 2017), to the robot during training, to force it to learn to react under a wide variety of observations.”
MethodRL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1611.02779 · Cited by 503 · see citing works
Levine, S., & Koltun, V. (2013) · Cited by 481 · see citing works
Self-citationResidual Reinforcement Learning for Robot Control
Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J. A., Solowjow, E., & Levine, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794127 · Cited by 480 · see citing works
Self-citationLeveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
Vecerik, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Rothörl, T., Lampe, T., & Riedmiller, M. (2017) · arXiv · doi:10.48550/arxiv.1707.08817 · Cited by 478 · see citing works
Relative Entropy Policy Search
Peters, J., Mülling, K., & Altün, Y. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7727 · Cited by 467 · see citing works
One-Shot Imitation LearningPreprint
Duan, Y., Andrychowicz, M., Stadie, B. C., Ho, O. J., Schneider, J., Sutskever, I., Abbeel, P., & Zaremba, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07326 · Cited by 449 · see citing works
Deep spatial autoencoders for visuomotor learning
Finn, C., Tan, X., Duan, Y., Darrell, T., Levine, S., & Abbeel, P. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487173 · Cited by 446 · see citing works
Self-citationStabilizing Off-Policy Q-Learning via Bootstrapping Error ReductionPreprint
Kumar, A., Fu, J., Tucker, G., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00949 · Cited by 445 · see citing works
Self-citationLearning to Walk Via Deep Reinforcement Learning
Haarnoja, T., Ha, S., Zhou, A., Tan, J., Tucker, G., & Levine, S. (2019) · Robotics: Science and Systems XV · doi:10.15607/rss.2019.xv.011 · Cited by 430 · see citing works
Self-citationJames, S., Wohlhart, P., Kalakrishnan, M., Kalashnikov, D., Irpan, A., Ibarz, J., Levine, S., Hadsell, R., & Bousmalis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01291 · Cited by 420 · see citing works
Self-citationLearning Neural Network Policies with Guided Policy Search under Unknown Dynamics
Levine, S., & Abbeel, P. (2014) · Cited by 379 · see citing works
Self-citationNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE
Hilton, W. (1983) · Cited by 376 · see citing works
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context VariablesPreprint
Rakelly, K., Zhou, A., Quillen, D., Finn, C., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08254 · Cited by 371 · see citing works
Self-citationLearning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning
Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv · doi:10.48550/arxiv.1803.11347 · Cited by 370 · see citing works
Self-citationAsymmetric Actor Critic for Image-Based Robot Learning
Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., & Abbeel, P. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.008 · Cited by 369 · see citing works
Lee, M. A., Zhu, Y., Srinivasan, K., Shah, P., Savarese, S., Fei-Fei, L., Garg, A., & Bohg, J. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793485 · Cited by 349 · see citing works
One-Shot Visual Imitation Learning via Meta-Learning
Finn, C., Yu, T., Zhang, T., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1709.04905 · Cited by 341 · see citing works
Self-citationDeepMPC: Learning Deep Latent Features for Model Predictive Control
Lenz, I., Knepper, R. A., & Saxena, A. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.012 · Cited by 339 · see citing works
Mahler, J., Matl, M., Liu, X., Li, A., Gealy, D. V., & Goldberg, K. Y. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460887 · Cited by 326 · see citing works
Robot trajectory optimization using approximate inference
Toussaint, M. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553508 · Cited by 325 · see citing works
Overlapping experiment infrastructure
Tang, D., Agarwal, A., O'Brien, D., & Meyer, M. (2010) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1835804.1835810 · Cited by 314 · see citing works
Learning by Playing - Solving Sparse Reward Tasks from ScratchPreprint
Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Van de Wiele, T., Mnih, V., Heess, N., & Springenberg, J. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10567 · Cited by 314 · see citing works
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic ControlPreprint
Ebert, F., Finn, C., Dasari, S., Xie, A., Lee, A. X., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00568 · Cited by 294 · see citing works
Self-citationRobot learning from demonstration by constructing skill trees
Konidaris, G., Kuindersma, S., Grupen, R. A., & Barto, A. G. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911428653 · Cited by 287 · see citing works
Learning contact-rich manipulation skills with guided policy search
Levine, S., Wagener, N., & Abbeel, P. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7138994 · Cited by 276 · see citing works
Self-citationModel-Ensemble Trust-Region Policy OptimizationPreprint
Kurutach, T., Clavera, I., Duan, Y., Tamar, A., & Abbeel, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10592 · Cited by 276 · see citing works
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
Zhao, C., Badrinarayanan, V., Lee, C., & Rabinovich, A. (2018) · International Conference on Machine Learning · Cited by 275 · see citing works
Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., & Hadsell, R. (2015) · arXiv · doi:10.48550/arxiv.1511.06295 · Cited by 270 · see citing works
Off-Policy Deep Reinforcement Learning without Exploration
Fujimoto, S. (2024) · TIB Data Manager · doi:10.57702/vkcnqiqb · Cited by 269 · see citing works
Active Preference-Based Learning of Reward Functions
Sadigh, D., Dragan, A. D., Sastry, S., & Seshia, S. A. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.053 · Cited by 260 · see citing works
Deep Dynamics Models for Learning Dexterous ManipulationPreprint
Nagabandi, A., Konoglie, K., Levine, S., & Kumar, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11652 · Cited by 245 · see citing works
Self-citationPreparing for the unknown: Learning a universal policy with online system identification
Yu, W., Tan, J., Karen Liu, C., & Turk, G. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.048 · Cited by 237 · see citing works
Self-citationModel predictive control using neural networks
Draeger, A., Engell, S., & Ranke, H. (1995) · IEEE Control Systems · doi:10.1109/37.466261 · Cited by 236 · see citing works
Learning Navigation Behaviors End-to-End With AutoRL
Chiang, H.-T. L., Faust, A., Fišer, M., & Francis, A. G., Jr. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2899918 · Cited by 226 · see citing works
(2021) · Journal of Machine Learning Research · Cited by 224 · see citing works
Recovery RL: Safe Reinforcement Learning With Learned Recovery Zones
Thananjeyan, B., Balakrishna, A., Nair, S., Luo, M., Srinivasan, K., Hwang, M., Gonzalez, J. E., Ibarz, J., Finn, C., & Goldberg, K. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3070252 · Cited by 218 · see citing works
Self-citationVariational Intrinsic ControlPreprint
Gregor, K., Rezende, D. J., & Wierstra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07507 · Cited by 213 · see citing works
Reinforcement Learning in Robotics: A Survey
Kober, J., & Peters, J. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-03194-1_2 · Cited by 213 · see citing works
Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
James, S., Davison, A. J., & Johns, E. (2017) · arXiv · doi:10.48550/arxiv.1707.02267 · Cited by 208 · see citing works
Learning symmetric and low-energy locomotion
Yu, W., Turk, G., & Liu, C. K. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201397 · Cited by 208 · see citing works
Actor-Mimic: Deep Multitask and Transfer Reinforcement LearningPreprint
Parisotto, E., Ba, J. L., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06342 · Cited by 198 · see citing works
End-To-End Robotic Reinforcement Learning without Reward Engineering
Singh, A., Yang, L., Finn, C., & Levine, S. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.073 · Cited by 198 · see citing works
Self-citationComposable Deep Reinforcement Learning for Robotic Manipulation
Haarnoja, T., Pong, V. H., Zhou, A., Dalal, M., Abbeel, P., & Levine, S. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460756 · Cited by 193 · see citing works
Self-citationDeep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards
Schoettler, G., Nair, A., Luo, J., Bahl, S., Ojea, J. A., Solowjow, E., & Levine, S. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341714 · Cited by 191 · see citing works
Self-citationLearning to be Safe: Deep RL with a Safety CriticPreprint
Srinivasan, K., Eysenbach, B., Ha, S., Tan, J., & Finn, C. (2020) · arXiv.org · doi:10.48550/arxiv.2010.14603 · Cited by 185 · see citing works
Self-citationDexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost
Zhu, H., Gupta, A., Rajeswaran, A., Levine, S., & Kumar, V. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794102 · Cited by 184 · see citing works
Self-citationDense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic ManipulationPreprint
Florence, P., Manuelli, L., & Tedrake, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08756 · Cited by 181 · see citing works
Long-Range Indoor Navigation With PRM-RL
Francis, A. G., Jr., Faust, A., Chiang, H.-T. L., Hsu, J., Kew, J. C., Fišer, M., & Lee, T.-W. E. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.2975428 · Cited by 172 · see citing works
RL-CycleGAN: Reinforcement Learning Aware Simulation-to-Real
Rao, K., Harris, C., Irpan, A., Levine, S., Ibarz, J., & Khansari, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01117 · Cited by 169 · see citing works
Self-citationSimple random search of static linear policies is competitive for reinforcement learning
Mania, H., Guy, A., & Recht, B. (2018) · neural information processing systems · Cited by 163 · see citing works
An Optimistic Perspective on Offline Reinforcement LearningPreprint
Agarwal, R., Schuurmans, D., & Norouzi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04543 · Cited by 162 · see citing works
Learning a visuomotor controller for real world robotic grasping using simulated depth imagesPreprint
Viereck, U., ten Pas, A., Saenko, K., & Platt, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04652 · Cited by 160 · see citing works
Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement LearningPreprint
Peng, X. B., Kumar, A., Zhang, G., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00177 · Cited by 158 · see citing works
Self-citationRoboNet: Large-Scale Multi-Robot Learning
Dasari, S., Ebert, F., Tian, S., Nair, S., Bucher, B., Schmeckpeper, K., Singh, S., Levine, S., & Finn, C. (2019) · Digital Eprints Services at ISB (DESI) (Indian School of Business) · doi:10.48550/arxiv.1910.11215 · Cited by 151 · see citing works
Self-citationCartman: The Low-Cost Cartesian Manipulator that Won the Amazon Robotics Challenge
Morrison, D., Tow, A. W., McTaggart, M., Smith, R. S., Kelly-Boxall, N., Wade-McCue, S., Erskine, J., Grinover, R., Gurman, A., Hunn, T., et al. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463191 · Cited by 145 · see citing works
Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement LearningPreprint
Chebotar, Y., Hausman, K., Zhang, M., Sukhatme, G. S., Schaal, S., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.03078 · Cited by 131 · see citing works
Self-citationPath integral guided policy search
Chebotar, Y., Kalakrishnan, M., Yahya, A., Li, A., Stefan Schaal, A., & Levine, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989384 · Cited by 124 · see citing works
Self-citationDeep predictive policy training using reinforcement learning
Ghadirzadeh, A., Maki, A., Kragic, D., & Bjorkman, M. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206046 · Cited by 113 · see citing works
Silver, T., Allen, K., Tenenbaum, J., & Kaelbling, L. (2018) · arXiv · doi:10.48550/arxiv.1812.06298 · Cited by 113 · see citing works
Self-Supervised Correspondence in Visuomotor Policy Learning
Florence, P., Manuelli, L., & Tedrake, R. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2956365 · Cited by 111 · see citing works
Data Efficient Reinforcement Learning for Legged RobotsPreprint
Yang, Y., Caluwaerts, K., Işçen, A., Zhang, T., Tan, J., & Sindhwani, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03613 · Cited by 108 · see citing works
Self-citationPolicies Modulating Trajectory GeneratorsPreprint
Işçen, A., Caluwaerts, K., Tan, J., Zhang, T., Coumans, E., Sindhwani, V., & Vanhoucke, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02812 · Cited by 96 · see citing works
Self-citationEX2: Exploration with Exemplar Models for Deep Reinforcement Learning
Fu, J., Co-Reyes, J. D., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01260 · Cited by 95 · see citing works
Self-citationBıyık, E., Losey, D. P., Palan, M., Landolfi, N. C., Shevchuk, G., & Sadigh, D. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211041652 · Cited by 95 · see citing works
On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference
Rawlik, K., Toussaint, M., & Vijayakumar, S. (2012) · Robotics: Science and Systems · doi:10.15607/rss.2012.viii.045 · Cited by 90 · see citing works
Learning to dress: Synthesizing human dressing motion via deep reinforcement learning
Clegg, A., Yu, W., Tan, J., Liu, C. K., & Turk, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275048 · Cited by 89 · see citing works
Self-citationImitation Learning from Imperfect DemonstrationPreprint
Wu, Y.-H., Charoenphakdee, N., Bao, H., Tangkaratt, V., & Sugiyama, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09387 · Cited by 89 · see citing works
Online control of simulated humanoids using particle belief propagation
Hämäläinen, P., Rajamäki, J., & Liu, C. K. (2015) · ACM Transactions on Graphics · doi:10.1145/2767002 · Cited by 88 · see citing works
Conference on artificial intelligence
(1987) · Computers in Biology and Medicine · doi:10.1016/0010-4825(87)90050-3 · Cited by 88 · see citing works
Learning Fast Adaptation With Meta Strategy Optimization
Yu, W., Tan, J., Bai, Y., Coumans, E., & Ha, S. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974685 · Cited by 86 · see citing works
Self-citationTask-Embedded Control Networks for Few-Shot Imitation LearningPreprint
James, S., Bloesch, M., & Davison, A. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03237 · Cited by 82 · see citing works
Guided Policy Search via Approximate Mirror DescentPreprint
Montgomery, W. H., & Levine, S. (2016) · arXiv (Cornell University) · Cited by 82 · see citing works
Self-citationAn Optimistic Perspective on Offline Deep Reinforcement Learning
Agarwal, R., Schuurmans, D., & Norouzi, M. (2020) · International Conference on Machine Learning · Cited by 82 · see citing works
Tan, J., Gu, Y., Liu, C. K., & Turk, G. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601121 · Cited by 76 · see citing works
Self-citationLeave no Trace: Learning to Reset for Safe and Autonomous Reinforcement LearningPreprint
Eysenbach, B., Gu, S., Ibarz, J., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06782 · Cited by 76 · see citing works
Self-citationImprovisation through Physical Understanding: Using Novel Objects As Tools with Visual Foresight
Xie, A., Ebert, F., Levine, S., & Finn, C. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.001 · Cited by 66 · see citing works
Self-citationCollaborative Evolutionary Reinforcement LearningPreprint
Khadka, S., Majumdar, S., Nassar, T., Dwiel, Z., Tumer, E. C., Miret, S., Liu, Y., & Tumer, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00976 · Cited by 63 · see citing works
Learning to Walk via Deep Reinforcement Learning
Haarnoja, T., Ha, S., Zhou, A., Tan, J., Tucker, G., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1812.11103 · Cited by 53 · see citing works
Self-citationLearning to sequence movement primitives from demonstrations
Manschitz, S., Kober, J., Gienger, M., & Peters, J. (2014) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2014.6943187 · Cited by 47 · see citing works
Learning Deep Visuomotor Policies for Dexterous Hand Manipulation
Jain, D., Li, A. R., Singhal, S., Rajeswaran, A., Kumar, V., & Todorov, E. V. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794033 · Cited by 45 · see citing works
Automated Deep Reinforcement Learning Environment for Hardware of a Modular Legged Robot
Ha, S., Kim, J., & Yamane, K. (2018) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/urai.2018.8442201 · Cited by 42 · see citing works
Few-Shot Goal Inference for Visuomotor Learning and PlanningPreprint
Xie, A., Singh, A., Levine, S., & Finn, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00482 · Cited by 42 · see citing works
Self-citationUnsupervised Curricula for Visual Meta-Reinforcement LearningPreprint
Jabri, A., Hsu, K., Eysenbach, B., Gupta, A., Levine, S., & Finn, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.04226 · Cited by 42 · see citing works
Self-citationSE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Control
Byravan, A., Leeb, F., Meier, F., & Fox, D. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461184 · Cited by 40 · see citing works
On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference
Rawlik, K., Toussaint, M., & Vijayakumar, S. (2013) · The MIT Press eBooks · doi:10.7551/mitpress/9816.003.0050 · Cited by 39 · see citing works
Learning sequential motor tasks
Daniel, C., Neumann, G. M., Kroemer, O., & Peters, J. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630937 · Cited by 38 · see citing works
The limits and potentials of deep learning for robotics
Sünderhauf, N., Brock, O., Scheirer, W., Hadsell, R., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M., et al. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364918770733 · Cited by 38 · see citing works
Ray Interference: a Source of Plateaus in Deep Reinforcement LearningPreprint
Schaul, T., Borsa, D., Modayil, J., & Pascanu, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.11455 · Cited by 33 · see citing works
Realtime State Estimation with Tactile and Visual Sensing. Application to Planar Manipulation
Yu, K., & Rodríguez, A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463183 · Cited by 33 · see citing works
Learning Image-Conditioned Dynamics Models for Control of Underactuated Legged Millirobots
Nagabandi, A., Yang, G., Asmar, T., Pandya, R., Kahn, G., Levine, S., & Fearing, R. S. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594193 · Cited by 32 · see citing works
Self-citationEnd-to-End Robotic Reinforcement Learning without Reward EngineeringPreprint
Singh, A., Yang, L., Hartikainen, K., Finn, C., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.07854 · Cited by 31 · see citing works
Self-citationValue constrained model-free continuous controlPreprint
Bohez, S., Abdolmaleki, A., Neunert, M., Buchli, J., Heess, N., & Hadsell, R. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04623 · Cited by 29 · see citing works
NoRML: No-Reward Meta Learning
Yang, Y., Caluwaerts, K., Işçen, A., Tan, J., & Finn, C. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/elvr9404 · Cited by 27 · see citing works
Self-citationMontgomery, W. H., Ajay, A., Finn, C., Abbeel, P., & Levine, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989383 · Cited by 26 · see citing works
Self-citationThinking While Moving: Deep Reinforcement Learning with Concurrent Control
Xiao, T., Jang, E., Kalashnikov, D., Levine, S., Ibarz, J., Hausman, K., & Herzog, A. (2020) · arXiv · doi:10.48550/arxiv.2004.06089 · Cited by 25 · see citing works
Self-citationExperience-Embedded Visual ForesightPreprint
Lin, Y.-C., Bauzá, M., & Isola, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.05071 · Cited by 20 · see citing works
Recovery RL: Safe Reinforcement Learning with Learned Recovery ZonesPreprint
Thananjeyan, B., Balakrishna, A., Nair, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15920 · Cited by 20 · see citing works
Simultaneously Learning Vision and Feature-Based Control Policies for Real-World Ball-In-A-Cup
Schwab, D., Springenberg, J. T., Martins, M. F., Neunert, M., Lampe, T., Abdolmaleki, A., Hertweck, T., Hafner, R., Nori, F., & Riedmiller, M. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.027 · Cited by 18 · see citing works
Modular Hopping and Running via Parallel Composition
De, A. (2017) · ScholarlyCommons (University of Pennsylvania) · Cited by 14 · see citing works
Learning Fast Adaptation with Meta Strategy OptimizationPreprint
Yu, W., Tan, J., Bai, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12995 · Cited by 12 · see citing works
Self-citationA Framework for Data-Driven RoboticsPreprint
Cabi, S., Colmenarejo, S. G., Novikov, A., Konyushkova, K., Reed, S., Jeong, R., Żołna, K., Aytar, Y., Budden, D. M., Vecerík, M., et al. (2019) · arXiv (Cornell University) · Cited by 10 · see citing works
Regrasping using Tactile Perception and Supervised Policy Learning
Hausman, K., Chebotar, Y., Kroemer, O., Sukhatme, G. S., & Stefan Schaal, A. (2017) · National Conference on Artificial Intelligence · Cited by 6 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
8 of the 193 references couldn’t be matched to a known paper.
Research works printed but not linked: 9
- Burda Y, Edwards H, Storkey A and Klimov O (2019) Exploration by random network distillation. In: International Conference on Learning Representations .
- Chen D, Zhou B, Koltun V and Kr¨ ahenb¨ uhl P (2020) Learning by cheating. In: Conference on Robot Learning . PMLR.
- Finn C, Yu T, Zhang T, Abbeel P and Levine S (2017b) OneShot Visual Imitation Learning via Meta-Learning. In: Conference on Robot Learning , Proceedings of Machine Learning Research , volume 78.
- Gandhi D, Pinto L and Gupta A (2017) Learning to fly by crashing. In: International Conference on Intelligent Robots and Systems . IEEE, pp. 3948-3955.
- Pathak D, Agrawal P, Efros AA and Darrell T (2017) Curiositydriven exploration by self-supervised prediction. In: Conference on Computer Vision and Pattern Recognition Workshops . IEEE, pp. 16-17.
- Peng XB, Abbeel P, Levine S and van de Panne M (2018a) Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions on Graphics (TOG) 37(4).
- Sermanet P, Lynch C, Chebotar Y, Hsu J, Jang E, Schaal S, Levine S and Brain G (2018) Time-contrastive networks: Selfsupervised learning from video. In: International Conference on Robotics and Automation . IEEE.
- Tedrake R, Zhang TW and Seung HS (2015) Learning to Walk in 20 minutes. In: Workshop on Adaptive and Learning Systems .
- Yu W, Turk G and Liu CK (2018) Learning symmetric and lowenergy locomotion. ACM Transactions on Graphics (TOG) 37(4).
Cited by
Sorted by citations
Manakitsa, N., Maraslidis, G. S., Moysis, L., & Fragulis, G. F. (2024) · Technologies · doi:10.3390/technologies12020015 · Cited by 326 · see citing works
Explainable Reinforcement Learning: A Survey and Comparative Review
Milani, S., Topin, N., Veloso, M., & Fang, F. (2023) · ACM Computing Surveys · doi:10.1145/3616864 · Cited by 289 · see citing works
Robot learning towards smart robotic manufacturing: A review
Liu, Z., Liu, Q., Xu, W., Wang, L., & Zhou, Z. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102360 · Cited by 233 · see citing works
Recent advancements of robotics in construction
Xiao, B., Chen, C., & Yin, X. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104591 · Cited by 204 · see citing works
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
Tang, C., Abbatematteo, B., Hu, J., Chandra, R., Martín-Martín, R., & Stone, P. (2024) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-030323-022510 · Cited by 217 · see citing works
Retzlaff, C. O., Das, S., Wayllace, C., Mousavi, P., Afshari, M. P., Yang, T., Saranti, A., Angerschmid, A., Taylor, M. E., & Holzinger, A. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15348 · Cited by 194 · see citing works
Learning agile soccer skills for a bipedal robot with deep reinforcement learning
Haarnoja, T., Moran, B., Lever, G., Huang, S. H., Tirumala, D., Humplik, J., Wulfmeier, M., Tunyasuvunakool, S., Siegel, N., Hafner, R., et al. (2024) · Science Robotics · doi:10.1126/scirobotics.adi8022 · Cited by 191 · see citing works
Borboni, A., Reddy, K. V. V., Elamvazuthi, I., Al-Quraishi, M. S., Natarajan, E., & Ali, S. S. A. (2023) · Machines · doi:10.3390/machines11010111 · Cited by 146 · see citing works
Xu, J., Li, B., Lu, B., Liu, Y., Dou, Q., & Heng, P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635867 · Cited by 107 · see citing works
Kabir, H., Tham, M., & Chang, Y. C. (2023) · Digital Communications and Networks · doi:10.1016/j.dcan.2023.05.006 · Cited by 119 · see citing works
The neuromechanics of animal locomotion: From biology to robotics and back
Ramdya, P., & Ijspeert, A. J. (2023) · Science Robotics · doi:10.1126/scirobotics.adg0279 · Cited by 92 · see citing works
del Real, A. J., Andreiana, D. S., Roldán, Á. O., Bustos, A. H., & Acevedo, L. (2022) · Applied Sciences · doi:10.3390/app122312377 · Cited by 97 · see citing works
Al-Hamadani, M. N. A., Fadhel, M. A., Alzubaidi, L., & Harangi, B. (2024) · Sensors · doi:10.3390/s24082461 · Cited by 100 · see citing works
Hobbs, K. L., Mote, M., Abate, M., Coogan, S., & Féron, É. (2023) · IEEE Control Systems · doi:10.1109/mcs.2023.3234380 · Cited by 90 · see citing works
Pore, A., Li, Z., Dall’Alba, D., Hernansanz, A., Momi, E. D., Menciassi, A., Casals, A., Dankelman, J., Fiorini, P., & Poorten, E. V. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3269384 · Cited by 77 · see citing works
Long, Y., Wang, W., Huang, T., Wang, Y., & Dou, Q. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3284380 · Cited by 75 · see citing works
Active learning in robotics: A review of control principles
Taylor, A., Berrueta, T. A., & Murphey, T. D. (2021) · Mechatronics · doi:10.1016/j.mechatronics.2021.102576 · Cited by 75 · see citing works
Intelligent Robots and Human-Robot Collaboration in the Construction Industry: A Review
Wei, H.-H., Zhang, Y., Sun, X., Chen, J., & Li, S. (2023) · Journal of Intelligent Construction · doi:10.26599/jic.2023.9180002 · Cited by 75 · see citing works
High-speed quadrupedal locomotion by imitation-relaxation reinforcement learning
Jin, Y., Liu, X., Shao, Y., Wang, H., & Yang, W. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00576-3 · Cited by 69 · see citing works
Taniguchi, T., Murata, S., Suzuki, M., Ognibene, D., Lanillos, P., Uğur, E., Jamone, L., Nakamura, T., Ciria, A., Lara, B., et al. (2023) · Advanced Robotics · doi:10.1080/01691864.2023.2225232 · Cited by 67 · see citing works
Human-in-the-Loop Robot Learning for Smart Manufacturing: A Human-Centric Perspective
Chen, H., Li, S., Fan, J., Duan, A., Yang, C., Navarro-Alarcon, D., & Zheng, P. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3528051 · Cited by 67 · see citing works
Chu, S., Lin, M., Li, D., Lin, R., & Xiao, S. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.120139 · Cited by 67 · see citing works
Huang, L., Zhu, Z., & Zou, Z. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104691 · Cited by 67 · see citing works
Li, Y., Song, Y., Sun, Y., & Zeng, M. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102518 · Cited by 63 · see citing works
Learning-based legged locomotion: State of the art and future perspectives
Ha, S., Lee, J., van de Panne, M., Xie, Z., Yu, W., & Khadiv, M. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649241312698 · Cited by 63 · see citing works
Huang, L., Ye, M., Xue, X., Wang, Y., Qiu, H., & Deng, X. (2022) · Wireless Networks · doi:10.1007/s11276-022-03066-x · Cited by 60 · see citing works
CybORG: A Gym for the Development of Autonomous Cyber AgentsPreprint
Standen, M., Lucas, M., Bowman, D. B., Richer, T. J., Kim, J., & Marriott, D. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09118 · Cited by 56 · see citing works
Dedeloudi, A., Weaver, E., & Lamprou, D. A. (2023) · International Journal of Pharmaceutics · doi:10.1016/j.ijpharm.2023.122818 · Cited by 58 · see citing works
A residual reinforcement learning method for robotic assembly using visual and force information
Zhang, Z., Wang, Y., Zhang, Z., Wang, L., Huang, H., & Cao, Q. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.11.008 · Cited by 55 · see citing works
Yin, Y., Chen, Z., Liu, G., Yin, J., & Guo, J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123202 · Cited by 57 · see citing works
Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments
Iyer, A., Grewal, K., Velu, A., Souza, L. O., Forest, J., & Ahmad, S. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.846219 · Cited by 55 · see citing works
A review on interaction control for contact robots through intent detection
Li, Y., Sena, A., Wang, Z., Xing, X., Babič, J., van Asseldonk, E. H. F., & Burdet, E. (2022) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/ac8193 · Cited by 54 · see citing works
Booth, S., Knox, W. B., Shah, J., Niekum, S., Stone, P., & Allievi, A. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25733 · Cited by 52 · see citing works
Noack, B. R. (2023) · Physics of Fluids · doi:10.1063/5.0152777 · Cited by 50 · see citing works
From screens to scenes: A survey of embodied AI in healthcare
Liu, Y., Cao, X., Chen, T., Jiang, Y., You, J., Wu, M., Wang, X., Feng, M., Jin, Y., & Chen, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103033 · Cited by 59 · see citing works
Robot Learning of Mobile Manipulation With Reachability Behavior Priors
Jauhri, S., Peters, J., & Chalvatzaki, G. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3188109 · Cited by 49 · see citing works
Learning from Demonstrations for Autonomous Soft-tissue Retraction
Pore, A., Tagliabue, E., Piccinelli, M., Dall’Alba, D., Casals, A., & Fiorini, P. (2021) · International Symposium on Medical Robotics · doi:10.1109/ismr48346.2021.9661514 · Cited by 48 · see citing works
Ning, G., Liang, H., Zhang, X., & Liao, H. (2023) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2023.3250767 · Cited by 47 · see citing works
Gao, Y., Qi, J., Sun, Y., Hu, X., Dong, Z., & Sun, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111506 · Cited by 47 · see citing works
Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach
Chen, Y., Cai, Y.-R., & Cheng, M.-Y. (2023) · Machines · doi:10.3390/machines11020275 · Cited by 50 · see citing works
Ji, Q., Fu, S., Tan, K., Muralidharan, S. T., Lagrelius, K., Danelia, D., Andrikopoulos, G., Wang, X. V., Wang, L., & Feng, L. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102382 · Cited by 46 · see citing works
Continual Semantic Segmentation with Automatic Memory Sample Selection
Zhu, L., Chen, T., Yin, J., See, S. C. W., & Liu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00301 · Cited by 46 · see citing works
Data-Driven Formation Control for Multiple Heterogeneous Vehicles in Air–Ground Coordination
Zhao, W., Liu, H., Wan, Y., & Lin, Z. (2022) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2022.3181254 · Cited by 44 · see citing works
An adaptable and personalized framework for top-N course recommendations in online learning
Amin, S., Uddin, M. I., Alarood, A. A., Mashwani, W. K., Alzahrani, A. O., & Alzahrani, H. A. (2024) · Scientific Reports · doi:10.1038/s41598-024-56497-1 · Cited by 44 · see citing works
Suárez, P., Alcántara-Ávila, F., Miró, A., Rabault, J., Font, B., Lehmkuhl, O., & Vinuesa, R. (2025) · Flow Turbulence and Combustion · doi:10.1007/s10494-025-00642-x · Cited by 44 · see citing works
Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning
Behrens, M. R., & Ruder, W. C. (2022) · Advanced Intelligent Systems · doi:10.1002/aisy.202200023 · Cited by 44 · see citing works
Khan, M. A., Asadi, H., Zhang, L., Qazani, M. R. C., Oladazimi, S., Loo, C. K., Lim, C. P., & Nahavandi, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123717 · Cited by 48 · see citing works
The Societal Implications of Deep Reinforcement Learning
Whittlestone, J., Arulkumaran, K., & Crosby, M. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12360 · Cited by 43 · see citing works
3D robotic navigation using a vision-based deep reinforcement learning model
Zieliński, P., & Markowska–Kaczmar, U. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107602 · Cited by 42 · see citing works
Lim, V., Huang, H., Chen, L. Y., Wang, J., Ichnowski, J., Seita, D., Laskey, M., & Goldberg, K. Y. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811651 · Cited by 53 · see citing works
Towards intelligent cooperative robotics in additive manufacturing: Past, present, and future
Rescsanski, S., Hebert, R., Haghighi, A., Tang, J., & Imani, F. (2025) · Robotics Comput. Integr. Manuf. · doi:10.1016/j.rcim.2024.102925 · Cited by 32 · see citing works
Machine learning meets advanced robotic manipulation
Nahavandi, S., Alizadehsani, R., Nahavandi, D., Lim, C. P., Kelly, K., & Bello, F. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102221 · Cited by 42 · see citing works
Reinforcement Learning Control of a Forestry Crane Manipulator
Andersson, J., Bodin, K., Lindmark, D., Servin, M., & Wallin, E. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636219 · Cited by 40 · see citing works
High-Tech Defense Industries: Developing Autonomous Intelligent Systems
dos Reis, J. C. G., Cohen, Y., Melão, N. F., Costa, J., & Jorge, D. (2021) · Applied Sciences · doi:10.3390/app11114920 · Cited by 44 · see citing works
Reinforcement-learning-based actuator selection method for active flow control
Paris, R., Beneddine, S., & Dandois, J. (2023) · Journal of Fluid Mechanics · doi:10.1017/jfm.2022.1043 · Cited by 37 · see citing works
Games for Artificial Intelligence Research: A Review and Perspectives
Hu, C., Zhao, Y., Wang, Z., Du, H., & Liu, J. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3410935 · Cited by 51 · see citing works
GNN-Empowered Effective Partial Observation MARL Method for AoI Management in Multi-UAV Network
Pan, Y., Wang, X., Xu, Z., Cheng, N., Xu, W., & Zhang, J.-J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3447774 · Cited by 37 · see citing works
Huang, T., Chen, K., Li, B., Liu, Y., & Dou, Q. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160327 · Cited by 34 · see citing works
General-purpose foundation models for increased autonomy in robot-assisted surgery
Schmidgall, S., Kim, J. W., Kuntz, A., Ghazi, A., & Krieger, A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00917-4 · Cited by 35 · see citing works
Gu, Z., Li, J., Shen, W., Yu, W., Xie, Z., McCrory, S., Cheng, X., Shamsah, A., Griffin, R. J., Liu, C. K., et al. (2026) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2025.3579247 · Cited by 55 · see citing works
Swarm Cooperative Navigation Using Centralized Training and Decentralized Execution
Azzam, R., Boiko, I. M., & Zweiri, Y. (2023) · Drones · doi:10.3390/drones7030193 · Cited by 33 · see citing works
Qashqai, P., Babaie, M., Zgheib, R., & Al‐Haddad, K. (2023) · IEEE Access · doi:10.1109/access.2023.3318264 · Cited by 35 · see citing works
Sim-to-real transfer of active suspension control using deep reinforcement learning
Wiberg, V., Wallin, E., Fälldin, A., Semberg, T., Rossander, M., Wadbro, E., & Servin, M. (2024) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2024.104731 · Cited by 34 · see citing works
Skill generalization of tubular object manipulation with tactile sensing and Sim2Real learning
Zhao, Y., Jing, X., Qian, K., Gomes, D. F., & Luo, S. (2022) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2022.104321 · Cited by 32 · see citing works
Port-Hamiltonian Neural ODE Networks on Lie Groups for Robot Dynamics Learning and Control
Duong, T., Altawaitan, A., Stanley, J., & Atanasov, N. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3428433 · Cited by 31 · see citing works
Kulkarni, P., Kober, J., Babuška, R., & Santina, C. D. (2021) · Advanced Intelligent Systems · doi:10.1002/aisy.202100095 · Cited by 31 · see citing works
Dynamic Navigation in Unconstrained Environments Using Reinforcement Learning Algorithms
Chronis, C., Anagnostopoulos, G., Politi, E., Dimitrakopoulos, G. J., & Varlamis, I. (2023) · IEEE Access · doi:10.1109/access.2023.3326435 · Cited by 33 · see citing works
Ren, L., Ning, X., & Wang, Z. X. (2022) · Computers & Industrial Engineering · doi:10.1016/j.cie.2022.108242 · Cited by 30 · see citing works
Deng, X., Liu, J., Gong, H., Gong, H., & Huang, J. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3224966 · Cited by 30 · see citing works
Liu, Q., Ji, Z., Xu, W., Liu, Z., Yao, B., & Zhou, Z. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121037 · Cited by 30 · see citing works
Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics
Rana, K., Dasagi, V., Haviland, J., Talbot, B., Milford, M. J., & Sünderhauf, N. (2023) · The International Journal of Robotics Research · doi:10.1177/02783649231167210 · Cited by 30 · see citing works
Eßer, J., Bach, N., Jestel, C., Urbann, O., & Kerner, S. (2022) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2022.3207664 · Cited by 30 · see citing works
Park, S.-Y., Lee, C., Jeong, S., Lee, J., Kim, D., Jang, Y., Seol, W., Kim, H., & Ahn, S. (2024) · International Journal of Precision Engineering and Manufacturing-Green Technology · doi:10.1007/s40684-023-00593-6 · Cited by 35 · see citing works
A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning
Sarhadi, P., Naeem, W., & Αθανασόπουλος, Ν. (2022) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2022.10.440 · Cited by 28 · see citing works
Yao, T., Wang, H., Lu, B., Ge, J., Pei, Z., Kowarschik, M., Sun, L., Seneviratne, L. D., & Qi, P. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3555559 · Cited by 27 · see citing works
AI Challenges for Society and Ethics
Whittlestone, J., & Clarke, S. (2022) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197579329.013.3 · Cited by 27 · see citing works
A Survey of the Potential Long-term Impacts of AI
Clarke, S., & Whittlestone, J. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534131 · Cited by 27 · see citing works
Hazrathosseini, A., & Afrapoli, A. M. (2024) · Mining Technology Transactions of the Institutions of Mining and Metallurgy · doi:10.1177/25726668231222998 · Cited by 26 · see citing works
Learning-Based Navigation and Collision Avoidance Through Reinforcement for UAVs
Azzam, R., Chehadeh, M. S., Hay, O. A., Humais, M. A., Boiko, I. M., & Zweiri, Y. (2023) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2023.3294889 · Cited by 25 · see citing works
Billard, A., Albu‐Schäffer, A., Beetz, M., Burgard, W., Corke, P., Ciocarlie, M., Dahiya, R., Kragić, D., Goldberg, K., Nagai, Y., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01050-6 · Cited by 25 · see citing works
Excavation Reinforcement Learning Using Geometric Representation
Lu, Q., Zhu, Y., & Zhang, L. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3150511 · Cited by 25 · see citing works
FED-UP: Federated Deep Reinforcement Learning-based UAV Path Planning against Hostile Defense System
Khalil, A. A., & Rahman, M. A. (2022) · International Conference on Network and Service Management (CNSM) · doi:10.23919/cnsm55787.2022.9964907 · Cited by 25 · see citing works
Resolving the associative learning paradox by category learning in pigeons
Wasserman, E. A., Kain, A. G., & O'Donoghue, E. M. (2023) · Current Biology · doi:10.1016/j.cub.2023.01.024 · Cited by 25 · see citing works
Identifying important sensory feedback for learning locomotion skills
Yu, W., Yang, C., McGreavy, C., Triantafyllidis, E., Bellegarda, G., Shafiee, M., Ijspeert, A. J., & Li, Z. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00701-w · Cited by 27 · see citing works
Abbas, A. N., Chasparis, G. C., & Kelleher, J. D. (2023) · Data & Knowledge Engineering · doi:10.1016/j.datak.2023.102240 · Cited by 25 · see citing works
Learn From Safe Experience: Safe Reinforcement Learning for Task Automation of Surgical Robot
Fan, K., Chen, Z., Ferrigno, G., & Momi, E. D. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3351797 · Cited by 25 · see citing works
Chen, W., Xu, J. L., Xiang, F., Yuan, X., Su, H., & Chen, R. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3352969 · Cited by 24 · see citing works
Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasets
Huber, J., Hélénon, F., Watrelot, H., Amar, F. B., & Doncieux, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610677 · Cited by 24 · see citing works
Lee, R., Hamaya, M., Murooka, T., Ijiri, Y., & Corke, P. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3130377 · Cited by 24 · see citing works
Learning Implicit Priors for Motion Optimization
Urain, J., Le, A. T., Lambert, A., Chalvatzaki, G., Boots, B., & Peters, J. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981264 · Cited by 24 · see citing works
Learning positioning policies for mobile manipulation operations with deep reinforcement learning
Iriondo, A., Lazkano, E., Ansuategi, A., Rivera, A., Lluvia, I., & Tubío, C. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-01815-8 · Cited by 23 · see citing works
Liu, P., Zhang, K., Tateo, D., Jauhri, S., Hu, Z., Peters, J., & Chalvatzaki, G. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161548 · Cited by 23 · see citing works
Effective Multi-Agent Deep Reinforcement Learning Control With Relative Entropy Regularization
Miao, C., Cui, Y., Li, H., & Wu, X. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3398712 · Cited by 23 · see citing works
Maximum diffusion reinforcement learning
Berrueta, T. A., Pinosky, A., & Murphey, T. D. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00829-3 · Cited by 24 · see citing works
Deep reinforcement learning in surgical robotics: Enhancing the automation level
Qian, C., & Ren, H. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-13271-1.00055-8 · Cited by 24 · see citing works
2-level reinforcement learning for ships on inland waterways: Path planning and following
Waltz, M., Paulig, N., & Okhrin, O. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126933 · Cited by 24 · see citing works
Active object tracking of free floating space manipulators based on deep reinforcement learning
Lei, W., Fu, H., & Sun, G. (2022) · Advances in Space Research · doi:10.1016/j.asr.2022.08.041 · Cited by 22 · see citing works
AdaptiveON: Adaptive Outdoor Local Navigation Method for Stable and Reliable Actions
Liang, J., Weerakoon, K., Guan, T., Karapetyan, N., & Manocha, D. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3229907 · Cited by 21 · see citing works
Comparative Study of Physics Engines for Robot Simulation with Mechanical Interaction
Yoon, J., Son, B., & Lee, D. (2023) · Applied Sciences · doi:10.3390/app13020680 · Cited by 22 · see citing works
BASE: Bridging the Gap between Cost and Latency for Query Optimization
Xu, C., Wang, Z., Liu, S., Li, Y., Zeng, K., Ding, B., Zhou, J., Su, H., & Zheng, K. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3594512.3594525 · Cited by 22 · see citing works
- Authors
- Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S.
- Affiliations
- Google (United States) (US)
- Everyday Robots, X, The Moonshot Factory, Mountain View, CA, USA
- Corresponding author
- Ibarz, Julian
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2021-01-31
- Volume
- 40
- Issue
- 4-5
- Pages
- 698-721
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W3126321819
- Cited by
- 570
- References
- 193
- Citations by year
- 2026: 63 · 2025: 123 · 2024: 145 · 2023: 112 · 2022: 89 · 2021: 38
- FWCI
- 33.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- The International Journal of Robotics Research (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, robot learning, embodied agents, physical robots, robot skill learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
