The DOI matches its Crossref record; 184 of the 193 references it cites were found in a registry or an index.
How to train your robot with deep reinforcement learning: lessons we have learned
Open accessAbstract
Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“POMDPs more accurately describe real-world applications of robotic systems [7] and are a major tool for training modern DRL and DIL end-to-end policy controllers.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, DRL is data-hungry and trained through trial and error which requires human involvement for constantly resetting the environment and preventing unsafe actions [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“(4) high-dimensional/continuous state settings exacerbate the challenge of exploration and further affect data efficiency.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We can also utilise demonstration data to further reduce the distribution shift, as it causes the model to learn the important part of the state space [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In recent years, data-driven approaches with deep learning (DL) [6], such as deep reinforcement learning (DRL) [7] and deep imitation learning (DIL) [8], have demonstrated many advantages in performing task-level and skill-level control problems (Section 3.3).”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“(3) DRL algorithms often come with a set of hyper-parameters, and the algorithms are often sensitive to these hyper-parameters and the application domain”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“(2) data efficiency (both sampling and learning efficiency) is a major concern for such complex settings”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“MBRL is generally more data-efficient than MFRL algorithms and it generalises better with large and diverse data [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“(1) theoretically, we have no guarantee of convergence of the DRL algorithms”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,043
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881
(2017) · Cited by 11,625
Research works printed but not linked: 61
- Achiam J, 2017, International Conference on Machine Learning
- Agarwal R, 2020, International Conference on Machine Learning
- Andrychowicz M, 2017, Advances in Neural Information Processing Systems, 5048
- Burda Y, 2019, International Conference on Learning Representations
- Cabi S, 2019, arXiv preprint arXiv:1909.12200 · arXiv
- Chebotar Y, 2017, International Conference on Machine Learning, 703
- Chen Z, 2018, International Conference on Machine Learning
- Clavera I, 2019, International Conference on Learning Representations
- Deisenroth M, 2011, International Conference on Machine Learning, 465
- Deisenroth MP, 2013, Foundations and Trends in Robotics, 2, 1
- Devlin J, 2019, NAACL-HLT
- Duan Y, 2017, Advances in Neural Information Processing Systems, 1087
- Eysenbach B, 2018, International Conference on Learning Representations
- Finn C, 2017, International Conference on Machine Learning
- Finn C, 2016, International Conference on Machine Learning, 49
- Finn C, 2017, Proceedings of Machine Learning Research, 78, 357
- Fox R, 2016, Conference on Uncertainty in Artificial Intelligence
- Fu J, 2017, Advances in Neural Information Processing Systems, 2577
- Fujimoto S, 2019, International Conference on Machine Learning
- Fujimoto S, 2018, International Conference on Machine Learning
- Garofolo JS, 1993, NASA STI/Recon Technical Report, 93
- Goodfellow I, 2014, Advances in Neural Information Processing Systems, 2672
- Gregor K, 2017, International Conference on Learning Representations, Workshop Track Proceedings
- Gu S, 2016, International Conference on Machine Learning, 2829
- Haarnoja T, 2017, International Conference on Machine Learning, 1352
- Haarnoja T, 2018, International Conference on Machine Learning
- Hausman K, 2017, AAAI Symposium on Interactive Multi-Sensory Object Perception for Embodied Agents
- Iscen A, 2018, Conference on Robot Learning
- Jabri A, 2019, Advances in Neural Information Processing Systems, 10519
- James S, 2018, Conference on Robot Learning
- James S, 2017, Conference on Robot Learning
- Kalashnikov D, 2018, Conference on Robot Learning, Proceedings of Machine Learning Research
- Khadka S, 2019, International Conference on Machine Learning
- Kroemer O, 2019, CoRR abs/1907.03146 · arXiv
- Kumar A, 2019, Advances in Neural Information Processing Systems, 32
- Kurutach T, 2018, International Conference on Learning Representations
- Levine S, 2014, Advances in Neural Information Processing Systems, 1071
- Levine S, 2016, The Journal of Machine Learning Research, 17, 1334
- Levine S, 2013, International Conference on Machine Learning
- Mania H, 2018, Advances in Neural Information Processing Systems
- Montgomery WH, 2016, Advances in Neural Information Processing Systems, 4008
- Nagabandi A, 2020, Conference on Robot Learning
- Osband I, 2016, Advances in Neural Information Processing Systems, 4026
- Parisotto E, 2016, CoRR
- Paszke A, 2017, Advances in Neural Information Processing Systems Workshop on Autodiff
- Pinto L, 2017, International Conference on Machine Learning
- Rakelly K, 2019, International Conference on Machine Learning
- Riedmiller M, 2018, International Conference on Machine Learning
- Schaul T, 2019, Multidisciplinary Conference on Reinforcement Learning and Decision Making
- Schoettler G, 2019, International Conference on Intelligent Robots and Systems
- Schulman J, 2015, International Conference on Machine Learning
- Sener O, 2018, Advances in Neural Information Processing Systems, 527
- Tedrake R, 2015, Workshop on Adaptive and Learning Systems
- Viereck U, 2017, Conference on Robot Learning
- Wu YH, 2019, International Conference on Machine Learning, Proceedings of Machine Learning Research
- Xiao T, 2020, International Conference on Learning Representations
- Xie A, 2019, arXiv preprint arXiv:1904.05538 · arXiv
- Xie A, 2018, Proceedings of Machine Learning Research, 87, 40
- Yang Y, 2019, AAMAS
- Yang Y, 2020, Conference on Robot Learning
- Yen-Chen L, 2020, Conference on Robot Learning
Cited by
Sorted by citations
Items 101–200 of 569
A human-centered safe robot reinforcement learning framework with interactive behaviors
Gu, S., Kshirsagar, A., Du, Y., Chen, G., Peters, J., & Knoll, A. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1280341 · Cited by 29
Cooperative Object Transportation Using Curriculum-Based Deep Reinforcement Learning
Eoh, G., & Park, T.-H. (2021) · Sensors · doi:10.3390/s21144780 · Cited by 22
Colonoscopy Navigation using End-to-End Deep Visuomotor Control: A User Study
Pore, A., Finocchiaro, M., Dall’Alba, D., Hernansanz, A., Ciuti, G., Arezzo, A., Menciassi, A., Casals, A., & Fiorini, P. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981480 · Cited by 23
A Learning-Based Two-Stage Method for Submillimeter Insertion Tasks With Only Visual Inputs
Zhao, J., Wang, Z., Zhao, L., & Liu, H. (2023) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2023.3299051 · Cited by 23
Axis-space framework for cable-driven soft continuum robot control via reinforcement learning
Wei, D., Zhou, J., Zhu, Y., Ma, J., & Ma, S. (2023) · Communications Engineering · doi:10.1038/s44172-023-00110-2 · Cited by 21
Digital evolution and twin miracle of sugarcane breeding
Wang, X., Wu, Q., Zeng, H., Xu, Y., Yang, X., Yi, X., Khalil, I., & Que, Y. (2024) · Field Crops Research · doi:10.1016/j.fcr.2024.109588 · Cited by 25
Wen, J., Li, Z., Xi, M., & He, J. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3540820 · Cited by 20
Zhang, D., Liang, J., Guo, K., Lu, S., Wang, Q., Xiong, R., Miao, Z., & Wang, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01607 · Cited by 20
Robust Deep Reinforcement Learning for Quadcopter Control
Deshpande, A. M., Minai, A. A., & Kumar, M. (2021) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2021.11.158 · Cited by 20
Hu, W., Shao, J., Jiao, Q., Wang, C., Cheng, J., Liu, Z., & Tan, J. (2022) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-022-01971-8 · Cited by 20
Self-citationRecent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspective
Yang, X., Ji, Z., Wu, J., & Lai, Y. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3277288 · Cited by 24
Few-Shot Video Classification via Representation Fusion and Promotion Learning
Xia, H., Li, K., Min, M. R., & Ding, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01769 · Cited by 20
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 27
“We can also utilise demonstration data to further reduce the distribution shift, as it causes the model to learn the important part of the state space [7].”
BackgroundMethodInfluentialHou, Z., Yang, W., Chen, R., Feng, P., & Xu, J. L. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3192435 · Cited by 19
Deep reinforcement learning for real-world quadrupedal locomotion: a comprehensive review
Zhang, H., Li, H., & Wang, D. (2022) · Intelligence & Robotics · doi:10.20517/ir.2022.20 · Cited by 22
Oliveira, P., Carvalho, J. M. S., & Faria, S. M. P. S. (2025) · Information · doi:10.3390/info16090764 · Cited by 29
Learning High Speed Precision Table Tennis on a Physical Robot
Ding, T., Graesser, L., Abeyruwan, S., D'Ambrosio, D. B., Shankar, A., Sermanet, P., Sanketi, P. R., & Lynch, C. (2022) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros47612.2022.9982205 · Cited by 31
Examining the simulation-to-reality gap of a wheel loader digging in deformable terrain
Aoshima, K., & Servin, M. (2024) · Multibody System Dynamics · doi:10.1007/s11044-024-10005-5 · Cited by 18
UAV Swarm Objectives: A Critical Analysis and Comprehensive Review
Kumar, P. A., Manoj, N., Sudheer, N., Bhat, P. P., Arya, A., & Sharma, R. (2024) · SN Computer Science · doi:10.1007/s42979-024-03156-x · Cited by 18
GraspLDM: Generative 6-DoF Grasp Synthesis Using Latent Diffusion Models
Barad, K. R., Orsula, A., Richard, A., Dentler, J., Olivares-Méndez, M. A., & Martínez, C. (2024) · IEEE Access · doi:10.1109/access.2024.3492118 · Cited by 23
Learning to Centralize Dual-Arm Assembly
Alles, M., & Aljalbout, E. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.830007 · Cited by 17
Dexterous manipulation of construction tools using anthropomorphic robotic hand
Huang, L., Cai, W., Zhu, Z., & Zou, Z. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.105133 · Cited by 19
Comprehensive Review of Robotics Operating System-Based Reinforcement Learning in Robotics
Aljamal, M., Patel, S. H., & Mahmood, A. (2025) · Applied Sciences · doi:10.3390/app15041840 · Cited by 21
Smart quadruped robotics: a systematic review of design, control, sensing and perception
Hamrani, A., Rayhan, M. M., Telusma, M., McDaniel, D., & Lagos, L. E. (2024) · Advanced Robotics · doi:10.1080/01691864.2024.2411684 · Cited by 17
A big-data-driven matching model based on deep reinforcement learning for cotton blending
Xia, H., Yuan, W., Jasimuddin, S. M., Zhang, Z., & Thomas, A. J. (2022) · International Journal of Production Research · doi:10.1080/00207543.2022.2153942 · Cited by 16
Sim-to-Real Transfer of Robotic Assembly with Visual Inputs Using CycleGAN and Force Control
Yuan, C., Shi, Y., Feng, Q., Chang, C., Liu, M., Chen, Z., Knoll, A., & Zhang, J. (2022) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio55434.2022.10011878 · Cited by 16
Supervised Learning from Data Streams: An Overview and Update
Read, J., & Žliobaitė, I. (2025) · ACM Computing Surveys · doi:10.1145/3737279 · Cited by 15
Correct Me If I'm Wrong: Using Non-Experts to Repair Reinforcement Learning Policies
van Waveren, S., Pek, C., Tůmová, J., & Leite, I. (2022) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889604 · Cited by 15
DeepCPG Policies for Robot Locomotion
Deshpande, A. M., Hurd, E., Minai, A. A., & Kumar, M. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3250393 · Cited by 17
An overview of reinforcement learning techniques
Pecioski, D., Gavriloski, V., Domazetovska, S., & Ignjatovska, A. A. (2023) · Mediterranean Conference on Embedded Computing (MECO) · doi:10.1109/meco58584.2023.10155066 · Cited by 19
Materializing Autonomy in Soft Robots across Scales
Berrueta, T. A., Murphey, T. D., & Truby, R. L. (2023) · Advanced Intelligent Systems · doi:10.1002/aisy.202300111 · Cited by 15
Chen, H., Kim, M.-C., Ko, Y., & Kim, C. (2023) · International Journal of Control Automation and Systems · doi:10.1007/s12555-023-0342-6 · Cited by 15
Li, C., Kreiman, G., & Ramanathan, S. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00854-2 · Cited by 15
Sampled-data funnel control and its use for safe continual learning
Lanza, L., Dennstädt, D., Worthmann, K., Schmitz, P., Şen, G. D., Trenn, S., & Schaller, M. (2024) · Systems & Control Letters · doi:10.1016/j.sysconle.2024.105892 · Cited by 15
Pavlichenko, D., & Behnke, S. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812023 · Cited by 17
Dromnelle, R., Renaudo, E., Chétouani, M., Maragos, P. A., Chatila, R., Girard, B., & Khamassi, M. (2022) · International Journal of Social Robotics · doi:10.1007/s12369-022-00942-6 · Cited by 15
Robot Imitation Learning From Image-Only Observation Without Real-World Interaction
Xu, X., You, M., Zhou, H., Qian, Z., & He, B. (2022) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2022.3217048 · Cited by 14
Learning to Assist Bimanual Teleoperation Using Interval Type-2 Polynomial Fuzzy Inference
Wang, Z., Fei, H., Huang, Y., Rouxel, Q., Xiao, B. J., Li, Z., & Burdet, E. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3272730 · Cited by 14
Yan, S., Zhang, P., Huang, S., Wang, J., Sun, H., Zhang, Y., & Tolba, A. K. R. (2023) · Electronics · doi:10.3390/electronics12112478 · Cited by 14
Advanced Power Converters and Learning in Diverse Robotic Innovation: A Review
Singh, R., Kurukuru, V. S. B., & Khan, M. A. (2023) · Energies · doi:10.3390/en16207156 · Cited by 16
evoBOT – Design and Learning-Based Control of a Two-Wheeled Compound Inverted Pendulum Robot
Klokowski, P., Eßer, J., Gramse, N., Pschera, B., Plitt, M., Feldmeier, F., Bajpai, S., Jestel, C., Bach, N., Urbann, O., et al. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342128 · Cited by 14
Learning Soft Millirobot Multimodal Locomotion with Sim‐to‐Real Transfer
Demir, S. Ö., Tiryaki, M. E., Karacakol, A. C., & Sitti, M. (2024) · Advanced Science · doi:10.1002/advs.202308881 · Cited by 15
Kang, G., Seong, H., Lee, D., & Shim, D. H. (2024) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2024.104760 · Cited by 14
Hierarchical intrinsically motivated agent planning behavior with dreaming in grid environments
Dzhivelikian, E., Latyshev, A., Kuderov, P., & Panov, A. I. (2022) · Brain Informatics · doi:10.1186/s40708-022-00156-6 · Cited by 14
Energy grid management system with anomaly detection and Q-learning decision modules
Syu, J.-H., Srivastava, G., Fojcik, M., Cupek, R., & Lin, J. C. (2023) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2023.108639 · Cited by 14
Offline Decentralized Multi-Agent Reinforcement Learning
Jiang, J., & Lu, Z. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230390 · Cited by 15
Optimization-Based Algebraic Multigrid Coarsening Using Reinforcement LearningPreprint
Taghibakhshi, A., MacLachlan, S., Olson, L. N., & West, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01854 · Cited by 14
The Developments and Challenges Toward Dexterous and Embodied Robotic Manipulation: A Survey
Li, G., Wang, R., Xu, P., Ye, Q., & Chen, J. (2025) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2025.3642671 · Cited by 33
Liu, X., Huang, C., Zhu, H., Wang, Z., Li, J., & Cangelosi, A. (2024) · IEEE Systems Man and Cybernetics Magazine · doi:10.1109/msmc.2023.3238855 · Cited by 16
Review of Reinforcement Learning for Robotic Grasping: Analysis and Recommendations
Sekkat, H., Moutik, O., Ourabah, L., Elkari, B., Chaibi, Y., & Tchakoucht, T. A. (2023) · Statistics Optimization & Information Computing · doi:10.19139/soic-2310-5070-1797 · Cited by 17
RL-NBV: A deep reinforcement learning based next-best-view method for unknown object reconstruction
Wang, T., Xi, W., Cheng, Y., Han, H., & Yang, Y. (2024) · Pattern Recognition Letters · doi:10.1016/j.patrec.2024.05.014 · Cited by 13
Contrastive Initial State Buffer for Reinforcement Learning
Messikommer, N., Song, Y., & Scaramuzza, D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610528 · Cited by 14
Mirra, G., & Pugnale, A. (2022) · Architecture Structures and Construction · doi:10.1007/s44150-022-00035-y · Cited by 14
Controlling Fleets of Autonomous Mobile Robots with Reinforcement Learning: A Brief Survey
Wesselhöft, M., Hinckeldeyn, J., & Kreutzfeldt, J. (2022) · Robotics · doi:10.3390/robotics11050085 · Cited by 13
Planar Robot Casting with Real2Sim2Real Self-Supervised LearningPreprint
Lim, V., Huang, H., Chen, L. Y., Wang, J., Ichnowski, J., Seita, D., Laskey, M., & Goldberg, K. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04814 · Cited by 13
TACT: Humanoid Whole-Body Contact Manipulation Through Deep Imitation Learning With Tactile Modality
Murooka, M., Hoshi, T., Fukumitsu, K., Masuda, S., Hamze, M., Sasaki, T., Morisawa, M., & Yoshida, E. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3580329 · Cited by 19
Lee, J., Rahman, A., Huang, S., Smith, A. D., & Katipamula, S. (2022) · Science and Technology for the Built Environment · doi:10.1080/23744731.2022.2094729 · Cited by 13
Pacini, F., Dini, P., & Fanucci, L. (2024) · Electronics · doi:10.3390/electronics13081507 · Cited by 12
A safe reinforcement learning algorithm for supervisory control of power plants
Sun, Y., Khairy, S., Vilim, R., Hu, R., & Dave, A. J. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112312 · Cited by 16
Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion
Smith, L., Cao, Y.-H., & Levine, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610485 · Cited by 28
Self-citationReactive Long Horizon Task Execution via Visual Skill and Precondition Models
Mukherjee, S., Paxton, C., Mousavian, A., Fishman, A., Likhachev, M. A., & Fox, D. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636037 · Cited by 12
Revisiting Human-Robot Teaching and Learning Through the Lens of Human Concept Learning
Booth, S., Sharma, S., Chung, S., Shah, J., & Glassman, E. L. (2022) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889398 · Cited by 13
Manipulation Learning on Humanoid Robots
Gams, A., Petrič, T., Nemec, B., & Ude, A. (2022) · Current Robotics Reports · doi:10.1007/s43154-022-00082-9 · Cited by 11
Robot Navigation in Crowded Environments: A Reinforcement Learning Approach
Caruso, M., Regolin, E., Verdù, F. J. C., Russo, S. A., Bortolussi, L., & Seriani, S. (2023) · Machines · doi:10.3390/machines11020268 · Cited by 10
Xia, W., Lu, Y., Xu, W., & Xu, X. W. (2024) · Manufacturing Letters · doi:10.1016/j.mfglet.2024.09.151 · Cited by 11
Optimal gait design for a soft quadruped robot via multi-fidelity Bayesian optimization
Tan, K., Niu, X., Ji, Q., Feng, L., & Törngren, M. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112568 · Cited by 11
Backdoor attacks against deep reinforcement learning based traffic signal control systems
Zhang, H., Gu, J., Zhang, Z., Du, L., Zhang, Y., Ren, Y., Zhang, J., & Li, H. (2022) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-022-01434-0 · Cited by 11
Flexible Gear Assembly with Visual Servoing and Force Feedback
Ming, J., Bargmann, D., Cao, H., & Caccamo, M. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341833 · Cited by 11
Learning Thin Deformable Object Manipulation With a Multisensory Integrated Soft Hand
Zhao, C., Jiang, C., Luo, L., Yuan, S., Chen, Q., & Yu, H. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3588448 · Cited by 12
Chen, F., Chen, H., Yu, T., Wang, R., Wang, Y., Zhang, X., Li, J., Liu, K., Hai, D., Bao, X., et al. (2026) · MedComm · doi:10.1002/mco2.70597 · Cited by 11
A Reinforcement Learning-Based Strategy of Path Following for Snake Robots with an Onboard Camera
Liu, L., Guo, X., & Fang, Y. (2022) · Sensors · doi:10.3390/s22249867 · Cited by 10
Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learning
Cao, H., Theile, M., Wyrwal, F. G., & Caccamo, M. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981565 · Cited by 10
Uncertain Pose Estimation during Contact Tasks using Differentiable Contact Features
Lee, D., Lee, J., & Lee, M. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.043 · Cited by 10
Tan, W., & Huang, D. (2024) · IEEE Access · doi:10.1109/access.2024.3435949 · Cited by 10
Hierarchical Rule-Base Reduction-Based ANFIS With Online Optimization Through DDPG
Juston, M. F. R., Dekhterman, S. R., Norris, W. R., Nottage, D., & Soylemezoglu, A. (2024) · IEEE transactions on fuzzy systems · doi:10.1109/tfuzz.2024.3449147 · Cited by 9
Learning explainable task-relevant state representation for model-free deep reinforcement learning
Zhao, T., Li, G., Zhao, T., Chen, Y., Xie, N., Niu, G., & Sugiyama, M. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106741 · Cited by 11
The $H_{\infty}$ Robust Stability and Performance Conditions for Uncertain Robot Manipulators
Song, G. I., Park, H. Y., & Kim, J. H. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124701 · Cited by 10
Research on Robotic Peg-in-Hole Assembly Method Based on Variable Admittance
Zhang, S., Wang, Y., Liang, S., Han, H., Jiang, Z., & Zhang, M. (2025) · Applied Sciences · doi:10.3390/app15042143 · Cited by 10
Tang, Z., Shi, Y., & Xu, X. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3253023 · Cited by 10
MultiROS: ROS-Based Robot Simulation Environment for Concurrent Deep Reinforcement Learning
Kapukotuwa, J., Lee, B., Devine, D. M., & Qiao, Y. (2022) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case49997.2022.9926475 · Cited by 9
Human-robot interaction for extraction of robotic disassembly information
Prioli, J. P. J., & Rickli, J. L. (2023) · International Journal of Computer Integrated Manufacturing · doi:10.1080/0951192x.2023.2257667 · Cited by 10
Noh, S., Lee, W., & Myung, H. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107202 · Cited by 9
Trivedi, A., Prajapati, S., Zolotas, M., Everett, M., & Padır, T. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3585315 · Cited by 10
Variable-Stiffness Underwater Robotic Systems: A Review
Lu, P., Dong, B., Gao, X. S., Zhang, F., Song, Y. Z., Liu, Z., & Zhang, Z. (2025) · Journal of Marine Science and Engineering · doi:10.3390/jmse13091805 · Cited by 9
Bridging language and action: A survey of language-conditioned robot manipulation
Yao, X., Zhou, H., Mees, O., Meng, Y., Xiao, T. G., Bisk, Y., Oh, J., Johns, E., Shridhar, M., Shah, D., et al. (2026) · The International Journal of Robotics Research · doi:10.1177/02783649261468360 · Cited by 56
Learning Collision-free and Torque-limited Robot Trajectories based on Alternative Safe Behaviors
Kiemel, J. C., & Kröger, T. (2022) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids53995.2022.10000077 · Cited by 8
Hong, Z., Lin, Q., & Hu, B. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3376579 · Cited by 12
Burst Imaging for Light-Constrained Structure-From-Motion
Ravendran, A., Bryson, M., & Dansereau, D. G. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3137520 · Cited by 9
Leveraging control priors in deep reinforcement learning for robotics
Rana, K. (2023) · Queensland University of Technology · doi:10.5204/thesis.eprints.238632 · Cited by 9
Strangeness-driven exploration in multi-agent reinforcement learning
Kim, J.-B., Choi, H.-B., & Han, Y. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106149 · Cited by 10
Design and Development of Cost-Effective Humanoid Robots for Enhanced Human–Robot Interaction
Salem, K. M., Mohamed, M. S., ElMessmary, M. H., Ehsan, A., Elgharib, A. O., & Elshimy, H. (2025) · Automation · doi:10.3390/automation6030041 · Cited by 8
Khôi, P. B., Giang, N. T., & Tan, H. V. (2021) · Indian Journal of Science and Technology · doi:10.17485/ijst/v14i30.1030 · Cited by 9
Zhang, Y., Qiao, J., Zhang, G., Tian, H., & Li, L. (2022) · Advanced Intelligent Systems · doi:10.1002/aisy.202200162 · Cited by 8
Talha, M., Hussein, A., & Hossny, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-22695-3_51 · Cited by 8
Vazquez-Armendariz, J., Olivas-Alanis, L. H., Mahan, T., Rodriguez, C. A., Groeber, M. A., Niezgoda, S. R., Morris, J. M., Emam, H. A., Skoracki, R. J., Cao, J., et al. (2023) · Integrating materials and manufacturing innovation · doi:10.1007/s40192-023-00298-3 · Cited by 8
Lawson, D., & Qureshi, A. H. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341628 · Cited by 10
Reliability assessment of off-policy deep reinforcement learning: A benchmark for aerodynamics
Berger, S., Ramo, A. A., Guillet, V., Lahire, T., Martin, B., Jardin, T., Rachelson, E., & Bauerheim, M. (2024) · Data-Centric Engineering · doi:10.1017/dce.2023.28 · Cited by 9
Yang, S., Barlow, M., Townsend, T., Liu, X., Samarasinghe, D., Lakshika, E., Moy, G., Lynar, T., & Turnbull, B. (2023) · IEEE Access · doi:10.1109/access.2023.3287100 · Cited by 10
FPGA-Accelerated Sim-to-Real Control Policy Learning for Robotic Arms
Guo, C., & Luk, W. W. (2024) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2024.3353690 · Cited by 8
da Cruz, L. C., Sierra-Franco, C. A., Silva-Calpa, G. F. M., & Raposo, A. B. (2021) · Annals of Computer Science and Information Systems · doi:10.15439/2021f86 · Cited by 8
Metadata
- Authors
- Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S.
- Affiliations
- Google (United States) (US)
- Everyday Robots, X, The Moonshot Factory, Mountain View, CA, USA
- Corresponding author
- Ibarz, Julian
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2021-01-31
- Volume
- 40
- Issue
- 4-5
- Pages
- 698-721
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W3126321819
- Cited by
- 570
- References
- 193
- Citations by year
- 2026: 63 · 2025: 123 · 2024: 145 · 2023: 112 · 2022: 89 · 2021: 38
- FWCI
- 33.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- The International Journal of Robotics Research (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, robot learning, embodied agents, physical robots, robot skill learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
