The DOI matches its Crossref record; 184 of the 193 references it cites were found in a registry or an index.
How to train your robot with deep reinforcement learning: lessons we have learned
Open accessAbstract
Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“POMDPs more accurately describe real-world applications of robotic systems [7] and are a major tool for training modern DRL and DIL end-to-end policy controllers.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, DRL is data-hungry and trained through trial and error which requires human involvement for constantly resetting the environment and preventing unsafe actions [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“(4) high-dimensional/continuous state settings exacerbate the challenge of exploration and further affect data efficiency.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We can also utilise demonstration data to further reduce the distribution shift, as it causes the model to learn the important part of the state space [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In recent years, data-driven approaches with deep learning (DL) [6], such as deep reinforcement learning (DRL) [7] and deep imitation learning (DIL) [8], have demonstrated many advantages in performing task-level and skill-level control problems (Section 3.3).”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“(3) DRL algorithms often come with a set of hyper-parameters, and the algorithms are often sensitive to these hyper-parameters and the application domain”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“(2) data efficiency (both sampling and learning efficiency) is a major concern for such complex settings”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“MBRL is generally more data-efficient than MFRL algorithms and it generalises better with large and diverse data [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“(1) theoretically, we have no guarantee of convergence of the DRL algorithms”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,043
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881
(2017) · Cited by 11,625
Research works printed but not linked: 61
- Achiam J, 2017, International Conference on Machine Learning
- Agarwal R, 2020, International Conference on Machine Learning
- Andrychowicz M, 2017, Advances in Neural Information Processing Systems, 5048
- Burda Y, 2019, International Conference on Learning Representations
- Cabi S, 2019, arXiv preprint arXiv:1909.12200 · arXiv
- Chebotar Y, 2017, International Conference on Machine Learning, 703
- Chen Z, 2018, International Conference on Machine Learning
- Clavera I, 2019, International Conference on Learning Representations
- Deisenroth M, 2011, International Conference on Machine Learning, 465
- Deisenroth MP, 2013, Foundations and Trends in Robotics, 2, 1
- Devlin J, 2019, NAACL-HLT
- Duan Y, 2017, Advances in Neural Information Processing Systems, 1087
- Eysenbach B, 2018, International Conference on Learning Representations
- Finn C, 2017, International Conference on Machine Learning
- Finn C, 2016, International Conference on Machine Learning, 49
- Finn C, 2017, Proceedings of Machine Learning Research, 78, 357
- Fox R, 2016, Conference on Uncertainty in Artificial Intelligence
- Fu J, 2017, Advances in Neural Information Processing Systems, 2577
- Fujimoto S, 2019, International Conference on Machine Learning
- Fujimoto S, 2018, International Conference on Machine Learning
- Garofolo JS, 1993, NASA STI/Recon Technical Report, 93
- Goodfellow I, 2014, Advances in Neural Information Processing Systems, 2672
- Gregor K, 2017, International Conference on Learning Representations, Workshop Track Proceedings
- Gu S, 2016, International Conference on Machine Learning, 2829
- Haarnoja T, 2017, International Conference on Machine Learning, 1352
- Haarnoja T, 2018, International Conference on Machine Learning
- Hausman K, 2017, AAAI Symposium on Interactive Multi-Sensory Object Perception for Embodied Agents
- Iscen A, 2018, Conference on Robot Learning
- Jabri A, 2019, Advances in Neural Information Processing Systems, 10519
- James S, 2018, Conference on Robot Learning
- James S, 2017, Conference on Robot Learning
- Kalashnikov D, 2018, Conference on Robot Learning, Proceedings of Machine Learning Research
- Khadka S, 2019, International Conference on Machine Learning
- Kroemer O, 2019, CoRR abs/1907.03146 · arXiv
- Kumar A, 2019, Advances in Neural Information Processing Systems, 32
- Kurutach T, 2018, International Conference on Learning Representations
- Levine S, 2014, Advances in Neural Information Processing Systems, 1071
- Levine S, 2016, The Journal of Machine Learning Research, 17, 1334
- Levine S, 2013, International Conference on Machine Learning
- Mania H, 2018, Advances in Neural Information Processing Systems
- Montgomery WH, 2016, Advances in Neural Information Processing Systems, 4008
- Nagabandi A, 2020, Conference on Robot Learning
- Osband I, 2016, Advances in Neural Information Processing Systems, 4026
- Parisotto E, 2016, CoRR
- Paszke A, 2017, Advances in Neural Information Processing Systems Workshop on Autodiff
- Pinto L, 2017, International Conference on Machine Learning
- Rakelly K, 2019, International Conference on Machine Learning
- Riedmiller M, 2018, International Conference on Machine Learning
- Schaul T, 2019, Multidisciplinary Conference on Reinforcement Learning and Decision Making
- Schoettler G, 2019, International Conference on Intelligent Robots and Systems
- Schulman J, 2015, International Conference on Machine Learning
- Sener O, 2018, Advances in Neural Information Processing Systems, 527
- Tedrake R, 2015, Workshop on Adaptive and Learning Systems
- Viereck U, 2017, Conference on Robot Learning
- Wu YH, 2019, International Conference on Machine Learning, Proceedings of Machine Learning Research
- Xiao T, 2020, International Conference on Learning Representations
- Xie A, 2019, arXiv preprint arXiv:1904.05538 · arXiv
- Xie A, 2018, Proceedings of Machine Learning Research, 87, 40
- Yang Y, 2019, AAMAS
- Yang Y, 2020, Conference on Robot Learning
- Yen-Chen L, 2020, Conference on Robot Learning
Cited by
Sorted by citations
Items 201–300 of 569
Better value estimation in Q-learning-based multi-agent reinforcement learning
Ding, L., Du, W., Zhang, J., Guo, L., Zhang, C., Jin, D., & Ding, S. (2023) · Soft Computing · doi:10.1007/s00500-023-09365-5 · Cited by 7
Luo, Y., Li, W., Wang, P., Duan, H., Wei, W., & Sun, J. (2024) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2024.3406730 · Cited by 7
Task and Motion Planning for Execution in the Real
Pan, T., Shome, R., & Kavraki, L. E. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3418550 · Cited by 7
Anticipatory Control on Human-Following Robots Using Online Deep-Model Predictive Control
Gui, S., & Luximon, Y. (2024) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2024.3419209 · Cited by 7
Sim-to-Real Transfer for a Robotics Task: Challenges and Lessons Learned
Rothert, J. J., Lang, S., Seidel, M., & Hanses, M. (2024) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa61755.2024.10711073 · Cited by 7
Dong, Q., Zeng, P., He, Y., Wan, G., & Dong, X. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3487484 · Cited by 7
Martínez, D., Riazuelo, L., & Montano, L. (2025) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2025.105020 · Cited by 9
Curriculum-guided skill learning for long-horizon robot manipulation tasks
Alves, J., Lau, N., & da Silva, F. M. T. P. (2025) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2025.105032 · Cited by 7
Chemotactic navigation in robotic swimmers via reset-free hierarchical reinforcement learning
Xiong, T., Liu, Z., Wang, Y., Ong, C. J., & Zhu, L. (2025) · Nature Communications · doi:10.1038/s41467-025-60646-z · Cited by 11
Lindemann, L., Zhao, Y., Yu, X., Pappas, G. J., & Deshmukh, J. V. (2025) · IEEE Control Systems · doi:10.1109/mcs.2025.3611545 · Cited by 66
Improvements in learning to control perched landings
Fletcher, L., Clarke, R., Richardson, T. S., & Hansen, M. (2022) · The Aeronautical Journal · doi:10.1017/aer.2022.48 · Cited by 7
Towards designing a generic and comprehensive deep reinforcement learning framework
Nguyen, N. D., Nguyen, T. T., Pham, N. T., Nguyen, V., Nguyen, D. T., Nguyen, T. T., Lim, C. P., Johnstone, M., Bhatti, A., Creighton, D., et al. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03550-z · Cited by 7
Extensible Hierarchical Multi-Agent Reinforcement-Learning Algorithm in Traffic Signal Control
Zhao, P., Yuan, Y., & Guo, T. (2022) · Applied Sciences · doi:10.3390/app122412783 · Cited by 7
Impact-Angle Constraint Guidance and Control Strategies Based on Deep Reinforcement Learning
Fan, J., Dou, D., & Ji, Y. (2023) · Aerospace · doi:10.3390/aerospace10110954 · Cited by 7
Sultana, S., Alam, M. M., Su’ud, M. M., Mustapha, J. C., & Prasad, M. (2024) · ACM Computing Surveys · doi:10.1145/3648357 · Cited by 8
Machine Learning Techniques for Vibration-Based Condition Monitoring-A Review
Shehata, A., & Mohammed, O. D. (2024) · Prognostics and System Health Management Conference · doi:10.1109/phm61473.2024.00049 · Cited by 7
Prediction of Robot Grasp Robustness using Artificial Intelligence Algorithms
Šegota, S. B., Anđelić, N., Car, Z., & Šercer, M. (2021) · Tehnicki vjesnik - Technical Gazette · doi:10.17559/tv-20210204092154 · Cited by 7
A Framework and Algorithm for Human-Robot Collaboration Based on Multimodal Reinforcement Learning
Cai, Z., Feng, Z., Zhou, L., Ai, C., Shao, H., & Yang, X. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/2341898 · Cited by 7
Traffic Signal Control with Successor Feature-Based Deep Reinforcement Learning Agent
Szőke, L., Aradi, S., & Bécsi, T. (2023) · Electronics · doi:10.3390/electronics12061442 · Cited by 7
Neutrons Sensitivity of Deep Reinforcement Learning Policies on EdgeAI Accelerators
Bodmann, P. R., Saveriano, M., Kritikakou, A. S., & Rech, P. (2024) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2024.3387087 · Cited by 11
Learning intraoperative organ manipulation with context-based reinforcement learning
D’Ettorre, C., Zirino, S., Dei, N. N., Stilli, A., Momi, E. D., & Stoyanov, D. V. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02630-2 · Cited by 7
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications
Liu, P., Ammar, H. B., Peters, J., & Tateo, D. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3567477 · Cited by 6
Human-AI collaboration in real-world complex environment with reinforcement learning
Islam, M. S., Das, S., Gottipati, S. K., Duguay, W., Mars, C., Arabneydi, J., Fagette, A., Guzdial, M., & Taylor, M. E. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11288-1 · Cited by 11
AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning
Alegre, L. N., Serifi, A., Grandia, R., Müller, D., Knoop, E., & Bächer, M. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730656 · Cited by 6
Manoharan, G., & Lee, H. (2025) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2025.3602899 · Cited by 6
Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion
Grossman, L., & Plancher, B. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160293 · Cited by 6
Real-Time Model-Free Deep Reinforcement Learning for Force Control of a Series Elastic Actuator
Sambhus, R., Gokce, A., Welch, S., Herron, C. W., & Leonessa, A. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341751 · Cited by 6
Robots in manufacturing: Programming, control, and safety standards
Lakshminarayanan, S., Kana, S., Bernabé, A. D. S., Turlapati, S. H., Accoto, D., & Campolo, D. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-13812-6.00011-7 · Cited by 6
Machine Learning-Driven Burrowing with a Snake-Like Robot
Even, S., Gordon, H., Yang, H., & Ozkan-Aydin, Y. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610264 · Cited by 6
Ye, M., Ma, H., Ren, Y., Zhang, C., Haidn, O. J., & Hu, X. (2025) · Engineering Applications of Computational Fluid Mechanics · doi:10.1080/19942060.2025.2460677 · Cited by 6
Quantifying the Sim2Real Gap: Model-Based Verification and Validation in Autonomous Ground Systems
Waheed, A., Areti, M., Gallantree, L., & Hasnain, Z. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3546126 · Cited by 6
Non-blocking Asynchronous Training for Reinforcement Learning in Real-World Environments
Böhm, P., Pounds, P., & Chapman, A. C. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981333 · Cited by 6
Q-learning based traffic signal control method for an isolated intersection
Ye, B., Wu, P., Wu, W., Li, L., Zhu, Y., & Chen, B. (2022) · China Automation Congress (CAC) · doi:10.1109/cac57257.2022.10054839 · Cited by 6
Double Graph Attention Networks for Visual Semantic Navigation
Lyu, Y., & Talebi, M. S. (2023) · Neural Processing Letters · doi:10.1007/s11063-023-11190-8 · Cited by 6
Irnawan, R., Ataka, A., Yasirroni, M., Putranto, L. M., Ali, H. R., Firmansyah, E., & Sarjiya, S. (2023) · Energies · doi:10.3390/en16145369 · Cited by 6
LAGOON: Language-Guided Motion Control
Xu, S., Wang, H., Ouyang, Y., Gao, J., Mei, Z., Yu, C., & Wu, Y. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610467 · Cited by 6
Sun, P., Yang, Z., Zhang, T., Guo, S., & Chen, F. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02527-8 · Cited by 6
Application of DQN Learning for Delayed Output Feedback Control of a Gait-Assist Hip Exoskeleton
Kalani, H., Tahamipour-Z., S. M., Kardan, I., & Akbarzadeh, A. (2021) · RSI International Conference on Robotics and Mechatronics (ICRoM) · doi:10.1109/icrom54204.2021.9663497 · Cited by 6
One-shot learning-driven autonomous robotic assembly via human-robot symbiotic interaction
Liu, Q., Ji, Z., Xu, W., Liu, Z., & Wang, L. (2025) · npj Advanced Manufacturing · doi:10.1038/s44334-025-00030-3 · Cited by 5
Lee, H., & Jeong, J. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1210442 · Cited by 5
Value-Informed Skill Chaining for Policy Learning of Long-Horizon Tasks with Surgical Robot
Huang, T., Chen, K., Wei, W., Li, J., Long, Y., & Dou, Q. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342180 · Cited by 5
Reinforcement Learning-Based Collision Avoidance of Kinova Gen3 Robot for Ball Balancing Task
Kadam, A. C., & Patil, S. M. (2024) · International Research Journal of Multidisciplinary Scope · doi:10.47857/irjms.2024.v05i01.0213 · Cited by 5
MFRLMO: Model-free reinforcement learning for multi-objective optimization of apache spark
Öztürk, M. M. (2024) · ICST Transactions on Scalable Information Systems · doi:10.4108/eetsis.4764 · Cited by 5
Data-driven control of nonlinear systems: An online sequential approach
Vu, M., Huang, Y., & Zeng, S. (2024) · Systems & Control Letters · doi:10.1016/j.sysconle.2024.105932 · Cited by 5
Intrinsically Motivated Self-supervised Learning in Reinforcement Learning
Zhao, Y., Du, C., Zhao, H., & Li, T. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812213 · Cited by 5
Template-Based Category-Agnostic Instance Detection for Robotic Manipulation
Hu, Z., Tan, R., Zhou, Y., Woon, J., & Lv, C. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3219021 · Cited by 5
Fast suction-grasp-difficulty estimation for high throughput plastic-waste sorting
Um, S., Kim, K., & Kim, S. (2023) · Journal of Mechanical Science and Technology · doi:10.1007/s12206-023-0135-0 · Cited by 5
Vision-Based High-Precision Assembly with Force Feedback
Chen, Y., Yu, J., Shen, L., Lin, Z., & Liu, Z. (2023) · International Conference on Control, Automation and Robotics (ICCAR) · doi:10.1109/iccar57134.2023.10151762 · Cited by 5
Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning
George, A., Bartsch, A., & Farimani, A. B. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161119 · Cited by 11
Long-Range Navigation in Complex and Dynamic Environments with Full-Stack S-DOVS
Martínez, D., Riazuelo, L., & Montano, L. (2023) · Applied Sciences · doi:10.3390/app13158925 · Cited by 5
Kandaswamy, A., & Schwaminger, S. P. (2024) · Crystals · doi:10.3390/cryst14060501 · Cited by 5
Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning
Jiang, K., Liu, W., Wang, Y., Dong, L., & Sun, C. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124281 · Cited by 5
Otto—Design and Control of an 8-DoF SEA-Driven Quadrupedal Robot
Scaldaferri, A., Tolomei, S., Iotti, F., Gambino, P., Pierallini, M., Angelini, F., & Garabini, M. (2025) · IEEE Open Journal of the Industrial Electronics Society · doi:10.1109/ojies.2025.3567112 · Cited by 5
Learning Robot Trajectories subject to Kinematic Joint Constraints
Kiemel, J. C., & Kröger, T. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561159 · Cited by 5
Learning to Climb: Constrained Contextual Bayesian Optimisation on a Multi-Modal Legged Robot
Chen, Y., Cao, J., & Rosendo, A. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3192798 · Cited by 5
Accelerated Continuous-Time Approximate Dynamic Programming via Data-Assisted Hybrid Control
Ochoa, D. E., & Poveda, J. I. (2022) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2022.07.371 · Cited by 5
Towards Robot-Aided Diver Navigation in Mapped Environments (ROADMAP)
Nađ, Đ., Ferreira, F., Kvasić, I., Mandić, L., Slošić, V., Walker, C. R., Orbaugh, D., & Anderson, I. A. (2022) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.1109/oceans47191.2022.9977173 · Cited by 5
Set-valued rigid-body dynamics for simultaneous, inelastic, frictional impacts
Halm, M., & Posa, M. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241236860 · Cited by 5
Survey on Multimodal Transformers for RobotsPreprint
Miyazawa, K., & Nagai, T. (2023) · doi:10.36227/techrxiv.21993317 · Cited by 5
Decentralised adaptive learning-based control of robot manipulators with unknown parameters
Sveen, E. M., Zhou, J., Ebbesen, M. K., & Poursina, M. (2025) · Journal of Automation and Intelligence · doi:10.1016/j.jai.2025.02.002 · Cited by 4
Cooperative Bearing-Only Target Pursuit via Multiagent Reinforcement Learning: Design and Experiment
Li, J., Wang, Z., Ding, S., Guo, S., & Zhao, S. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247029 · Cited by 4
Autonomous navigation of intelligent microrobotic swarms in unknown environments
An, X., Luo, S., Zhang, H., Yang, Q., Ma, Y., Wang, B., Du, J., & Wang, Q. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01252-6 · Cited by 11
Navigation of Tendon-driven Flexible Robotic Endoscope through Deep Reinforcement Learning
Ng, C. K., Gao, H., Ren, T.-A., Lai, J., & Ren, H. (2024) · IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO) · doi:10.1109/arso60199.2024.10557794 · Cited by 5
Optimal Dynamic Regimes for CO Oxidation Discovered by Reinforcement Learning
Lifar, M., Tereshchenko, A. A., Bulgakov, A. N., Guda, S. A., Guda, A. A., & Soldatov, A. V. (2024) · ACS Omega · doi:10.1021/acsomega.3c10422 · Cited by 4
SwarmRL: building the future of smart active systems
Tovey, S., Lohrmann, C., Merkt, T., Zimmer, D., Nikolaou, K., Koppenhöfer, S., Bushmakina, A., Scheunemann, J., & Holm, C. (2025) · The European Physical Journal E · doi:10.1140/epje/s10189-025-00477-4 · Cited by 9
Adversarial environment design for crowd navigation based on deep reinforcement learning
Kim, J., Hwang, H.-S., & Seok, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111621 · Cited by 5
Design and Deployment of Swarm Engineering Systems: Insights, Challenges, and Innovations
Kazim, R. M., Wang, G., Ming, Z., Allen, J. K., & Mistree, F. (2025) · IEEE Access · doi:10.1109/access.2025.3592950 · Cited by 4
Q-learning with Long-term Action-space Shaping to Model Complex Behavior for Autonomous Lane Changes
Kalweit, G., Hügle, M., Werling, M., & Boedecker, J. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636668 · Cited by 4
2-D Air Combat Maneuver Decision Using Reinforcement Learning
Taşbaş, A. S., & Aydinli, S. U. (2021) · International Conference on Engineering and Emerging Technologies (ICEET) · doi:10.1109/iceet53442.2021.9659753 · Cited by 4
Fletcher, L., Clarke, R. J., Richardson, T. S., & Hansen, M. (2022) · AIAA SciTech Forum · doi:10.2514/6.2022-1288 · Cited by 4
Online beam orbit correction of MEBT in CiADS based on multi-agent reinforcement learning algorithm
Yang, X., Chen, Y., Wang, J., Zheng, H., Liu, H., Zhou, D., He, Y., Wang, Z., & Zhou, Q. (2022) · Annals of Nuclear Energy · doi:10.1016/j.anucene.2022.109346 · Cited by 4
Igoe, C., Pande, S., Venkatraman, S., & Schneider, J. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161395 · Cited by 4
Bálint, B. A., Lämmle, A., Salteris, Y., & Tenbrock, P. G. (2023) · Procedia CIRP · doi:10.1016/j.procir.2023.03.149 · Cited by 4
The Role of Time Delay in Sim2real Transfer of Reinforcement Learning for Unmanned Aerial Vehicles
Elocla, N. M., Chehadeh, M. S., Boiko, I. M., Swei, S. S.-M., & Zweiri, Y. (2023) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar58858.2023.10406926 · Cited by 4
Liang, Z., Zhao, K., Xie, J., & Zhang, Z. (2024) · ISA Transactions · doi:10.1016/j.isatra.2024.04.029 · Cited by 6
Kins, R., Möbitz, C., & Gries, T. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-024-02405-3 · Cited by 4
Santelices, I. B., Landry, C., Arami, A., & Peterson, S. D. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3423698 · Cited by 5
Abbatematteo, B., Rosen, E., Thompson, S., Akbulut, T., Rammohan, S., & Konidaris, G. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610846 · Cited by 4
Liu, X., Wang, S., Li, X., & Cui, Z. (2024) · Symmetry · doi:10.3390/sym16091227 · Cited by 4
Liu, Y., Jiang, X. Y., Li, X., Sun, B., Qian, S., & Wu, Y. (2024) · Proceedings of the Institution of Mechanical Engineers Part I Journal of Systems and Control Engineering · doi:10.1177/09596518241274000 · Cited by 4
The Design of Sports Games under the Internet of Things Fitness by Deep Reinforcement Learning
Wang, X., Liu, C., & Sun, L. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/4623561 · Cited by 4
RetractedThe pigeon as a machine: Complex category structures can be acquired by a simple associative model
Turner, B. M., & Wasserman, E. A. (2023) · iScience · doi:10.1016/j.isci.2023.107998 · Cited by 4
Zheng, Y., Tao, J., Sun, Q., Sun, H., Chen, Z., & Sun, M. (2023) · Sustainability · doi:10.3390/su151914452 · Cited by 4
Ahmad, M. F. (2023) · International Journal of Information Technology · doi:10.1007/s41870-023-01507-0 · Cited by 4
Optical Tactile Sim-to-Real Policy Transfer via Real-to-Sim Tactile Image Translation.Preprint
Church, A., Lloyd, J. W., Hadsell, R. T., & Lepora, N. F. (2021) · arXiv (Cornell University) · Cited by 4
Luis, J. J. G., Crawley, E. F., & Cameron, B. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.03015 · Cited by 4
What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural NetworksPreprint
Luck, K. S., Calandra, R., & Mistry, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02371 · Cited by 4
Artificial intelligent focusing of a microbeam system based on reinforcement learning
Li, Y., Mou, H., Zhang, W., Guo, J., An, S., Du, G., & Liu, X. (2025) · The European Physical Journal Plus · doi:10.1140/epjp/s13360-025-06221-3 · Cited by 3
Zhao, Y., & Pei, Y. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.114111 · Cited by 3
Machine Learning Achieves Accurate and Smooth Navigation Control for Microrobots
Li, Y., Ge, R., Wang, Y., Yangmin, L. I., & Yang, L. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3650946 · Cited by 3
Deep Reinforcement Learning for Temperature Control of a Two-Way SMA-Actuated Tendon-Driven Gripper
Thien, P., Le, Q. N., Park, H., Kim, H., Shim, S., Park, K., & Kim, Y. (2026) · Actuators · doi:10.3390/act15010037 · Cited by 3
AI without borders: The rise of cross-disciplinary machine learning
Wibawa, A. P., Herdianto, R., Handayani, A. N., Utama, A. B. P., Aripriharta, Dwiyanto, F. A., & Dreżewski, R. (2026) · Telematics and Informatics Reports · doi:10.1016/j.teler.2026.100294 · Cited by 4
He, Q., Li, W., Dai, G., Chen, H., Liu, Q., Tian, X., You, J., Cui, W., Triantafyllou, M. S., & Fan, D. (2026) · Soft Robotics · doi:10.1177/21695172261417752 · Cited by 3
Zhang, X., Qi, Z., Ma, L., & Zhang, G. (2026) · Journal of Knowledge Management · doi:10.1108/jkm-10-2024-1249 · Cited by 5
Method of Controlling a Tankendo Robot Using a Mecanum Wheel Trolley Robot
Zhao, C., Shibuya, T., Igo, N., & Hoshino, K. (2022) · International Conference on Intelligent Information Technology (ICIIT) · doi:10.1145/3524889.3524910 · Cited by 3
Design and Research of Robot Image Perception System Based on Artificial Intelligence
Guo, L., Cheng, K., & Jia, L. (2023) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2493/1/012014 · Cited by 3
Inverted Pendulum Control: A Comparative Study from Conventional Control to Reinforcement Learning
Ataka, A., Sandiwan, A. P., Tnunay, H., Utomo, D. R., & Cahyadi, A. I. (2023) · Jurnal Nasional Teknik Elektro dan Teknologi Informasi (JNTETI) · doi:10.22146/jnteti.v12i3.7065 · Cited by 3
Baykal, Y., & Başpınar, B. (2023) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc58513.2023.10311295 · Cited by 3
Mandić, L., Mišković, N., & Nađ, Đ. (2023) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.10.464 · Cited by 3
Metadata
- Authors
- Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S.
- Affiliations
- Google (United States) (US)
- Everyday Robots, X, The Moonshot Factory, Mountain View, CA, USA
- Corresponding author
- Ibarz, Julian
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2021-01-31
- Volume
- 40
- Issue
- 4-5
- Pages
- 698-721
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W3126321819
- Cited by
- 570
- References
- 193
- Citations by year
- 2026: 63 · 2025: 123 · 2024: 145 · 2023: 112 · 2022: 89 · 2021: 38
- FWCI
- 33.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- The International Journal of Robotics Research (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, robot learning, embodied agents, physical robots, robot skill learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
