The DOI matches its Crossref record; 184 of the 193 references it cites were found in a registry or an index.
How to train your robot with deep reinforcement learning: lessons we have learned
Open accessAbstract
Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“POMDPs more accurately describe real-world applications of robotic systems [7] and are a major tool for training modern DRL and DIL end-to-end policy controllers.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, DRL is data-hungry and trained through trial and error which requires human involvement for constantly resetting the environment and preventing unsafe actions [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“(4) high-dimensional/continuous state settings exacerbate the challenge of exploration and further affect data efficiency.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We can also utilise demonstration data to further reduce the distribution shift, as it causes the model to learn the important part of the state space [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In recent years, data-driven approaches with deep learning (DL) [6], such as deep reinforcement learning (DRL) [7] and deep imitation learning (DIL) [8], have demonstrated many advantages in performing task-level and skill-level control problems (Section 3.3).”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“(3) DRL algorithms often come with a set of hyper-parameters, and the algorithms are often sensitive to these hyper-parameters and the application domain”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“(2) data efficiency (both sampling and learning efficiency) is a major concern for such complex settings”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“MBRL is generally more data-efficient than MFRL algorithms and it generalises better with large and diverse data [7].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“(1) theoretically, we have no guarantee of convergence of the DRL algorithms”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,043
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881
(2017) · Cited by 11,625
Research works printed but not linked: 61
- Achiam J, 2017, International Conference on Machine Learning
- Agarwal R, 2020, International Conference on Machine Learning
- Andrychowicz M, 2017, Advances in Neural Information Processing Systems, 5048
- Burda Y, 2019, International Conference on Learning Representations
- Cabi S, 2019, arXiv preprint arXiv:1909.12200 · arXiv
- Chebotar Y, 2017, International Conference on Machine Learning, 703
- Chen Z, 2018, International Conference on Machine Learning
- Clavera I, 2019, International Conference on Learning Representations
- Deisenroth M, 2011, International Conference on Machine Learning, 465
- Deisenroth MP, 2013, Foundations and Trends in Robotics, 2, 1
- Devlin J, 2019, NAACL-HLT
- Duan Y, 2017, Advances in Neural Information Processing Systems, 1087
- Eysenbach B, 2018, International Conference on Learning Representations
- Finn C, 2017, International Conference on Machine Learning
- Finn C, 2016, International Conference on Machine Learning, 49
- Finn C, 2017, Proceedings of Machine Learning Research, 78, 357
- Fox R, 2016, Conference on Uncertainty in Artificial Intelligence
- Fu J, 2017, Advances in Neural Information Processing Systems, 2577
- Fujimoto S, 2019, International Conference on Machine Learning
- Fujimoto S, 2018, International Conference on Machine Learning
- Garofolo JS, 1993, NASA STI/Recon Technical Report, 93
- Goodfellow I, 2014, Advances in Neural Information Processing Systems, 2672
- Gregor K, 2017, International Conference on Learning Representations, Workshop Track Proceedings
- Gu S, 2016, International Conference on Machine Learning, 2829
- Haarnoja T, 2017, International Conference on Machine Learning, 1352
- Haarnoja T, 2018, International Conference on Machine Learning
- Hausman K, 2017, AAAI Symposium on Interactive Multi-Sensory Object Perception for Embodied Agents
- Iscen A, 2018, Conference on Robot Learning
- Jabri A, 2019, Advances in Neural Information Processing Systems, 10519
- James S, 2018, Conference on Robot Learning
- James S, 2017, Conference on Robot Learning
- Kalashnikov D, 2018, Conference on Robot Learning, Proceedings of Machine Learning Research
- Khadka S, 2019, International Conference on Machine Learning
- Kroemer O, 2019, CoRR abs/1907.03146 · arXiv
- Kumar A, 2019, Advances in Neural Information Processing Systems, 32
- Kurutach T, 2018, International Conference on Learning Representations
- Levine S, 2014, Advances in Neural Information Processing Systems, 1071
- Levine S, 2016, The Journal of Machine Learning Research, 17, 1334
- Levine S, 2013, International Conference on Machine Learning
- Mania H, 2018, Advances in Neural Information Processing Systems
- Montgomery WH, 2016, Advances in Neural Information Processing Systems, 4008
- Nagabandi A, 2020, Conference on Robot Learning
- Osband I, 2016, Advances in Neural Information Processing Systems, 4026
- Parisotto E, 2016, CoRR
- Paszke A, 2017, Advances in Neural Information Processing Systems Workshop on Autodiff
- Pinto L, 2017, International Conference on Machine Learning
- Rakelly K, 2019, International Conference on Machine Learning
- Riedmiller M, 2018, International Conference on Machine Learning
- Schaul T, 2019, Multidisciplinary Conference on Reinforcement Learning and Decision Making
- Schoettler G, 2019, International Conference on Intelligent Robots and Systems
- Schulman J, 2015, International Conference on Machine Learning
- Sener O, 2018, Advances in Neural Information Processing Systems, 527
- Tedrake R, 2015, Workshop on Adaptive and Learning Systems
- Viereck U, 2017, Conference on Robot Learning
- Wu YH, 2019, International Conference on Machine Learning, Proceedings of Machine Learning Research
- Xiao T, 2020, International Conference on Learning Representations
- Xie A, 2019, arXiv preprint arXiv:1904.05538 · arXiv
- Xie A, 2018, Proceedings of Machine Learning Research, 87, 40
- Yang Y, 2019, AAMAS
- Yang Y, 2020, Conference on Robot Learning
- Yen-Chen L, 2020, Conference on Robot Learning
Cited by
Sorted by citations
Manakitsa, N., Maraslidis, G. S., Moysis, L., & Fragulis, G. F. (2024) · Technologies · doi:10.3390/technologies12020015 · Cited by 355
Explainable Reinforcement Learning: A Survey and Comparative Review
Milani, S., Topin, N., Veloso, M., & Fang, F. (2023) · ACM Computing Surveys · doi:10.1145/3616864 · Cited by 347
Robot learning towards smart robotic manufacturing: A review
Liu, Z., Liu, Q., Xu, W., Wang, L., & Zhou, Z. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102360 · Cited by 251
Recent advancements of robotics in construction
Xiao, B., Chen, C., & Yin, X. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104591 · Cited by 204
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
Tang, C., Abbatematteo, B., Hu, J., Chandra, R., Martín-Martín, R., & Stone, P. (2024) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-030323-022510 · Cited by 276
Retzlaff, C. O., Das, S., Wayllace, C., Mousavi, P., Afshari, M. P., Yang, T., Saranti, A., Angerschmid, A., Taylor, M. E., & Holzinger, A. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15348 · Cited by 224
Learning agile soccer skills for a bipedal robot with deep reinforcement learning
Haarnoja, T., Moran, B., Lever, G., Huang, S. H., Tirumala, D., Humplik, J., Wulfmeier, M., Tunyasuvunakool, S., Siegel, N., Hafner, R., et al. (2024) · Science Robotics · doi:10.1126/scirobotics.adi8022 · Cited by 253
Borboni, A., Reddy, K. V. V., Elamvazuthi, I., Al-Quraishi, M. S., Natarajan, E., & Ali, S. S. A. (2023) · Machines · doi:10.3390/machines11010111 · Cited by 146
Xu, J., Li, B., Lu, B., Liu, Y., Dou, Q., & Heng, P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635867 · Cited by 107
Kabir, H., Tham, M., & Chang, Y. C. (2023) · Digital Communications and Networks · doi:10.1016/j.dcan.2023.05.006 · Cited by 126
The neuromechanics of animal locomotion: From biology to robotics and back
Ramdya, P., & Ijspeert, A. J. (2023) · Science Robotics · doi:10.1126/scirobotics.adg0279 · Cited by 92
del Real, A. J., Andreiana, D. S., Roldán, Á. O., Bustos, A. H., & Acevedo, L. (2022) · Applied Sciences · doi:10.3390/app122312377 · Cited by 112
Al-Hamadani, M. N. A., Fadhel, M. A., Alzubaidi, L., & Harangi, B. (2024) · Sensors · doi:10.3390/s24082461 · Cited by 109
Hobbs, K. L., Mote, M., Abate, M., Coogan, S., & Féron, É. (2023) · IEEE Control Systems · doi:10.1109/mcs.2023.3234380 · Cited by 117
Pore, A., Li, Z., Dall’Alba, D., Hernansanz, A., Momi, E. D., Menciassi, A., Casals, A., Dankelman, J., Fiorini, P., & Poorten, E. V. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3269384 · Cited by 78
Long, Y., Wang, W., Huang, T., Wang, Y., & Dou, Q. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3284380 · Cited by 78
Active learning in robotics: A review of control principles
Taylor, A., Berrueta, T. A., & Murphey, T. D. (2021) · Mechatronics · doi:10.1016/j.mechatronics.2021.102576 · Cited by 85
Intelligent Robots and Human-Robot Collaboration in the Construction Industry: A Review
Wei, H.-H., Zhang, Y., Sun, X., Chen, J., & Li, S. (2023) · Journal of Intelligent Construction · doi:10.26599/jic.2023.9180002 · Cited by 75
High-speed quadrupedal locomotion by imitation-relaxation reinforcement learning
Jin, Y., Liu, X., Shao, Y., Wang, H., & Yang, W. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00576-3 · Cited by 70
Taniguchi, T., Murata, S., Suzuki, M., Ognibene, D., Lanillos, P., Uğur, E., Jamone, L., Nakamura, T., Ciria, A., Lara, B., et al. (2023) · Advanced Robotics · doi:10.1080/01691864.2023.2225232 · Cited by 79
Human-in-the-Loop Robot Learning for Smart Manufacturing: A Human-Centric Perspective
Chen, H., Li, S., Fan, J., Duan, A., Yang, C., Navarro-Alarcon, D., & Zheng, P. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3528051 · Cited by 67
Chu, S., Lin, M., Li, D., Lin, R., & Xiao, S. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.120139 · Cited by 70
Huang, L., Zhu, Z., & Zou, Z. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104691 · Cited by 67
Li, Y., Song, Y., Sun, Y., & Zeng, M. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102518 · Cited by 67
Learning-based legged locomotion: State of the art and future perspectives
Ha, S., Lee, J., van de Panne, M., Xie, Z., Yu, W., & Khadiv, M. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649241312698 · Cited by 76
Huang, L., Ye, M., Xue, X., Wang, Y., Qiu, H., & Deng, X. (2022) · Wireless Networks · doi:10.1007/s11276-022-03066-x · Cited by 67
CybORG: A Gym for the Development of Autonomous Cyber AgentsPreprint
Standen, M., Lucas, M., Bowman, D. B., Richer, T. J., Kim, J., & Marriott, D. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09118 · Cited by 56
Dedeloudi, A., Weaver, E., & Lamprou, D. A. (2023) · International Journal of Pharmaceutics · doi:10.1016/j.ijpharm.2023.122818 · Cited by 58
A residual reinforcement learning method for robotic assembly using visual and force information
Zhang, Z., Wang, Y., Zhang, Z., Wang, L., Huang, H., & Cao, Q. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.11.008 · Cited by 55
Yin, Y., Chen, Z., Liu, G., Yin, J., & Guo, J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123202 · Cited by 57
Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments
Iyer, A., Grewal, K., Velu, A., Souza, L. O., Forest, J., & Ahmad, S. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.846219 · Cited by 63
A review on interaction control for contact robots through intent detection
Li, Y., Sena, A., Wang, Z., Xing, X., Babič, J., van Asseldonk, E. H. F., & Burdet, E. (2022) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/ac8193 · Cited by 54
Booth, S., Knox, W. B., Shah, J., Niekum, S., Stone, P., & Allievi, A. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25733 · Cited by 62
Noack, B. R. (2023) · Physics of Fluids · doi:10.1063/5.0152777 · Cited by 53
From screens to scenes: A survey of embodied AI in healthcare
Liu, Y., Cao, X., Chen, T., Jiang, Y., You, J., Wu, M., Wang, X., Feng, M., Jin, Y., & Chen, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103033 · Cited by 76
Robot Learning of Mobile Manipulation With Reachability Behavior Priors
Jauhri, S., Peters, J., & Chalvatzaki, G. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3188109 · Cited by 49
Learning from Demonstrations for Autonomous Soft-tissue Retraction
Pore, A., Tagliabue, E., Piccinelli, M., Dall’Alba, D., Casals, A., & Fiorini, P. (2021) · International Symposium on Medical Robotics · doi:10.1109/ismr48346.2021.9661514 · Cited by 48
Ning, G., Liang, H., Zhang, X., & Liao, H. (2023) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2023.3250767 · Cited by 47
Gao, Y., Qi, J., Sun, Y., Hu, X., Dong, Z., & Sun, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111506 · Cited by 48
Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach
Chen, Y., Cai, Y.-R., & Cheng, M.-Y. (2023) · Machines · doi:10.3390/machines11020275 · Cited by 50
Ji, Q., Fu, S., Tan, K., Muralidharan, S. T., Lagrelius, K., Danelia, D., Andrikopoulos, G., Wang, X. V., Wang, L., & Feng, L. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102382 · Cited by 46
Continual Semantic Segmentation with Automatic Memory Sample Selection
Zhu, L., Chen, T., Yin, J., See, S. C. W., & Liu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00301 · Cited by 51
Data-Driven Formation Control for Multiple Heterogeneous Vehicles in Air–Ground Coordination
Zhao, W., Liu, H., Wan, Y., & Lin, Z. (2022) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2022.3181254 · Cited by 44
An adaptable and personalized framework for top-N course recommendations in online learning
Amin, S., Uddin, M. I., Alarood, A. A., Mashwani, W. K., Alzahrani, A. O., & Alzahrani, H. A. (2024) · Scientific Reports · doi:10.1038/s41598-024-56497-1 · Cited by 44
Suárez, P., Alcántara-Ávila, F., Miró, A., Rabault, J., Font, B., Lehmkuhl, O., & Vinuesa, R. (2025) · Flow Turbulence and Combustion · doi:10.1007/s10494-025-00642-x · Cited by 52
Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning
Behrens, M. R., & Ruder, W. C. (2022) · Advanced Intelligent Systems · doi:10.1002/aisy.202200023 · Cited by 46
Khan, M. A., Asadi, H., Zhang, L., Qazani, M. R. C., Oladazimi, S., Loo, C. K., Lim, C. P., & Nahavandi, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123717 · Cited by 51
The Societal Implications of Deep Reinforcement Learning
Whittlestone, J., Arulkumaran, K., & Crosby, M. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12360 · Cited by 47
3D robotic navigation using a vision-based deep reinforcement learning model
Zieliński, P., & Markowska–Kaczmar, U. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107602 · Cited by 42
Lim, V., Huang, H., Chen, L. Y., Wang, J., Ichnowski, J., Seita, D., Laskey, M., & Goldberg, K. Y. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811651 · Cited by 53
Towards intelligent cooperative robotics in additive manufacturing: Past, present, and future
Rescsanski, S., Hebert, R. J., Haghighi, A., Tang, J., & Imani, F. (2024) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2024.102925 · Cited by 42
Machine learning meets advanced robotic manipulation
Nahavandi, S., Alizadehsani, R., Nahavandi, D., Lim, C. P., Kelly, K., & Bello, F. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102221 · Cited by 46
Reinforcement Learning Control of a Forestry Crane Manipulator
Andersson, J., Bodin, K., Lindmark, D., Servin, M., & Wallin, E. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636219 · Cited by 40
High-Tech Defense Industries: Developing Autonomous Intelligent Systems
dos Reis, J. C. G., Cohen, Y., Melão, N. F., Costa, J., & Jorge, D. (2021) · Applied Sciences · doi:10.3390/app11114920 · Cited by 52
Reinforcement-learning-based actuator selection method for active flow control
Paris, R., Beneddine, S., & Dandois, J. (2023) · Journal of Fluid Mechanics · doi:10.1017/jfm.2022.1043 · Cited by 37
Games for Artificial Intelligence Research: A Review and Perspectives
Hu, C., Zhao, Y., Wang, Z., Du, H., & Liu, J. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3410935 · Cited by 56
GNN-Empowered Effective Partial Observation MARL Method for AoI Management in Multi-UAV Network
Pan, Y., Wang, X., Xu, Z., Cheng, N., Xu, W., & Zhang, J.-J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3447774 · Cited by 42
Huang, T., Chen, K., Li, B., Liu, Y., & Dou, Q. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160327 · Cited by 34
General-purpose foundation models for increased autonomy in robot-assisted surgery
Schmidgall, S., Kim, J. W., Kuntz, A., Ghazi, A., & Krieger, A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00917-4 · Cited by 39
Gu, Z., Li, J., Shen, W., Yu, W., Xie, Z., McCrory, S., Cheng, X., Shamsah, A., Griffin, R. J., Liu, C. K., et al. (2026) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2025.3579247 · Cited by 130
Swarm Cooperative Navigation Using Centralized Training and Decentralized Execution
Azzam, R., Boiko, I. M., & Zweiri, Y. (2023) · Drones · doi:10.3390/drones7030193 · Cited by 33
Qashqai, P., Babaie, M., Zgheib, R., & Al‐Haddad, K. (2023) · IEEE Access · doi:10.1109/access.2023.3318264 · Cited by 35
Sim-to-real transfer of active suspension control using deep reinforcement learning
Wiberg, V., Wallin, E., Fälldin, A., Semberg, T., Rossander, M., Wadbro, E., & Servin, M. (2024) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2024.104731 · Cited by 34
Skill generalization of tubular object manipulation with tactile sensing and Sim2Real learning
Zhao, Y., Jing, X., Qian, K., Gomes, D. F., & Luo, S. (2022) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2022.104321 · Cited by 32
Port-Hamiltonian Neural ODE Networks on Lie Groups for Robot Dynamics Learning and Control
Duong, T., Altawaitan, A., Stanley, J., & Atanasov, N. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3428433 · Cited by 31
Kulkarni, P., Kober, J., Babuška, R., & Santina, C. D. (2021) · Advanced Intelligent Systems · doi:10.1002/aisy.202100095 · Cited by 31
Dynamic Navigation in Unconstrained Environments Using Reinforcement Learning Algorithms
Chronis, C., Anagnostopoulos, G., Politi, E., Dimitrakopoulos, G. J., & Varlamis, I. (2023) · IEEE Access · doi:10.1109/access.2023.3326435 · Cited by 33
Ren, L., Ning, X., & Wang, Z. X. (2022) · Computers & Industrial Engineering · doi:10.1016/j.cie.2022.108242 · Cited by 30
Deng, X., Liu, J., Gong, H., Gong, H., & Huang, J. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3224966 · Cited by 30
Liu, Q., Ji, Z., Xu, W., Liu, Z., Yao, B., & Zhou, Z. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121037 · Cited by 30
Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics
Rana, K., Dasagi, V., Haviland, J., Talbot, B., Milford, M. J., & Sünderhauf, N. (2023) · The International Journal of Robotics Research · doi:10.1177/02783649231167210 · Cited by 40
Eßer, J., Bach, N., Jestel, C., Urbann, O., & Kerner, S. (2022) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2022.3207664 · Cited by 34
Park, S.-Y., Lee, C., Jeong, S., Lee, J., Kim, D., Jang, Y., Seol, W., Kim, H., & Ahn, S. (2024) · International Journal of Precision Engineering and Manufacturing-Green Technology · doi:10.1007/s40684-023-00593-6 · Cited by 35
A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning
Sarhadi, P., Naeem, W., & Αθανασόπουλος, Ν. (2022) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2022.10.440 · Cited by 28
Yao, T., Wang, H., Lu, B., Ge, J., Pei, Z., Kowarschik, M., Sun, L., Seneviratne, L. D., & Qi, P. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3555559 · Cited by 27
AI Challenges for Society and Ethics
Whittlestone, J., & Clarke, S. (2022) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197579329.013.3 · Cited by 35
A Survey of the Potential Long-term Impacts of AI
Clarke, S., & Whittlestone, J. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534131 · Cited by 41
Hazrathosseini, A., & Afrapoli, A. M. (2024) · Mining Technology Transactions of the Institutions of Mining and Metallurgy · doi:10.1177/25726668231222998 · Cited by 37
Learning-Based Navigation and Collision Avoidance Through Reinforcement for UAVs
Azzam, R., Chehadeh, M. S., Hay, O. A., Humais, M. A., Boiko, I. M., & Zweiri, Y. (2023) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2023.3294889 · Cited by 25
Billard, A., Albu‐Schäffer, A., Beetz, M., Burgard, W., Corke, P., Ciocarlie, M., Dahiya, R., Kragić, D., Goldberg, K., Nagai, Y., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01050-6 · Cited by 25
Excavation Reinforcement Learning Using Geometric Representation
Lu, Q., Zhu, Y., & Zhang, L. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3150511 · Cited by 25
FED-UP: Federated Deep Reinforcement Learning-based UAV Path Planning against Hostile Defense System
Khalil, A. A., & Rahman, M. A. (2022) · International Conference on Network and Service Management (CNSM) · doi:10.23919/cnsm55787.2022.9964907 · Cited by 25
Resolving the associative learning paradox by category learning in pigeons
Wasserman, E. A., Kain, A. G., & O'Donoghue, E. M. (2023) · Current Biology · doi:10.1016/j.cub.2023.01.024 · Cited by 25
Identifying important sensory feedback for learning locomotion skills
Yu, W., Yang, C., McGreavy, C., Triantafyllidis, E., Bellegarda, G., Shafiee, M., Ijspeert, A. J., & Li, Z. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00701-w · Cited by 29
Abbas, A. N., Chasparis, G. C., & Kelleher, J. D. (2023) · Data & Knowledge Engineering · doi:10.1016/j.datak.2023.102240 · Cited by 28
Learn From Safe Experience: Safe Reinforcement Learning for Task Automation of Surgical Robot
Fan, K., Chen, Z., Ferrigno, G., & Momi, E. D. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3351797 · Cited by 25
Chen, W., Xu, J. L., Xiang, F., Yuan, X., Su, H., & Chen, R. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3352969 · Cited by 34
Domain Randomization for Sim2real Transfer of Automatically Generated Grasping Datasets
Huber, J., Hélénon, F., Watrelot, H., Amar, F. B., & Doncieux, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610677 · Cited by 30
Lee, R., Hamaya, M., Murooka, T., Ijiri, Y., & Corke, P. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3130377 · Cited by 24
Learning Implicit Priors for Motion Optimization
Urain, J., Le, A. T., Lambert, A., Chalvatzaki, G., Boots, B., & Peters, J. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981264 · Cited by 25
Learning positioning policies for mobile manipulation operations with deep reinforcement learning
Iriondo, A., Lazkano, E., Ansuategi, A., Rivera, A., Lluvia, I., & Tubío, C. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-01815-8 · Cited by 23
Liu, P., Zhang, K., Tateo, D., Jauhri, S., Hu, Z., Peters, J., & Chalvatzaki, G. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161548 · Cited by 23
Effective Multi-Agent Deep Reinforcement Learning Control With Relative Entropy Regularization
Miao, C., Cui, Y., Li, H., & Wu, X. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3398712 · Cited by 25
Maximum diffusion reinforcement learning
Berrueta, T. A., Pinosky, A., & Murphey, T. D. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00829-3 · Cited by 24
Deep reinforcement learning in surgical robotics: Enhancing the automation level
Qian, C., & Ren, H. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-13271-1.00055-8 · Cited by 24
2-level reinforcement learning for ships on inland waterways: Path planning and following
Waltz, M., Paulig, N., & Okhrin, O. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126933 · Cited by 24
Active object tracking of free floating space manipulators based on deep reinforcement learning
Lei, W., Fu, H., & Sun, G. (2022) · Advances in Space Research · doi:10.1016/j.asr.2022.08.041 · Cited by 22
AdaptiveON: Adaptive Outdoor Local Navigation Method for Stable and Reliable Actions
Liang, J., Weerakoon, K., Guan, T., Karapetyan, N., & Manocha, D. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3229907 · Cited by 24
Comparative Study of Physics Engines for Robot Simulation with Mechanical Interaction
Yoon, J., Son, B., & Lee, D. (2023) · Applied Sciences · doi:10.3390/app13020680 · Cited by 22
BASE: Bridging the Gap between Cost and Latency for Query Optimization
Xu, C., Wang, Z., Liu, S., Li, Y., Zeng, K., Ding, B., Zhou, J., Su, H., & Zheng, K. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3594512.3594525 · Cited by 22
Metadata
- Authors
- Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S.
- Affiliations
- Google (United States) (US)
- Everyday Robots, X, The Moonshot Factory, Mountain View, CA, USA
- Corresponding author
- Ibarz, Julian
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2021-01-31
- Volume
- 40
- Issue
- 4-5
- Pages
- 698-721
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W3126321819
- Cited by
- 570
- References
- 193
- Citations by year
- 2026: 63 · 2025: 123 · 2024: 145 · 2023: 112 · 2022: 89 · 2021: 38
- FWCI
- 33.92
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- The International Journal of Robotics Research (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, robot learning, embodied agents, physical robots, robot skill learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
