The DOI matches its Crossref record; 40 of the 44 references it cites were found in a registry or an index.
Overcoming Exploration in Reinforcement Learning with Demonstrations
Open accessAbstract
Exploration in environments with sparse rewards has been a persistent problem in reinforcement learning (RL). Many tasks are natural to specify with a sparse reward, and manually shaping a reward function can result in suboptimal performance. However, finding a non-zero reward is exponentially more difficult with increasing task horizon or action dimensionality. This puts many real-world tasks out of practical reach of RL methods. In this work, we use demonstrations to overcome the exploration problem and successfully learn to perform long-horizon, multi-step robotics tasks with continuous control such as stacking blocks with a robot arm. Our method, which builds on top of Deep Deterministic Policy Gradients and Hindsight Experience Replay, provides an order of magnitude of speedup over RL on simulated robotics tasks. It is simple to implement and makes only the additional assumption that we can collect a small set of demonstrations. Furthermore, our method is able to solve tasks not solvable by either RL or behavior cloning alone, and often ends up outperforming the demonstrator policy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Demonstration learning (DemoL) algorithms often used in robotics applications are DQfD [218], DDPGfD [219] and Q-filtered BC [220].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [220]
The cited source says what the claim attributes to it.
In the citing paper
“Building upon DDPG [124] and HER [182], Nair et al. (2018) [220] propose a method called Q-filtered Behaviour Cloning (QfBC) that also applies the BC loss to learn demonstration data to solve long-horizon, sparse-reward and goal-conditioned tasks.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [220]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Probabilistic roadmaps for path planning in high-dimensional configuration spaces
Kavraki, L. E., Svestka, P., Latombe, J.-C., & Overmars, M. H. (1996) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.508439 · Cited by 6,343 · cited 1×
“More recent work on task and motion planning considers both logical and physical aspects of the task [27], [28], [29], but requires domainspecific engineering.”
BackgroundMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Self-citationEnd to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
Flagged first, then by citations
4 of the 44 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- hester, 2017, Learning from demonstrations for real world reinforcement learning
- ng, 2000, Algorithms for Inverse Reinforcement Learning, International Conference on Machine Learning (ICML)
- ziebart, 2008, Maximum Entropy Inverse Reinforcement Learning, AAAI Conference on Artificial Intelligence, 1433
Cited by
Sorted by citations
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
Continuous control for robot based on deep reinforcement learning
Zhang, S. (2019) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/90191 · Cited by 961
Peng, X. B., Abbeel, P., Levine, S., & van de Panne, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201311 · Cited by 946
Self-citationTransfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., & Levine, S. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.049 · Cited by 847
Recent Advances in Robot Learning from Demonstration
Ravichandar, H., Polydoros, A. S., Chernova, S., & Billard, A. (2020) · Annu. Rev. Control. Robotics Auton. Syst. · doi:10.1146/annurev-control-100819-063206 · Cited by 947
How to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570
Exploration in deep reinforcement learning: A survey
Ladosz, P., Weng, L., Kim, M., & Oh, H. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.003 · Cited by 525
Off-Policy Deep Reinforcement Learning without ExplorationPreprint
Fujimoto, S., Meger, D., & Precup, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02900 · Cited by 646
Residual Reinforcement Learning for Robot Control
Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J. A., Solowjow, E., & Levine, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794127 · Cited by 480
Self-citationPRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning
Sartoretti, G., Kerr, J. H., Shi, Y., Wagner, G., Kumar, T. K. S., Koenig, S., & Choset, H. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2903261 · Cited by 426
Learning by Playing - Solving Sparse Reward Tasks from ScratchPreprint
Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Van de Wiele, T., Mnih, V., Heess, N., & Springenberg, J. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10567 · Cited by 334
Deep reinforcement learning in computer vision: a comprehensive survey
Le, N., Rathour, V. S., Yamazaki, K., Luu, K., & Savvides, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10061-9 · Cited by 271
Reinforcement and Imitation Learning for Diverse Visuomotor Skills
Zhu, Y., Wang, Z., Merel, J., Rusu, A. A., Erez, T., Cabi, S., Tunyasuvunakool, S., Kramár, J., Hadsell, R. T., de Freitas, N., et al. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.009 · Cited by 330
A Minimalist Approach to Offline Reinforcement LearningPreprint
Fujimoto, S., & Gu, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06860 · Cited by 267
A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation
Han, D., Mulyana, B., Stankovic, V., & Cheng, S. (2023) · Sensors · doi:10.3390/s23073762 · Cited by 253
Deep reinforcement learning in medical imaging: A literature review
Zhou, S., Le, N., Luu, K., Nguyen, H. V., & Ayache, N. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102193 · Cited by 227
Wu, J., Huang, Z., Hu, Z., & Lv, C. (2023) · Engineering · doi:10.1016/j.eng.2022.05.017 · Cited by 205
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · RePEc: Research Papers in Economics · Cited by 201
Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards
Schoettler, G., Nair, A., Luo, J., Bahl, S., Ojea, J. A., Solowjow, E., & Levine, S. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341714 · Cited by 191
Self-citationVariable Compliance Control for Robotic Peg-in-Hole Assembly: A Deep-Reinforcement-Learning Approach
Beltran-Hernandez, C. C., Petit, D., Ramírez-Alpizar, I. G., & Harada, K. (2020) · Applied Sciences · doi:10.3390/app10196923 · Cited by 189
Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost
Zhu, H., Gupta, A., Rajeswaran, A., Levine, S., & Kumar, V. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794102 · Cited by 184
Decentralized Multi-Agent Pursuit Using Deep Reinforcement Learning
de Souza, C., Newbury, R., Cosgun, A., Castillo, P., Vidolov, B., & Kuli, D. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3068952 · Cited by 192
Xu, B., Rathod, D., Zhang, D., Yebi, A., Zhang, X., Li, X., & Filipi, Z. S. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.114200 · Cited by 182
Retzlaff, C. O., Das, S., Wayllace, C., Mousavi, P., Afshari, M. P., Yang, T., Saranti, A., Angerschmid, A., Taylor, M. E., & Holzinger, A. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15348 · Cited by 224
AWAC: Accelerating Online Reinforcement Learning with Offline Datasets
Nair, A., Gupta, A., Dalal, M., & Levine, S. (2020) · arXiv · doi:10.48550/arxiv.2006.09359 · Cited by 175
Self-citationReinforcement Learning for Mobile Robotics Exploration: A Survey
Garaffa, L. C., Basso, M., Konzen, A. A., & de Freitas, E. P. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3124466 · Cited by 178
Deep Reinforcement Learning for Traffic Signal Control: A Review
Rasheed, F., Yau, K. A., Noor, R. M., Wu, C., & Low, Y.-C. (2020) · IEEE Access · doi:10.1109/access.2020.3034141 · Cited by 189
Deep learning in robotics: survey on model structures and training strategies
Karoly, A. I., Galambos, P., Kuti, J., & Rudas, I. J. (2021) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2020.3018325 · Cited by 146
Dorokhova, M., Martinson, Y., Ballif, C., & Wyrsch, N. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.117504 · Cited by 143
Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach
Liu, Y., Liu, Q., Zhao, H., Pan, Z., & Liu, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5587 · Cited by 141
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
Mandlekar, A., Zhu, Y., Garg, A., Booher, J., Spero, M., Tung, A., Gao, J., Emmons, J., Gupta, A., Orbay, E., et al. (2018) · arXiv · doi:10.48550/arxiv.1811.02790 · Cited by 134
Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges
Bhagat, S., Banerjee, H., Tse, Z. T. H., & Ren, H. (2019) · Robotics · doi:10.3390/robotics8010004 · Cited by 139
Wang, Z., Novikov, A., Zolna, K., Springenberg, J. T., Reed, S., Shahriari, B., Siegel, N., Merel, J., Gulcehre, C., Heess, N., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.15134 · Cited by 128
Prioritized Experience-Based Reinforcement Learning With Human Guidance for Autonomous Driving
Wu, J., Huang, Z., Huang, W., & Lv, C. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3177685 · Cited by 123
Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration
Johns, E. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9560942 · Cited by 119
Goal-Conditioned Imitation LearningPreprint
Ding, Y., Florensa, C., Phielipp, M., & Abbeel, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05838 · Cited by 128
Self-citationSearch on the Replay Buffer: Bridging Planning and Reinforcement Learning
Eysenbach, B., Salakhutdinov, R., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1906.05253 · Cited by 123
A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning
Vecerík, M., Sushkov, O. O., Barker, D., Rothörl, T., Hester, T., & Scholz, J. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794074 · Cited by 113
Reward learning from human preferences and demonstrations in Atari
Ibarz, B., Leike, J., Pohlen, T., Irving, G., Legg, S., & Amodei, D. (2018) · arXiv · doi:10.48550/arxiv.1811.06521 · Cited by 117
Goal-Conditioned Reinforcement Learning: Problems and Solutions
Liu, M., Zhu, M., & Zhang, W. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/770 · Cited by 115
“Researchers used to evaluate their image algorithms on simulation environments, as in Warde-Farley et al. ; Lee et al. ; Chane-Sane et al. [2019; 2020; 2021], and also in real-world robotics applications [Nair et al. , 2018b; Nair et al. , 2020b].”
BackgroundMethodDynamic Cloth Manipulation with Deep Reinforcement Learning
Jangir, R., Alenya, G., & Torras, C. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9196659 · Cited by 114
Human-Guided Reinforcement Learning With Sim-to-Real Transfer for Autonomous Navigation
Wu, J., Zhou, Y., Yang, H., Huang, Z., & Lv, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3314762 · Cited by 110
Model-free reinforcement learning from expert demonstrations: a survey
Ramírez, J., Yu, W., & Perrusquía, A. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10085-1 · Cited by 115
Xu, J., Li, B., Lu, B., Liu, Y., Dou, Q., & Heng, P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635867 · Cited by 107
GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning AlgorithmsPreprint
Colas, C., Sigaud, O., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05054 · Cited by 124
Mohammed, M. Q., Kwek, L. C., & Chua, S. C. (2020) · IEEE Access · doi:10.1109/access.2020.3027923 · Cited by 111
Privacy Preserving Load Control of Residential Microgrid via Deep Reinforcement Learning
Qin, Z., Liu, D., Hua, H., & Cao, J. (2021) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2021.3088290 · Cited by 105
CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement LearningPreprint
Colas, C., Fournier, P., Sigaud, O., Chétouani, M., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06284 · Cited by 104
Policy Optimization with Demonstrations
Kang, B., Jie, Z., & Feng, J. (2018) · International Conference on Machine Learning · Cited by 106
Nguyen, K., Dey, D., Brockett, C., & Dolan, B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01281 · Cited by 112
Hierarchical Reinforcement Learning With Universal Policies for Multistep Robotic Manipulation
Yang, X., Ji, Z., Wu, J., Lai, Y., Wei, C., Liu, G., & Setchi, R. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3059912 · Cited by 94
Hierarchical Imitation and Reinforcement LearningPreprint
Le, H. M., Jiang, N., Agarwal, A., Dudı́k, M., Yue, Y., & Daumé, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00590 · Cited by 107
Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks
Nasiriany, S., Liu, H., & Zhu, Y. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9812140 · Cited by 91
Huang, Z., Sheng, Z., Ma, C., & Chen, S. (2024) · Communications in Transportation Research · doi:10.1016/j.commtr.2024.100127 · Cited by 88
Meta-Reinforcement Learning for Robotic Industrial Insertion Tasks
Schoettler, G., Nair, A., Ojea, J. A., Levine, S., & Solowjow, E. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340848 · Cited by 88
Self-citationLuo, J., Sushkov, O. O., Pevceviciute, R., Lian, W., Su, C., Vecerík, M., Ye, N., Stefan Schaal, A., & Scholz, J. (2021) · Robotics: Science and Systems · doi:10.15607/rss.2021.xvii.088 · Cited by 88
Learning Montezuma's Revenge from a Single DemonstrationPreprint
Salimans, T., & Chen, R. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03381 · Cited by 152
Liu, B., Wang, L., Liu, M., & Xu, C. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2976321 · Cited by 83
Thananjeyan, B., Balakrishna, A., Rosolia, U., Li, F., McAllister, R., Gonzalez, J. E., Levine, S., Borrelli, F., & Goldberg, K. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2976272 · Cited by 83
A review of recent trend in motion planning of industrial robots
Tamizi, M. G., Yaghoubi, M., & Najjaran, H. (2023) · International Journal of Intelligent Robotics and Applications · doi:10.1007/s41315-023-00274-2 · Cited by 83
Observe and Look Further: Achieving Consistent Performance on AtariPreprint
Pohlen, T., Piot, B., Hester, T., Azar, M. G., Horgan, D., Budden, D. M., Barth-Maron, G., van Hasselt, H. P., Quan, J., Vecerík, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11593 · Cited by 81
Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation
Antotsiou, D., Garcia-Hernando, G., & Kim, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11024-6_19 · Cited by 80
Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning
Li, R., Jabri, A., Darrell, T., & Agrawal, P. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197468 · Cited by 79
Mandlekar, A., Ramos, F. T., Boots, B., Savarese, S., Fei-Fei, L., Garg, A., & Fox, D. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196935 · Cited by 88
Imitating Latent Policies from Observation
Edwards, A. D., Sahni, H., Schroecker, Y., & Isbell, C. L. (2018) · arXiv · doi:10.48550/arxiv.1805.07914 · Cited by 79
Deep Reinforcement Learning With Optimized Reward Functions for Robotic Trajectory Planning
Xie, J., Shao, Z., Li, Y., Guan, Y., & Tan, J. (2019) · IEEE Access · doi:10.1109/access.2019.2932257 · Cited by 78
Keller, B., Draelos, M., Zhou, K. C., Qian, R., Kuo, A. N., Konidaris, G., Hauser, K. K., & Izatt, J. A. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.2980158 · Cited by 82
Colas, C., Karch, T., Sigaud, O., & Oudeyer, P. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13554 · Cited by 137
VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation
Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A., Jamali, N., Yamane, K., Iba, S., & Goldberg, K. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.034 · Cited by 74
Lee, H.-R., & Lee, T. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2020.09.018 · Cited by 71
AGIL: Learning Attention from Human for Visuomotor Tasks
Zhang, R., Liu, Z., Zhang, L., Whritner, J. A., Muller, K., Hayhoe, M., & Ballard, D. H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_41 · Cited by 75
Xiang, X., & Foo, S. Y. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030029 · Cited by 77
Q-Attention: Enabling Efficient Learning for Vision-Based Robotic Manipulation
James, S., & Davison, A. J. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3140817 · Cited by 70
Open-Sourced Reinforcement Learning Environments for Surgical Robotics
Richter, F., Orosco, R. K., & Yip, M. C. (2019) · arXiv · doi:10.48550/arxiv.1903.02090 · Cited by 92
Improvisation through Physical Understanding: Using Novel Objects As Tools with Visual Foresight
Xie, A., Ebert, F., Levine, S., & Finn, C. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.001 · Cited by 88
Deep Learning in Diverse Intelligent Sensor Based Systems
Zhu, Y., Wang, M., Yin, X., Zhang, J., Meijering, E. H. W., & Hu, J. (2022) · Sensors · doi:10.3390/s23010062 · Cited by 73
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to ImitationPreprint
Tai, L., Zhang, J., Liu, M., Boedecker, J., & Burgard, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.07139 · Cited by 111
Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving
Cao, Z., Bıyık, E., Wang, W. Z., Raventos, A., Gaidon, A., Rosman, G., & Sadigh, D. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.039 · Cited by 65
Bimanual Regrasping for Suture Needles using Reinforcement Learning for Rapid Motion Planning
Chiu, Z.-Y., Richter, F., Funk, E. K., Orosco, R. K., & Yip, M. C. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561673 · Cited by 62
Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation
Jeong, R., Aytar, Y., Khosid, D., Zhou, Y., Kay, J., Lampe, T., Bousmalis, K., & Nori, F. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197326 · Cited by 60
Model-based Deep Reinforcement Learning for Dynamic Portfolio OptimizationPreprint
Yu, P., Sern, L. J., Kulyatin, I., Shi, Z., & Dasgupta, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08740 · Cited by 106
Robot learning from demonstration for path planning: A review
Xie, Z., Zhang, Q., Jiang, Z., & Liu, H. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1648-4 · Cited by 58
Xie, Y., & Zhao, X. (2021) · Physics of Fluids · doi:10.1063/5.0037334 · Cited by 57
From Model-Centric to Data-Centric AI: A Paradigm Shift or Rather a Complementary Approach?
Hamid, O. H. (2022) · International Conference on Information Technology Trends (ITT) · doi:10.1109/itt56123.2022.9863935 · Cited by 57
Lee, M. A., Florensa, C., Tremblay, J., Ratliff, N., Garg, A., Ramos, F., & Fox, D. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197125 · Cited by 46
Exact-K Recommendation via Maximal Clique Optimization
Gong, Y., Zhu, Y., Duan, L., Liu, Q., Guan, Z., Sun, F., Ou, W., & Zhu, K. Q. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330832 · Cited by 56
Reinforcement Learning for Statistical Process Control in Manufacturing
Viharos, Z. J., & Jakab, R. (2021) · Measurement · doi:10.1016/j.measurement.2021.109616 · Cited by 55
Surgical embodied intelligence for generalized task autonomy in laparoscopic robot-assisted surgery
Long, Y., Lin, A., Kwok, D. H. C., Zhang, L., Yang, Z., Shi, K., Song, L., Fu, J., Lin, H., Wang, W., et al. (2025) · Science Robotics · doi:10.1126/scirobotics.adt3093 · Cited by 64
Task-Oriented Deep Reinforcement Learning for Robotic Skill Acquisition and Control
Xiang, G., & Su, J. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2949596 · Cited by 56
Accelerating Reinforcement Learning for Reaching Using Continuous Curriculum Learning
Luo, S., Kasaei, H., & Schomaker, L. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207427 · Cited by 56
Divide-and-Conquer Reinforcement LearningPreprint
Ghosh, D., Singh, A., Rajeswaran, A., Kumar, V., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.09874 · Cited by 53
Parrot: Data-Driven Behavioral Priors for Reinforcement LearningPreprint
Singh, A., Liu, H., Zhou, G., Yu, A., Rhinehart, N., & Levine, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.10024 · Cited by 101
Li, X., Liang, X., Wang, X., Wang, R., Shu, L., & Xu, W. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110543 · Cited by 51
Bayesian Gaussian Mixture Model for Robotic Policy Imitation
Pignat, E., & Calinon, S. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2932610 · Cited by 50
Overcoming model bias for robust offline deep reinforcement learning
Swazinna, P., Udluft, S., & Runkler, T. A. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104366 · Cited by 52
Making Efficient Use of Demonstrations to Solve Hard Exploration ProblemsPreprint
Paine, T. L., Gülçehre, Ç., Shahriari, B., Denil, M., Hoffman, M., Soyer, H., Tanburn, R., Kapturowski, S., Rabinowitz, N. C., Williams, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01387 · Cited by 49
Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
Wang, L., Liu, J., Shao, H., Wang, W., Chen, R., Liu, Y., & Waslander, S. L. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.102 · Cited by 72
Rewards Prediction-Based Credit Assignment for Reinforcement Learning With Sparse Binary Rewards
Seo, M., Vecchietti, L. F., Lee, S., & Har, D. (2019) · IEEE Access · doi:10.1109/access.2019.2936863 · Cited by 50
DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable FeedbackPreprint
Arakawa, R., Kobayashi, S., Unno, Y., Tsuboi, Y., & Maeda, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11748 · Cited by 81
Metadata
- Authors
- Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P.
- Affiliations
- OpenAI (United States) (US)
- ORCID
- Abbeel, Pieter
- Venue
- 2018 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 6292-6299
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2756826236
- Cited by
- 712
- References
- 44
- Citations by year
- 2026: 24 · 2025: 72 · 2024: 86 · 2023: 111 · 2022: 93 · 2021: 117 · 2020: 105 · 2019: 67 · 2018: 35
- FWCI
- 33.08
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- exploration in reinforcement learning, sparse rewards, demonstrations, deep deterministic policy gradient, hindsight experience replay, long-horizon tasks, continuous control, robotic manipulation, block stacking, robot manipulators, behavior cloning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
