The DOI matches its Crossref record; 40 of the 44 references it cites were found in a registry or an index.
Overcoming Exploration in Reinforcement Learning with Demonstrations
Open accessAbstract
Exploration in environments with sparse rewards has been a persistent problem in reinforcement learning (RL). Many tasks are natural to specify with a sparse reward, and manually shaping a reward function can result in suboptimal performance. However, finding a non-zero reward is exponentially more difficult with increasing task horizon or action dimensionality. This puts many real-world tasks out of practical reach of RL methods. In this work, we use demonstrations to overcome the exploration problem and successfully learn to perform long-horizon, multi-step robotics tasks with continuous control such as stacking blocks with a robot arm. Our method, which builds on top of Deep Deterministic Policy Gradients and Hindsight Experience Replay, provides an order of magnitude of speedup over RL on simulated robotics tasks. It is simple to implement and makes only the additional assumption that we can collect a small set of demonstrations. Furthermore, our method is able to solve tasks not solvable by either RL or behavior cloning alone, and often ends up outperforming the demonstrator policy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Demonstration learning (DemoL) algorithms often used in robotics applications are DQfD [218], DDPGfD [219] and Q-filtered BC [220].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [220]
The cited source says what the claim attributes to it.
In the citing paper
“Building upon DDPG [124] and HER [182], Nair et al. (2018) [220] propose a method called Q-filtered Behaviour Cloning (QfBC) that also applies the BC loss to learn demonstration data to solve long-horizon, sparse-reward and goal-conditioned tasks.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [220]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Probabilistic roadmaps for path planning in high-dimensional configuration spaces
Kavraki, L. E., Svestka, P., Latombe, J.-C., & Overmars, M. H. (1996) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.508439 · Cited by 6,343 · cited 1×
“More recent work on task and motion planning considers both logical and physical aspects of the task [27], [28], [29], but requires domainspecific engineering.”
BackgroundMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Self-citationEnd to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
Flagged first, then by citations
4 of the 44 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- hester, 2017, Learning from demonstrations for real world reinforcement learning
- ng, 2000, Algorithms for Inverse Reinforcement Learning, International Conference on Machine Learning (ICML)
- ziebart, 2008, Maximum Entropy Inverse Reinforcement Learning, AAAI Conference on Artificial Intelligence, 1433
Cited by
Sorted by citations
Items 101–200 of 710
Niu, H., Li, S., & Li, J. (2022) · International Conference on Information and Knowledge Management · doi:10.1145/3511808.3557363 · Cited by 59
Real-time artificial intelligence for accelerator control: A study at the Fermilab Booster
John, J. M. S., Herwig, T. C., Kafkes, D., Mitrevski, J., Pellico, W. A., Perdue, G., Quintero-Parra, A., Schupbach, B., Seiya, K., Tran, N. V., et al. (2021) · Physical Review Accelerators and Beams · doi:10.1103/physrevaccelbeams.24.104601 · Cited by 54
Visuospatial foresight for physical sequential fabric manipulation
Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A. K., Jamali, N., Yamane, K., Iba, S., & Goldberg, K. (2022) · Autonomous Robots · doi:10.1007/s10514-021-10001-0 · Cited by 45
Disagreement-Regularized Imitation Learning
Brantley, K., Sun, W. Y., & Henaff, M. (2020) · International Conference on Learning Representations · Cited by 45
Imitation Learning with Concurrent Actions in 3D Games
Harmer, J., Gisslén, L., del Val, J., Holst, H., Bergdahl, J., Olsson, T., Sjöö, K., & Nordin, M. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490398 · Cited by 48
Learning to combine primitive skills: A step towards versatile robotic manipulation §
Strudel, R., Pashevich, A., Kalevatykh, I., Laptev, I., Šivic, J., & Schmid, C. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196619 · Cited by 51
Luo, Y., Wang, Y., Dong, K., Zhang, Q., Cheng, E., Sun, Z., & Song, B. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126620 · Cited by 46
Certified reinforcement learning with logic guidance
Hasanbeig, H., Kroening, D., & Abate, A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103949 · Cited by 67
Learning Deep Visuomotor Policies for Dexterous Hand Manipulation
Jain, D., Li, A. R., Singhal, S., Rajeswaran, A., Kumar, V., & Todorov, E. V. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794033 · Cited by 49
Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning
Qiao, Z., Tyree, Z., Mudalige, P., Schneider, J., & Dolan, J. M. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341496 · Cited by 45
Learning and Adapting Agile Locomotion Skills by Transferring Experience
Smith, L. E., Kew, J. C., Li, T., Luu, L., Peng, X. B., Ha, S., Tan, J., & Levine, S. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.051 · Cited by 60
Robertshaw, H. H., Karstensen, L., Jackson, B. M., Sadati, S. M. H., Rhode, K., Sebastien Ourselin, A., Granados, A., & Booth, T. C. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1239374 · Cited by 47
A Systematic Literature Review on AI Safety: Identifying Trends, Challenges, and Future Directions
Salhab, W., Ameyed, D., Jaafar, F., & Mcheick, H. (2024) · IEEE Access · doi:10.1109/access.2024.3440647 · Cited by 50
Data-Centric and Model-Centric AI: Twin Drivers of Compact and Robust Industry 4.0 Solutions
Hamid, O. H. (2023) · Applied Sciences · doi:10.3390/app13052753 · Cited by 46
C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters
Dou, Z., Chen, X., Fan, Q., Komura, T., & Wang, W. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618205 · Cited by 44
Learning Self-Imitating Diverse PoliciesPreprint
Gangwani, T., Liu, Q., & Peng, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10309 · Cited by 56
Xu, C., Zhang, T., Zhang, D., Zhang, D., & Han, J. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3383716 · Cited by 39
Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation LearningPreprint
Sun, W. Y., Bagnell, J. A., & Boots, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11240 · Cited by 61
A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning
Liu, L., Liu, Q., Song, Y. H., Pang, B. J., Yuan, X., & Xu, Q. (2021) · Applied Sciences · doi:10.3390/app11041816 · Cited by 41
Complex Robotic Manipulation via Graph-Based Hindsight Goal Generation
Bing, Z., Brucker, M., Morin, F. O., Li, R., Su, X., Huang, K., & Knoll, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3088947 · Cited by 37
Learning Deformable Object Manipulation From Expert Demonstrations
Salhotra, G., Liu, I.-C. A., Dominguez-Kuhne, M., & Sukhatme, G. S. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3187843 · Cited by 37
Deep Reinforcement Learning for the Navigation of Neurovascular Catheters
Behr, T., Pusch, T., Siegfarth, M., Hüsener, D., Mörschel, T., & Karstensen, L. (2019) · Current Directions in Biomedical Engineering · doi:10.1515/cdbme-2019-0002 · Cited by 37
Sample-efficient Reinforcement Learning in Robotic Table Tennis
Tebbe, J., Krauch, L., Gao, Y., & Zell, A. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560764 · Cited by 39
Reinforcement learning for robotic manipulation using simulated locomotion demonstrations
Kilinc, O., & Montana, G. (2021) · Machine Learning · doi:10.1007/s10994-021-06116-1 · Cited by 39
Visual Tracking by Means of Deep Reinforcement Learning and an Expert Demonstrator
Dunnhofer, M., Martinel, N., Foresti, G. L., & Micheloni, C. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00282 · Cited by 35
Rule-Aware Reinforcement Learning for Knowledge Graph Reasoning
Hou, Z., Jin, X., Li, Z., & Bai, L. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.412 · Cited by 35
Learning Task-Parameterized Skills From Few Demonstrations
Zhu, J., Gienger, M., & Kober, J. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3150013 · Cited by 34
Huang, T., Chen, K., Li, B., Liu, Y., & Dou, Q. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160327 · Cited by 34
Wang, J., Zhang, P., & Wang, Y. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110604 · Cited by 35
UAV Swarm Cooperative Dynamic Target Search: A MAPPO-Based Discrete Optimal Control Method
Wei, D., Zhang, L., Liu, Q., Chen, H. P., & Huang, J. (2024) · Drones · doi:10.3390/drones8060214 · Cited by 39
Marzari, L., Pore, A., Dall’Alba, D., Aragón-Camarasa, G., Farinelli, A., & Fiorini, P. (2021) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar53236.2021.9659344 · Cited by 34
Wang, C., Wang, J., Ma, Z., Xu, M., Qi, K., Ji, Z., & Wei, C. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3425755 · Cited by 33
Rate-Splitting for Intelligent Reflecting Surface-Aided Multiuser VR Streaming
Huang, R., Wong, V. W. S., & Schober, R. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3240710 · Cited by 33
Cai, M., Aasi, E., Belta, C. A., & Vasile, C.-I. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3246844 · Cited by 32
Hossain, R. R., Yin, T., Du, Y., Huang, R., Tan, J., Yu, W., Liu, Y., & Huang, Q. (2023) · Machine Learning · doi:10.1007/s10994-023-06422-w · Cited by 32
Trajectory Optimization for Unknown Constrained Systems using Reinforcement Learning
Ota, K., Jha, D. K., Oiki, T., Miura, M., Nammoto, T., Nikovski, D. N., & Mariyama, T. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968010 · Cited by 32
Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers
Wang, G., Xin, M., Wu, W., Liu, Z., & Wang, H. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3201705 · Cited by 32
Chen, J., Ma, R., & Oyekan, J. O. (2023) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104489 · Cited by 31
Lassoued, S., & Schwung, A. (2024) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2024.04.028 · Cited by 31
Hu, Z., Gao, X., Wan, K., Neretin, E. S., & Li, J. (2022) · Chinese Journal of Aeronautics · doi:10.1016/j.cja.2022.09.008 · Cited by 31
Multiagent Deep Reinforcement Learning With Demonstration Cloning for Target Localization
Alagha, A., Mizouni, R., Bentahar, J., Otrok, H., & Singh, S. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3262663 · Cited by 31
Hybrid Trajectory and Force Learning of Complex Assembly Tasks: A Combined Learning Framework
Wang, Y., Beltran-Hernandez, C. C., Wan, W., & Harada, K. (2021) · IEEE Access · doi:10.1109/access.2021.3073711 · Cited by 30
An Adaptive Imitation Learning Framework for Robotic Complex Contact-Rich Insertion Tasks
Wang, Y., Beltran-Hernandez, C. C., Wan, W., & Harada, K. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.777363 · Cited by 30
Chen, L., Lei, Y., Jin, S., Zhang, Y., & Zhang, L. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3400189 · Cited by 35
Few-Shot Bayesian Imitation Learning with Logical Program Policies
Silver, T., Allen, K. R., Lew, A. K., Kaelbling, L. P., & Tenenbaum, J. B. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6587 · Cited by 30
A Framework for Learning From Demonstration With Minimal Human Effort
Rigter, M. P., Lacerda, B., & Hawes, N. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2970619 · Cited by 29
Qiu, Y., Jin, Y., Yu, L., Wang, J., Wang, Y., & Zhang, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3240671 · Cited by 29
Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous DrivingPreprint
Wu, J., Huang, Z., Huang, C., Hu, Z., Hang, P., Xing, Y., & Lv, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07246 · Cited by 46
Behavior Planning at Urban Intersections through Hierarchical Reinforcement Learning
Qiao, Z., Schneider, J., & Dolan, J. M. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561095 · Cited by 32
Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
Zhan, A., Zhao, R., Pinto, L., Abbeel, P., & Laskin, M. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981055 · Cited by 58
Self-citationReinforcement Learning with Videos: Combining Offline Observations with InteractionPreprint
Schmeckpeper, K., Rybkin, O., Daniilidis, K., Levine, S., & Finn, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06507 · Cited by 57
Robotic Imitation of Human Assembly Skills Using Hybrid Trajectory and Force Learning
Wang, Y., Beltran-Hernandez, C. C., Wan, W., & Harada, K. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561619 · Cited by 27
Luo, S., & Schomaker, L. (2023) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2023.104545 · Cited by 27
Adaptive Hybrid Optimization Learning-Based Accurate Motion Planning of Multi-Joint Arm
Bai, C., Zhang, J., Guo, J., & Yue, C. P. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3262109 · Cited by 26
Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation
Khandate, G., Shang, S., Chang, E. T., Saidi, T. L., Adams, J., & Ciocarlie, M. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.020 · Cited by 26
UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking
Wei, M., Wang, S., Zheng, J., & Chen, D. (2018) · IEEE Access · doi:10.1109/access.2018.2872751 · Cited by 27
CURIOUS: Intrinsically Motivated Multi-Task Multi-Goal Reinforcement Learning
Colas, C., Fournier, P., Sigaud, O., & Oudeyer, P. (2018) · HAL AMU · Cited by 25
Investigating exploration for deep reinforcement learning of concentric tube robot control
Iyengar, K., Dwyer, G., & Stoyanov, D. V. (2020) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-020-02194-z · Cited by 27
A conceptual framework for externally-influenced agents: an assisted reinforcement learning review
Bignold, A., Cruz, F., Taylor, M. E., Brys, T., Dazeley, R., Vamplew, P., & Foale, C. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03489-y · Cited by 24
Crash mitigation controller for unavoidable T-bone collisions using reinforcement learning
Hou, X., Zhang, J., He, C., Li, C., Ji, Y., & Han, J. (2022) · ISA Transactions · doi:10.1016/j.isatra.2022.03.021 · Cited by 24
Learning Skills From Demonstrations: A Trend From Motion Primitives to Experience Abstraction
Tavassoli, M., Katyara, S., Pozzi, M., Deshpande, N., Caldwell, D. G., & Prattichizzo, D. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3296166 · Cited by 24
Autonomous Reinforcement Learning of Multiple Interrelated Tasks
Santucci, V. G., Baldassarre, G., & Cartoni, E. (2019) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2019.8850713 · Cited by 24
Hierarchical Deep Q-Network from imperfect demonstrations in Minecraft
Skrynnik, A., Staroverov, A., Aitygulov, E., Aksenov, K., Davydov, V., & Panov, A. I. (2020) · Cognitive Systems Research · doi:10.1016/j.cogsys.2020.08.012 · Cited by 24
Tan, M., Zhao, J., Liu, X., Su, Y., Wang, L., Wang, R., & Dai, Z. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109579 · Cited by 23
Secondary crash mitigation controller after rear-end collisions using reinforcement learning
Hou, X., Gan, M., Zhang, J., Zhao, S., & Ji, Y. (2023) · Advanced Engineering Informatics · doi:10.1016/j.aei.2023.102176 · Cited by 23
RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration
Pavse, B. S., Torabi, F., Hanna, J. P., Warnell, G., & Stone, P. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3010750 · Cited by 25
Accelerated Robot Learning via Human Brain Signals
Akinola, I., Wang, Z., Shi, J., He, X., Lapborisuth, P., Xu, J., Watkins-Valls, D., Sajda, P., & Allen, P. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196566 · Cited by 23
DeepMNavigate: Deep Reinforced Multi-Robot Navigation Unifying Local & Global Collision Avoidance
Tan, Q., Fan, T., Pan, J., & Manocha, D. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341805 · Cited by 23
Backplay: "Man muss immer umkehren"Preprint
Resnick, C., Răileanu, R., Kapoor, S., Peysakhovich, A., Cho, K., & Bruna, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06919 · Cited by 47
Deep Reinforcement Learning based Local Planner for UAV Obstacle Avoidance using Demonstration DataPreprint
He, L., Aouf, N., Whidborne, J. F., & Song, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02521 · Cited by 23
Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control
Hermann, L., Argus, M. J., Eitel, A., Amiranashvili, A., Burgard, W., & Brox, T. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197108 · Cited by 23
Learning synergies based in‐hand manipulation with reward shaping
Deng, Z., & Zhang, J. W. (2020) · CAAI Transactions on Intelligence Technology · doi:10.1049/trit.2019.0094 · Cited by 23
Knowledge-Guided Reinforcement Learning Control for Robotic Lower Limb Prosthesis
Gao, X. H., Si, J., Wen, Y., Li, M., & Huang, H. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196749 · Cited by 22
Planning-Augmented Hierarchical Reinforcement Learning
Gieselmann, R., & Pokorny, F. T. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3071062 · Cited by 23
Hierarchical Policies for Cluttered-Scene Grasping With Latent Plans
Wang, L., Meng, X., Yu, X., & Fox, D. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3143198 · Cited by 22
Zhang, Z., Hong, J., Enayati, A. M. S., & Najjaran, H. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3468770 · Cited by 21
Añazco, E. V., Lopez, P. R., Park, N., Oh, J.-H., Ryu, G., Al–antari, M. A., & Kim, T. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01870-6 · Cited by 21
Xu, D., Zhu, F., Liu, Q., & Zhao, P. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115680 · Cited by 21
Choi, C.-W., & Kim, J. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111739 · Cited by 21
Li, W., Wang, Y., Liang, Y., & Pham, D. T. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102625 · Cited by 21
Kinose, A., & Taniguchi, T. (2020) · Advanced Robotics · doi:10.1080/01691864.2020.1778521 · Cited by 22
Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras
Akinola, I., Varley, J., & Kalashnikov, D. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197181 · Cited by 21
Zhang, Y., Yan, C., Xiao, J., & Feroskhan, M. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3464510 · Cited by 20
Coordinating Disaster Emergency Response with Heuristic Reinforcement Learning
Yang, Z., Nguyen, L., Zhu, J., Pan, Z., Li, J., & Jin, F. (2020) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam49781.2020.9381416 · Cited by 20
Meta-Residual Policy Learning: Zero-Trial Robot Skill Adaptation via Knowledge Fusion
Peng, H., Lu, T., Cui, S., Wei, J., Cai, Y., & Wang, S. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3146916 · Cited by 20
AMBF-RL: A real-time simulation based Reinforcement Learning toolkit for Medical Robotics
Varier, V. M., Rajamani, D. K., Tavakkolmoghaddam, F., Munawar, A., & Fischer, G. S. (2022) · International Symposium on Medical Robotics · doi:10.1109/ismr48347.2022.9807609 · Cited by 20
Reinforcement learning from expert demonstrations with application to redundant robot control
Ramírez, J., & Yu, W. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105753 · Cited by 22
Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspective
Yang, X., Ji, Z., Wu, J., & Lai, Y. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3277288 · Cited by 24
Hou, Y., Liang, X., Lv, M., Yang, Q., & Li, Y. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106703 · Cited by 21
A Q-learning approach to the continuous control problem of robot inverted pendulum balancing
Safeea, M., & Neto, P. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200313 · Cited by 20
Zhou, T., Wang, L., Chen, R., Wang, W., & Liu, Y. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341449 · Cited by 22
Lee, H.-R., & Lee, T. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/hxwh4898 · Cited by 19
Hybrid Reinforcement Learning with Expert State Sequences
Guo, X., Chang, S., Yu, M., Tesauro, G. J., & Campbell, M. S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013739 · Cited by 23
Matheron, G., Perrin, N., & Sigaud, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8_24 · Cited by 21
Augmenting GAIL with BC for sample efficient imitation learningPreprint
Jena, R., Liu, C., & Sycara, K. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.07798 · Cited by 21
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 27
“Demonstration learning (DemoL) algorithms often used in robotics applications are DQfD [218], DDPGfD [219] and Q-filtered BC [220].”
MethodLearning contact-rich whole-body manipulation with example-guided reinforcement learning
Barreiros, J. A. L., Önol, A. Ö., Zhang, M., Creasey, S., Goncalves, A., Beaulieu, A., Bhat, A. D., Tsui, K., & Alspach, A. N. (2025) · Science Robotics · doi:10.1126/scirobotics.ads6790 · Cited by 22
Reward Learning From Very Few Demonstrations
Eteke, C., Kebüde, D., & Akgün, B. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.3038698 · Cited by 19
Lin, G., Huang, P., Wang, M., Xu, Y., Zhang, R., & Zhu, L. (2022) · Agronomy · doi:10.3390/agronomy12092157 · Cited by 19
Shi, J., Zhang, T., Zhan, J., Chen, S., Xin, J., & Zheng, N. (2023) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55152.2023.10186577 · Cited by 19
Metadata
- Authors
- Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P.
- Affiliations
- OpenAI (United States) (US)
- ORCID
- Abbeel, Pieter
- Venue
- 2018 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 6292-6299
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2756826236
- Cited by
- 712
- References
- 44
- Citations by year
- 2026: 24 · 2025: 72 · 2024: 86 · 2023: 111 · 2022: 93 · 2021: 117 · 2020: 105 · 2019: 67 · 2018: 35
- FWCI
- 33.08
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- exploration in reinforcement learning, sparse rewards, demonstrations, deep deterministic policy gradient, hindsight experience replay, long-horizon tasks, continuous control, robotic manipulation, block stacking, robot manipulators, behavior cloning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review