Cited by 712Reinforcement Learning in RoboticsRobot Manipulation and LearningDomain Adaptation and Few-Shot Learning

The DOI matches its Crossref record; 40 of the 44 references it cites were found in a registry or an index.

Overcoming Exploration in Reinforcement Learning with Demonstrations

Open access

Nair, Ashvin, McGrew, Bob, Andrychowicz, Marcin, Zaremba, Wojciech, Abbeel, Pieter·2018·2018 IEEE International Conference on Robotics and Automation (ICRA)

DOI 10.1109/icra.2018.8463162

Open this paper in CoChat — free account Open PDF

Abstract

Exploration in environments with sparse rewards has been a persistent problem in reinforcement learning (RL). Many tasks are natural to specify with a sparse reward, and manually shaping a reward function can result in suboptimal performance. However, finding a non-zero reward is exponentially more difficult with increasing task horizon or action dimensionality. This puts many real-world tasks out of practical reach of RL methods. In this work, we use demonstrations to overcome the exploration problem and successfully learn to perform long-horizon, multi-step robotics tasks with continuous control such as stacking blocks with a robot arm. Our method, which builds on top of Deep Deterministic Policy Gradients and Hindsight Experience Replay, provides an order of magnitude of speedup over RL on simulated robotics tasks. It is simple to implement and makes only the additional assumption that we can collect a small set of demonstrations. Furthermore, our method is able to solve tasks not solvable by either RL or behavior cloning alone, and often ends up outperforming the demonstrator policy.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/icra.2018.8463162

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Probabilistic roadmaps for path planning in high-dimensional configuration spaces
Kavraki, L. E., Svestka, P., Latombe, J.-C., & Overmars, M. H. (1996) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.508439 · Cited by 6,343 · cited 1×
“More recent work on task and motion planning considers both logical and physical aspects of the task [27], [28], [29], but requires domainspecific engineering.”
Background
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Self-citation
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198

Flagged first, then by citations

Show all 44 references

4 of the 44 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

hester, 2017, Learning from demonstrations for real world reinforcement learning
ng, 2000, Algorithms for Inverse Reinforcement Learning, International Conference on Machine Learning (ICML)
ziebart, 2008, Maximum Entropy Inverse Reinforcement Learning, AAAI Conference on Artificial Intelligence, 1433

Sorted by citations

Items 301–400 of 710

SimStu-Transformer: A Transformer-Based Approach to Simulating Student Behaviour
Li, Z., Shi, L., Cristea, A. I., Zhou, Y., Xiao, C., & Pan, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-11647-6_67 · Cited by 9
Urban Traffic Signal Control with Reinforcement Learning from Demonstration Data
Wang, M., Wu, L., Li, J., Wu, D., & Ma, C. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892538 · Cited by 9
Intrinsically motivated reinforcement learning based recommendation with counterfactual data augmentation
Chen, X., Wang, S., Qi, L., Li, Y., & Yao, L. (2023) · World Wide Web · doi:10.1007/s11280-023-01187-7 · Cited by 9
State Representation Learning from Demonstration
Merckling, A., Coninx, A., Cressot, L., Doncieux, S., & Perrin, N. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64580-9_26 · Cited by 9
Hierarchical Policy Learning is Sensitive to Goal Space DesignPreprint
Dwiel, Z., Candadai, M., Phielipp, M., & Bansal, A. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01537 · Cited by 9
Learning Sparse Rewarded Tasks from Sub-Optimal DemonstrationsPreprint
Zhu, Z., Lin, K., Dai, B., & Zhou, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00530 · Cited by 9
Policy learning in SE(3) action spacesPreprint
Wang, D., Kohler, C., & Platt, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02798 · Cited by 11
Leveraging control priors in deep reinforcement learning for robotics
Rana, K. (2023) · Queensland University of Technology · doi:10.5204/thesis.eprints.238632 · Cited by 9
Learning Multi-Stage Tasks with One Demonstration via Self-Replay
Di Palo, N., & Johns, E. (2021) · arXiv · doi:10.48550/arxiv.2111.07447 · Cited by 8
“3 - Reinforcement Learning from Demonstrations: we developed Deep Deterministic Policy Gra300 dient from Demonstrations (DDPGfD) [22, 23], that takes human demonstrations to train a policy, 301”
Method
Efficient Stacking and Grasping in Unstructured Environments
Wang, F., Liu, Y., Shi, M., Chen, C., Liu, S., & Zhu, J. (2024) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-024-02078-3 · Cited by 9
Learn-Gen-Plan: Bridging the Gap Between Vision Language Models and Real-World Long-Horizon Dexterous Manipulations
Peng, H., Cui, S., Wei, J., Lu, T., Cai, Y., & Wang, S. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3567609 · Cited by 10
Interactive imitation learning for dexterous robotic manipulation: challenges and perspectives—a survey
Welte, E., & Rayyes, R. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1682437 · Cited by 11
Unobstructed Programming-by-Demonstration for Force-Based Assembly Utilizing External Force-Torque Sensors
Bargmann, D., Tenbrock, P. G., Halt, L., Nägele, F., Kraus, W., & Huber, M. F. (2021) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc52423.2021.9658707 · Cited by 8
Highlights of Lane-Free Automated Vehicle Traffic with Nudging
Papageorgiou, M., Typaldos, P., Theodosis, D., Chalkiadakis, G., Chrysomallis, I., Dabestani, N., Karafyllis, I., Malekzadeh, M., Naderi, M., Papamichail, I., et al. (2024) · Springer tracts on transportation and traffic · doi:10.1007/978-3-031-64769-7_6 · Cited by 8
Demonstration actor critic
Liu, G., Zhao, L., Zhang, P., Bian, J., Qin, T., Yu, N., & Liu, T. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.12.116 · Cited by 9
Relative control of an underactuated spacecraft using reinforcement learning
Khoroshylov, S. V., & Redka, M. O. (2020) · Technical mechanics · doi:10.15407/itm2020.04.043 · Cited by 8
OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching
Hoshino, H., Ota, K., Kanezaki, A., & Yokota, R. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811660 · Cited by 8
Learning From Guided Play: Improving Exploration for Adversarial Imitation Learning With Simple Auxiliary Tasks
Ablett, T., Chan, B., & Kelly, J. W. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3236882 · Cited by 8
HTTP adaptive streaming scheme based on reinforcement learning with edge computing assistance
Kim, M., & Chung, K. (2022) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2023.103604 · Cited by 8
Learning from Demonstration Based on a Classification of Task Parameters and Trajectory Optimization
Vidaković, J., Jerbić, B., Šekoranja, B., Švaco, M., & Šuligoj, F. (2019) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-019-01101-2 · Cited by 8
Addressing Sample Inefficiency and Reward Bias in Inverse Reinforcement Learning.Preprint
Kostrikov, I., Agrawal, K. K., Levine, S., & Tompson, J. (2018) · arXiv (Cornell University) · Cited by 8
ZPD Teaching Strategies for Deep Reinforcement Learning from DemonstrationsPreprint
Seita, D., Chan, D. M., Rao, R., Tang, C., Mandi, Z., & Canny, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12154 · Cited by 8
Exploration-efficient Deep Reinforcement Learning with Demonstration Guidance for Robot ControlPreprint
Lin, K. C., Gong, L., Li, X., Sun, T., Chen, B., Liu, C., Zhang, Z., Pu, J., & Zhang, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.12089 · Cited by 7
Adaptive Procedural Task Generation for Hard-Exploration ProblemsPreprint
Fang, K., Zhu, Y., Savarese, S., & Fei-Fei, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00350 · Cited by 13
Locally Persistent Exploration in Continuous Control Tasks with Sparse RewardsPreprint
Amin, S., Gomrokchi, M., Aboutalebi, H., Satija, H., & Precup, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.13658 · Cited by 14
Trust-Calibrated Human-in-the-Loop Reinforcement Learning for Safe and Efficient Autonomous Navigation
Hu, D. T., Zhou, G., Wu, J., & Huang, C. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3588176 · Cited by 7
A Sample Efficiency Improved Method via Hierarchical Reinforcement Learning Networks
Chen, Q., Dallas, E., Shahverdi, P., Korneder, J., Rawashdeh, O. A., & Louie, W.-Y. G. (2022) · IEEE RO-MAN · doi:10.1109/ro-man53752.2022.9900738 · Cited by 7
Sample-Efficient Multi-Agent Reinforcement Learning with Demonstrations for Flocking Control
Qiu, Y., Zhan, Y., Jin, Y., Wang, J., & Zhang, X. (2022) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2022-fall57202.2022.10012835 · Cited by 7
Application of Reinforcement Learning to UR10 Positioning for Prioritized Multi-Step Inspection in NVIDIA Omniverse
Bahrpeyma, F., Sunilkumar, A., & Reichelt, D. (2023) · IEEE Symposium on Industrial Electronics & Applications (ISIEA) · doi:10.1109/isiea58478.2023.10212317 · Cited by 7
Learning Prehensile Dexterity by Imitating and Emulating State-Only Observations
Han, Y., Chen, Z., Williams, K. A., & Ravichandar, H. C. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3443595 · Cited by 7
Optimizing Crowd-Aware Multi-Agent Path Finding through Local Communication with Graph Neural Networks
Pham, P., & Bera, A. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801969 · Cited by 7
Cooperative and Competitive Reinforcement and Imitation Learning for a Mixture of Heterogeneous Learning Modules
Uchibe, E. (2018) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2018.00061 · Cited by 7
Vision-Based Deep Reinforcement Learning For UR5 Robot Motion Control
Jiang, R., Wang, Z., He, B., & Di, Z. (2021) · International Conference on Consumer Electronics and Computer Engineering (ICCECE) · doi:10.1109/iccece51280.2021.9342134 · Cited by 7
Causal Based Action Selection Policy for Reinforcement Learning
Feliciano-Avelino, I., Méndez-Molina, A., Morales, E. F., & Sucar, L. E. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89817-5_16 · Cited by 7
Transfer Learning and Curriculum Learning in Sokoban
Yang, Z., Preuß, M., & Plaat, A. (2022) · Communications in computer and information science · doi:10.1007/978-3-030-93842-0_11 · Cited by 7
Imitation Learning
Ding, Z. (2020) · Springer eBooks · doi:10.1007/978-981-15-4095-0_8 · Cited by 7
Exploiting Abstract Symmetries in Reinforcement Learning for Complex Environments
Gupta, K., & Najjaran, H. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811652 · Cited by 7
Reinforcement Learning With Adaptive Policy Gradient Transfer Across Heterogeneous Problems
Zhang, G., Feng, L., Wang, Y., Li, M., Xie, H. L., & Tan, K. C. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3361860 · Cited by 7
Stage-Wise Learning of Reaching Using Little Prior Knowledge
Bourdonnaye, F. D. L., Teulière, C., Triesch, J., & Château, T. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00110 · Cited by 7
Learning a Behavioral Repertoire from Demonstrations
Justesen, N., González-Duque, M., Cabarcas, D., Mouret, J.-B., & Risi, S. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231897 · Cited by 7
Attentive Task-Net: Self Supervised Task-Attention Network for Imitation Learning using Video Demonstration
Ramachandruni, K., Madhu Babu, V., Majumder, A., Dutta, S., & Kumar, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197544 · Cited by 7
Learning intraoperative organ manipulation with context-based reinforcement learning
D’Ettorre, C., Zirino, S., Dei, N. N., Stilli, A., Momi, E. D., & Stoyanov, D. V. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02630-2 · Cited by 7
Learning from Ambiguous Demonstrations with Self-Explanation Guided Reinforcement Learning
Zha, Y., Guan, L., & Subbarao Kambhampati, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28907 · Cited by 8
Transferring Human Manipulation Knowledge to Robots with Inverse Reinforcement Learning
Hansen, E. B., Andersen, R. E., Madsen, S., & Bøgh, S. (2020) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii46433.2020.9025873 · Cited by 7
Sample-Efficient Learning of Nonprehensile Manipulation Policies via Physics-Based Informed State DistributionsPreprint
Pinto, L., Mandalika, A., Hou, B., & Srinivasa, S. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10654 · Cited by 7
Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative SamplingPreprint
Luo, Y., Xu, H., & Ma, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.05634 · Cited by 7
Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale StudyPreprint
Luo, J., Sushkov, O. O., Pevceviciute, R., Lian, W., Su, C., Vecerík, M., Ye, N., Stefan Schaal, A., & Scholz, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11512 · Cited by 7
RoboAuditor: Goal-Oriented Robotic System for Assessing Energy-intensive Indoor Appliance via Visual Language Models
Cai, W., Huang, L., & Zou, Z. (2023) · ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys) · doi:10.1145/3600100.3623739 · Cited by 7
Multi-objective Cross-task Learning via Goal-conditioned GPT-based Decision Transformers for Surgical Robot Task Automation
Fu, J., Long, Y., Chen, K., Wei, W., & Dou, Q. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611051 · Cited by 9
Domain Adapting Deep Reinforcement Learning for Real-World Speech Emotion Recognition
Rajapakshe, T., Rana, R. K., Khalifa, S., & Schuller, B. W. (2024) · IEEE Access · doi:10.1109/access.2024.3519761 · Cited by 7
Human-AI collaboration in real-world complex environment with reinforcement learning
Islam, M. S., Das, S., Gottipati, S. K., Duguay, W., Mars, C., Arabneydi, J., Fagette, A., Guzdial, M., & Taylor, M. E. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11288-1 · Cited by 11
RIRL: A Recurrent Imitation and Reinforcement Learning Method for Long-Horizon Robotic Tasks
Yu, Z., Zhang, J., Mao, S., Periaswamy, S. C., & Patton, J. (2022) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc49033.2022.9700600 · Cited by 6
Task Phasing: Automated Curriculum Learning from Demonstrations
Bajaj, V., Sharon, G., & Stone, P. (2023) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v33i1.27235 · Cited by 6
Augmentation Enables One-Shot Generalization in Learning from Demonstration for Contact-Rich Manipulation
Li, X., Baum, M., & Brock, O. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341625 · Cited by 6
Real-time power scheduling through reinforcement learning from demonstrations
Liu, S., Liu, J., Yang, N., Huang, Y., Jiang, Q., & Gao, Y. (2024) · Electric Power Systems Research · doi:10.1016/j.epsr.2024.110638 · Cited by 6
Achieving Safe Deep Reinforcement Learning via Environment Comprehension Mechanism
Pai, P., Fei, Z., Liu, Q., Zhao, P., & Wu, W. (2021) · Chinese Journal of Electronics · doi:10.1049/cje.2021.07.025 · Cited by 6
LORM: a novel reinforcement learning framework for biped gait control
Zhang, W., Jiang, Y., Farrukh, F. U. D., Zhang, C., Zhang, D., & Wang, G. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.927 · Cited by 6
An Underwater Soft Claw Based on Bionic Principle
Sun, Y., Han, F., Zhang, D., Zhao, X., & Ye, D. (2021) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio54168.2021.9739295 · Cited by 6
Incorporating External Knowledge Reasoning for Vision-and-Language Navigation with Assistant’s Help
Li, X., Zhang, Y., Yuan, W., & Luo, J. (2022) · Applied Sciences · doi:10.3390/app12147053 · Cited by 6
Analyzing and Overcoming Degradation in Warm-Start Reinforcement Learning
Wexler, B., Sarafian, E., & Kraus, S. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981286 · Cited by 6
Towards Improving Exploration in Self-Imitation Learning using Intrinsic Motivation
Andrés, A., Villar-Rodríguez, E., & Ser, J. D. (2022) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci51031.2022.10022199 · Cited by 6
A Fast Online Planning Under Partial Observability Using Information Entropy Rewards
Chen, Y., Liu, J., Lan, L., Zhang, H., Miao, Z., & Wang, Y. (2023) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2023.3248086 · Cited by 6
A Penetration Method for UAV Based on Distributed Reinforcement Learning and Demonstrations
Li, K., Wang, Y., Zhuang, X., Yin, H., Liu, X., & Li, H. (2023) · Drones · doi:10.3390/drones7040232 · Cited by 6
Seq2Seq Imitation Learning for Tactile Feedback-based Manipulation
Yang, W., Angleraud, A., Pieters, R., Pajarinen, J., & Kämäräinen, J. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161145 · Cited by 8
Blockchain-Assisted Demonstration Cloning for Multiagent Deep Reinforcement Learning
Alagha, A., Bentahar, J., Otrok, H., Singh, S., & Mizouni, R. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3316078 · Cited by 6
Deep Reinforcement Learning With Explicit Context Representation
Munguia‐Galeano, F., Tan, A., & Ji, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3325633 · Cited by 7
Automatic enhancement of vascular configuration for self-healing concrete through reinforcement learning approach
Wan, Z., Xu, Y., Chang, Z., Liang, M., & Šavija, B. (2023) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2023.134592 · Cited by 6
Hybrid of Reinforcement and Imitation Learning for Human-Like Agents
Dossa, R. F. J., Lian, X., Hirokazu, N., Matsubara, T., & Uehara, K. (2020) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2019edp7298 · Cited by 6
Improving environmental awareness for autonomous vehicles
Peixoto, M. J. P., & Azim, A. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03468-6 · Cited by 6
BulletArm: An Open-Source Robotic Manipulation Benchmark and Learning Framework
Wang, D., Kohler, C., Zhu, X., Jia, M., & Platt, R. (2023) · Springer proceedings in advanced robotics · doi:10.1007/978-3-031-25555-7_23 · Cited by 8
Equivalence of the Projected Forward Dynamics and the Dynamically Consistent Inverse Solution
Moura, J., Ivan, V. K., Erden, M. S., & Vijayakumar, S. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.036 · Cited by 6
Human-in-the-Loop Methods for Data-Driven and Reinforcement Learning SystemsPreprint
Goecks, V. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.13221 · Cited by 14
Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative ModelsPreprint
Wu, Y., Mozifian, M., & Shkurti, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.01298 · Cited by 6
Wish you were here: Hindsight Goal Selection for long-horizon dexterous manipulationPreprint
Davchev, T., Sushkov, O. O., Regli, J.-B., Stefan Schaal, A., Aytar, Y., Wulfmeier, M., & Scholz, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00597 · Cited by 6
Abstract Demonstrations and Adaptive Exploration for Efficient and Stable Multi-step Sparse Reward Reinforcement Learning
Yang, X., Ji, Z., Wu, J., & Lai, Y. (2022) · International Conference on Automation and Computing (ICAC) · doi:10.1109/icac55051.2022.9911100 · Cited by 6
Extended residual learning with one-shot imitation learning for robotic assembly in semi-structured environment
Wang, C., Su, C., Sun, B., Chen, G., & Xie, L. (2024) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1355170 · Cited by 5
Robust Visuomotor Control for Humanoid Loco-Manipulation Using Hybrid Reinforcement Learning
Wang, C., Huang, Q., Chen, X., Zhang, Z., & Shi, J. (2025) · Biomimetics · doi:10.3390/biomimetics10070469 · Cited by 5
Safe Multi-Agent Navigation Guided by Goal-Conditioned Safe Reinforcement Learning
Feng, M., Parimi, V., & Williams, B. C. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127461 · Cited by 5
Value-Informed Skill Chaining for Policy Learning of Long-Horizon Tasks with Surgical Robot
Huang, T., Chen, K., Wei, W., Li, J., Long, Y., & Dou, Q. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342180 · Cited by 5
D-MARL: A Dynamic Communication-Based Action Space Enhancement for Multi Agent Reinforcement Learning Exploration of Large Scale Unknown Environments
Calzolari, G., Sumathy, V., Kanellakis, C., & Nikolakopoulos, G. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801319 · Cited by 5
Explicit-Implicit Subgoal Planning for Long-Horizon Tasks With Sparse Rewards
Wang, F., Duan, A., Zhou, P., Huo, S., Guo, G., Yang, C., & Navarro-Alarcón, D. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3574162 · Cited by 6
JAVRIS: Joint Artificial Visual Prediction and Control for Remote-(Robot) Interaction Systems
Grohmann, A. I., Busch, J. V. S., Perez, A. B., Lehmann, C., & Fitzek, F. H. P. (2022) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc49033.2022.9700532 · Cited by 5
Deep multi-agent fusion Q-Network for graph generation
Rassil, A., Chougrad, H., & Zouaki, H. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110509 · Cited by 6
Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning
George, A., Bartsch, A., & Farimani, A. B. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161119 · Cited by 11
Interpretable Imitation Learning with Symbolic Rewards
Bougie, N., Onishi, T., & Tsuruoka, Y. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3627822 · Cited by 9
Exploring the Use of Deep Reinforcement Learning Algorithms for Wound-Approaching Trajectories in Robot-Assisted Minimally Invasive Surgery
Caianiello, M., Iacono, C., Imperato, A., & Ficuciello, F. (2023) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar58858.2023.10406708 · Cited by 5
Goal exploration augmentation via pre-trained skills for sparse-reward long-horizon goal-conditioned reinforcement learning
Wu, L., & Chen, K. (2024) · Machine Learning · doi:10.1007/s10994-023-06503-w · Cited by 5
Open-Source Reinforcement Learning Environments Implemented in MuJoCo with Franka Manipulator
Xu, Z., Li, Y., Yang, X., Zhao, Z., Zhuang, L., & Zhao, J. (2024) · IEEE International Conference on Advanced Intelligent Mechatronics (AIM) · doi:10.1109/aim55361.2024.10636979 · Cited by 5
Reinforcement learning based automatic block decomposition of solid models for hexahedral meshing
Zhang, S., Guan, Z., Wang, X., Tan, P., & Jiang, H. (2025) · Computer-Aided Design · doi:10.1016/j.cad.2025.103850 · Cited by 5
Intelligent Eco-Driving Control for Urban CAVs Using a Model-Based Controller Assisted Deep Reinforcement Learning
Li, J., Wu, X., Bai, X.-X. ‘., Liu, Y., & Xu, M. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3559916 · Cited by 5
Deep Reinforcement Learning with Feedback-based Exploration
Scholten, J., Wout, D., Celemin, C. E., & Kober, J. (2019) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029503 · Cited by 7
Parallel Multi-Environment Shaping Algorithm for Complex Multi-step Task
Ma, C., Li, Z., Lin, D., & Zhang, J. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.04.070 · Cited by 5
POPO: Pessimistic Offline Policy Optimization
He, Q., Hou, X., & Liu, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747886 · Cited by 6
Deep Reinforcement Learning for Stock Recommendation
Shen, Y., Liu, T., Liu, W., Xu, R., Li, Z., & Wang, J. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2050/1/012012 · Cited by 5
Learning Design and Construction with Varying-Sized Materials via Prioritized Memory Resets
Li, Y., Kong, T., Li, L., & Wu, Y. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811624 · Cited by 5
Learning good policies from suboptimal demonstrations
Li, Y., Kash, I. A., & Hofmann, K. (2018) · European Workshop on Reinforcement Learning · Cited by 5
Match Plan Generation in Web Search with Parameterized Action Reinforcement Learning
Luo, Z., Zhao, L., Cheng, W., Chen, S., Chen, Q., Xue, H., Wang, H., Liu, C., Yang, M., & Zhang, L. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449862 · Cited by 5
Safer Deep RL with Shallow MCTS: A Case Study in PommermanPreprint
Kartal, B., Hernández-Leal, P., Gao, C., & Taylor, M. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05759 · Cited by 7
Physically Embedded Planning Problems: New Challenges for Reinforcement LearningPreprint
Mirza, M., Jaegle, A., Hunt, J. J., Guez, A., Tunyasuvunakool, S., Muldal, A., Weber, T., Karkus, P., Racanière, S., Buesing, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05524 · Cited by 12
Tolerance-Guided Policy Learning for Adaptable and Transferrable Delicate Industrial InsertionPreprint
Niu, B., Wang, C., & Liu, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.02303 · Cited by 5

Authors: Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P.
Affiliations: OpenAI (United States) (US)
ORCID: Abbeel, Pieter
Venue: 2018 IEEE International Conference on Robotics and Automation (ICRA)
Publisher: IEEE
Published: 2018-05
Pages: 6292-6299
ISSN: 1050-4729 (ISSN-L)
DOI: 10.1109/icra.2018.8463162
OpenAlex: W2756826236
Cited by: 712
References: 44
Citations by year: 2026: 24 · 2025: 72 · 2024: 86 · 2023: 111 · 2022: 93 · 2021: 117 · 2020: 105 · 2019: 67 · 2018: 35
FWCI: 33.08
Type: conference paper
Language: en
Open access: green
arxiv.org
Locations: Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords: exploration in reinforcement learning, sparse rewards, demonstrations, deep deterministic policy gradient, hindsight experience replay, long-horizon tasks, continuous control, robotic manipulation, block stacking, robot manipulators, behavior cloning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Overcoming Exploration in Reinforcement Learning with Demonstrations

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks