The DOI matches its DataCite record; 41 of the 42 references it cites were found in a registry or an index, and 1 are not checked yet.
Exploration by Random Network Distillation
Open accessAbstract
We introduce an exploration bonus for deep reinforcement learning methods that is easy to implement and adds minimal overhead to the computation performed. The bonus is the error of a neural network predicting features of the observations given by a fixed randomly initialized neural network. We also introduce a method to flexibly combine intrinsic and extrinsic rewards. We find that the random network distillation (RND) bonus combined with this increased flexibility enables significant progress on several hard exploration Atari games. In particular we establish state of the art performance on Montezuma's Revenge, a game famously difficult for deep reinforcement learning methods. To the best of our knowledge, this is the first method that achieves better than average human performance on this game without using demonstrations or having access to the underlying state of the game, and occasionally completes the first level.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Solving these problems requires a good global exploration strategy [211] that can better balance long-term and short-term environment information [195].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [211]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Specifically, although our experiments showed strong empirical gains in simplified environments, the current approach is vulnerable to the noisy TV problem [19], where novelty-driven level prioritization may focus on unpredictable noise elements of the environment, rather than beneficial learning experiences.”
From Improving Environment Novelty Quantification for Effective Unsupervised Environment Design (2024) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This solution limits the agent looping over distinct states or getting stuck in a state like the noisy TV problem [25].”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Since the feedback is infinite, the agent may loop over novel states or get stuck in a novel state [25].”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [25]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Self-citationPlaying Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,705
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317
Random Features for Large-Scale Kernel Machines
Rahimi, A., & Recht, B. (2007) · Cited by 2,783
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153
Rainbow: Combining Improvements in Deep Reinforcement Learning
Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., & Silver, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11796 · Cited by 1,846
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815
Research works printed but not linked: 13
- Yusuf Aytar, Tobias Pfaff, David Budden, Tom Le Paine, Ziyu Wang, and Nando de Freitas. Playing hard exploration games by watching YouTube. arXiv preprint arXiv:1805.11592 , 2018. · arXiv
- Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function. arXiv preprint , 2018.
- Justin Fu, John D Co-Reyes, and Sergey Levine. EX2: Exploration with exemplar models for deep reinforcement learning. NIPS , 2017.
- Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control. ICLR Workshop , 2017.
- Nick Haber, Damian Mrowca, Li Fei-Fei, and Daniel LK Yamins. Learning to play with intrinsicallymotivated self-aware agents. arXiv preprint arXiv:1802.07442 , 2018. · arXiv
- Rein Houthooft, Xi Chen, Yan Duan, John Schulman, Filip De Turck, and Pieter Abbeel. VIME: Variational information maximizing exploration. In NIPS , 2016.
- Manuel Lopes, Tobias Lang, Marc Toussaint, and Pierre-Yves Oudeyer. Exploration in model-based reinforcement learning by empirically estimating learning progress. In NIPS , 2012.
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nature , 518(7540):529-533, February 2015.
- Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In ICML , 2016.
- Junhyuk Oh, Yijie Guo, Satinder Singh, and Honglak Lee. Self-imitation learning. arXiv preprint arXiv:1806.05635 , 2018. · arXiv
- OpenAI, :, M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba. Learning Dexterous In-Hand Manipulation. ArXiv e-prints , August 2018.
- Georg Ostrovski, Marc G Bellemare, Aaron van den Oord, and R´ emi Munos. Count-based exploration with neural density models. International Conference for Machine Learning , 2018.
- Zichao Yang, Marcin Moczulski, Misha Denil, Nando de Freitas, Alex Smola, Le Song, and Ziyu Wang. Deep fried convnets. In Proceedings of the IEEE International Conference on Computer Vision , pp. 1476-1483, 2015.
Cited by
Sorted by citations
Items 501–533 of 533
Li, K., Wang, Q., Gong, M., Li, J., & Shi, H. (2023) · International Symposium on Computer Engineering and Intelligent Communications (ISCEIC) · doi:10.1109/isceic59030.2023.10271221
Emergence of tool construction and tool use through hierarchical reinforcement learning
Li, Q., & Choe, Y. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-96104-2.00008-7
Hu, M. Y. (2023) · Apress eBooks · doi:10.1007/978-1-4842-9606-6_13
Park, S., & Yoo, J. (2023) · Transactions of Korean Society of Automotive Engineers · doi:10.7467/ksae.2023.31.12.949
Uwano, F., Satoshi, H., & Takadama, K. (2024) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2024.p0380
Wang, J., Liu, B., Chen, J., Lei, T., & Ni, K. (2023) · International Conference on Information Education and Artificial Intelligence (ICIEAI) · doi:10.1145/3660043.3660202
Improving Deep Reinforcement Learning with Intrinsic Rewards via Self-Imitation Learning
Xu, M., & Zhao, Q. (2024) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac63405.2024.10598576
Model Based Reinforcement Learning Pre-Trained with Various State Data
Ono, M., & Ichise, R. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00169
Fear based Intrinsic Reward as a Barrier Function for Continuous Reinforcement Learning
Sanchez, R., Sahin, F., & Heard, J. R. (2024) · International Conference on System of Systems Engineering · doi:10.1109/sose62659.2024.10620974
An Efficient Multi-UAV Policy Learning Method under Binary Rewards
Chen, Y., & Li, J. (2024) · International Symposium on Intelligent Robotics and Systems (ISoIRS) · doi:10.1109/isoirs63136.2024.00021
Efficient Adversarially Guided Actor–Critic
Xu, M., Ge, S. S., Zhao, D., & Zhao, Q. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3453444
Autoencoder Reconstruction Model for Long-Horizon Exploration
Yan, R., Wu, Y., Gan, Y., Yang, Y., Yu, Z., Liu, Z., Zhang, X., Liang, L., & Cai, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650970
Absorb What You Need: Accelerating Exploration via Valuable Knowledge Extraction
Wu, Y., Wu, Z., Yan, R., Tao, P., & Xing, J. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651098
Progressively Learning to Reach Remote Goals by Continuously Updating Boundary Goals
Shao, M., Zhu, H., Zhao, D., Han, K., Jiang, F., Liu, S., & Zhang, W. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3428323
CuDA2: An Approach for Incorporating Traitor Agents Into Cooperative Multiagent Systems
Chen, Z., Liao, Y., Zhao, Y., Dai, Z., & Zhao, J. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3485726 · Cited by 1
Reward Shaping Based on Optimal-Policy-Free
Sang, J., Wang, Y., Khan, Z. A., & Zhou, X. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3489415
Can curiosity enhance the quality of on-demand delivery services?
Yoshioka, R., Moriyama, K., & Sakurai, Y. (2024) · IEEE International Conference on Agents (ICA) · doi:10.1109/ica63002.2024.00019
Ji, X., & Yi, J. (2024) · International Joint Conference on Robotics and Artificial Intelligence (JCRAI) · doi:10.1145/3696474.3698030
Novelty-based Sample Reuse for Continuous Robotics Control
Duan, K., Yang, K., Liu, H., & Wang, X. (2024) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio64047.2024.10907722 · Cited by 1
Ma, Z., Cui, Y., & Huang, Y. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919663
GOME-NGU: Visual Navigation Under Sparse Reward via Goal-Oriented Memory Encoder With Never Give Up
Lee, J. S., & Moon, J. (2025) · IEEE Access · doi:10.1109/access.2025.3556894
Umbrella Reinforcement Learning - A Tool for Hard Problems
Nuzhin, E. E., & Brilliantov, N. V. (2024) · International Conference on Robotics, Automation and Artificial Intelligence (RAAI) · doi:10.1109/raai64504.2024.10949570
Skill Disentanglement in Reproducing Kernel Hilbert Space
Dave, V., & Rueckert, E. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i15.33774
From LLMs to Randomness: Analyzing Program Input Efficacy With Resource and Language Metrics
Black, G., Yocam, E., Vaidyan, V. M., Comert, G., & Wang, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3571205
Multi-Agent Reinforcement Learning Based on Dual Model Loss Prediction
Niu, S., Hou, J., Zhan, C., & Liu, Q. (2025) · International Symposium on Autonomous Systems (ISAS) · doi:10.1109/icaisisas64483.2025.11051544
Role-Aware Representation and Uncertainty-Driven Policy Regularization for UAV Swarm Air Combat
Yang, Q., Li, D., Yan, H., Li, F., & Jia, J. (2026) · International Journal of Aeronautical and Space Sciences · doi:10.1007/s42405-026-01284-7
Baba in Wonderland: Online Self-Supervised Dynamics Discovery for Executable World Models
Seo, S., Han, D., Noh, S., & Kang, H. (2026) · arXiv · doi:10.48550/arxiv.2605.16725
ProcgenOOD: a benchmark for out-of-distribution generalization in reinforcement learning
Corder, K., Park, S. J., & Vindiola, M. M. (2026) · Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications VIII · doi:10.1117/12.3093362
Solving Rubik's Cube with a Robot Hand
OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.07113 · Cited by 602
Influence-Based Multi-Agent Exploration
Wang, T., Wang, J., Wu, Y., & Zhang, C. (2019) · arXiv · doi:10.48550/arxiv.1910.05512 · Cited by 69
A Survey of Exploration Methods in Reinforcement Learning
Amin, S., Gomrokchi, M., Satija, H., van Hoof, H., & Precup, D. (2021) · arXiv · doi:10.48550/arxiv.2109.00157 · Cited by 41
Exploration and preference satisfaction trade-off in reward-free learning
Sajid, N., Tigas, P., Zakharov, A., Fountas, Z., & Friston, K. (2021) · arXiv · doi:10.48550/arxiv.2106.04316 · Cited by 11
Metadata
- Authors
- Burda, Y., Edwards, H., Storkey, A., & Klimov, O.
- Affiliations
- OpenAI (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1810.12894
- OpenAlex
- W2899205164
- Cited by
- 527
- References
- 41
- Citations by year
- 2026: 2 · 2025: 35 · 2024: 75 · 2023: 77 · 2022: 79 · 2021: 115 · 2020: 87 · 2019: 52 · 2018: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- random network distillation, exploration bonus, deep reinforcement learning, intrinsic rewards, Atari games, Montezuma's Revenge, feature prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
