The DOI matches its Crossref record; 185 of the 185 references it cites were found in a registry or an index.
Exploration in deep reinforcement learning: A survey
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Solving these problems requires a good global exploration strategy [211] that can better balance long-term and short-term environment information [195].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [195]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While underappreciated in current reasoning literature, the exploration component of RL has long been recognized as a critical driver for emergent behavior in traditional RL (Yue et al., 2025; Silver et al., 2016; Ladosz et al., 2022; Pourcel et al., 2024).”
From Absolute Zero: Reinforced Self-play Reasoning with Zero Data (2025) · cites this paper as Ladosz et al., 2022
In the citing paper
“This subdomain is dedicated to developing AI systems capable of smooth, natural, and context-aware human-computer interactions, by focusing on dialogue modeling, question answering, user intent recognition, and multi-turn context tracking [204], [205], [206], [207].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [206]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“impressive performance in Atari games [2], Mujoco simulator [3], controllers tuning [4], autonomous landing [5], self-driving cars [6] and healthcare [7].”
BackgroundReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“The most common technique for exploration in reinforcement learning is random exploration [1].”
MethodLabaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
Proceedings of IEEE International Conference on Computer Vision
(1995) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466933 · Cited by 13,117
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,869
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · cited 1×
“impressive performance in Atari games [2], Mujoco simulator [3], controllers tuning [4], autonomous landing [5], self-driving cars [6] and healthcare [7].”
BackgroundSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · cited 1×
“A general overview of reinforcement learning can be found in [23].”
BackgroundProceedings. 1985 IEEE International Conference on Robotics and Automation
(1985) · doi:10.1109/robot.1985 · Cited by 3,692
2009 International Joint Conference on Artificial Intelligence
(2009) · doi:10.1109/jcai15434.2009 · Cited by 2,846
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
Similarity estimation techniques from rounding algorithms
Charikar, M. (2002) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/509907.509965 · Cited by 2,263 · cited 2×
“Here, a technique called SimHash [63] was used, which rep-”
MethodThe Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161 · cited 1×
“The Atari games benchmark are a set of 57 Atari games combined under the Atari Learning Environment (ALE) [25].”
BackgroundCuriosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · cited 2×
“A state representation method derived from inverse dynamic features (IDF) was used in [41].”
BackgroundMethod2011 International Joint Conference on Neural Networks
(2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2102870 · Cited by 1,772
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471 · cited 2×
“An approach using the adaptive resonance theorem (ART) [150] was presented in [151] and was later extended in [152].”
MethodNguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306
Abandoning Objectives: Evolution Through the Search for Novelty Alone
Lehman, J., & Stanley, K. O. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00025 · Cited by 1,216 · cited 2×
“One of the earliest methods called novelty search was devised in [74] and [75].”
BackgroundMethodZaki, M. J., & Meira, W. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810114 · Cited by 1,165
(2015) · Journal of Machine Learning Research · Cited by 1,119
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1703.03864 · Cited by 1,027 · cited 1×
“Note that evolutionary approaches are generally not considered as the part of reinforcement learning but can be used to solve the same type of problems [72, 73].”
BackgroundProceedings Of The International Joint Conference On Neural Networks 2003
(2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2003.1223944 · Cited by 953
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic MotivationPreprint
Kulkarni, T. D., Narasimhan, K., Saeedi, A., & Tenenbaum, J. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06057 · Cited by 909
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889 · cited 2×
“(DQfD) [128] differs in two aspects from [127].”
BackgroundNoisy Networks for ExplorationPreprint
Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., Pietquin, O., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.10295 · Cited by 880
Ensemble of exemplar-SVMs for object detection and beyond
Malisiewicz, T., Gupta, A., & Efros, A. A. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126229 · Cited by 865 · cited 1×
“One example method is to use exemplar model [69] developed in [70].”
MethodLearning to Navigate in Complex EnvironmentsPreprint
Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A. J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03673 · Cited by 847 · cited 2×
“[48] A3C Prediction error DM Lab: Random Goal 96 (LSTM-A3C 65) DM Lab images Ac MB C/ C Tang et al.”
MethodSuch, F. P., Madhavan, V., Conti, E., Lehman, J., Stanley, K. O., & Clune, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06567 · Cited by 846
Deep Exploration via Bootstrapped DQNPreprint
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 837 · cited 2×
“[123] DQN Uncertainty Atari: James Bond 1000 (DQN 600) Atari imgaes Q MB D/ D Tang and Agrawal [114] DDPG Uncertainty Mujoco: sparse mountaincar 0.”
BackgroundMethodWhat is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811 · cited 1×
“There are different ways of classifying intrinsic rewards [32, 16].”
BackgroundReinforcement Learning in Healthcare: A Survey
Yu, C., Liu, J., Nemati, S., & Yin, G. (2021) · ACM Computing Surveys · doi:10.1145/3477600 · Cited by 783
Data Mining and Analysis: Fundamental Concepts and Algorithms
Zaki, M. J., & Meira, W. (2014) · Cited by 780
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747
(2016) · neural information processing systems · Cited by 735
Overcoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712
FeUdal Networks for Hierarchical Reinforcement LearningPreprint
Vezhnevets, A. S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., & Kavukcuoglu, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01161 · Cited by 693 · cited 1×
“An approach in which both goal creation and selection mechanisms are devised by a meta-controller was proposed in [96].”
BackgroundViZDoom: A Doom-based AI research platform for visual reinforcement learning
Kempka, M., Wydmuch, M., Runc, G., Toczek, J., & Jaśkowski, W. (2016) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2016.7860433 · Cited by 624 · cited 5×
“Similar to VizDoom, it is also from the first-person perspective, and the image is the state space.”
BackgroundInfluentialCurious model-building control systems
Schmidhuber, J. (1991) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1991.170605 · Cited by 601 · cited 2×
“The simplest method of this type was described in [10, 11].”
BackgroundMethodVIME: Variational Information Maximizing ExplorationPreprint
Houthooft, R., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09674 · Cited by 597
Conference on Neural Information Processing Systems
Liu, L., Zhou, T., Long, G., Jiang, J., & Zhang, C. (2019) · UTS ePRESS (University of Technology Sydney) · Cited by 593
2004 IEEE International Joint Conference on Neural Networks
(2004) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1381025 · Cited by 574
Parameter Space Noise for ExplorationPreprint
Plappert, M., Houthooft, R., Dhariwal, P., Sidor, S., Chen, R. Y., Chen, X., Asfour, T., Abbeel, P., & Andrychowicz, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.01905 · Cited by 574 · cited 2×
“noise is by comparing the decision made by the noisy and noiseless policy [156].”
Background2011 IEEE International Conference on Robotics and Automation
(2009) · IEEE Transactions on Haptics · doi:10.1109/toh.2009.52 · Cited by 540
Exploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527 · cited 8×
“Stanton and Clune [51] agent position, reward grid A2C Prediction error Atari: Montezuma Revenge 3200 (A2C 0) Atari images Ac MB D/ D Achiam and Sastry [33] TRPO Prediction error Mujoco: halfcheetah 80 (VIME 40); Atari: Venture 400 (VIME 0) Atari RAM states/ Mujoco joints angles Ac MB C/ C Li et al. [44] A2C Predictio…”
BackgroundInfluentialQuality Diversity: A New Frontier for Evolutionary Computation
Pugh, J. K., Soros, L. B., & Stanley, K. O. (2016) · Frontiers in Robotics and AI · doi:10.3389/frobt.2016.00040 · Cited by 514 · cited 1×
“More detailed reviews of exploration in evolution strategies can be found in [78] and [79].”
Background#Exploration: A Study of Count-Based Exploration for Deep Reinforcement LearningPreprint
Tang, H., Houthooft, R., Foote, D., Stooke, A., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04717 · Cited by 504 · cited 2×
“One of the popular methods of this type was proposed in [62], where static hashing was used.”
BackgroundMethodLeveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
Vecerik, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Rothörl, T., Lampe, T., & Riedmiller, M. (2017) · arXiv · doi:10.48550/arxiv.1707.08817 · Cited by 478
Incentivizing Exploration In Reinforcement Learning With Deep Predictive ModelsPreprint
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00814 · Cited by 462
Variational Information Maximisation for Intrinsically Motivated Reinforcement LearningPreprint
Mohamed, S., & Rezende, D. J. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.08731 · Cited by 460 · cited 1×
“In [57], maximising mutual information between a sequence of actions leads to a state that is rewarded.”
BackgroundA Possibility for Implementing Curiosity and Boredom in Model-Building Neural Controllers
Schmidhuber, J. (1991) · The MIT Press eBooks · doi:10.7551/mitpress/3115.003.0030 · Cited by 455 · cited 2×
“The simplest method of this type was described in [10, 11].”
BackgroundMethodLarge-Scale Study of Curiosity-Driven Learning
Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., & Efros, A. A. (2018) · arXiv · doi:10.48550/arxiv.1808.04355 · Cited by 401 · cited 1×
“IDF was compared against the state prediction method and random representation in [42] with the following conclusions: IDF had the best performance and it scales the best to the unseen environments.”
MethodCount-Based Exploration with Neural Density Models
Ostrovski, G., Bellemare, M. G., Oord, A. V. D., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.01310 · Cited by 377
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.00909 · Cited by 366 · cited 2×
“[18] considered an application to multi-agent problems, Levine [19] provided a tutorial and extensive comparison with probabilistic inference methods and [20] provided an extensive description of the key breakthrough methods”
A Bayesian Framework for Reinforcement Learning
Strens, M. J. A. (2000) · Cited by 361
Electrospun nanofibers: Internal structure and intrinsic orientation
Dersch, R., Liu, T., Schaper, A., Greiner, A., & Wendorff, J. H. (2003) · Journal of Polymer Science Part A Polymer Chemistry · doi:10.1002/pola.10609 · Cited by 354
Learning by Playing - Solving Sparse Reward Tasks from ScratchPreprint
Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Van de Wiele, T., Mnih, V., Heess, N., & Springenberg, J. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10567 · Cited by 334
IJCAI-83: International joint conference on artificial intelligence
(1982) · Artificial Intelligence · doi:10.1016/0004-3702(82)90046-7 · Cited by 325
Safe Exploration in Continuous Action SpacesPreprint
Dalal, G., Dvijotham, K., Vecerík, M., Hester, T., Păduraru, C., & Tassa, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.08757 · Cited by 284
Agent57: Outperforming the Atari Human BenchmarkPreprint
Badia, A. P., Piot, B., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, D., & Blundell, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13350 · Cited by 277
Reverse Curriculum Generation for Reinforcement LearningPreprint
Florensa, C., Held, D., Wulfmeier, M., Zhang, M., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05300 · Cited by 277
Deep Q-learning from DemonstrationsPreprint
Hester, T., Vecerík, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Dulac-Arnold, G., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03732 · Cited by 276
Adaptive ε-Greedy Exploration in Reinforcement Learning Based on Value Differences
Tokic, M. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-16111-7_23 · Cited by 266
Conti, E., Madhavan, V., Such, F. P., Lehman, J., Stanley, K. O., & Clune, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06560 · Cited by 264 · cited 4×
“Initially, novelty search was used as a stand-alone technique; however, recently, combining it with other techniques [77, 76] has shown more promise.”
BackgroundMethodInfluentialEncouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study
Mouret, J.-B., & Doncieux, S. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00048 · Cited by 263 · cited 1×
“More detailed reviews of exploration in evolution strategies can be found in [78] and [79].”
Background2022 IEEE International Conference on Robotics and Biomimetics
(2022) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio55434.2022.10011704 · Cited by 262
Episodic Curiosity through ReachabilityPreprint
Savinov, N., Raichuk, A., Marinier, R., Vincent, D., Pollefeys, M., Lillicrap, T., & Gelly, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02274 · Cited by 258 · cited 2×
“Prediction error methods have also shown the ability to solve the couch-potato (noisy-TV) problem by storing observations in a memory buffer [60].”
BackgroundTrial without Error: Towards Safe Reinforcement Learning via Human Intervention
Saunders, W., Sastry, G., Stuhlmüller, A., & Evans, O. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/mzmg4429 · Cited by 257
Randomized Prior Functions for Deep Reinforcement LearningPreprint
Osband, I., Aslanides, J., & Cassirer, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03335 · Cited by 236 · cited 1×
“A similar approach was considered in [40].”
Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum LearningPreprint
Forestier, S., Portelas, R., Mollard, Y., & Oudeyer, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02190 · Cited by 219
Near-Bayesian exploration in polynomial time
Kolter, J. Z., & Ng, A. Y. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553441 · Cited by 218
Model-Based Active ExplorationPreprint
Shyam, P., Jaśkowski, W., & Gomez, F. (2018) · International Conference on Machine Learning · doi:10.48550/arxiv.1810.12162 · Cited by 207 · cited 2×
“A similar approach was developed in [125], using the information gain among the environmental models to decide where to go.”
BackgroundMethodThe Malmo platform for artificial intelligence experimentation
Johnson, M., Hofmann, K., Hutton, T. J., & Bignell, D. E. (2016) · International Joint Conference on Artificial Intelligence · Cited by 205
Go-Explore: a New Approach for Hard-Exploration Problems
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2019) · arXiv · doi:10.48550/arxiv.1901.10995 · Cited by 205
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · RePEc: Research Papers in Economics · Cited by 201
Self-Supervised Exploration via DisagreementPreprint
Pathak, D., Gandhi, D., & Gupta, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04161 · Cited by 190 · cited 1×
“Another approximation could be to use ensembles of models as proposed in [45].”
MethodSelf-Imitation LearningPreprint
Oh, J., Guo, Y., Singh, S. P., & Lee, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05635 · Cited by 184 · cited 2×
“[88] A2C Goals to Explore from Atari: Montezuma Revenge 2500 (A2C 0) Atari images Ac MF D/ D Guo et al.”
BackgroundA survey on intrinsic motivation in reinforcement learningPreprint
Aubret, A., Matignon, L., & Hassas, S. (2019) · arXiv.org · doi:10.48550/arxiv.1908.06976 · Cited by 175 · cited 4×
“Here, we largely follow the classification of [16] with the following categories: (i) prediction error methods, (ii) count-based methods and (iii) memory methods.”
BackgroundMethodInfluentialDeep Exploration via Randomized Value FunctionsPreprint
Osband, I., Roy, B. V., Russo, D., & Wen, Z. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07608 · Cited by 175 · cited 2×
“[113] LSVI Optimistic Cartpole Swing up: 600 (DQN 0) State Id Ac MF D/ D Nikolov et al.”
BackgroundNever Give Up: Learning Directed Exploration Strategies
Badia, A. P., Sprechmann, P., Vitvitskyi, A., Guo, D., Piot, B., Kapturowski, S., Tieleman, O., Arjovsky, M., Pritzel, A., Bolt, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2002.06038 · Cited by 170
Generalization and Exploration via Randomized Value FunctionsPreprint
Osband, I., Roy, B. V., & Wen, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0635 · Cited by 169 · cited 2×
“[112] LSVI Optimistic Tetris: 5000 (LSVI 4000) Hand tuned 22 features Ac MF D/ D”
BackgroundAn information-theoretic approach to curiosity-driven reinforcement learning
Still, S., & Precup, D. (2012) · Theory in Biosciences · doi:10.1007/s12064-011-0142-z · Cited by 165 · cited 1×
“The general framework for these types of approaches was presented in [54, 55].”
BackgroundLearning Montezuma's Revenge from a Single DemonstrationPreprint
Salimans, T., & Chen, R. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03381 · Cited by 152 · cited 2×
“Salimans and Chen [131] single demonstration PPO Imitation with Exploration Strategy Atari: Montezuma Revenge with distraction 74500 (Playing by youtube 41098) Atari images Ac MF D/ D”
BackgroundMethodSurprise-Based Intrinsic Motivation for Deep Reinforcement Learning
Achiam, J., & Sastry, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01732 · Cited by 139
RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated EnvironmentsPreprint
Răileanu, R., & Rocktäschel, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.12292 · Cited by 131 · cited 4×
“IDF was utilised in [43], where the Euclidean distance between two consecutive state representations was used as an intrinsic reward, as shown”
BackgroundMethodInfluentialBehaviour Suite for Reinforcement LearningPreprint
Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepesvári, C., Singh, S. P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03568 · Cited by 126
GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning AlgorithmsPreprint
Colas, C., Sigaud, O., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05054 · Cited by 124 · cited 2×
“[115] split the exploration into two phases: (i) explore randomly and (ii) compare experiences to an expert-created imitation to determine the good behavior.”
BackgroundCount-Based Exploration in Feature Space for Reinforcement Learning
Martin, J., Sasikumar, S. N., Everitt, T., & Hütter, M. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/344 · Cited by 117 · cited 2×
“[68] BlobPROST features SARSAe Countbased Atari: Montezuma Revenge 2000 (SARSA 200) BlobPROST features Q MB D/ D”
BackgroundSafe Exploration in Finite Markov Decision Processes with Gaussian ProcessesPreprint
Turchetta, M., Berkenkamp, F., & Krause, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04753 · Cited by 116 · cited 3×
“This can be either in the form of a mathematical model [139] or heuristic rules [140].”
BackgroundReinforcement Learning in Healthcare: A SurveyPreprint
Yu, C., Liu, J., & Nemati, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.08796 · Cited by 114
Trial without Error: Towards Safe Reinforcement Learning via Human InterventionPreprint
Saunders, W., Sastry, G., Stuhlmueller, A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05173 · Cited by 111 · cited 2×
“[138] human intervention data DQN Human Designer Knowledge Atari: Space Invaders 0 catastrophes (DQN 800000) Atari images Q MB D/ D”
BackgroundCURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement LearningPreprint
Colas, C., Fournier, P., Sigaud, O., Chétouani, M., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06284 · Cited by 104
Eigenoption Discovery through the Deep Successor RepresentationPreprint
Machado, M. C., Rosenbaum, C., Guo, X., Liu, M., Tesauro, G. J., & Campbell, M. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.11089 · Cited by 102 · cited 2×
“This was later improved in [103] by replacing the handcrafted adjacency matrix with successor representations.”
BackgroundMethodEX2: Exploration with Exemplar Models for Deep Reinforcement Learning
Fu, J., Co-Reyes, J. D., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01260 · Cited by 95 · cited 2×
“[70] state encoder TRPO Memory Mujoco: SparseHalfCheetah 173.”
BackgroundMethodEfficient Bayes-Adaptive Reinforcement Learning using Sample-Based SearchPreprint
Guez, A., Silver, D., & Dayan, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.3109 · Cited by 94 · cited 2×
“[120] guess of a prior Policy learning Uncertainty Dearden Maze: 965.”
BackgroundMethodA Laplacian Framework for Option Discovery in Reinforcement LearningPreprint
Machado, M. C., Bellemare, M. G., & Bowling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00956 · Cited by 93 · cited 2×
“[101] hadcrafted features itself Exploratory Goal 4-room domain: 1 Enumarated state id Ac MB D/ D”
BackgroundMethodPlaying hard exploration games by watching YouTubePreprint
Aytar, Y., Pfaff, T., Budden, D. M., Paine, T. L., Wang, Z., & de Freitas, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11592 · Cited by 93 · cited 4×
“[26] youtube embbeding IMPALA Imitation with Exploration Strategy Atari: Montezuma’s Revenge 80k (DqfD 4k) Atari Images Ac MF D/ D”
BackgroundMethodInfluentialSkew-Fit: State-Covering Self-Supervised Reinforcement LearningPreprint
Pong, V. H., Dalal, M., Lin, S., Nair, A., Bahl, S., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03698 · Cited by 90
Safe Exploration of State and Action Spaces in Reinforcement Learning
Garcia, J., & Fernandez, F. (2012) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3761 · Cited by 85 · cited 2×
“Garcia and Fernandez [134] baseline behaviour Human Designer Knowledge car parking problem 6.”
BackgroundExploring Parameter Space in Reinforcement Learning
Rückstieß, T., Sehnke, F., Schaul, T., Wierstra, D., Sun, Y., & Schmidhuber, J. (2010) · Paladyn Journal of Behavioral Robotics · doi:10.2478/s13230-010-0002-4 · Cited by 81 · cited 2×
“The easiest method of including the noise is to include a fixed amount of noise [153].”
MethodProceedings of the ... International Joint Conference on Autonomous Agents and Multiagent Systems
Press, A. (2002) · ACM Press eBooks · Cited by 81
EMI: Exploration with Mutual InformationPreprint
Kim, H., Kim, J., Jeong, Y., Levine, S., & Song, H. O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01176 · Cited by 81
State-Dependent Exploration for Policy Gradient Methods
Rückstieß, T., Felder, M., & Schmidhuber, J. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87481-2_16 · Cited by 78 · cited 1×
“In [157], the problem of assigning rewards when the same state is present multiple times is discussed.”
BackgroundMachine Learning and Knowledge Discovery in Databases
Balcázar, J. L., Sebag, M., Gionis, A., et al. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15880-3 · Cited by 78
Diversity-Driven Exploration Strategy for Deep Reinforcement Learning
Hong, Z.-W., Shann, T.-Y., Su, S.-Y., Chang, Y.-H., & Lee, C.-Y. (2018) · arXiv · doi:10.48550/arxiv.1802.04564 · Cited by 77 · cited 2×
“The diversity among policies was measured in [80].”
BackgroundHow novelty search escapes the deceptive trap of learning to learn
Risi, S., Vanderbleek, S. D., Hughes, C. E., & Stanley, K. O. (2009) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1569901.1569923 · Cited by 75 · cited 2×
“[75] measure of policies distance NEAT Evolution strategies T-maze: solved after 50,000 evaluations enumarated state id Ac MF D/ D”
BackgroundMethodThe Uncertainty Bellman Equation and ExplorationPreprint
O’Donoghue, B., Osband, I., Munos, R., & Mnih, V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05380 · Cited by 74 · cited 2×
“[121] derived Bayesian uncertainty such that it can be computed using the Bellman principle and the output of the neural network.”
BackgroundMethodCombating Reinforcement Learning's Sisyphean Curse with Intrinsic FearPreprint
Lipton, Z. C., Azizzadenesheli, K., Kumar, A., Li, L., Gao, J., & Deng, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01211 · Cited by 73
Information-theoretic approach to interactive learning
Still, S. (2009) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/85/28005 · Cited by 72 · cited 1×
“The general framework for these types of approaches was presented in [54, 55].”
BackgroundUnsupervised Learning of Goal Spaces for Intrinsically Motivated Goal ExplorationPreprint
Péré, A., Forestier, S., Sigaud, O., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00781 · Cited by 70 · cited 1×
“[95] IMGEP Exploratory Goal Mujoco (KLC): ArmArrow 7.”
BackgroundContingency-Aware Exploration in Reinforcement LearningPreprint
Choi, J., Guo, Y., Moczulski, M., Oh, J., Wu, N., Norouzi, M., & Lee, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01483 · Cited by 69 · cited 1×
“To generate an even more compact representation, in [64], the state was represented as the learned xy position of an agent.”
BackgroundAlignment of Requirements Specification and Testing: A Systematic Mapping Study
Barmi, Z. A., Ebrahimi, A. H., & FELDT, R. H. (2011) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2011.58 · Cited by 68
Exploration from Demonstration for Interactive Reinforcement Learning
Subramanian, K. A., Isbell, C. L., & Thomaz, A. L. (2016) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2936924.2936990 · Cited by 68
Random curiosity-driven exploration in deep reinforcement learning
Li, J., Shi, X., Li, J., Zhang, X., & Wang, J.-Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.08.024 · Cited by 68 · cited 2×
“[44] A2C Prediction error Atari: Asterix 500000 (RND 10000) Atari images Ac MB D/ D Kim et al.”
BackgroundThe Uncertainty Bellman Equation and Exploration.
O’Donoghue, B., Osband, I., Munos, R., & Mnih, V. (2018) · International Conference on Machine Learning · Cited by 67
Deep Reinforcement Learning for NLP
Wang, W. Y., Li, J., & He, X. (2018) · doi:10.18653/v1/p18-5007 · Cited by 63
Folkers, A. (2019) · BesMasters · doi:10.1007/978-3-658-28886-0_3 · Cited by 60
Deep Reinforcement Learning: A State-of-the-Art Walkthrough
Lazaridis, A., Fachantidis, A., & Vlahavas, I. P. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12412 · Cited by 59 · cited 1×
“[18] considered an application to multi-agent problems, Levine [19] provided a tutorial and extensive comparison with probabilistic inference methods and [20] provided an extensive description of the key breakthrough methods”
BackgroundVisual novelty, curiosity, and intrinsic reward in machine learning and the brain
Jaegle, A., Mehrpour, V., & Rust, N. C. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.08.004 · Cited by 56
Algorithm-Directed Exploration for Model-Based Reinforcement Learning in Factored MDPs
Guestrin, C., Patrascu, R., & Schuurmans, D. (2002) · Cited by 49
Making Efficient Use of Demonstrations to Solve Hard Exploration ProblemsPreprint
Paine, T. L., Gülçehre, Ç., Shahriari, B., Denil, M., Hoffman, M., Soyer, H., Tanburn, R., Kapturowski, S., Rabinowitz, N. C., Williams, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01387 · Cited by 49
Verifiably safe exploration for end-to-end reinforcement learning
Hunt, N., Fulton, N., Magliacane, S., Hoang, T. N., Das, S., & Solar-Lezama, A. (2021) · International Conference on Hybrid Systems: computation and control · doi:10.1145/3447928.3456653 · Cited by 48
Curiosity-Driven Experience Prioritization via Density Estimation
Zhao, R., & Tresp, V. (2019) · arXiv · doi:10.48550/arxiv.1902.08039 · Cited by 47 · cited 1×
“The density models include context tree switching (CTS) [14], Gaussian mixture models [66] or PixelCNN [67].”
MethodForward-Backward Reinforcement LearningPreprint
Edwards, A. D., Downs, L., & Davidson, J. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10227 · Cited by 47
Model-based reinforcement learning with parametrized physical models and optimism-driven exploration
Xie, C., Patil, S., Moldovan, T. M., Levine, S., & Abbeel, P. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487172 · Cited by 45 · cited 2×
“The linear Gaussian algorithm can also be used as a model of the reward [110].”
BackgroundMethodAutonomous Quadrotor Landing using Deep Reinforcement LearningPreprint
Polvara, R., Patacchiola, M., Sharma, S. K., Wan, J., Manning, A. J., Sutton, R., & Cangelosi, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.03339 · Cited by 40
See, Hear, Explore: Curiosity via Audio-Visual AssociationPreprint
Dean, V., Tulsiani, S., & Gupta, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.03669 · Cited by 37
Fast and slow curiosity for high-level exploration in reinforcement learning
Bougie, N., & Ichise, R. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01849-3 · Cited by 35 · cited 2×
“Improvements to autoencoder-based approaches were proposed in [38, 39], where a slow-trained autoencoder was added.”
BackgroundA Critical Investigation of Deep Reinforcement Learning for NavigationPreprint
Dhiman, V., Banerjee, S., Griffin, B., Siskind, J. M., & Corso, J. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.02274 · Cited by 35
Stanton, C., & Clune, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00553 · Cited by 34
Proceedings. The 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology
(2005) · IEEE/WIC/ACM International Conference on Intelligent Agent Technology · doi:10.1109/iat.2005.146 · Cited by 31
Why is Posterior Sampling Better than Optimism for Reinforcement Learning?Preprint
Osband, I., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.00215 · Cited by 30 · cited 2×
“Tabular summary of optimistic and uncertainty approaches is shown in Table 4 and have been extensively compared in [108].”
MethodQuality Diversity Through Surprise
Gravina, D., Liapis, A., & Yannakakis, G. N. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2018.2877215 · Cited by 30 · cited 4×
“Initially, novelty search was used as a stand-alone technique; however, recently, combining it with other techniques [77, 76] has shown more promise.”
BackgroundMethodInfluentialInformation-Directed Exploration for Deep Reinforcement LearningPreprint
Nikolov, N., Kirschner, J., Berkenkamp, F., & Krause, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07544 · Cited by 30 · cited 2×
“[122] bDQN and C51 Uncertainty 55 atari games: 1058% of reference human performance Atari images Q MB D/ D”
BackgroundKhamassi, M., Velentzas, G., Tsitsimis, T., & Tzafestas, C. S. (2017) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2017.33 · Cited by 29 · cited 2×
“[148] used long-term and short-term reward averages to control exploration and exploitation.”
BackgroundGaussian Processes for Sample Efficient Reinforcement Learning with RMAX-Like Exploration
Jung, T., & Stone, P. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15880-3_44 · Cited by 27
Curiosity-Bottleneck: Exploration By Distilling Task-Specific Novelty.
Kim, Y., Nam, W., Kim, H., Kim, J., & Kim, G. (2019) · KAIST Institutional Repository (KAIST) · Cited by 27
Fast active learning for pure exploration in reinforcement learningPreprint
Ménard, P., Domingues, O. D., Jönsson, A., Kaufmann, E., Leurent, E., & Valko, M. (2020) · Repositori digital de la UPF (Universitat Pompeu Fabra) · doi:10.48550/arxiv.2007.13442 · Cited by 25
ACDER: Augmented Curiosity-Driven Experience Replay
Li, B., Lü, T., Li, J., Lu, N., Cai, Y., & Wang, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197421 · Cited by 24 · cited 1×
“Positions in a 3D space can also be used [50].”
BackgroundSuccessor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning
Janz, D., Hron, J., Mazur, P., Hofmann, K., Hernández-Lobato, J. M., & Tschiatschek, S. (2018) · arXiv · doi:10.48550/arxiv.1810.06530 · Cited by 23 · cited 2×
“[116] DQN Uncertainty 49 Atari games: 77.”
MethodExploration by Distributional Reinforcement Learning
Tang, Y., & Agrawal, S. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/376 · Cited by 22
Curiosity-Driven Reinforcement Learning with Homeostatic Regulation
Magrans, I., & Ryota Kanai, C. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489075 · Cited by 22
Exploratory Gradient Boosting for Reinforcement Learning in Complex DomainsPreprint
Abel, D. L., Agarwal, A., Díaz, F., Krishnamurthy, A., & Schapire, R. E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.04119 · Cited by 21
Matheron, G., Perrin, N., & Sigaud, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8_24 · Cited by 21 · cited 2×
“[85] state storage method DDPG Goals to Explore from Maze: reach reward after 146k (TD3 never) x-y position Ac MB C/ C”
BackgroundMethodAkiyama, T., Hachiya, H., & Sugiyama, M. (2010) · Neural Networks · doi:10.1016/j.neunet.2009.12.010 · Cited by 19 · cited 2×
“Alternatively, policies can be sampled [118].”
BackgroundPearce, T., Anastassacos, N., Zaki, M., & Neely, A. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11324 · Cited by 19
Automatic Gain Tuning Method of a Quad-Rotor Geometric Attitude Controller Using A3C
Lee, S., & Bang, H. (2019) · International Journal of Aeronautical and Space Sciences · doi:10.1007/s42405-019-00233-x · Cited by 19 · cited 1×
“impressive performance in Atari games [2], Mujoco simulator [3], controllers tuning [4], autonomous landing [5], self-driving cars [6] and healthcare [7].”
BackgroundCount-based exploration with the successor representation
Machado, M. C., Bellemare, M. G., & Bowling, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5955 · Cited by 19
Curiosity-Driven Exploration for Off-Policy Reinforcement Learning Methods
Li, B., Lü, T., Li, J., Lu, N., Cai, Y., & Wang, S. (2019) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio49542.2019.8961529 · Cited by 19 · cited 2×
“[34] DDPG Prediction error Robot: FetchPush 1 (DDPG 0) Robot joints angles Ac MB C/ C Raileanu and Rocktäschel [43] IMPALA Prediction error Vizdoom: 0.”
BackgroundAsymmetric self-play for automatic goal discovery in robotic manipulation
OpenAI, O., Plappert, M., Sampedro, R., Xu, T., Akkaya, I., Kosaraju, V., Welinder, P., D'Sa, R., Petron, A., Pinto, H. P. D. O., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.04882 · Cited by 18
Explicit Explore-Exploit Algorithms in Continuous State SpacesPreprint
Henaff, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00617 · Cited by 18 · cited 2×
“Another ensemble model was presented in [126].”
BackgroundAdaptive exploration for continual reinforcement learning
Stulp, F. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385818 · Cited by 17 · cited 2×
“Stulp [117] PI(2) Uncertainty Ball batting: learned after 20 steps Robot joints angles Ac MB C/ C Janz et al.”
BackgroundMethodMemory Based Trajectory-conditioned Policies for Learning from Sparse RewardsPreprint
Guo, Y., Choi, J., Moczulski, M., Feng, S., Samy Bengio, A., Norouzi, M., & Lee, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.10247 · Cited by 16
Conservative Exploration in Reinforcement LearningPreprint
Garcelon, E., Ghavamzadeh, M., Lazaric, A., & Pirotta, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.03218 · Cited by 15 · cited 2×
“12, while in [136], agents are expected to perform no worse than the a priori known baseline.”
BackgroundReinforcement learning with internal-dynamics-based exploration using a chaotic neural network
Shibata, K., & Sakashita, Y. (2015) · doi:10.1109/ijcnn.2015.7280430 · Cited by 14 · cited 3×
“In [154], chaotic networks were used to induce the noise in the network.”
BackgroundMethodInfluentialLearning exploration strategies in model-based reinforcement learning
Hester, T., Lopes, M. C., & Stone, P. (2013) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/xlfe9254 · Cited by 14 · cited 2×
“In [97], a multi-arm bandit-based method to choose one strategy from a group of hand-designed strategies was proposed.”
BackgroundMethodDirected Exploration for Reinforcement LearningPreprint
Guo, Z. D., & Brunskill, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07805 · Cited by 14
An ant system based exploration-exploitation for reinforcement learning
Chang, H. S. (2004) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1400937 · Cited by 13 · cited 1×
“[149] used multiple agents (ants) to adjust the exploration parameters.”
MethodCuriosity-driven reinforcement learning with homeostatic regulationPreprint
de Abril, I. M., & Kanai, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.07440 · Cited by 13 · cited 1×
“information gain was combined with the state prediction error into a single intrinsic reward in [58, 59].”
BackgroundGenerative adversarial exploration for reinforcement learning
Hong, W., Zhu, M., Liu, M., Zhang, W., Zhou, M. Q., Yu, Y. H., & Sun, P. (2019) · International Conference on Distributed Artificial Intelligence · doi:10.1145/3356464.3357706 · Cited by 13 · cited 2×
“[36] DQN Prediction error Atari: Montezuma Revenge 200 (DQN 0) Enumarated state id/ Atari Image Q MB D/ D”
BackgroundMethodAdaptive Procedural Task Generation for Hard-Exploration ProblemsPreprint
Fang, K., Zhu, Y., Savarese, S., & Fei-Fei, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00350 · Cited by 13 · cited 3×
“igated in [94] or [104], but these approaches rely on the ability to parametrize the task.”
BackgroundInfluentialSelf-Imitation Learning via Trajectory-Conditioned Policy for Hard-Exploration TasksPreprint
Guo, Y., Choi, J., Moczulski, M., Samy Bengio, A., Norouzi, M., & Lee, H. (2019) · arXiv (Cornell University) · Cited by 12
GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning
Kovač, G., Laversanne-Finot, A., & Oudeyer, P. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3216911 · Cited by 12
Learning Transferable Domain Priors for Safe Exploration in Reinforcement Learning
Karimpanal, T. G., Rana, S., Gupta, S., Tran, T., & Venkatesh, S. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207344 · Cited by 11
Artificial Neural Networks and Machine Learning – ICANN 2020
Riccio Salvatore, D., Deyan, D., Giorgio, J., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8 · Cited by 11
Knowledge-Based Exploration for Reinforcement Learning in Self-Organizing Neural Networks
Teng, T.-H., & Tan, A. (2012) · IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2012.154 · Cited by 10 · cited 1×
“An approach using the adaptive resonance theorem (ART) [150] was presented in [151] and was later extended in [152].”
MethodAdaptive exploration in reinforcement learning
Patrascu, R., & Stacey, D. (2003) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1999.833417 · Cited by 10 · cited 2×
“One technique to adapt the exploration rate is by simply considering a reward and adjusting the random exploration parameter accordingly, as described in [144].”
BackgroundMethodDead-ends and Secure Exploration in Reinforcement Learning
Fatemi, M., Sharma, S., van Seijen, H., & Kahou, S. E. (2019) · International Conference on Machine Learning · Cited by 10
On Hard Exploration for Reinforcement Learning: a Case Study in Pommerman
Gao, C., Kartal, B., Hernandez-Leal, P., & Taylor, M. E. (2019) · Artificial Intelligence and Interactive Digital Entertainment Conference · doi:10.48550/arxiv.1907.11788 · Cited by 10 · cited 3×
“This can be either in the form of a mathematical model [139] or heuristic rules [140].”
BackgroundMethodAutomatic Abstraction in Reinforcement Learning Using Ant System Algorithm
Ghafoorian, M., Taghizadeh, N., & Beigy, H. (2013) · National Conference on Artificial Intelligence · Cited by 9
Learning-Driven Exploration for Reinforcement Learning
Usama, M., & Chang, D. E. (2021) · International Conference on Control, Automation and Systems (ICCAS) · doi:10.23919/iccas52745.2021.9649810 · Cited by 9
On Hard Exploration for Reinforcement Learning: A Case Study in Pommerman
Gao, C., Kartal, B., Hernández-Leal, P., & Taylor, M. E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v15i1.5220 · Cited by 9
Exploiting Action-Value Uncertainty to Drive Exploration in Reinforcement Learning
D’Eramo, C., Cini, A., & Restelli, M. (2019) · doi:10.1109/ijcnn.2019.8852326 · Cited by 8 · cited 2×
“Bootstrapped deep-Q networks (DQN) and Thomson sampling were utilised in [111].”
BackgroundMethodLearning Abstract Models for Strategic Exploration and Fast Reward TransferPreprint
Liu, E. Z., Keramati, R., Seshadri, S., Guu, K., Pasupat, P., Brunskill, E., & Liang, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05896 · Cited by 7
Probabilistic Guided Exploration for Reinforcement Learning in Self-Organizing Neural Networks
Wang, P., Zhou, W. J., Wang, D., & Tan, A. (2018) · IEEE International Conference on Agents (ICA) · doi:10.1109/agents.2018.8460067 · Cited by 5 · cited 3×
“An approach using the adaptive resonance theorem (ART) [150] was presented in [151] and was later extended in [152].”
BackgroundMethodExplicit Planning for Efficient Exploration in Reinforcement Learning
Zhang, L., Tang, K., & Yao, X. (2019) · neural information processing systems · Cited by 5
Towards High-Level Intrinsic Exploration in Reinforcement Learning
Bougie, N., & Ichise, R. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/733 · Cited by 5
Stochastic Curiosity Maximizing Exploration
Chien, J., & Hsu, P.-C. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207295 · Cited by 5 · cited 3×
“information gain was combined with the state prediction error into a single intrinsic reward in [58, 59].”
BackgroundInfluentialExploration Conscious Reinforcement Learning RevisitedPreprint
Shani, L., Efroni, Y., & Mannor, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05551 · Cited by 4
Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement LearningPreprint
Cohen, A., Yu, L., Qiao, X., & Tong, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00828 · Cited by 2
Curiosity-driven Exploration by Self-supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · arXiv · doi:10.48550/arxiv.1705.05363 · cited 2×
“A state representation method derived from inverse dynamic features (IDF) was used in [41].”
BackgroundMethodEcoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2020) · arXiv · doi:10.48550/arxiv.2004.12919
GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning
Kovač, G., Laversanne-Finot, A., & Oudeyer, P.-Y. (2020) · arXiv · doi:10.48550/arxiv.2008.04388
Verifiably Safe Exploration for End-to-End Reinforcement Learning
Hunt, N., Fulton, N., Magliacane, S., Hoang, N., Das, S., & Solar-Lezama, A. (2020) · arXiv · doi:10.48550/arxiv.2007.01223
Learning-Driven Exploration for Reinforcement Learning
Usama, M., & Chang, D. E. (2019) · arXiv · doi:10.48550/arxiv.1906.06890
Flagged first, then by citations
Research works printed but not linked: 1
- T. Gangwani, Q. Liu, J. Peng, Learning Self-Imitating Diverse Policies, in: 7th International Conference on Learning Representations, ICLR 2019, 2019.
Cited by
Sorted by citations
Review of large vision models and visual prompt engineering
Wang, J., Liu, Z., Zhao, L., Wu, Z., Ma, C., Yu, S., Dai, H., Yang, Q., Liu, Y., Zhang, S., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100047 · Cited by 257
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3236361 · Cited by 245
“Work [21] is a concurrent work.”
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2109.06668 · Cited by 192
“Work [21] is a concurrent work.”
AI for science: Predicting infectious diseases
Zhao, P., Li, S., Cao, Z., Hu, P. J., Wang, J., Xiang, Y., Xie, D. G., & Lu, X. (2024) · Journal of Safety Science and Resilience · doi:10.1016/j.jnlssr.2024.02.002 · Cited by 193
Remote Sensing Image Classification: A Comprehensive Review and Applications
Mehmood, M., Shahzad, A., Zafar, B., Shabbir, A., & Ali, N. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/5880959 · Cited by 160
Metadata
- Authors
- Ladosz, P., Weng, L., Kim, M., & Oh, H.
- Affiliations
- OpenAI (United States) (US)
- Ulsan National Institute of Science and Technology (KR)
- Corresponding author
- Oh, Hyondong
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2022-09
- Volume
- 85
- Pages
- 1-22
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W4220747123
- Cited by
- 525
- References
- 185
- Citations by year
- 2026: 112 · 2025: 198 · 2024: 149 · 2023: 59 · 2022: 7
- FWCI
- 33.97
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Information Fusion (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Research Explorer (The University of Manchester) (submittedVersion, cc-by-nc-nd)
- Scholarworks@UNIST (Ulsan National Institute of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Ministry of Science, ICT and Future Planning (2020M3C1C1A01082375, 2020R1F1A1049066)
- National Research Foundation of Korea
- Ministry of Education (2020R1A6A1A03040570)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- safe exploration, random exploration, reward learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
