Exploration in deep reinforcement learning: A survey
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Solving detachment and derailment requires a good global exploration strategy that can better balance long-term and short-term environment information.
“Solving these problems requires a good global exploration strategy [211] that can better balance long-term and short-term environment information [195].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [195]
The citation to this survey attaches to the need for a good global exploration strategy, not to the later claim about balancing long- and short-term information. Section 2.2 (p. 4) says exploration is useful when local minima or flat rewards obstruct a solution, and the Introduction (p. 2) calls for more sophisticated exploration algorithms. Our copy does not connect a global strategy specifically to solving detachment and derailment. (LLM, conf=0.76)
Experimental — accuracy not yet measuredThe exploration component of RL has long been recognized as a critical driver for emergent behavior in traditional RL.
“While underappreciated in current reasoning literature, the exploration component of RL has long been recognized as a critical driver for emergent behavior in traditional RL (Yue et al., 2025; Silver et al., 2016; Ladosz et al., 2022; Pourcel et al., 2024).”
From Absolute Zero: Reinforced Self-play Reasoning with Zero Data (2025) · cites this paper as Ladosz et al., 2022
Conversational AI is dedicated to developing AI systems capable of smooth, natural, and context-aware human-computer interactions, focusing on dialogue modeling, question answering, user intent recognition, and multi-turn context tracking.
“This subdomain is dedicated to developing AI systems capable of smooth, natural, and context-aware human-computer interactions, by focusing on dialogue modeling, question answering, user intent recognition, and multi-turn context tracking [204], [205], [206], [207].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [206]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1× · see citing works
“impressive performance in Atari games [2], Mujoco simulator [3], controllers tuning [4], autonomous landing [5], self-driving cars [6] and healthcare [7].”
BackgroundReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1× · see citing works
“The most common technique for exploration in reinforcement learning is random exploration [1].”
MethodLabaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,100 · see citing works
Proceedings of IEEE International Conference on Computer Vision
(1995) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466933 · Cited by 13,117 · see citing works
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,869 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · cited 1× · see citing works
“impressive performance in Atari games [2], Mujoco simulator [3], controllers tuning [4], autonomous landing [5], self-driving cars [6] and healthcare [7].”
BackgroundSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · see citing works
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · cited 1× · see citing works
“A general overview of reinforcement learning can be found in [23].”
BackgroundProceedings. 1985 IEEE International Conference on Robotics and Automation
(1985) · doi:10.1109/robot.1985 · Cited by 3,692 · see citing works
2009 International Joint Conference on Artificial Intelligence
(2009) · doi:10.1109/jcai15434.2009 · Cited by 2,846 · see citing works
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342 · see citing works
Similarity estimation techniques from rounding algorithms
Charikar, M. (2002) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/509907.509965 · Cited by 2,263 · cited 2× · see citing works
“Here, a technique called SimHash [63] was used, which rep-”
MethodThe Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161 · cited 1× · see citing works
“The Atari games benchmark are a set of 57 Atari games combined under the Atari Learning Environment (ALE) [25].”
BackgroundCuriosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · cited 2× · see citing works
“A state representation method derived from inverse dynamic features (IDF) was used in [41].”
BackgroundMethod2011 International Joint Conference on Neural Networks
(2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2102870 · Cited by 1,772 · see citing works
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471 · cited 2× · see citing works
“An approach using the adaptive resonance theorem (ART) [150] was presented in [151] and was later extended in [152].”
MethodNguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306 · see citing works
Abandoning Objectives: Evolution Through the Search for Novelty Alone
Lehman, J., & Stanley, K. O. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00025 · Cited by 1,216 · cited 2× · see citing works
“One of the earliest methods called novelty search was devised in [74] and [75].”
BackgroundMethodZaki, M. J., & Meira, W. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810114 · Cited by 1,156 · see citing works
(2015) · Journal of Machine Learning Research · Cited by 1,119 · see citing works
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1703.03864 · Cited by 1,027 · cited 1× · see citing works
“Note that evolutionary approaches are generally not considered as the part of reinforcement learning but can be used to solve the same type of problems [72, 73].”
BackgroundProceedings Of The International Joint Conference On Neural Networks 2003
(2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2003.1223944 · Cited by 953 · see citing works
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic MotivationPreprint
Kulkarni, T. D., Narasimhan, K., Saeedi, A., & Tenenbaum, J. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06057 · Cited by 895 · see citing works
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889 · cited 2× · see citing works
“(DQfD) [128] differs in two aspects from [127].”
BackgroundEnsemble of exemplar-SVMs for object detection and beyond
Malisiewicz, T., Gupta, A., & Efros, A. A. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126229 · Cited by 865 · cited 1× · see citing works
“One example method is to use exemplar model [69] developed in [70].”
MethodReinforcement Learning in Healthcare: A Survey
Yu, C., Liu, J.-M., Nemati, S., & Yin, G. (2019) · ACM Computing Surveys · doi:10.1145/3477600 · Cited by 851 · see citing works
Deep Exploration via Bootstrapped DQNPreprint
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 837 · cited 2× · see citing works
“[123] DQN Uncertainty Atari: James Bond 1000 (DQN 600) Atari imgaes Q MB D/ D Tang and Agrawal [114] DDPG Uncertainty Mujoco: sparse mountaincar 0.”
BackgroundMethodWhat is intrinsic motivation? A typology of computational approaches
Oudeyer, P.-Y. (2007) · Frontiers in Neurorobotics · doi:10.3389/neuro.12.006.2007 · Cited by 811 · cited 1× · see citing works
“There are different ways of classifying intrinsic rewards [32, 16].”
BackgroundData Mining and Analysis: Fundamental Concepts and Algorithms
Zaki, M. J., & Meira, W. (2014) · Cited by 780 · see citing works
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747 · see citing works
(2016) · neural information processing systems · Cited by 735 · see citing works
Curious model-building control systems
Schmidhuber, J. (1991) · [Proceedings] 1991 IEEE International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.170605 · Cited by 733 · cited 2× · see citing works
“The simplest method of this type was described in [10, 11].”
BackgroundMethodOvercoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712 · see citing works
Such, F. P., Madhavan, V., Conti, E., Lehman, J., Stanley, K. O., & Clune, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06567 · Cited by 689 · see citing works
Learning to Navigate in Complex EnvironmentsPreprint
Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A. J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03673 · Cited by 679 · cited 2× · see citing works
“[48] A3C Prediction error DM Lab: Random Goal 96 (LSTM-A3C 65) DM Lab images Ac MB C/ C Tang et al.”
MethodA possibility for implementing curiosity and boredom in model-building neural controllers
Schmidhuber, J. (1991) · The MIT Press eBooks · doi:10.7551/mitpress/3115.003.0030 · Cited by 669 · cited 2× · see citing works
“The simplest method of this type was described in [10, 11].”
BackgroundMethodNoisy Networks for ExplorationPreprint
Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., Pietquin, O., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.10295 · Cited by 664 · see citing works
FeUdal Networks for Hierarchical Reinforcement LearningPreprint
Vezhnevets, A. S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., & Kavukcuoglu, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01161 · Cited by 608 · cited 1× · see citing works
“An approach in which both goal creation and selection mechanisms are devised by a meta-controller was proposed in [96].”
BackgroundConference on Neural Information Processing Systems
Liu, L., Zhou, T., Long, G., Jiang, J., & Zhang, C. (2019) · UTS ePRESS (University of Technology Sydney) · Cited by 593 · see citing works
2004 IEEE International Joint Conference on Neural Networks
(2004) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1381025 · Cited by 574 · see citing works
ViZDoom: A Doom-based AI research platform for visual reinforcement learning
Kempka, M., Wydmuch, M., Runc, G., Toczek, J., & Jaśkowski, W. (2016) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2016.7860433 · Cited by 550 · cited 5× · see citing works
“Similar to VizDoom, it is also from the first-person perspective, and the image is the state space.”
BackgroundInfluentialIncentivizing Exploration In Reinforcement Learning With Deep Predictive ModelsPreprint
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv.org · doi:10.48550/arxiv.1507.00814 · Cited by 545 · see citing works
2011 IEEE International Conference on Robotics and Automation
(2009) · IEEE Transactions on Haptics · doi:10.1109/toh.2009.52 · Cited by 540 · see citing works
Exploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527 · cited 8× · see citing works
“Stanton and Clune [51] agent position, reward grid A2C Prediction error Atari: Montezuma Revenge 3200 (A2C 0) Atari images Ac MB D/ D Achiam and Sastry [33] TRPO Prediction error Mujoco: halfcheetah 80 (VIME 40); Atari: Venture 400 (VIME 0) Atari RAM states/ Mujoco joints angles Ac MB C/ C Li et al. [44] A2C Predictio…”
BackgroundInfluentialQuality Diversity: A New Frontier for Evolutionary Computation
Pugh, J. K., Soros, L. B., & Stanley, K. O. (2016) · Frontiers in Robotics and AI · doi:10.3389/frobt.2016.00040 · Cited by 514 · cited 1× · see citing works
“More detailed reviews of exploration in evolution strategies can be found in [78] and [79].”
Background#Exploration: A Study of Count-Based Exploration for Deep Reinforcement LearningPreprint
Tang, H., Houthooft, R., Foote, D., Stooke, A., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04717 · Cited by 486 · cited 2× · see citing works
“One of the popular methods of this type was proposed in [62], where static hashing was used.”
BackgroundMethodLeveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
Vecerik, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Rothörl, T., Lampe, T., & Riedmiller, M. (2017) · arXiv · doi:10.48550/arxiv.1707.08817 · Cited by 478 · see citing works
Variational Information Maximisation for Intrinsically Motivated Reinforcement LearningPreprint
Mohamed, S., & Rezende, D. J. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.08731 · Cited by 460 · cited 1× · see citing works
“In [57], maximising mutual information between a sequence of actions leads to a state that is rewarded.”
BackgroundParameter Space Noise for ExplorationPreprint
Plappert, M., Houthooft, R., Dhariwal, P., Sidor, S., Chen, R. Y., Chen, X., Asfour, T., Abbeel, P., & Andrychowicz, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.01905 · Cited by 423 · cited 2× · see citing works
“noise is by comparing the decision made by the noisy and noiseless policy [156].”
BackgroundLarge-Scale Study of Curiosity-Driven Learning
Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., & Efros, A. A. (2018) · arXiv · doi:10.48550/arxiv.1808.04355 · Cited by 401 · cited 1× · see citing works
“IDF was compared against the state prediction method and random representation in [42] with the following conclusions: IDF had the best performance and it scales the best to the unseen environments.”
MethodCount-Based Exploration with Neural Density Models
Ostrovski, G., Bellemare, M. G., Oord, A. V. D., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.01310 · Cited by 377 · see citing works
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.00909 · Cited by 366 · cited 2× · see citing works
“[18] considered an application to multi-agent problems, Levine [19] provided a tutorial and extensive comparison with probabilistic inference methods and [20] provided an extensive description of the key breakthrough methods”
A Bayesian Framework for Reinforcement Learning
Strens, M. J. A. (2000) · Cited by 361 · see citing works
Electrospun nanofibers: Internal structure and intrinsic orientation
Dersch, R., Liu, T., Schaper, A., Greiner, A., & Wendorff, J. H. (2003) · Journal of Polymer Science Part A Polymer Chemistry · doi:10.1002/pola.10609 · Cited by 354 · see citing works
IJCAI-83: International joint conference on artificial intelligence
(1982) · Artificial Intelligence · doi:10.1016/0004-3702(82)90046-7 · Cited by 325 · see citing works
Learning by Playing - Solving Sparse Reward Tasks from ScratchPreprint
Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Van de Wiele, T., Mnih, V., Heess, N., & Springenberg, J. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10567 · Cited by 314 · see citing works
Safe Exploration in Continuous Action SpacesPreprint
Dalal, G., Dvijotham, K., Vecerík, M., Hester, T., Păduraru, C., & Tassa, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.08757 · Cited by 284 · see citing works
Reverse Curriculum Generation for Reinforcement LearningPreprint
Florensa, C., Held, D., Wulfmeier, M., Zhang, M., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05300 · Cited by 277 · see citing works
Deep Q-learning from DemonstrationsPreprint
Hester, T., Vecerík, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Dulac-Arnold, G., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03732 · Cited by 276 · see citing works
Adaptive ε-Greedy Exploration in Reinforcement Learning Based on Value Differences
Tokic, M. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-16111-7_23 · Cited by 266 · see citing works
2022 IEEE International Conference on Robotics and Biomimetics
(2022) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio55434.2022.10011704 · Cited by 262 · see citing works
Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study
Mouret, J.-B., & Doncieux, S. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00048 · Cited by 254 · cited 1× · see citing works
“More detailed reviews of exploration in evolution strategies can be found in [78] and [79].”
BackgroundEpisodic Curiosity through ReachabilityPreprint
Savinov, N., Raichuk, A., Marinier, R., Vincent, D., Pollefeys, M., Lillicrap, T., & Gelly, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02274 · Cited by 231 · cited 2× · see citing works
“Prediction error methods have also shown the ability to solve the couch-potato (noisy-TV) problem by storing observations in a memory buffer [60].”
BackgroundRIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated EnvironmentsPreprint
Raileanu, R., & Rocktäschel, T. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2002.12292 · Cited by 224 · cited 4× · see citing works
“IDF was utilised in [43], where the Euclidean distance between two consecutive state representations was used as an intrinsic reward, as shown”
BackgroundMethodInfluentialNear-Bayesian exploration in polynomial time
Kolter, J. Z., & Ng, A. Y. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553441 · Cited by 218 · see citing works
Agent57: Outperforming the Atari Human BenchmarkPreprint
Badia, A. P., Piot, B., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, D., & Blundell, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13350 · Cited by 216 · see citing works
Model-Based Active ExplorationPreprint
Shyam, P., Jaśkowski, W., & Gomez, F. (2018) · International Conference on Machine Learning · doi:10.48550/arxiv.1810.12162 · Cited by 207 · cited 2× · see citing works
“A similar approach was developed in [125], using the information gain among the environmental models to decide where to go.”
BackgroundMethodThe Malmo platform for artificial intelligence experimentation
Johnson, M., Hofmann, K., Hutton, T. J., & Bignell, D. E. (2016) · International Joint Conference on Artificial Intelligence · Cited by 205 · see citing works
Go-Explore: a New Approach for Hard-Exploration Problems
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2019) · arXiv · doi:10.48550/arxiv.1901.10995 · Cited by 205 · see citing works
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · RePEc: Research Papers in Economics · Cited by 201 · see citing works
Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum LearningPreprint
Forestier, S., Portelas, R., Mollard, Y., & Oudeyer, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02190 · Cited by 200 · see citing works
Self-Supervised Exploration via DisagreementPreprint
Pathak, D., Gandhi, D., & Gupta, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04161 · Cited by 190 · cited 1× · see citing works
“Another approximation could be to use ensembles of models as proposed in [45].”
MethodTrial without Error: Towards Safe Reinforcement Learning via Human Intervention
Saunders, W., Sastry, G., Stuhlmüller, A., & Evans, O. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/mzmg4429 · Cited by 188 · see citing works
Self-Imitation LearningPreprint
Oh, J., Guo, Y., Singh, S. P., & Lee, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05635 · Cited by 184 · cited 2× · see citing works
“[88] A2C Goals to Explore from Atari: Montezuma Revenge 2500 (A2C 0) Atari images Ac MF D/ D Guo et al.”
BackgroundConti, E., Madhavan, V., Such, F. P., Lehman, J., Stanley, K. O., & Clune, J. (2018) · neural information processing systems · Cited by 182 · see citing works
A survey on intrinsic motivation in reinforcement learningPreprint
Aubret, A., Matignon, L., & Hassas, S. (2019) · arXiv.org · doi:10.48550/arxiv.1908.06976 · Cited by 175 · cited 4× · see citing works
“Here, we largely follow the classification of [16] with the following categories: (i) prediction error methods, (ii) count-based methods and (iii) memory methods.”
BackgroundMethodInfluentialNever Give Up: Learning Directed Exploration Strategies
Badia, A. P., Sprechmann, P., Vitvitskyi, A., Guo, D., Piot, B., Kapturowski, S., Tieleman, O., Arjovsky, M., Pritzel, A., Bolt, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2002.06038 · Cited by 170 · see citing works
Generalization and Exploration via Randomized Value FunctionsPreprint
Osband, I., Roy, B. V., & Wen, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0635 · Cited by 169 · cited 2× · see citing works
“[112] LSVI Optimistic Tetris: 5000 (LSVI 4000) Hand tuned 22 features Ac MF D/ D”
BackgroundAn information-theoretic approach to curiosity-driven reinforcement learning
Still, S., & Precup, D. (2012) · Theory in Biosciences · doi:10.1007/s12064-011-0142-z · Cited by 165 · cited 1× · see citing works
“The general framework for these types of approaches was presented in [54, 55].”
BackgroundDeep Exploration via Randomized Value FunctionsPreprint
Osband, I., Roy, B. V., Russo, D., & Wen, Z. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07608 · Cited by 153 · cited 2× · see citing works
“[113] LSVI Optimistic Cartpole Swing up: 600 (DQN 0) State Id Ac MF D/ D Nikolov et al.”
BackgroundSurprise-Based Intrinsic Motivation for Deep Reinforcement Learning
Achiam, J., & Sastry, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01732 · Cited by 139 · see citing works
Randomized Prior Functions for Deep Reinforcement LearningPreprint
Osband, I., Aslanides, J., & Cassirer, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03335 · Cited by 124 · cited 1× · see citing works
“A similar approach was considered in [40].”
GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning AlgorithmsPreprint
Colas, C., Sigaud, O., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05054 · Cited by 118 · cited 2× · see citing works
“[115] split the exploration into two phases: (i) explore randomly and (ii) compare experiences to an expert-created imitation to determine the good behavior.”
BackgroundSafe Exploration in Finite Markov Decision Processes with Gaussian ProcessesPreprint
Turchetta, M., Berkenkamp, F., & Krause, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04753 · Cited by 116 · cited 3× · see citing works
“This can be either in the form of a mathematical model [139] or heuristic rules [140].”
BackgroundReinforcement Learning in Healthcare: A SurveyPreprint
Yu, C., Liu, J., & Nemati, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.08796 · Cited by 114 · see citing works
Trial without Error: Towards Safe Reinforcement Learning via Human InterventionPreprint
Saunders, W., Sastry, G., Stuhlmueller, A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05173 · Cited by 111 · cited 2× · see citing works
“[138] human intervention data DQN Human Designer Knowledge Atari: Space Invaders 0 catastrophes (DQN 800000) Atari images Q MB D/ D”
BackgroundConti, E., Madhavan, V., Such, F. P., Lehman, J., Stanley, K. O., & Clune, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06560 · Cited by 108 · cited 4× · see citing works
“Initially, novelty search was used as a stand-alone technique; however, recently, combining it with other techniques [77, 76] has shown more promise.”
BackgroundMethodInfluentialCURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement LearningPreprint
Colas, C., Fournier, P., Sigaud, O., Chétouani, M., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06284 · Cited by 104 · see citing works
Learning Montezuma's Revenge from a Single DemonstrationPreprint
Salimans, T., & Chen, R. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03381 · Cited by 102 · cited 2× · see citing works
“Salimans and Chen [131] single demonstration PPO Imitation with Exploration Strategy Atari: Montezuma Revenge with distraction 74500 (Playing by youtube 41098) Atari images Ac MF D/ D”
BackgroundMethodEX2: Exploration with Exemplar Models for Deep Reinforcement Learning
Fu, J., Co-Reyes, J. D., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01260 · Cited by 95 · cited 2× · see citing works
“[70] state encoder TRPO Memory Mujoco: SparseHalfCheetah 173.”
BackgroundMethodEfficient Bayes-Adaptive Reinforcement Learning using Sample-Based SearchPreprint
Guez, A., Silver, D., & Dayan, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.3109 · Cited by 94 · cited 2× · see citing works
“[120] guess of a prior Policy learning Uncertainty Dearden Maze: 965.”
BackgroundMethodA Laplacian Framework for Option Discovery in Reinforcement LearningPreprint
Machado, M. C., Bellemare, M. G., & Bowling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00956 · Cited by 93 · cited 2× · see citing works
“[101] hadcrafted features itself Exploratory Goal 4-room domain: 1 Enumarated state id Ac MB D/ D”
BackgroundMethodPlaying hard exploration games by watching YouTubePreprint
Aytar, Y., Pfaff, T., Budden, D. M., Paine, T. L., Wang, Z., & de Freitas, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11592 · Cited by 93 · cited 4× · see citing works
“[26] youtube embbeding IMPALA Imitation with Exploration Strategy Atari: Montezuma’s Revenge 80k (DqfD 4k) Atari Images Ac MF D/ D”
BackgroundMethodInfluentialSkew-Fit: State-Covering Self-Supervised Reinforcement LearningPreprint
Pong, V. H., Dalal, M., Lin, S., Nair, A., Bahl, S., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03698 · Cited by 90 · see citing works
Count-Based Exploration in Feature Space for Reinforcement Learning
Martin, J., Sasikumar, S. N., Everitt, T., & Hütter, M. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/344 · Cited by 87 · cited 2× · see citing works
“[68] BlobPROST features SARSAe Countbased Atari: Montezuma Revenge 2000 (SARSA 200) BlobPROST features Q MB D/ D”
BackgroundSafe Exploration of State and Action Spaces in Reinforcement Learning
Garcia, J., & Fernandez, F. (2012) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3761 · Cited by 85 · cited 2× · see citing works
“Garcia and Fernandez [134] baseline behaviour Human Designer Knowledge car parking problem 6.”
BackgroundExploring Parameter Space in Reinforcement Learning
Rückstieß, T., Sehnke, F., Schaul, T., Wierstra, D., Sun, Y., & Schmidhuber, J. (2010) · Paladyn Journal of Behavioral Robotics · doi:10.2478/s13230-010-0002-4 · Cited by 81 · cited 2× · see citing works
“The easiest method of including the noise is to include a fixed amount of noise [153].”
MethodCuriosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural NetworksPreprint
Houthooft, R., Chen, X., Duan, Y., Schulman, J. D., de Turck, F., & Abbeel, P. (2016) · arXiv.org · doi:10.48550/arxiv.1605.09674 · Cited by 81 · see citing works
Proceedings of the ... International Joint Conference on Autonomous Agents and Multiagent Systems
Press, A. (2002) · ACM Press eBooks · Cited by 81 · see citing works
State-Dependent Exploration for Policy Gradient Methods
Rückstieß, T., Felder, M., & Schmidhuber, J. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87481-2_16 · Cited by 78 · cited 1× · see citing works
“In [157], the problem of assigning rewards when the same state is present multiple times is discussed.”
BackgroundMachine Learning and Knowledge Discovery in Databases
Balcázar, J. L., Sebag, M., Gionis, A., et al. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15880-3 · Cited by 78 · see citing works
Diversity-Driven Exploration Strategy for Deep Reinforcement Learning
Hong, Z.-W., Shann, T.-Y., Su, S.-Y., Chang, Y.-H., & Lee, C.-Y. (2018) · arXiv · doi:10.48550/arxiv.1802.04564 · Cited by 77 · cited 2× · see citing works
“The diversity among policies was measured in [80].”
BackgroundHow novelty search escapes the deceptive trap of learning to learn
Risi, S., Vanderbleek, S. D., Hughes, C. E., & Stanley, K. O. (2009) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1569901.1569923 · Cited by 75 · cited 2× · see citing works
“[75] measure of policies distance NEAT Evolution strategies T-maze: solved after 50,000 evaluations enumarated state id Ac MF D/ D”
BackgroundMethodInformation-theoretic approach to interactive learning
Still, S. (2009) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/85/28005 · Cited by 72 · cited 1× · see citing works
“The general framework for these types of approaches was presented in [54, 55].”
BackgroundEigenoption Discovery through the Deep Successor RepresentationPreprint
Machado, M. C., Rosenbaum, C., Guo, X., Liu, M., Tesauro, G. J., & Campbell, M. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.11089 · Cited by 71 · cited 2× · see citing works
“This was later improved in [103] by replacing the handcrafted adjacency matrix with successor representations.”
BackgroundMethodUnsupervised Learning of Goal Spaces for Intrinsically Motivated Goal ExplorationPreprint
Péré, A., Forestier, S., Sigaud, O., & Oudeyer, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00781 · Cited by 70 · cited 1× · see citing works
“[95] IMGEP Exploratory Goal Mujoco (KLC): ArmArrow 7.”
BackgroundAlignment of Requirements Specification and Testing: A Systematic Mapping Study
Barmi, Z. A., Ebrahimi, A. H., & FELDT, R. H. (2011) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2011.58 · Cited by 68 · see citing works
Exploration from Demonstration for Interactive Reinforcement Learning
Subramanian, K. A., Isbell, C. L., & Thomaz, A. L. (2016) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2936924.2936990 · Cited by 68 · see citing works
Random curiosity-driven exploration in deep reinforcement learning
Li, J., Shi, X., Li, J., Zhang, X., & Wang, J.-Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.08.024 · Cited by 68 · cited 2× · see citing works
“[44] A2C Prediction error Atari: Asterix 500000 (RND 10000) Atari images Ac MB D/ D Kim et al.”
BackgroundThe Uncertainty Bellman Equation and Exploration.
O’Donoghue, B., Osband, I., Munos, R., & Mnih, V. (2018) · International Conference on Machine Learning · Cited by 67 · see citing works
EMI: Exploration with Mutual InformationPreprint
Kim, H., Kim, J., Jeong, Y., Levine, S., & Song, H. O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01176 · Cited by 65 · see citing works
Folkers, A. (2019) · BesMasters · doi:10.1007/978-3-658-28886-0_3 · Cited by 60 · see citing works
Deep Reinforcement Learning: A State-of-the-Art Walkthrough
Lazaridis, A., Fachantidis, A., & Vlahavas, I. P. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12412 · Cited by 58 · cited 1× · see citing works
“[18] considered an application to multi-agent problems, Levine [19] provided a tutorial and extensive comparison with probabilistic inference methods and [20] provided an extensive description of the key breakthrough methods”
BackgroundBehaviour Suite for Reinforcement LearningPreprint
Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepesvári, C., Singh, S. P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03568 · Cited by 58 · see citing works
Deep Reinforcement Learning for NLP
Wang, W. Y., Li, J., & He, X. (2018) · doi:10.18653/v1/p18-5007 · Cited by 57 · see citing works
Contingency-Aware Exploration in Reinforcement LearningPreprint
Choi, J., Guo, Y., Moczulski, M., Oh, J., Wu, N., Norouzi, M., & Lee, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01483 · Cited by 57 · cited 1× · see citing works
“To generate an even more compact representation, in [64], the state was represented as the learned xy position of an agent.”
BackgroundVisual novelty, curiosity, and intrinsic reward in machine learning and the brain
Jaegle, A., Mehrpour, V., & Rust, N. C. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.08.004 · Cited by 54 · see citing works
The Uncertainty Bellman Equation and ExplorationPreprint
O’Donoghue, B., Osband, I., Munos, R., & Mnih, V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05380 · Cited by 53 · cited 2× · see citing works
“[121] derived Bayesian uncertainty such that it can be computed using the Bellman principle and the output of the neural network.”
BackgroundMethodAlgorithm-Directed Exploration for Model-Based Reinforcement Learning in Factored MDPs
Guestrin, C., Patrascu, R., & Schuurmans, D. (2002) · Cited by 49 · see citing works
Making Efficient Use of Demonstrations to Solve Hard Exploration ProblemsPreprint
Paine, T. L., Gülçehre, Ç., Shahriari, B., Denil, M., Hoffman, M., Soyer, H., Tanburn, R., Kapturowski, S., Rabinowitz, N. C., Williams, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01387 · Cited by 49 · see citing works
Verifiably safe exploration for end-to-end reinforcement learning
Hunt, N., Fulton, N., Magliacane, S., Hoang, T. N., Das, S., & Solar-Lezama, A. (2021) · International Conference on Hybrid Systems: computation and control · doi:10.1145/3447928.3456653 · Cited by 48 · see citing works
Combating Reinforcement Learning's Sisyphean Curse with Intrinsic FearPreprint
Lipton, Z. C., Azizzadenesheli, K., Kumar, A., Li, L., Gao, J., & Deng, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01211 · Cited by 47 · see citing works
Curiosity-Driven Experience Prioritization via Density Estimation
Zhao, R., & Tresp, V. (2019) · arXiv · doi:10.48550/arxiv.1902.08039 · Cited by 47 · cited 1× · see citing works
“The density models include context tree switching (CTS) [14], Gaussian mixture models [66] or PixelCNN [67].”
MethodModel-based reinforcement learning with parametrized physical models and optimism-driven exploration
Xie, C., Patil, S., Moldovan, T. M., Levine, S., & Abbeel, P. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487172 · Cited by 45 · cited 2× · see citing works
“The linear Gaussian algorithm can also be used as a model of the reward [110].”
BackgroundMethodStanton, C., & Clune, J. (2018) · arXiv.org · doi:10.48550/arxiv.1806.00553 · Cited by 42 · see citing works
Autonomous Quadrotor Landing using Deep Reinforcement LearningPreprint
Polvara, R., Patacchiola, M., Sharma, S. K., Wan, J., Manning, A. J., Sutton, R., & Cangelosi, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.03339 · Cited by 40 · see citing works
See, Hear, Explore: Curiosity via Audio-Visual AssociationPreprint
Dean, V., Tulsiani, S., & Gupta, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.03669 · Cited by 37 · see citing works
Fast and slow curiosity for high-level exploration in reinforcement learning
Bougie, N., & Ichise, R. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01849-3 · Cited by 33 · cited 2× · see citing works
“Improvements to autoencoder-based approaches were proposed in [38, 39], where a slow-trained autoencoder was added.”
BackgroundProceedings. The 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology
(2005) · IEEE/WIC/ACM International Conference on Intelligent Agent Technology · doi:10.1109/iat.2005.146 · Cited by 31 · see citing works
Why is Posterior Sampling Better than Optimism for Reinforcement Learning?Preprint
Osband, I., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.00215 · Cited by 30 · cited 2× · see citing works
“Tabular summary of optimistic and uncertainty approaches is shown in Table 4 and have been extensively compared in [108].”
MethodQuality Diversity Through Surprise
Gravina, D., Liapis, A., & Yannakakis, G. N. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2018.2877215 · Cited by 30 · cited 4× · see citing works
“Initially, novelty search was used as a stand-alone technique; however, recently, combining it with other techniques [77, 76] has shown more promise.”
BackgroundMethodInfluentialA Critical Investigation of Deep Reinforcement Learning for NavigationPreprint
Dhiman, V., Banerjee, S., Griffin, B., Siskind, J. M., & Corso, J. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.02274 · Cited by 30 · see citing works
Information-Directed Exploration for Deep Reinforcement LearningPreprint
Nikolov, N., Kirschner, J., Berkenkamp, F., & Krause, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07544 · Cited by 30 · cited 2× · see citing works
“[122] bDQN and C51 Uncertainty 55 atari games: 1058% of reference human performance Atari images Q MB D/ D”
BackgroundKhamassi, M., Velentzas, G., Tsitsimis, T., & Tzafestas, C. S. (2017) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc.2017.33 · Cited by 29 · cited 2× · see citing works
“[148] used long-term and short-term reward averages to control exploration and exploitation.”
BackgroundGaussian Processes for Sample Efficient Reinforcement Learning with RMAX-Like Exploration
Jung, T., & Stone, P. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15880-3_44 · Cited by 27 · see citing works
Curiosity-Bottleneck: Exploration By Distilling Task-Specific Novelty.
Kim, Y., Nam, W., Kim, H., Kim, J., & Kim, G. (2019) · KAIST Institutional Repository (KAIST) · Cited by 27 · see citing works
Fast active learning for pure exploration in reinforcement learningPreprint
Ménard, P., Domingues, O. D., Jönsson, A., Kaufmann, E., Leurent, E., & Valko, M. (2020) · Repositori digital de la UPF (Universitat Pompeu Fabra) · doi:10.48550/arxiv.2007.13442 · Cited by 25 · see citing works
ACDER: Augmented Curiosity-Driven Experience Replay
Li, B., Lü, T., Li, J., Lu, N., Cai, Y., & Wang, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197421 · Cited by 24 · cited 1× · see citing works
“Positions in a 3D space can also be used [50].”
BackgroundSuccessor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning
Janz, D., Hron, J., Mazur, P., Hofmann, K., Hernández-Lobato, J. M., & Tschiatschek, S. (2018) · arXiv · doi:10.48550/arxiv.1810.06530 · Cited by 23 · cited 2× · see citing works
“[116] DQN Uncertainty 49 Atari games: 77.”
MethodExploration by Distributional Reinforcement Learning
Tang, Y., & Agrawal, S. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/376 · Cited by 22 · see citing works
Curiosity-Driven Reinforcement Learning with Homeostatic Regulation
Magrans, I., & Ryota Kanai, C. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489075 · Cited by 22 · see citing works
Exploratory Gradient Boosting for Reinforcement Learning in Complex DomainsPreprint
Abel, D. L., Agarwal, A., Díaz, F., Krishnamurthy, A., & Schapire, R. E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.04119 · Cited by 21 · see citing works
Matheron, G., Perrin, N., & Sigaud, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8_24 · Cited by 21 · cited 2× · see citing works
“[85] state storage method DDPG Goals to Explore from Maze: reach reward after 146k (TD3 never) x-y position Ac MB C/ C”
BackgroundMethodAkiyama, T., Hachiya, H., & Sugiyama, M. (2010) · Neural Networks · doi:10.1016/j.neunet.2009.12.010 · Cited by 19 · cited 2× · see citing works
“Alternatively, policies can be sampled [118].”
BackgroundPearce, T., Anastassacos, N., Zaki, M., & Neely, A. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11324 · Cited by 19 · see citing works
Automatic Gain Tuning Method of a Quad-Rotor Geometric Attitude Controller Using A3C
Lee, S., & Bang, H. (2019) · International Journal of Aeronautical and Space Sciences · doi:10.1007/s42405-019-00233-x · Cited by 19 · cited 1× · see citing works
“impressive performance in Atari games [2], Mujoco simulator [3], controllers tuning [4], autonomous landing [5], self-driving cars [6] and healthcare [7].”
BackgroundCount-based exploration with the successor representation
Machado, M. C., Bellemare, M. G., & Bowling, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5955 · Cited by 19 · see citing works
Curiosity-Driven Exploration for Off-Policy Reinforcement Learning Methods
Li, B., Lü, T., Li, J., Lu, N., Cai, Y., & Wang, S. (2019) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio49542.2019.8961529 · Cited by 19 · cited 2× · see citing works
“[34] DDPG Prediction error Robot: FetchPush 1 (DDPG 0) Robot joints angles Ac MB C/ C Raileanu and Rocktäschel [43] IMPALA Prediction error Vizdoom: 0.”
BackgroundAsymmetric self-play for automatic goal discovery in robotic manipulation
OpenAI, O., Plappert, M., Sampedro, R., Xu, T., Akkaya, I., Kosaraju, V., Welinder, P., D'Sa, R., Petron, A., Pinto, H. P. D. O., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.04882 · Cited by 18 · see citing works
Adaptive exploration for continual reinforcement learning
Stulp, F. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385818 · Cited by 17 · cited 2× · see citing works
“Stulp [117] PI(2) Uncertainty Ball batting: learned after 20 steps Robot joints angles Ac MB C/ C Janz et al.”
BackgroundMethodForward-Backward Reinforcement LearningPreprint
Edwards, A. D., Downs, L., & Davidson, J. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10227 · Cited by 17 · see citing works
Explicit Explore-Exploit Algorithms in Continuous State SpacesPreprint
Henaff, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00617 · Cited by 17 · cited 2× · see citing works
“Another ensemble model was presented in [126].”
BackgroundMemory Based Trajectory-conditioned Policies for Learning from Sparse RewardsPreprint
Guo, Y., Choi, J., Moczulski, M., Feng, S., Samy Bengio, A., Norouzi, M., & Lee, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.10247 · Cited by 16 · see citing works
Conservative Exploration in Reinforcement LearningPreprint
Garcelon, E., Ghavamzadeh, M., Lazaric, A., & Pirotta, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.03218 · Cited by 15 · cited 2× · see citing works
“12, while in [136], agents are expected to perform no worse than the a priori known baseline.”
BackgroundReinforcement learning with internal-dynamics-based exploration using a chaotic neural network
Shibata, K., & Sakashita, Y. (2015) · doi:10.1109/ijcnn.2015.7280430 · Cited by 14 · cited 3× · see citing works
“In [154], chaotic networks were used to induce the noise in the network.”
BackgroundMethodInfluentialLearning exploration strategies in model-based reinforcement learning
Hester, T., Lopes, M. C., & Stone, P. (2013) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/xlfe9254 · Cited by 14 · cited 2× · see citing works
“In [97], a multi-arm bandit-based method to choose one strategy from a group of hand-designed strategies was proposed.”
BackgroundMethodAn ant system based exploration-exploitation for reinforcement learning
Chang, H. S. (2004) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1400937 · Cited by 13 · cited 1× · see citing works
“[149] used multiple agents (ants) to adjust the exploration parameters.”
MethodCuriosity-driven reinforcement learning with homeostatic regulationPreprint
de Abril, I. M., & Kanai, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.07440 · Cited by 13 · cited 1× · see citing works
“information gain was combined with the state prediction error into a single intrinsic reward in [58, 59].”
BackgroundSelf-Imitation Learning via Trajectory-Conditioned Policy for Hard-Exploration TasksPreprint
Guo, Y., Choi, J., Moczulski, M., Samy Bengio, A., Norouzi, M., & Lee, H. (2019) · arXiv (Cornell University) · Cited by 12 · see citing works
GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning
Kovač, G., Laversanne-Finot, A., & Oudeyer, P. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3216911 · Cited by 12 · see citing works
Generative adversarial exploration for reinforcement learning
Hong, W., Zhu, M., Liu, M., Zhang, W., Zhou, M. Q., Yu, Y. H., & Sun, P. (2019) · International Conference on Distributed Artificial Intelligence · doi:10.1145/3356464.3357706 · Cited by 11 · cited 2× · see citing works
“[36] DQN Prediction error Atari: Montezuma Revenge 200 (DQN 0) Enumarated state id/ Atari Image Q MB D/ D”
BackgroundMethodArtificial Neural Networks and Machine Learning – ICANN 2020
Riccio Salvatore, D., Deyan, D., Giorgio, J., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8 · Cited by 11 · see citing works
Knowledge-Based Exploration for Reinforcement Learning in Self-Organizing Neural Networks
Teng, T.-H., & Tan, A. (2012) · IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2012.154 · Cited by 10 · cited 1× · see citing works
“An approach using the adaptive resonance theorem (ART) [150] was presented in [151] and was later extended in [152].”
MethodAdaptive exploration in reinforcement learning
Patrascu, R., & Stacey, D. (2003) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1999.833417 · Cited by 10 · cited 2× · see citing works
“One technique to adapt the exploration rate is by simply considering a reward and adjusting the random exploration parameter accordingly, as described in [144].”
BackgroundMethodDead-ends and Secure Exploration in Reinforcement Learning
Fatemi, M., Sharma, S., van Seijen, H., & Kahou, S. E. (2019) · International Conference on Machine Learning · Cited by 10 · see citing works
Directed Exploration for Reinforcement LearningPreprint
Guo, Z. D., & Brunskill, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07805 · Cited by 10 · see citing works
Adaptive Procedural Task Generation for Hard-Exploration ProblemsPreprint
Fang, K., Zhu, Y., Savarese, S., & Fei-Fei, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00350 · Cited by 10 · cited 3× · see citing works
“igated in [94] or [104], but these approaches rely on the ability to parametrize the task.”
BackgroundInfluentialOn Hard Exploration for Reinforcement Learning: a Case Study in Pommerman
Gao, C., Kartal, B., Hernandez-Leal, P., & Taylor, M. E. (2019) · Artificial Intelligence and Interactive Digital Entertainment Conference · doi:10.48550/arxiv.1907.11788 · Cited by 10 · cited 3×
“This can be either in the form of a mathematical model [139] or heuristic rules [140].”
BackgroundMethodAutomatic Abstraction in Reinforcement Learning Using Ant System Algorithm
Ghafoorian, M., Taghizadeh, N., & Beigy, H. (2013) · National Conference on Artificial Intelligence · Cited by 9 · see citing works
Learning-Driven Exploration for Reinforcement Learning
Usama, M., & Chang, D. E. (2021) · International Conference on Control, Automation and Systems (ICCAS) · doi:10.23919/iccas52745.2021.9649810 · Cited by 9 · see citing works
Learning Transferable Domain Priors for Safe Exploration in Reinforcement Learning
Karimpanal, T. G., Rana, S., Gupta, S., Tran, T., & Venkatesh, S. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207344 · Cited by 9 · see citing works
On Hard Exploration for Reinforcement Learning: A Case Study in Pommerman
Gao, C., Kartal, B., Hernández-Leal, P., & Taylor, M. E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v15i1.5220 · Cited by 9 · see citing works
Exploiting Action-Value Uncertainty to Drive Exploration in Reinforcement Learning
D’Eramo, C., Cini, A., & Restelli, M. (2019) · doi:10.1109/ijcnn.2019.8852326 · Cited by 8 · cited 2× · see citing works
“Bootstrapped deep-Q networks (DQN) and Thomson sampling were utilised in [111].”
BackgroundMethodProbabilistic Guided Exploration for Reinforcement Learning in Self-Organizing Neural Networks
Wang, P., Zhou, W. J., Wang, D., & Tan, A. (2018) · IEEE International Conference on Agents (ICA) · doi:10.1109/agents.2018.8460067 · Cited by 5 · cited 3× · see citing works
“An approach using the adaptive resonance theorem (ART) [150] was presented in [151] and was later extended in [152].”
BackgroundMethodExplicit Planning for Efficient Exploration in Reinforcement Learning
Zhang, L., Tang, K., & Yao, X. (2019) · neural information processing systems · Cited by 5 · see citing works
Towards High-Level Intrinsic Exploration in Reinforcement Learning
Bougie, N., & Ichise, R. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/733 · Cited by 5 · see citing works
Stochastic Curiosity Maximizing Exploration
Chien, J., & Hsu, P.-C. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207295 · Cited by 5 · cited 3× · see citing works
“information gain was combined with the state prediction error into a single intrinsic reward in [58, 59].”
BackgroundInfluentialExploration Conscious Reinforcement Learning RevisitedPreprint
Shani, L., Efroni, Y., & Mannor, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05551 · Cited by 4 · see citing works
Learning Abstract Models for Strategic Exploration and Fast Reward TransferPreprint
Liu, E. Z., Keramati, R., Seshadri, S., Guu, K., Pasupat, P., Brunskill, E., & Liang, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05896 · Cited by 3 · see citing works
Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement LearningPreprint
Cohen, A., Yu, L., Qiao, X., & Tong, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00828 · Cited by 2 · see citing works
Curiosity-driven Exploration by Self-supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · arXiv · doi:10.48550/arxiv.1705.05363 · cited 2×
“A state representation method derived from inverse dynamic features (IDF) was used in [41].”
BackgroundMethodEcoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2020) · arXiv · doi:10.48550/arxiv.2004.12919
GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning
Kovač, G., Laversanne-Finot, A., & Oudeyer, P.-Y. (2020) · arXiv · doi:10.48550/arxiv.2008.04388
Verifiably Safe Exploration for End-to-End Reinforcement Learning
Hunt, N., Fulton, N., Magliacane, S., Hoang, N., Das, S., & Solar-Lezama, A. (2020) · arXiv · doi:10.48550/arxiv.2007.01223
Learning-Driven Exploration for Reinforcement Learning
Usama, M., & Chang, D. E. (2019) · arXiv · doi:10.48550/arxiv.1906.06890
Flagged first, then by citations
Research works printed but not linked: 4
- [12][12] B. Eysenbach, A. Gupta, J. Ibarz, S. Levine, Diversity Is All You Need, 7th International Conference on Learning Representations, ICLR 2019 (2019). URL: https://openreview.net/pdf?id=SJx63jRqFm . -
- [24][24] Exploration, https://dictionary.cambridge.org/dictionary/ english/exploration , 2020. Accessed: 2020-04-09. -
- [83][83] T. Gangwani, Q. Liu, J. Peng, Learning Self-Imitating Diverse Policies, 7th International Conference on Learning Representations, ICLR 2019 (2019). -
- [89][89] Y. Guo, J. Choi, M. Moczulski, S. Feng, S. Bengio, M. Norouzi, H. Lee, Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards, Conference on Neural Information Processing Systems, NeurIPS 2020 (2020). -
Cited by
Sorted by citations
Review of large vision models and visual prompt engineering
Wang, J., Liu, Z., Zhao, L., Wu, Z., Ma, C., Yu, S., Dai, H., Yang, Q., Liu, Y., Zhang, S., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100047 · Cited by 231 · see citing works
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3236361 · Cited by 209 · see citing works
“Work [21] is a concurrent work.”
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2109.06668 · Cited by 192
“Work [21] is a concurrent work.”
AI for science: Predicting infectious diseases
Zhao, P., Li, S., Cao, Z., Hu, P. J., Wang, J., Xiang, Y., Xie, D. G., & Lu, X. (2024) · Journal of Safety Science and Resilience · doi:10.1016/j.jnlssr.2024.02.002 · Cited by 168 · see citing works
Remote Sensing Image Classification: A Comprehensive Review and Applications
Mehmood, M., Shahzad, A., Zafar, B., Shabbir, A., & Ali, N. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/5880959 · Cited by 160 · see citing works
He, H., Meng, X., Wang, Y., Khajepour, A., An, X., Wang, R., & Sun, F. (2023) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2023.114248 · Cited by 148 · see citing works
Zhang, Y., Zhao, W., Wang, J., & Yuan, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128423 · Cited by 120 · see citing works
Dynamic path planning of mobile robots using adaptive dynamic programming
Li, X., Wang, L., An, Y., Huang, Q.-L., Cui, Y., & Hu, H. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121112 · Cited by 107 · see citing works
“The reinforcement learning methods have excellent performance, but designing an appropriate reward function can be difficult and the training process requires significant computational power (Ladosz et al., 2022).”
Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Majid, A. Y., Majid, A. Y., Saaybi, S., van Rietbergen, T., François-Lavet, V., Prasad, R. R. V., & Verhoeven, C. J. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264540 · Cited by 108 · see citing works
“Consequently, it is hard for the agent to learn action–reward associations to approach optimal policy [72], [74], [75], [76].”
BackgroundProcopio, A., Cesarelli, G., Donisi, L., Merola, A., Amato, F., & Cosentino, C. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107681 · Cited by 111 · see citing works
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102 · see citing works
“• Reinforcement Learning: Reinforcement learning, characterized by its adaptability and optimization capabilities, has become increasingly vital in decision-making and autonomous systems [176], [177].”
Backgrounddel Real, A. J., Andreiana, D. S., Roldán, Á. O., Bustos, A. H., & Acevedo, L. (2022) · Applied Sciences · doi:10.3390/app122312377 · Cited by 97 · see citing works
“Thanks to this methodology, striking DRL techniques and algorithms are identified, which will lead to the improvement in the efficiency of industrial processes in the near future.”
MethodDevelopments in Image Processing Using Deep Learning and Reinforcement Learning
Valente, J., António, J., Mora, C., & Jardim, S. (2023) · Journal of Imaging · doi:10.3390/jimaging9100207 · Cited by 95 · see citing works
Ma, H., Fan, C., Zhang, Y., Wang, Q., Yu, K., & Ma, Z. (2025) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2025.112770 · Cited by 92 · see citing works
Oyewola, D. O., Akinwunmi, S. A., & Omotehinwa, T. O. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111290 · Cited by 83 · see citing works
Liu, J., Guan, S., Zou, Q., Wu, H., Tiwari, P., & Ding, Y. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111329 · Cited by 75 · see citing works
Multi-modal policy fusion for end-to-end autonomous driving
Huang, Z., Sun, S., Zhao, J., & Mao, L. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101834 · Cited by 74 · see citing works
del Río, A., Jiménez, D., & Serrano, J. (2024) · IEEE Access · doi:10.1109/access.2024.3472473 · Cited by 76 · see citing works
“Exploration strategies are crucial in RL to balance exploration and exploitation [66].”
BackgroundCheng, N., Wang, X., Li, Z., Yin, Z., Luan, T. H., & Shen, X. (2024) · IEEE Network · doi:10.1109/mnet.2024.3438543 · Cited by 58 · see citing works
“Therefore, n-step learning leads to an overestimation of the rewards, as highlighted in [12].”
BackgroundKahil, H., Sharma, S., Välisuo, P. O., & Elmusrati, M. S. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.125734 · Cited by 58 · see citing works
A residual reinforcement learning method for robotic assembly using visual and force information
Zhang, Z., Wang, Y., Zhang, Z., Wang, L., Huang, H., & Cao, Q. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.11.008 · Cited by 55 · see citing works
Deep Reinforcement Learning based real-time open-pit mining truck dispatching system
Noriega, R., Pourrahimian, Y., & Askari-Nasab, H. (2024) · Computers & Operations Research · doi:10.1016/j.cor.2024.106815 · Cited by 55 · see citing works
Zhou, B., Zhou, Q., Hu, S., Ma, D., Jin, S., & Lee, D. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3352730 · Cited by 54 · see citing works
“RL is distinguished by its ‘trial-and-error’ approach, rooted in the Markov Decision Process (MDP) and steered by a system of rewards [20].”
BackgroundCui, Z., Guan, W., & Zhang, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124906 · Cited by 55 · see citing works
Zhou, Z., Wen, Z., Cui, J., Luan, H., Chen, Y., & Lin, D. (2025) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2025.110885 · Cited by 53 · see citing works
Adaptive urban traffic signal control based on enhanced deep reinforcement learning
Cai, C., & Wei, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-64885-w · Cited by 52 · see citing works
Li, X., Liang, X., Wang, X., Wang, R., Shu, L., & Xu, W. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110543 · Cited by 51 · see citing works
Reinforcement Learning for Autonomous Process Control in Industry 4.0: Advantages and Challenges
Nievas, N., Pagès‐Bernaus, A., Bonada, F., Echeverria, L., & Domingo, X. (2024) · Applied Artificial Intelligence · doi:10.1080/08839514.2024.2383101 · Cited by 54 · see citing works
Liang, Y., Tang, H., Wu, H., Wang, Y., & Jiao, P. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3445350 · Cited by 50 · see citing works
“DRL is celebrated for its adaptive learning capabilities and proficiency in making near-optimal decisions based on environmental interactions, thereby excelling in scenarios where conventional algorithms falter [29].”
Adam, M., & Baroudi, U. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3506214 · Cited by 46 · see citing works
Artificial Intelligence in Cybersecurity: A Comprehensive Review and Future Direction
Ofusori, L. O., Bokaba, T., & Mhlongo, S. (2024) · Applied Artificial Intelligence · doi:10.1080/08839514.2024.2439609 · Cited by 46 · see citing works
“According to Camacho (2024), the most common AI techniques used in cybersecurity include ML and DL. ML is a subfield of AI that focuses on the development of algorithms and statistical models that enable computers to learn from and make predictions or decisions based on data (Ladosz et al. 2022).”
A critical review of safe reinforcement learning strategies in power and energy systems
Bui, V., Mohammadi, S., Das, S., Hussain, A., Hollweg, G. V., & Su, W. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110091 · Cited by 46 · see citing works
Thermal Modeling and Thermal-Aware Energy Saving Methods for Cloud Data Centers: A Review
Lin, J., Lin, W., Huang, H., Lin, W., & Li, K. (2023) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2023.3346332 · Cited by 43 · see citing works
“To solve this problem, a multi-agent cooperative control framework [131] can be used to realize the joint control of multiple cooling units, which can maintain the global cooling supply-demand balance while considering the local thermal fluctuations.”
MethodA Systematic Literature Review on AI Safety: Identifying Trends, Challenges, and Future Directions
Salhab, W., Ameyed, D., Jaafar, F., & Mcheick, H. (2024) · IEEE Access · doi:10.1109/access.2024.3440647 · Cited by 43 · see citing works
“However, in situations where incentives are few or where mistakes are undesirable, and safety is a top priority, exploration becomes difficult [30], [31].”
BackgroundExploring reinforcement learning in process control: a comprehensive survey
Rajasekhar, N., Radhakrishnan, T. K., & Samsudeen, N. (2025) · International Journal of Systems Science · doi:10.1080/00207721.2025.2469821 · Cited by 45 · see citing works
“• Challenge: Ensuring that RL agents operate safely and stably in critical applications like healthcare or autonomous systems is crucial (Kordabad et al., 2022; Ladosz et al., 2022).”
Deep reinforcement learning-based local path planning in dynamic environments for mobile robot
Tao, B., & Kim, J. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102254 · Cited by 42 · see citing works
Aghasi, A., Jamshidi, K., Bohlooli, A., & Javadi, B. (2023) · Computer Networks · doi:10.1016/j.comnet.2023.109624 · Cited by 42 · see citing works
Ginzburg-Ganz, E., Segev, I., Balabanov, A., Segev, E., Naveh, S. K., Machlev, R., Belikov, J., Katzir, L., Keren, S., & Levron, Y. (2024) · Energies · doi:10.3390/en17215307 · Cited by 37 · see citing works
Li, Y., Jiang, Y., Lu, J., & Tan, C. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111485 · Cited by 35 · see citing works
A social path to human-like artificial intelligence
Duéñez‐Guzmán, E. A., Sadedin, S., Wang, J. X., McKee, K. R., & Leibo, J. Z. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00754-x · Cited by 39 · see citing works
“Sub-stantial research effort has been devoted to balancing this trade-off 20,21 , and some bounds are known 22,23 .”
BackgroundQin, Y., Liu, X., Li, X., & Mao, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113595 · Cited by 34 · see citing works
Zhang, Y., Huang, J., He, L., Zhao, D., & Zhao, Y. (2024) · Sustainable Energy & Fuels · doi:10.1039/d3se01403g · Cited by 33 · see citing works
Deep Reinforcement Learning Model for Stock Portfolio Management Based on Data Fusion
Li, H., Wu, R., & Hai, M. (2024) · Neural Processing Letters · doi:10.1007/s11063-024-11582-4 · Cited by 31 · see citing works
Pan, J., & Wei, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124845 · Cited by 31 · see citing works
Huang, D., Zhao, H. P., Tian, W., & Chen, K. (2024) · Computers & Industrial Engineering · doi:10.1016/j.cie.2024.110768 · Cited by 31 · see citing works
Ruan, S., & Lu, K. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100463 · Cited by 30 · see citing works
Autonomous morphing strategy for a long-range aircraft using reinforcement learning
Zhang, B., Guo, J., Wang, H., & Tang, S. (2024) · Aerospace Science and Technology · doi:10.1016/j.ast.2024.109087 · Cited by 28 · see citing works
Evolution-Guided Adaptive Dynamic Programming for Nonlinear Optimal Control
Wang, D., Huang, H., Liu, D., Zhao, M., & Qiao, J. (2024) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2024.3417230 · Cited by 27 · see citing works
“These challenges encompass the issue of gradient vanishing [6] and the absence of effective exploration [7].”
BackgroundAuto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft
Li, H., Yang, X., Wang, Z., Zhu, X., Zhou, J., Qiao, Y., Wang, X., Li, H., Lu, L., & Dai, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01554 · Cited by 28 · see citing works
“Efficient Learning in Sparse Reward Tasks is a long-standing challenge in RL due to the lack of effective learning signals [26].”
Chen, C., Zhou, J., Wang, H., Fan, Y., Song, X., Xie, J., Bäck, T., & Wang, H. (2024) · Microsystems & Nanoengineering · doi:10.1038/s41378-024-00792-4 · Cited by 27 · see citing works
“3, the rewards are 0 for all actions except the last action, which is typical in RL 28,29 .”
BackgroundZhu, Z., Dong, S., Zhang, H., Parker, W. J., Yin, R., Bai, X., Yu, Z., Wang, J., Gao, Y., & Ren, H. (2025) · Bioresource Technology · doi:10.1016/j.biortech.2025.132210 · Cited by 27 · see citing works
Fan, J., Zhang, X., & Zou, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120630 · Cited by 27 · see citing works
Özalp, R., Uçar, A., & Guzelis, C. (2024) · IEEE Access · doi:10.1109/access.2024.3385426 · Cited by 27 · see citing works
“The algorithms face specific challenges, such as the need for expert knowledge to determine the appropriate reward function [19], to model the complex environment [20], and to construct the proper algorithm for complex tasks [21].”
BackgroundXu, H., Xuan, J., Zhang, G., & Lü, J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127716 · Cited by 27 · see citing works
Synergistic integration of metaheuristics and machine learning: latest advances and emerging trends
Zhang, R., Wang, J., Liu, C., Su, K., Ishibuchi, H., & Jin, Y. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11266-y · Cited by 31 · see citing works
Surprise and novelty in the brain
Modirshanechi, A., Becker, S., Brea, J., & Gerstner, W. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2023.102758 · Cited by 29 · see citing works
“Intuitive usage of ‘surprise’ and ‘novelty’ is common practice in neuroscience [5], psychology [6], and machine learning [7].”
Edge-Cloud Collaborative Streaming Video Analytics With Multi-Agent Deep Reinforcement Learning
Qian, B., Xuan, Y., Wu, D., Wen, Z., Yang, R., He, S., Chen, J., & Ranjan, R. K. (2024) · IEEE Network · doi:10.1109/mnet.2024.3398724 · Cited by 25 · see citing works
“There have been many techniques for enabling efficient model exploration during training, as reflected in [12].”
MethodA decentralized path planning model based on deep reinforcement learning
Guo, D., Ji, S., Yao, Y., & Chen, C. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109276 · Cited by 24 · see citing works
Predictive control of power demand peak regulation based on deep reinforcement learning
Fu, Q., Liu, L., Zhao, L., Wang, Y., Zheng, Y., Lu, Y., & Chen, J. (2023) · Journal of Building Engineering · doi:10.1016/j.jobe.2023.106992 · Cited by 22 · see citing works
Jeung, M., Jang, J., Yoon, K., & Baek, S. (2023) · Journal of Hydrology · doi:10.1016/j.jhydrol.2023.129973 · Cited by 24 · see citing works
Multi-correntropy fusion based fuzzy system for predicting DNA N4-methylcytosine sites
Ding, Y., Tiwari, P., Guo, F., & Zou, Q. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101911 · Cited by 21 · see citing works
A Survey of Visual Affordance Recognition Based on Deep Learning
Chen, D., Kong, D., Li, J., Wang, S., & Yin, B. (2023) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2023.3291558 · Cited by 20 · see citing works
Autonomous UAV-based surveillance system for multi-target detection using reinforcement learning
Salameh, H. B., Hussienat, A., Alhafnawi, M., & Al-Ajlouni, A. (2024) · Cluster Computing · doi:10.1007/s10586-024-04452-0 · Cited by 20 · see citing works
“…method initially assigns random actions to all states and converges on the action values, dynamically allocating learning times for various actions [38, 39]. blackIn this study, a convergence-based algorithm serves as an exploration strategy, with the aim of striking a har-monious balance between…”
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
“Exploration is a huge field in RL literature, and we refer readers to related reviews [193–195].”
BackgroundChen, Y., Guo, W., Liu, J., Shen, S., Lin, J., & Cui, D. (2024) · Measurement and Control · doi:10.1177/00202940231216543 · Cited by 21 · see citing works
Autonomous Landing on a Moving Platform Using Vision-Based Deep Reinforcement Learning
Ładosz, P., Mammadov, M., Shin, H., Shin, W., & Oh, H. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3379837 · Cited by 19 · see citing works
“For an overview and explanation of reinforcement learning, the reader is invited to [24], [25].”
BackgroundData-driven AI algorithms for construction machinery
Liang, K., Zhao, J., Zhang, Z., Guan, W., Pan, M., & Li, M. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105648 · Cited by 19 · see citing works
Multi-Feature Supervised Reinforcement Learning for Stock Trading
Fu, K., Yu, Y., & Li, B. (2023) · IEEE Access · doi:10.1109/access.2023.3298821 · Cited by 18 · see citing works
“The agent will continue to select actions according to the principle of maximizing the cumulative reward value, and finally achieve the goal [28], [29].”
BackgroundRDG-TE: Link reliability-aware DRL-GNN-based traffic engineering in SDN
Farhan, M., Shah, N., Wang, L., Muntean, G., & Song, H. H. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125963 · Cited by 18 · see citing works
Advanced Persistent Threats (APT) Attribution Using Deep Reinforcement Learning
Basnet, A. S., Ghanem, M. C., Dunsin, D., Kheddar, H., & Sowinski-Mydlarz, W. (2025) · Digital Threats Research and Practice · doi:10.1145/3736654 · Cited by 18 · see citing works
Learning-based hybrid algorithms for container relocation problem with storage plan
Wang, P., Xu, Q., Li, Y., Chen, Q., Tao, J., Qin, W., Huang, H., & Zou, Y. (2025) · Transportation Research Part E Logistics and Transportation Review · doi:10.1016/j.tre.2025.104048 · Cited by 17 · see citing works
Deep reinforcement learning in agent-based simulations for optimal media planning
Vargas-Pérez, V. A., Mesejo, P., Chica, M., & Cordón, Ó. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.10.029 · Cited by 17 · see citing works
Wang, S., Khan, A., Lin, Y., Jiang, Z., Tang, H., Alomar, S. Y., Sanaullah, M., & Bhatti, U. A. (2023) · Frontiers in Plant Science · doi:10.3389/fpls.2023.1142957 · Cited by 17 · see citing works
“Therefore, the state, action, and reward design will significantly influence DRL’s optimization quality (Ladosz et al., 2022).”
BackgroundRGBManip: Monocular Image-based Robotic Manipulation through Active Object Pose Estimation
An, B., Geng, Y., Chen, K., Li, X., Dou, Q., & Dong, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610690 · Cited by 17 · see citing works
Hierarchical Diffusion Policy: Manipulation Trajectory Generation via Contact Guidance
Wang, D., Liu, C., Chang, F., & Xu, Y. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3547272 · Cited by 16 · see citing works
“The exploration mechanism enables the agent to learn complex operational skills, but also brings low sample efficiency and weak generalization [2].”
Efficient Q-learning hyperparameter tuning using FOX optimization algorithm
Jumaah, M. A., Ali, Y. H., & Rashid, T. A. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.104341 · Cited by 16 · see citing works
Fu, H., Zheng, L., Zhang, W., Feng, Y., Zhu, L., Long, Y., & Li, J. (2025) · Agriculture · doi:10.3390/agriculture15090943 · Cited by 15 · see citing works
Unlocking Terahertz technology with machine learning: A comprehensive review
Cacciari, I., & Ranfagni, A. (2025) · Journal of Applied Physics · doi:10.1063/5.0272428 · Cited by 17 · see citing works
Zhang, W., Wang, Y., & Zhang, Y. (2025) · Astrodynamics · doi:10.1007/s42064-024-0246-0 · Cited by 15 · see citing works
“It will guide the training direction of the agent and is of great significance to the convergence speed of the algorithm and the generation of interception strategy [31, 32].”
Li, Z., Chen, W., Yang, J., & Yan, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126281 · Cited by 15 · see citing works
“While there has been a significant amount of research on active RL algorithms, such as [23,22,20,15], they are not directly applicable to autonomous seeking and estimation.”
InfluentialRLS-DTS: Reinforcement-Learning Linguistic Steganalysis in Distribution-Transformed Scenario
Wang, Y., Zhang, R., & Liu, J. (2023) · IEEE Signal Processing Letters · doi:10.1109/lsp.2023.3310380 · Cited by 15 · see citing works
“Once the observation space is constructed, the agent (steganalyzer) interacts with the observation space [24] to optimize the extraction of steganalysis features.”
MethodEnd-to-end multi-sensor fusion method based on deep reinforcement learning in UASNs
Zheng, L., Liu, M., Zhang, S., Liu, Z., & Dong, S. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.117904 · Cited by 15 · see citing works
Ding, C., Wang, J., Zhang, S., Yang, S., & Ma, Y. (2025) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2025.111043 · Cited by 14 · see citing works
Machine Learning-Assisted Polymer and Polymer Composite Design for Additive Manufacturing
Gyabaah, K. Y., Mahoney, B., Martey, A. K., Yan, C., Mensah, P. F., & Li, G. (2026) · AI Materials · doi:10.3390/aimater1010002 · Cited by 14 · see citing works
Li, Z., Jiang, H., & Liu, Y. (2023) · Measurement Science and Technology · doi:10.1088/1361-6501/acf23d · Cited by 14 · see citing works
“Moreover, the efficiency of data utilisation remains a considerable problem, especially in high-dimensional and complex data environments [30].”
Absolute Zero: Reinforced Self-play Reasoning with Zero Data
Zhao, A., Wu, Y., Yue, Y., Wu, T., Xu, Q., Yue, Y., Lin, M., Wang, S., Wu, Q., Zheng, Z., et al. (2025) · arXiv · Cited by 13 · see citing works
Machine learning approaches for active queue management: A survey, taxonomy, and future directions
Toopchinezhad, M. P., & Ahmadi, M. (2025) · Computer Networks · doi:10.1016/j.comnet.2025.111174 · Cited by 14 · see citing works
Hu, W., Zhou, Y., & Ho, H. W. (2024) · Electronics · doi:10.3390/electronics13122423 · Cited by 13 · see citing works
“…to exploiting learned knowledge, taking the action with argmax In both the original DQN and DDQN algorithms, the agent utilizes the ϵ -greedy policy [11], choosing the action argmax a ′ Q ( s t , a ′ ; θ ) with a probability of 1 − ϵ and opting for a random action with a probability of ϵ [37,38].”
MethodA digital twin dynamic migration method for industrial mobile robots
Wang, Y., & Zhao, X. (2024) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2024.102864 · Cited by 13 · see citing works
Liu, S., Zou, S., Chang, X., Liu, H., Zhang, L., & Deng, X. (2025) · Applied Intelligence · doi:10.1007/s10489-024-06036-2 · Cited by 13 · see citing works
“This enables the UAV to receive appropriate reward feedback after executing each action, effectively addressing the sparse reward problem in DRL [21].”
Comprehensive review of reinforcement learning for medical ultrasound imaging
Elmekki, H., Islam, S., Alagha, A., Sami, H., Spilkin, A., Zakeri, E., Zanuttini, A., Bentahar, J., Kadem, L., Xie, W., et al. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11268-w · Cited by 14 · see citing works
Luo, L., Yan, X., Wu, Q., & Sheng, V. S. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2024.9010141 · Cited by 13 · see citing works
Novelty as a drive of human exploration in complex stochastic environments
Modirshanechi, A., Lin, W.-H., He, X., Herzog, M. H., & Gerstner, W. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2502193122 · Cited by 16 · see citing works
Huang, D., Zhao, H. P., Cao, J., Chen, K., & Zhang, L. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113128 · Cited by 12 · see citing works
Ensemble Learning based on CNN and Transformer Models for Leaf Diseases Classification
Li, L.-H., & Tanone, R. (2024) · International Conference on Ubiquitous Information Management and Communication (IMCOM) · doi:10.1109/imcom60618.2024.10418393 · Cited by 12 · see citing works
“CNN [22] is one of the deep learning based model for solving computer vision problems such as image classification.”
MethodYu, C., Tsai, J., & Chang, Y.-T. (2024) · Electronics · doi:10.3390/electronics13234716 · Cited by 12 · see citing works
Liang, Z., Gu, Y., Li, P., & Tao, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126723 · Cited by 12 · see citing works
Li, Y., Xu, Y., Li, W., Li, G., Feng, Z., Liu, S., Du, J., & Li, X. (2025) · IEEE Transactions on Communications · doi:10.1109/tcomm.2025.3552297 · Cited by 13 · see citing works
“The ε -greedy strategy is a balancing act between exploration [21] and exploitation, which is crucial for learning optimal policies.”
Kim, J., Shin, W., Yim, J., Kwon, D., Kwon, D., & Lee, J. (2024) · Advanced Intelligent Systems · doi:10.1002/aisy.202300763 · Cited by 12 · see citing works
Artificial intelligence in disaster management: achievements, challenges, and prospects
Xu, J., Nair, D. J., & Waller, S. T. (2025) · Natural Hazards · doi:10.1007/s11069-025-07667-5 · Cited by 12 · see citing works
- Authors
- Ladosz, P., Weng, L., Kim, M., & Oh, H.
- Affiliations
- OpenAI (United States) (US)
- Ulsan National Institute of Science and Technology (KR)
- Corresponding author
- Oh, Hyondong
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2022-09
- Volume
- 85
- Pages
- 1-22
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W4220747123
- Cited by
- 525
- References
- 185
- Citations by year
- 2026: 112 · 2025: 198 · 2024: 149 · 2023: 59 · 2022: 7
- FWCI
- 33.97
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Information Fusion (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Research Explorer (The University of Manchester) (submittedVersion, cc-by-nc-nd)
- Scholarworks@UNIST (Ulsan National Institute of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Ministry of Science, ICT and Future Planning (2020M3C1C1A01082375, 2020R1F1A1049066)
- National Research Foundation of Korea
- Ministry of Education (2020R1A6A1A03040570)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- safe exploration, random exploration, reward learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
