Deep Reinforcement Learning: A Brief Survey
Open accessThe DOI matches its Crossref record; 177 of the 182 references it cites were found in a registry or an index.
Abstract
Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higher-level understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via RL. To conclude, we describe several current areas of research within the field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Foundational surveys on RL [7,72] provide broad overviews of classical and deep RL, respectively, but do not focus on human feedback or LLMs.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [7]
In the citing paper
“Classical approaches employ reinforcement learning (RL) [6, 7] and imitation learning [8-10] that operate on primitive actions, which could be challenging for systematic exploration [11-15], interpretability [16-18], and generalization [19-21].”
From Voyager: An Open-Ended Embodied Agent with Large Language Models (2023) · cites this paper as [7]
In the citing paper
“This can be frequently modeled as decision making and learning in a Markov decision process (MDP) [160].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [160]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 2×
“The workhorse of DRL, however, remains backpropagation [162, 111].”
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 10×
“[34] train locally linear models for use with the NAF algorithm— the continuous equivalent of the DQN [71]—to improve the algorithm’s sample complexity in the robotic domain where”
InfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“(temporal) credit assignment problem [115].”
InfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 1×
“[22] devised a method for calculating gradients to optimize stochastic policies by “reparameterizing” [30], [60] the stochasticity away from the network, thereby allowing standard gradients to be used (instead of the high-variance REINFORCE estimator [97]).”
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 7×
“dynamic programming over all possible actions (Figure 3, top left), sample trajectories for heuristic search (as was done by AlphaGo [108]), or even perform an exhaustive search (Figure 3, top right).”
InfluentialDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“technique known as distillation [40] has been utilised to both transfer policies learned by large DQNs to smaller DQNs, and transfer policies learned by several DQNs trained on separate games to one single DQN [85, 95].”
Cited by
Items 801–900 of 1,021
Metacognition for Radar Coexistence
Martone, A. F., Sherbondy, K. D., Kovarskiy, J. A., Kirk, B. H., Thornton, C. E., Owen, J. W., Ravenscroft, B., Egbert, A., Goad, A. C., Dockendorf, A., et al. (2020) · IEEE International Radar Conference · doi:10.1109/radar42522.2020.9114775 · Cited by 45
“The goal of RL is to find a policy (cid:2024) , which is a rule that the radar utilizes to select actions given observed states, and can be found directly with a dynamic programming approach or indirectly via function approximation with deep neural networks [21].”
BackgroundInterterminal Truck Routing Optimization Using Deep Reinforcement Learning
Adi, T. N., Iskandar, Y. A., & Bae, H. (2020) · Sensors · doi:10.3390/s20205794 · Cited by 45
Practical Deep Learning Architecture Optimization
Wistuba, M. (2018) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00037 · Cited by 45
“Deep learning methods are very successful in various applications such as machine translation [1], image and speech recognition [2]–[4] or reinforcement learning in general [5].”
AI-Based Network-Aware Service Function Chain Migration in 5G and Beyond Networks
Addad, R. A., Dutra, D. L. C., Taleb, T., & Flinck, H. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3074618 · Cited by 49
“…a current state s t , then executes an action a t , and observes a new state s t +1 along with receiving results in the form of a reward r t +1 , i.e., sometimes referred to as punishment, to automatically adjust the strategy/policy π ( s t , a t ) for carrying out an optimal behavior [34].”
BackgroundResearch developments in adaptive intelligent vibration control of smart civil structures
Saeed, M. U., Sun, Z., & Elias, S. (2021) · Journal of low frequency noise, vibration and active control · doi:10.1177/14613484211032758 · Cited by 47
The NetHack Learning Environment
Küttler, H., Nardelli, N., Miller, A. H., Raileanu, R., Selvatici, M., Grefenstette, E., & Rocktäschel, T. (2020) · arXiv · doi:10.48550/arxiv.2006.13760 · Cited by 44
“ALE [9] provides a unified interface to Atari 2600 games, which enables testing of RL algorithms on high-dimensional visual observations quickly and cheaply, resulting in numerous Deep RL publications over the years [4].”
BackgroundHuangfu, B., Cui, C., Zhang, C., & Xu, L. (2022) · IEEE Journal of Emerging and Selected Topics in Power Electronics · doi:10.1109/jestpe.2022.3189078 · Cited by 46
“Reinforcement learning (RL), as a popular data-driven approach, is widely utilized in autonomous driving [13], robotics [14], motor control [15], and other fields to achieve optimal system control performance.”
BackgroundTang, T., Li, C., & Liu, F. (2023) · Computer Communications · doi:10.1016/j.comcom.2023.06.021 · Cited by 45
A Survey on Collaborative Learning for Intelligent Autonomous Systems
dos Anjos, J. C. S., Matteussi, K. J., Orlandi, F. C., Barbosa, J. L. V., Silva, J. S., Bittencourt, L. F., & Geyer, C. F. R. (2023) · ACM Computing Surveys · doi:10.1145/3625544 · Cited by 53
“The reinforcement rule is used to compute the expectation gradient of a function � for a random variable � related to vector parameters [3].”
BackgroundMethodA Systematic Literature Review on Machine Learning in Shared Mobility
Teusch, J., Gremmel, J. N., Koetsier, C., Johora, F. T., Sester, M., Woisetschläger, D. M., & Müller, J. P. (2023) · IEEE Open Journal of Intelligent Transportation Systems · doi:10.1109/ojits.2023.3334393 · Cited by 51
“However, with the availability of large datasets [219], [220], environments’ complex external and spatial effects, and with the developments of Deep-RL, model-free approaches surged in popularity [221].”
BackgroundPternea, M., Singh, P., Chakraborty, A., Oruganti, Y. D., Milletari, M., Bapat, S., & Jiang, K. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15960 · Cited by 59
Basile, G., Lui, D. G., Petrillo, A., & Santini, S. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108120 · Cited by 45
Robust Knowledge Adaptation for Dynamic Graph Neural Networks
Li, H., Li, C., Feng, K., Yuan, Y., Wang, G., & Zha, H. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3388453 · Cited by 51
“There are mainly two lines of methods in reinforcement learning [72], [73]: policy-based methods and value-based methods.”
MethodOpportunistic Fluid Antenna Multiple Access via Team-Inspired Reinforcement Learning
Waqar, N., Wong, K., Chae, C., Murch, R., Jin, S., & Sharples, A. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3387855 · Cited by 54
“Recent advances in artificial intelligence (AI), particularly the resurgence of RL via deep RL (DRL), have opened the path to achieving asymptotically optimal solutions in dynamic network environments [34], [35].”
BackgroundDeep reinforcement learning for robotic bipedal locomotion: a brief survey
Bao, L., Humphreys, J., Peng, T., & Zhou, C. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11451-z · Cited by 49
“…three main types; Section IV outlines key limitations and challenges, linking them to the preceding discussions; Section V explores potential pathways, opportunities, and two proposed conceptual models that extend the end-to-end and hierarchical paradigms; finally, Section VI concludes the survey.”
Exploration-based learning of a stabilizing controller predicts locomotor adaptation
Seethapathi, N., Clark, B. C., & Srinivasan, M. (2024) · Nature Communications · doi:10.1038/s41467-024-53416-w · Cited by 49
Exploring AI in metasurface structures with forward and inverse design
Yang, G., Xiao, Q., Zhang, Z., Yu, Z., Wang, X., & Lu, Q. (2025) · iScience · doi:10.1016/j.isci.2025.111995 · Cited by 54
Temiz, M., Zhang, Y., Fu, Y., Zhang, C., Meng, C., Kaplan, O., & Masouros, C. (2025) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2025.3586560 · Cited by 51
“DRL can effectively operate with real-time communication data, radar data, or visual data to learn sophisticated tasks, as encountered in communication networks [73], autonomous driving [74], and real-time control [70].”
Machine Learning Tips and Tricks for Power Line Communications
Tonello, A. M., Letizia, N. A., Righini, D., & Marcuzzi, F. (2019) · IEEE Access · doi:10.1109/access.2019.2923321 · Cited by 48
DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning
Jaunet, T., Vuillemot, R., & Wolf, C. (2020) · Computer Graphics Forum · doi:10.1111/cgf.13962 · Cited by 51
QoS and Jamming-Aware Wireless Networking Using Deep Reinforcement Learning
Abuzainab, N., Erpek, T., Davaslıoğlu, K., Sagduyu, Y. E., Shi, Y., Mackey, S. J., Patel, M. P., Panettieri, F., Qureshi, M. A., Isler, V., et al. (2019) · IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom47813.2019.9020985 · Cited by 44
“The main drawback of conventional reinforcement learning algorithms is that the computational overhead involving maintenance of the Q-table grows significantly with the state and action space [7], which limits their practicality.”
BackgroundZhang, T., Xiao, M., Zou, Y., & Xiao, J. (2019) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-019-04614-0 · Cited by 44
A Multi-Agent Reinforcement Learning Approach for Capacity Sharing in Multi-Tenant Scenarios
Vilà, I., Pérez-Romero, J., Sallent, O., & Umbert, A. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3099557 · Cited by 44
“episode, like in Monte Carlo simulations [36].”
DeepHealth: Review and challenges of artificial intelligence in health informaticsPreprint
Kwak, G. H., & Hui, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00384 · Cited by 76
Decision Making in Monopoly Using a Hybrid Deep Reinforcement Learning Approach
Bonjour, T., Haliem, M., Alsalem, A. O., Thomas, S., Li, H., Aggarwal, V., Kejriwal, M., & Bhargava, B. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3166555 · Cited by 47
“DRL [18] makes use of deep neural networks to approximate the optimal policy or the value function to deal with the limitations of traditional RL methods.”
MethodModeling Subsurface Performance of a Geothermal Reservoir Using Machine Learning
Duplyakin, D., Beckers, K. F., Siler, D. L., Martin, M. J., & Johnston, H. (2022) · Energies · doi:10.3390/en15030967 · Cited by 43
Lin, X., An, D., Cui, F., & Zhang, F. (2023) · Frontiers in Energy Research · doi:10.3389/fenrg.2022.1104989 · Cited by 49
“DRL combines the featureextraction capacity of neural networks with the decisionmaking capability of reinforcement learning in unknown environments to achieve direct control from state to action (Arulkumaran et al., 2017).”
BackgroundMulti-fidelity reinforcement learning framework for shape optimization
Bhola, S., Pawar, S. A., Balaprakash, P., & Maulik, R. (2023) · Journal of Computational Physics · doi:10.1016/j.jcp.2023.112018 · Cited by 51
“Similarly, the expected return starting in a state s t , taking an action a t , and thereafter following a policy π is called as the Q-value function and can be written as Optimal decision-making policy can be learned via value-based methods [42] or policy-based methods [43].”
MethodZhang, H., Zhao, H., Liu, R., Gao, X., & Xu, S. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3376256 · Cited by 48
Deep-VFog: When Artificial Intelligence Meets Fog Computing in V2X
Rihan, M., Elwekeil, M., Yang, Y., Huang, L., Xu, C., & Selim, M. M. (2021) · IEEE Systems Journal · doi:10.1109/jsyst.2020.3009998 · Cited by 41
Jagannath, J., Jagannath, A., Furman, S., & Gwin, T. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-77939-9_2 · Cited by 48
Du, Z., Deng, Y., Guo, W., Nallanathan, A., & Wu, Q. (2020) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2020.3015184 · Cited by 49
“RL Mechanism The RL mechanisms in DRL can be broadly classified into three types [1]: the value function-based method, policy-based method, and actor–critic method.”
BackgroundMethodInfluentialSun, L., Wang, Y., Swindlehurst, A. L., & Tang, X. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036954 · Cited by 49
“To address the first issue, we borrow the idea of the policy gradient from reinforcement learning [47] to modify the objective function as follows so that the generator is trainable: where b is a bias, and P θ g ( ˜ q i | s ) represents the probability of producing ˜ q i conditioned on channel input…”
BackgroundCococcioni, M., Rossi, F., Ruffaldi, E., Saponara, S., & de Dinechin, B. D. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2988436 · Cited by 49
Egger, J., Pepe, A., Gsaxner, C., Jin, Y., Li, J., & Kern, R. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.773 · Cited by 52
On Reward Shaping for Mobile Robot Navigation: A Reinforcement Learning and SLAM Based ApproachPreprint
Botteghi, N., Sırmaçek, B., Mustafa, K. A., Poel, M., & Stramigioli, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04109 · Cited by 43
“The interaction process can be modelled as a Markov Decision Process (MDP) [8], M = ( S , A , P , R , γ ) , where S is the state space, A is a set of actions, P ( s t +1 | s t , a t ) is the probability transition distribution, R ( s t , a t ) is the reward function based on the state-action pair…”
Comprehensive Review of Deep Reinforcement Learning Methods and Applications in EconomicsPreprint
Mosavi, A. H., Ghamisi, P., Faghan, Y., Duan, P., Ardabili, S., Salwana, E., & Band, S. S. (2020) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/2d5g7 · Cited by 43
FreeKD: Free-direction Knowledge Distillation for Graph Neural Networks
Feng, K., Li, C., Yuan, Y., & Wang, G. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539320 · Cited by 67
“Reinforcement learning aims at training agents to make optimal decisions by learning from interactions with the environment [2].”
BackgroundMethodInfluentialDeepGPS: Deep Learning Enhanced GPS Positioning in Urban Canyons
Liu, Z., Liu, J., Xu, X., & Wu, K. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3208240 · Cited by 42
“Inspired by deep reinforcement learning theory that employs deep neural networks as function approximators to learn Q functions [13], we instead propose to utilize a powerful deep neural network to approximate mapping function f for better describing complex relationships between GPS estimations…”
MethodTransfer reinforcement learning method with multi-label learning for compound fault recognition
Wang, Z., Zhang, Q., Tang, L., Shi, T., & Xuan, J. (2022) · Advanced Engineering Informatics · doi:10.1016/j.aei.2022.101818 · Cited by 42
Goby, N., Brandt, T., & Neumann, D. (2023) · Computers & Industrial Engineering · doi:10.1016/j.cie.2023.109165 · Cited by 48
Hong, S. M., Abbas, A., Kim, S., Kwon, D. H., Yoon, N., Yun, D., Lee, S., Pachepsky, Y. A., Pyo, J., & Cho, K. H. (2023) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2023.105805 · Cited by 44
Lai, L., Dong, Y., Andriotis, C. P., Wang, A., & Lei, X. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105302 · Cited by 48
Pritee, Z. T., Anik, M. H., Alam, S. B., Jim, J. R., Kabir, M. M., & Mridha, M. F. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103747 · Cited by 52
Hybrid Machine Learning and Reinforcement Learning Framework for Adaptive UAV Obstacle Avoidance
Skarka, W., & Ashfaq, R. (2024) · Aerospace · doi:10.3390/aerospace11110870 · Cited by 48
“A simulated environment is created, incorporating adversary elements using the STAGE Scenario software tool, which features a comprehensive database including battlefield entities like radar and missiles [82].”
Machine Learning Applications in Energy Harvesting Internet of Things Networks: A Review
Alamu, O., Olwal, T. O., & Migabo, E. (2025) · IEEE Access · doi:10.1109/access.2024.3525263 · Cited by 48
“The policy-based DRL chooses a parameterized policy and uses either gradient-based or gradient-free optimization methods to update the parameters [140].”
Stigmergic Independent Reinforcement Learning for Multiagent Collaboration
Xu, X., Li, R., Zhao, Z., & Zhang, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3056418 · Cited by 42
“Guiding autonomous agents to act optimally through trialand-error interaction with the corresponding environment is the primary goal in the field of artificial intelligence and is regarded as one of the most important objectives of reinforcement learning (RL) [9]–[11].”
BackgroundDong, T., Qi, Q., Wang, J., Liu, A. X., Sun, H., Zhuang, Z., & Liao, J. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3077249 · Cited by 47
“With the growth of network size (e.g., the increasing of network devices, links or servers), DRL algorithms usually require a huge sampling cost to achieve convergence [21].”
Multiaccess Point Coordination for Next-Gen Wi-Fi Networks Aided by Deep Reinforcement Learning
Zhang, L., Yin, H., Roy, S., & Cao, L. (2022) · IEEE Systems Journal · doi:10.1109/jsyst.2022.3183199 · Cited by 50
“, the curse of dimensionality [23]; thus, the well-known DQN is pro-”
BackgroundA Review of Intelligent Computation Offloading in Multiaccess Edge Computing
Jin, H., Gregory, M. A., & Li, S. (2022) · IEEE Access · doi:10.1109/access.2022.3187701 · Cited by 44
Crude oil price prediction using deep reinforcement learning
Liang, X., Luo, P., Li, X., Wang, X., & Shu, L. (2023) · Resources Policy · doi:10.1016/j.resourpol.2023.103363 · Cited by 42
Human-in-the-loop reinforcement learning in continuous-action space
Luo, B., Wu, Z., Zhou, F., & Wang, B.-C. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3289315 · Cited by 41
“Digital Object Identifier 10.1109/TNNLS.2023.3289315 [8], [9] also require a large amount of training data to learn the optimal policy, where the main reason is the state and action spaces are usually high-dimensional or continuous [10], [11], [12] in practice.”
BackgroundInfluentialFan, W., Zhao, X., Li, Q., Derr, T., Ma, Y., Liu, H., Wang, J., & Tang, J. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3272652 · Cited by 41
Gharagozlou, H., Mohammadzadeh, J., Bastanfard, A., & Ghidary, S. S. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/7839840 · Cited by 46
“[98] is a subfield of machine learning that solves a problem by making successive decisions [99, 100].”
BackgroundWang, Q., Hong, Z., & Zhong, Y. (2022) · Biomimetic Intelligence and Robotics · doi:10.1016/j.birob.2022.100066 · Cited by 41
Yuan, Y., Liu, M., Zhuo, X., Wei, Y. D., Tu, X., & Qu, F. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3232614 · Cited by 43
“It can be seen as a Markov decision process ( S , A , P , R ) , where S , A , P , and R are the set of states, actions, state transition probabilities, and rewards, respectively [33], [34].”
BackgroundWan, J., Duan, Y., Gui, X., Liu, C., Li, L., & Ma, Z. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3234545 · Cited by 45
“In addition, recent breakthroughs in Deep RL (DRL) [11] successfully addressed “the curse of dimensionaility” and enabled previously intractable problems.”
BackgroundModeling, Replicating, and Predicting Human Behavior: A Survey
Fuchs, A., Passarella, A., & Conti, M. (2023) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3580492 · Cited by 50
“This is referred to as Deep RL and is commonly used in the case of continuous and/or large domains such as robotic control or video games [10].”
MethodDissem, M., Amayri, M., & Bouguila, N. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3360882 · Cited by 41
“The agent aims to maximize its cumulative reward over time by discovering optimal policies through trial and error [21].”
BackgroundDeep Reinforcement Learning in Human Activity Recognition: A Survey and Outlook
Nikpour, B., Sinodinos, D., & Armanfard, N. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3360990 · Cited by 50
“DRL is a group of RL techniques that use deep neural networks, which are powerful function approximators, to deal with high-dimensional action/state spaces [24], [26], [34].”
Zhao, Y., Zhang, Y., Guo, L., Ding, S., & Wang, X. (2025) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2025.112645 · Cited by 41
Alginahi, Y. M., Sabri, O., & Said, W. (2025) · Machines · doi:10.3390/machines13121140 · Cited by 42
“Approaches such as Q-learning, DQN, PPO, and SAC have been widely explored, alongside specialized frameworks such as MARL, Offline RL, and Digital Twin–Integrated RL [1,2,8].”
InfluentialBayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Search
Pautrat, R., Chatzilygeroudis, K. I., & Mouret, J.-B. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463197 · Cited by 50
Continual Reinforcement Learning in 3D Non-stationary Environments
Lomonaco, V., Desai, K., Culurciello, E., & Maltoni, D. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00132 · Cited by 56
“In the last decade we have witnessed a renewed interest and major progresses in reinforcement learning (RL) especially due to recent deep learning developments [3].”
BackgroundMulti Type Mean Field Reinforcement Learning
Subramanian, S. G., Poupart, P., Taylor, M. E., & Hegde, N. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/lvmd6651 · Cited by 41
A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning
Liu, L., Liu, Q., Song, Y. H., Pang, B. J., Yuan, X., & Xu, Q. (2021) · Applied Sciences · doi:10.3390/app11041816 · Cited by 50
Toward Using Reinforcement Learning for Trigger Selection in Network Slice Mobility
Addad, R. A., Dutra, D. L. C., Taleb, T., & Flinck, H. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3078501 · Cited by 43
“This process keeps repeating while adjusting the policy (cid:25) ( s t ; a t ) until the convergence phase, i.e., optimal policy [48].”
BackgroundGuo, B., Wang, S., Ding, Y. C., Wang, G., He, S., Zhang, D., & He, T. (2021) · IEEE Real-Time Systems Symposium (RTSS) · doi:10.1109/rtss52674.2021.00026 · Cited by 41
“We have witnessed unprecedented advances in applying reinforcement learning algorithms to complicated order dispatch in real-world systems [25], [26], [32].”
An Adaptive Clipping Approach for Proximal Policy OptimizationPreprint
Chen, G., Peng, Y., & Zhang, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06461 · Cited by 51
“…numerous innovative RL algorithms have been proposed in recent years for training deep neural networks (DNN) that can solve difficult decision making problems modelled as Markov Decision Processes (MDPs) [Arulkumaran et al. , 2017; Wang et al. , 2015; Duan et al. , 2016; Mnih et al. , 2015 ] .”
BackgroundDeep Reinforcement Learning-Based Robust Protection in DER-Rich Distribution Grids
Wu, D., Kalathil, D. M., Begovic, M. M., Ding, K. Q., & Xie, L. (2022) · IEEE Open Access Journal of Power and Energy · doi:10.1109/oajpe.2022.3161904 · Cited by 40
“RL algorithms using deep neural networks [10], known as Deep RL algorithms, have made significant achievements in the past few years in areas like robotics, games, and autonomous driving [11].”
MethodWhen Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward
Song, J., Lyu, D., Zhang, Z., Wang, Z., Zhang, T., & Ma, L. (2022) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3510457.3513049 · Cited by 46
“Among different methods, deep reinforcement learning (DRL) is considered as the stateof-the-art, and has succeeded in various application domains [3].”
MethodHolistic resource management in UAV-assisted wireless networks: An optimization perspective
Taimoor, S., Ferdouse, L., & Ejaz, W. (2022) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2022.103439 · Cited by 43
Zhang, Y., Zhu, J., Wang, H., Shen, X., Wang, B., & Dong, Y. (2022) · Remote Sensing · doi:10.3390/rs14163947 · Cited by 43
“In order to further enhance the performance of intelligent agents, deep reinforcement learning (DRL) combines deep neural networks (DNNs) and RL to learn successful policies directly from high-dimensionality inputs [32,33].”
BackgroundKlingebiel, J., Salamon, M., Bogdanov, P., Venzik, V., Vering, C., & Müller, D. (2022) · Energy and Buildings · doi:10.1016/j.enbuild.2023.113397 · Cited by 40
Cost-aware cloud workflow scheduling using DRL and simulated annealing
Gu, Y., Cheng, F., Yang, L., Xu, J., Chen, X., & Cheng, L. (2024) · Digital Communications and Networks · doi:10.1016/j.dcan.2023.12.009 · Cited by 40
Klar, M., Schworm, P., Wu, X., Simon, P., Glatt, M. F., Ravani, B., & Aurich, J. C. (2024) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2024.04.007 · Cited by 40
DRL-GAN: A Hybrid Approach for Binary and Multiclass Network Intrusion Detection
Strickland, C., Zakar, M., Saha, C., Nejad, S. S., Tasnim, N., Lizotte, D. J., & Haque, A. (2024) · Sensors · doi:10.3390/s24092746 · Cited by 45
“DRL combines function approximation and target optimization, mapping states and actions to the rewards they lead to [3].”
Paraschos, P. D., & Koulouriotis, D. E. (2024) · Computers & Industrial Engineering · doi:10.1016/j.cie.2024.110656 · Cited by 42
Soft robots built for extreme environments
Kulkarni, M., Edward, S., Golecki, T., Kaehr, B., & Golecki, H. M. (2025) · Soft Science · doi:10.20517/ss.2023.51 · Cited by 47
A robotic shared control teleoperation method based on learning from demonstrations
Xi, B., Wang, S., Ye, X., Cai, Y., Lu, T., & Wang, R. (2019) · International Journal of Advanced Robotic Systems · doi:10.1177/1729881419857428 · Cited by 40
A Deep Reinforcement Learning Approach to the Optimization of Data Center Task Scheduling
Che, H., Bai, Z., Zuo, R., & Li, H. (2020) · Complexity · doi:10.1155/2020/3046769 · Cited by 40
Security Hardening of Botnet Detectors Using Generative Adversarial Networks
Randhawa, R. H., Aslam, N., Alauthman, M., Rafiq, H., & Comeau, F. (2021) · IEEE Access · doi:10.1109/access.2021.3083421 · Cited by 47
“To address the problem of online learning, researchers used DRL [17], [18].”
MethodInfluentialDeep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networks
Nasir, Y. S., & Guo, D. (2020) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/ieeeconf51394.2020.9443301 · Cited by 46
“Two main approaches to train agents with model-free reinforcement learning are value function and policy search based methods [9].”
BackgroundFedDSR: Daily Schedule Recommendation in a Federated Deep Reinforcement Learning Framework
Huang, W. X., Liu, J., Li, T., Huang, T., Ji, S., & Wan, J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3130265 · Cited by 46
“It combines the perceptual capabilities of deep learning with the decision-making capabilities of reinforcement learning in a generalized form and enables direct control from raw input to output by means of end-to-end learning [25].”
BackgroundTactile Based Intelligence Touch Technology in IoT Configured WCN in B5G/6G-A Survey
Gupta, M., Jha, R. K., & Jain, S. (2022) · IEEE Access · doi:10.1109/access.2022.3148473 · Cited by 42
A Unifying Framework for Reinforcement Learning and Planning
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.908353 · Cited by 63
“Surveys of deep reinforcement learning are provided by Fran¸cois-Lavet et al. (2018) and Arulkumaran et al. (2017).”
BackgroundRecognition of grape leaf diseases using MobileNetV3 and deep transfer learning
Yin, X., Li, W., Li, Z., & Yi, L. (2022) · International journal of agricultural and biological engineering · doi:10.25165/j.ijabe.20221503.7062 · Cited by 48
gym-gazebo2, a toolkit for reinforcement learning using ROS 2 and GazeboPreprint
Lopez, N. G., Nuin, Y., Moral, E. B., Juan, L. U. S., Rueda, A. S., Vilches, V. M., & Kojcev, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.06278 · Cited by 51
“Acquiring large amounts of sampling data can be computationally expensive and timeconsuming [14].”
BackgroundReinforcement Learning for Intelligent Healthcare Systems: A Comprehensive SurveyPreprint
Abdellatif, A. A., Mhaisen, N., Chkirbene, Z., Mohamed, A. M. S., Erbad, A., & Guizani, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04087 · Cited by 44
“Conventional RL schemes utilized a lookup table to store the obtained rewards [8], however, this was sufficient for straightforward learning models with a small state space [9].”
Primer on artificial intelligence
Shi, Y., Yang, K., Yang, Z., & Zhou, Y. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-823817-2.00011-5 · Cited by 40
Data-driven selection of constitutive models via rheology-informed neural networks (RhINNs)
Saadat, M., Mahmoudabadbozchelou, M., & Jamali, S. (2022) · Rheologica Acta · doi:10.1007/s00397-022-01357-w · Cited by 39
“…the complex relations between the inputs and outputs by forming a data-driven framework have been widely used in semi-supervised (e.g., deep reinforcement learning (Arulkumaran et al. 2017)) and unsupervised (e.g., dimensionality reduction through self-organizing maps (Kohonen 2001)) tasks.”
BackgroundMethodA Comprehensive Review on Cyber-Attack Detection and Control of Microgrid Systems
Shafei, H., Li, L., & Aguilera, R. P. (2023) · Power systems · doi:10.1007/978-3-031-20360-2_1 · Cited by 44
Timely Data Collection for UAV-Based IoT Networks: A Deep Reinforcement Learning Approach
Hu, Y., Liu, Y., Kaushik, A., Masouros, C., & Thompson, J. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3265935 · Cited by 41
“Using the DRL algorithm [24], an agent can obtain this ability by continuously interactive learning from the environment.”
MethodWei, P., Zeng, Y., Yan, B., Zhou, J., & Nikougoftar, E. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.04.002 · Cited by 39
“As with most DRL approaches, A3C also has a Markov property (Arulkumaran et al. 2017).”
InfluentialHu, D. T., Xie, H., Song, K., Zhang, Y., & Yan, L. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121227 · Cited by 41
A Survey of Using Machine Learning in IoT Security and the Challenges Faced by Researchers
Harahsheh, K., & Chen, C. (2023) · Informatica · doi:10.31449/inf.v47i6.4635 · Cited by 49
Recent Advances in Deep Learning for Channel Coding: A Survey
Matsumine, T., & Ochiai, H. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3472094 · Cited by 46
“More details on DRL can be found, for example, in [114]–[117].”
General-purpose foundation models for increased autonomy in robot-assisted surgery
Schmidgall, S., Kim, J. W., Kuntz, A., Ghazi, A., & Krieger, A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00917-4 · Cited by 60
Liu, C., Chen, Y., & Xu, X. (2024) · Composite Structures · doi:10.1016/j.compstruct.2024.118727 · Cited by 43
Insights into user behavioral-based insider threat detection: systematic review
Kamatchi, K., & Uma, E. (2025) · International Journal of Information Security · doi:10.1007/s10207-025-01002-6 · Cited by 47
Metadata
- Authors
- Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A.
- Affiliations
- University of Oxford (GB)
- Institution of Engineering and Technology (GB)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-11
- Volume
- 34
- Issue
- 6
- Pages
- 26-38
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W3100789280
- Cited by
- 4,733
- References
- 182
- Citations by year
- 2026: 444 · 2025: 855 · 2024: 806 · 2023: 749 · 2022: 590 · 2021: 511 · 2020: 388 · 2019: 260 · 2018: 122 · 2017: 6
- FWCI
- 126.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, value-based methods, deep Q-network, trust region policy optimization, asynchronous advantage actor-critic, visual understanding, autonomous systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.