Deep Reinforcement Learning: A Brief Survey
Open accessThe DOI matches its Crossref record; 177 of the 182 references it cites were found in a registry or an index.
Abstract
Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higher-level understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via RL. To conclude, we describe several current areas of research within the field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Foundational surveys on RL [7,72] provide broad overviews of classical and deep RL, respectively, but do not focus on human feedback or LLMs.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [7]
In the citing paper
“Classical approaches employ reinforcement learning (RL) [6, 7] and imitation learning [8-10] that operate on primitive actions, which could be challenging for systematic exploration [11-15], interpretability [16-18], and generalization [19-21].”
From Voyager: An Open-Ended Embodied Agent with Large Language Models (2023) · cites this paper as [7]
In the citing paper
“This can be frequently modeled as decision making and learning in a Markov decision process (MDP) [160].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [160]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 2×
“The workhorse of DRL, however, remains backpropagation [162, 111].”
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 10×
“[34] train locally linear models for use with the NAF algorithm— the continuous equivalent of the DQN [71]—to improve the algorithm’s sample complexity in the robotic domain where”
InfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“(temporal) credit assignment problem [115].”
InfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 1×
“[22] devised a method for calculating gradients to optimize stochastic policies by “reparameterizing” [30], [60] the stochasticity away from the network, thereby allowing standard gradients to be used (instead of the high-variance REINFORCE estimator [97]).”
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 7×
“dynamic programming over all possible actions (Figure 3, top left), sample trajectories for heuristic search (as was done by AlphaGo [108]), or even perform an exhaustive search (Figure 3, top right).”
InfluentialDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“technique known as distillation [40] has been utilised to both transfer policies learned by large DQNs to smaller DQNs, and transfer policies learned by several DQNs trained on separate games to one single DQN [85, 95].”
Cited by
Items 201–300 of 1,021
Learning Combinatorial Optimization on Graphs: A Survey With Applications to Networking
Vesselinova, N., Steinert, R., Perez-Ramirez, D. F., & Boman, M. (2020) · IEEE Access · doi:10.1109/access.2020.3004964 · Cited by 164
“[65].”
Machine Learning for Security and the Internet of Things: The Good, the Bad, and the Ugly
Liang, F., Hatcher, W. G., Liao, W., Gao, W., & Yu, W. (2019) · IEEE Access · doi:10.1109/access.2019.2948912 · Cited by 164
“The key component of the value function is the state-action value function, known as the quality function [37].”
BackgroundMobile-robotic machining for large complex components: A review study
Tao, B., Zhao, X., & Ding, H. (2019) · Science China Technological Sciences · doi:10.1007/s11431-019-9510-1 · Cited by 164
Glavic, M. (2019) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2019.09.008 · Cited by 163
Singh, M., Kumar, A., Khanna, N. N., Laird, J. R., Nicolaides, A., Faa, G., Johri, A. M., Mantella, L. E., Fernandes, J. F. E., Teji, J. S., et al. (2024) · eClinicalMedicine · doi:10.1016/j.eclinm.2024.102660 · Cited by 163
A Survey of Multi-Task Deep Reinforcement Learning
Vithayathil Varghese, N., & Mahmoud, Q. H. (2020) · Electronics · doi:10.3390/electronics9091363 · Cited by 163
“A second survey analyzed how deep reinforcement learning could revolutionize the field of artificial intelligence and pave the steps towards the development of intelligent agents that can understand the visual world in a better way [12].”
Active Mapping and Robot Exploration: A Survey
Lluvia, I., Lazkano, E., & Ansuategi, A. (2021) · Sensors · doi:10.3390/s21072445 · Cited by 201
Task offloading strategies for mobile edge computing: A survey
Dong, S., Tang, J., Abbas, K., Hou, R., Kamruzzaman, J., Rutkowski, L., & Buyya, R. (2024) · Computer Networks · doi:10.1016/j.comnet.2024.110791 · Cited by 162
“The Bell-man equation is frequently employed to resolve recursive MDP problems [53].”
MethodTowards artificial intelligence enabled 6G: State of the art, challenges, and opportunities
Zhang, S., & Zhu, D. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107556 · Cited by 162
Autonomous Highway Driving using Deep Reinforcement Learning
Nageshrao, S. P., Tseng, E., & Filev, D. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8914621 · Cited by 184
“This can be mainly attributed to augmenting RL with deep learning techniques [15].”
BackgroundFoundation models in smart agriculture: Basics, opportunities, and challenges
Li, J., Xu, M., Xiang, L., Chen, D., Zhuang, W., Yin, X., & Li, Z. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109032 · Cited by 194
Enabling AI in Future Wireless Networks: A Data Life Cycle Perspective
Nguyen, D. C., Cheng, P., Ding, M., Lopez-Perez, D., Pathirana, P. N., Li, J., Seneviratne, A., Li, Y., & Poor, H. V. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3024783 · Cited by 160
Machine learning for intelligent optical networks: A comprehensive survey
Gu, R., Yang, Z., & Ji, Y. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102576 · Cited by 160
Artificial intelligence enabled software‐defined networking: a comprehensive overview
Latah, M., & Toker, L. (2019) · IET Networks · doi:10.1049/iet-net.2018.5082 · Cited by 160
Cross-Type Transfer for Deep Reinforcement Learning Based Hybrid Electric Vehicle Energy Management
Lian, R., Tan, H., Peng, J., Li, Q., & Wu, Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2999263 · Cited by 160
“nious combination of policy search methods and value function has enabled actor-critic algorithm to learn faster [24].”
BackgroundData-driven dynamic resource scheduling for network slicing: A Deep reinforcement learning approach
Wang, H., Wu, Y., Min, G., Xu, J., & Tang, P. (2019) · Information Sciences · doi:10.1016/j.ins.2019.05.012 · Cited by 160
“Combining the use of deep learning with RL generates the concept of Deep Reinforcement Learning (DRL) [4], which enables RL to scale to the decision-making problems that were previously intractable, i.e., the settings with high-dimensional state and action spaces.”
MethodAlmehdhar, M., Albaseer, A. M., Khan, M. A., Abdallah, M. M., Menouar, H., Al-Kuwari, S. M., & Al‐Fuqaha, A. (2024) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2024.3422253 · Cited by 183
Zhang, H., Liu, R., Kaushik, A., & Gao, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3233383 · Cited by 159
“DRL can not only provide solutions to large-scale complex problems by exploiting the function approximation properties of deep neural network (DNN) but also receive reward feedback from the environment to dynamically adjust its policy in a model-free framework [27].”
BackgroundKnowledge-integrated machine learning for materials: lessons from gameplaying and robotics
Hippalgaonkar, K., Li, Q., Wang, X., Fisher, J. S., Kirkpatrick, J., & Buonassisi, T. (2023) · Nature Reviews Materials · doi:10.1038/s41578-022-00513-1 · Cited by 174
Thornton, C. E., Kozy, M. A., Buehrer, R. M., Martone, A. F., & Sherbondy, K. D. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.3019605 · Cited by 156
“Nonlinear function approximation using deep learning models can enable RL to scale to decision-making problems that were previously intractable [17].”
BackgroundReinforcement Learning Approaches in Social Robotics
Akalın, N., & Loutfi, A. (2021) · Sensors · doi:10.3390/s21041292 · Cited by 193
“In these methods, the policy is parameterized πθ (s,a) = P(a |s;θ ) and the focus is the policy optimization to find a θ that maximizes the expected return Eπ [R |θ ] by using gradient-based or gradient-free optimisation [3].”
Arwa, E. O., & Folly, K. A. (2020) · IEEE Access · doi:10.1109/access.2020.3038735 · Cited by 146
“There are two learning methods that produce intrinsic motivation in the agent, namely, curiosity-driven learning and hindsight experience replay [92].”
BackgroundInfluentialApplications and Techniques of Machine Learning in Cancer Classification: A Systematic Review
Yaqoob, A., Aziz, R. M., & Verma, N. K. (2023) · Human-Centric Intelligent Systems · doi:10.1007/s44230-023-00041-3 · Cited by 164
“Value-based methods include Q-Learning and SARSA [100].”
Deep Reinforcement Learning for Mobile Edge Caching: Review, New Features, and Open Issues
Zhu, H., Cao, Y., Wang, W., Jiang, T., & Jin, S. (2018) · IEEE Network · doi:10.1109/mnet.2018.1800109 · Cited by 153
“By combining RL deep learning, deep reinforcement learning DRL) enables agents to optimize their control in environment by automatically learning knowledge directly from raw, high-dimensional observations [3].”
BackgroundDQNViz: A Visual Analytics Approach to Understand Deep Q-Networks
Wang, J., Gou, L., Shen, H., & Yang, H. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864504 · Cited by 167
“Demonstrating the action/movement/reward patterns of the agent is a strong need from the experts [4, 5], given that few existing tools are readily applicable for this purpose.”
Liu, Z., Yao, C., Yu, H., & Wu, T. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.02.068 · Cited by 149
The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning
Yu, J., Su, Y., & Liao, Y. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.00063 · Cited by 152
“The DRL method has obvious advantages in path planning and requires less prior information about the environment (Wulfmeier et al., 2017; Zheng and Liu, 2020).”
BackgroundA survey of inverse reinforcement learning
Adams, S., Cody, T., & Beling, P. A. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10108-x · Cited by 221
“For (2) (3) more information on RL, see (Arulkumaran et al. 2017; Garcıa and Fernández 2015; Ghavamzadeh et al. 2015).”
BackgroundThe Advent of Generative Chemistry
Vanhaelen, Q., Lin, Y.-C., & Zhavoronkov, A. (2020) · ACS Medicinal Chemistry Letters · doi:10.1021/acsmedchemlett.0c00088 · Cited by 151
Chien, C., Lin, Y.-S., & Lin, S.-K. (2020) · International Journal of Production Research · doi:10.1080/00207543.2020.1733125 · Cited by 157
“Since RL learns optimal control policies through completely raw inputs, the memory complexity and computational complexity become limitations for RL (Arulkumaran et al. 2017).”
BackgroundDong, L., He, Z., Song, C., & Sun, C. (2023) · Journal of Systems Engineering and Electronics · doi:10.23919/jsee.2023.000051 · Cited by 168
Caching Transient Data for Internet of Things: A Deep Reinforcement Learning Approach
Zhu, H., Cao, Y., Wei, X., Wang, W., Jiang, T., & Jin, S. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2882583 · Cited by 159
“Neural networks (NNs) have recently been applied successfully to solve large-scale RL problems [23], [24], by utilizing the advantages of deep NNs on automatically learning low-dimensional feature representations.”
BackgroundModelling building HVAC control strategies using a deep reinforcement learning approach
Nguyen, A. T., Pham, D. H., Oo, B. L., Santamouris, M., Ahn, Y., & Lim, B. T. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.114065 · Cited by 158
Zeng, N., Li, H., Wang, Z., Liu, W., Liu, S., Alsaadi, F. E., & Liu, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.04.001 · Cited by 147
“Nevertheless, traditional RL algorithms have the problem of lack of scalability [2].”
BackgroundMethodAlkinani, M. H., Khan, W. Z., & Arshad, Q. (2020) · IEEE Access · doi:10.1109/access.2020.2999829 · Cited by 149
“Since, there are major gaps between simulated and real environments that make it difficult to train models, DRL works very well in closed environments like video games, but it is difficult to apply to real-world environments [74]–[76].”
BackgroundA Brief Survey of Machine Learning and Deep Learning Techniques for E-Commerce Research
Zhang, X., Guo, F., Chen, T., Pan, L., Beliakov, G., & Wu, J. (2023) · Journal of Theoretical and Applied Electronic Commerce Research · doi:10.3390/jtaer18040110 · Cited by 121
He, Y., Fang, J., Yu, F. R., & Leung, V. C. M. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3415661 · Cited by 189
FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented MicroservicesPreprint
Qiu, H., Banerjee, S. S., Jha, S., Kalbarczyk, Z., & Iyer, R. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.08509 · Cited by 446
“Two main categories of approaches are proposed for policy learning: value-based methods and policy based methods [5].”
A Review on Computational Intelligence Techniques in Cloud and Edge Computing
Asim, M., Wang, Y., Wang, K., & Huang, P.-Q. (2020) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2020.3007905 · Cited by 158
“learning algorithm by combining DL with RL [132]–[134].”
MethodFederated Multiagent Actor–Critic Learning for Age Sensitive Mobile-Edge Computing
Zhu, Z., Wan, S., Fan, P., & Letaief, K. B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3078514 · Cited by 156
“The employ of target nets improves the stability and convergence of replay training [56].”
Leveraging Deep Reinforcement Learning for Traffic Engineering: A Survey
Xiao, Y., Liu, J., Wu, J., & Ansari, N. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3102580 · Cited by 162
Wu, J., Sun, Y., Li, D., Shi, J., Li, X., Gao, L., Yu, L., Han, G., & Wu, J. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3297837 · Cited by 147
Wu, Y., Tan, H., Qin, L., & Ran, B. (2020) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2020.102649 · Cited by 165
Cyber-security and reinforcement learning — A brief survey
Adawadkar, A. M. K., & Kulkarni, N. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105116 · Cited by 172
Tran-Dang, H., Bhardwaj, S., Rahim, T., Musaddiq, A., & Kim, D.-S. (2022) · Journal of Communications and Networks · doi:10.23919/jcn.2021.000041 · Cited by 126
Jamalipour, A., & Murali, S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3126811 · Cited by 159
The AI Revolution: Opportunities and Challenges for the Finance SectorPreprint
Maple, C. R., Szpruch, Ł., Epiphaniou, G., Staykova, K., Singh, S., Penwarden, W., Wen, Y., Wang, Z., Hariharan, J., & Avramovic, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.16538 · Cited by 180
“The RL problem can be formalised as a Markov Decision Process (MDP), consisting of a set of states, actions, transition probabilities, and rewards (Arulkumaran et al., 2017).”
BackgroundApplications of Deep Learning in Intelligent Transportation Systems
Haghighat, A. K., Mouli, V. R., Chakraborty, P., Esfandiari, Y., Arabi, S., & Sharma, A. (2020) · Journal of Big Data Analytics in Transportation · doi:10.1007/s42421-020-00020-1 · Cited by 155
ReLMoGen: Integrating Motion Generation in Reinforcement Learning for Mobile Manipulation
Xia, F., Li, C., Martin-Martin, R., Litany, O., Toshev, A., & Savarese, S. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561315 · Cited by 89
“Moreover, MGs excel at answering “how” to move to a point, but not “where” a a 0 , a 0 ( a ,... .. � R 0 a 0 ,a ) ( s 0 ( s 0 )) Base or Arm Subgoal Base or Arm Subgoal to move to achieve the task based on current observations, where deep RL [3, 4] has shown strong results.”
BackgroundDigital twin-driven deep reinforcement learning for adaptive task allocation in robotic construction
Lee, D., Lee, S. H., Masoud, N., Krishnan, M. S., & Li, V. C. (2022) · Advanced Engineering Informatics · doi:10.1016/j.aei.2022.101710 · Cited by 143
A Metaverse-Based Teaching Building Evacuation Training System With Deep Reinforcement Learning
Gu, J., Wang, J., Guo, X., Liu, G., Qin, S., & Bi, Z. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2022.3231299 · Cited by 143
“In recent studies, researchers combined the deep neural network (DNN) with RL to come up with DRL for dealing with high-dimensional data [45].”
BackgroundEr, M. J., Gong, H., Liu, Y., & Liu, T. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3312268 · Cited by 143
“Due to the powerful function approximation ability of deep NNs, researchers have combined deep learning with reinforcement learning to propose deep reinforcement learning methods [84].”
BackgroundLevels of explainable artificial intelligence for human-aligned conversational explanations
Dazeley, R., Vamplew, P., Foale, C., Young, C., Aryal, S., & Cruz, F. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103525 · Cited by 122
Deep reinforcement learning in transportation research: A review
Farazi, N. P., Zou, B., Ahamed, T., & Barua, L. (2021) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2021.100425 · Cited by 202
Mattera, G., Nele, L., & Paolella, D. (2023) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02085-5 · Cited by 149
Data-Driven Dynamical Control for Bottom-up Energy Internet System
Hua, H., Qin, Z., Dong, N., Qin, Y., Ye, M., Wang, Z., Chen, X., & Cao, J. (2021) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2021.3110294 · Cited by 140
“Compared with the traditional multi-stage optimization methods, DRL has unparalleled advantages in end-to-end training, utilizing global information and continuous control [32].”
MethodAlwarafy, A., Abdallah, M. M., Çiftler, B. S., Al‐Fuqaha, A., & Hamdi, M. (2022) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2022.3153226 · Cited by 151
“It is mainly utilized to enhance the learning rate of RL algorithms and address wireless communication and networking problems having high dimensionality [16], [17], [35], [36].”
BackgroundA Brief Review of Machine Learning Algorithms in Forest Fires Science
Alkhatib, R., Sahwan, W., Alkhatieb, A., & Schütt, B. (2023) · Applied Sciences · doi:10.3390/app13148275 · Cited by 148
Machine Learning in Beyond 5G/6G Networks—State-of-the-Art and Future Trends
Rekkas, V. P., Sotiroudis, S. P., Sarigiannidis, P. G., Wan, S., Karagiannidis, G. K., & Goudos, S. K. (2021) · Electronics · doi:10.3390/electronics10222786 · Cited by 155
“Deep learning enables RL to operate in high-dimensional state and action spaces and can now be used for complex decision-making problems [31,32].”
BackgroundReinforcement Learning Methods for Computation Offloading: A Systematic Review
Zabihi, Z., Moghadam, A. M. E., & Rezvani, M. H. (2023) · ACM Computing Surveys · doi:10.1145/3603703 · Cited by 157
Feature-based aggregation and deep reinforcement learning: a survey and some new implementations
Bertsekas, D. P. (2018) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2018.7511249 · Cited by 159
Path Planning of Coastal Ships Based on Optimized DQN Reward Function
Guo, S., Zhang, X., Du, Y., Zheng, Y., & Cao, Z. (2021) · Journal of Marine Science and Engineering · doi:10.3390/jmse9020210 · Cited by 137
A Survey of Machine Learning Applications to Handover Management in 5G and Beyond
Mollel, M. S., Abubakar, A. I., Öztürk, M., Kaijage, S. F., Kisangiri, M. F., Hussain, S., Imran, M. A., & Abbasi, Q. H. (2021) · IEEE Access · doi:10.1109/access.2021.3067503 · Cited by 163
“Q -learning, SARSA), policy-based (e.g. policy gradient, proximal policy optimization (PPO) and actor-critic (A2C)) [121], [122].”
Artificial intelligence explainability: the technical and ethical dimensions
McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0363 · Cited by 136
Deep Reinforcement Learning for Adaptive Caching in Hierarchical Content Delivery Networks
Sadeghi, A., Wang, G., & Giannakis, G. B. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2936193 · Cited by 143
“[36]. Deep learning approaches on the other hand, have recently demonstrated remarkable potential in applications such as object detection, speech recognition, and language translation, to name a few [41]. This is because DNNs are capable of extracting compact low-dimensional features from highdimensional data. Weddin…”
BackgroundLiu, X.-B., Fan, K., Huang, X., Ge, J., Liu, Y., & Kang, H. (2024) · Chemical Engineering Journal · doi:10.1016/j.cej.2024.151625 · Cited by 155
Exposing GAN-synthesized Faces Using Landmark Locations
Yang, X., Li, Y., Qi, H., & Lyu, S. (2019) · Proceedings of the ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3335203.3335724 · Cited by 124
“The recent developments in deep neural networks [1, 3, 4, 21], and in particular, the generative adversary networks (GANs) [5], have spawned a new type of image synthesis methods that can produce images with high levels of realism.”
BackgroundMethodFrom inverse optimal control to inverse reinforcement learning: A historical review
Azar, N. A., Shahmansoorian, A., & Davoudi, M. (2020) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2020.06.001 · Cited by 166
Blockchain-based recommender systems: Applications, challenges and future opportunities
Himeur, Y., Sayed, A., Alsalemi, A., Bensaali, F., Amira, A., Varlamis, I., Eirinaki, M., Sardianos, C., & Dimitrakopoulos, G. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100439 · Cited by 155
“In this context, a RS can be represented as a Markov decision process (MDP), where reinforcement learning (RL) algorithms can be deployed for solving it Arulkumaran et al. (2017); Zhao et al. (2018). Typically, the combination of RL and deep learning has led to the paradigm of deep reinforcement learning (DRL), which…”
BackgroundMethodInfluentialDRSIR: A Deep Reinforcement Learning Approach for Routing in Software-Defined Networking
Casas-Velasco, D. M., Rendón, O. M. C., & da Fonseca, N. L. S. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3132491 · Cited by 150
“Nonetheless, RL-based solutions usually lead to a significant increase in the learning process when handling large action and state spaces, requiring the RL agent to make several interactions with the environment to converge towards reliable estimations [19].”
BackgroundZhang, G., Chang, F., Jin, J., Yang, F., & Huang, H. (2024) · Accident Analysis & Prevention · doi:10.1016/j.aap.2023.107451 · Cited by 148
A reinforcement learning approach to personalized learning recommendation systems
Tang, X., Chen, Y., Li, X., Liu, J., & Ying, Z. (2018) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12144 · Cited by 151
Dynamic Service Function Chain Embedding for NFV-Enabled IoT: A Deep Reinforcement Learning Approach
Fu, X., Yu, F. R., Wang, J., Qi, Q., & Liao, J. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2946797 · Cited by 146
“In this paper, with recent advances in deep reinforcement learning (DRL) [13], we present a DRL-based SFC embedding scheme in NFV-enabled IoT.”
MethodZamfirache, I. A., Precup, R., Roman, R., & Petriu, E. M. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120112 · Cited by 143
An Overview on IEEE 802.11bf: WLAN Sensing
Du, R. Y., Hua, H., Xie, H., Song, X., Lyu, Z., Hu, M., Narengerile, Xin, Y., McCann, S., Montemurro, M., et al. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3408899 · Cited by 171
“…possible that a task-oriented feedback overhead control system could be built based on several AI models such as deep reinforcement learning (DRL) [111], [112], deep neural network (DNN) [113], and long short-term memory (LSTM) [114], so that the sensing performance can be ensured with minimum…”
Zeb, S., Mahmood, A., Hassan, S. A., Piran, M. J., Gidlund, M., & Guizani, M. (2022) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103309 · Cited by 132
Qian, B., Su, J., Wen, Z., Jha, D. N., Li, Y., Guan, Y., Puthal, D., James, P., Yang, R., Zomaya, A. Y., et al. (2021) · ACM Computing Surveys · doi:10.1145/3398020 · Cited by 132
A Hybrid DQN and Optimization Approach for Strategy and Resource Allocation in MEC Networks
Wu, Y.-C., Dinh, T. Q., Fu, Y., Lin, C., & Quek, T. Q. S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3057882 · Cited by 137
“The corresponding performances have been validated in various communication problems [23], [24].”
BackgroundSee, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion
Fazeli, N., Oller, M., Wu, J., Wu, Z., Tenenbaum, J. B., & Rodríguez, A. (2019) · Science Robotics · doi:10.1126/scirobotics.aav3123 · Cited by 161
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010009 · Cited by 129
Specialized Deep Neural Networks for Battery Health Prognostics: Opportunities and Challenges
Zhao, J., Han, X., Ouyang, M., & Burke, A. F. (2023) · Journal of Energy Chemistry · doi:10.1016/j.jechem.2023.08.047 · Cited by 129
Machine Learning in IoT Security: Current Solutions and Future Challenges
Hussain, F., Hussain, R., Hassan, S. A., & Hossain, E. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986444 · Cited by 129
“Most recent work in DRL can be found in [72], [73].”
BackgroundLi, B., Wu, G., He, Y., Fan, M., & Pedrycz, W. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105677 · Cited by 148
“Constraint (8) is used to ensure decision variable x ki,j is a binary.”
A Survey on Reinforcement Learning for Recommender Systems
Lin, Y., Liu, Y., Lin, F., Zou, L., Wu, P., Zeng, W., Chen, H., & Miao, C. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3280161 · Cited by 159
“In particular, deep RL (DRL) [16], the combination of traditional RL with deep learning methods, is competent to learn from historical data with enormous state and action spaces to address large-scale problems.”
BackgroundA survey on deep reinforcement learning for audio-based applications
Latif, S., Cuayáhuitl, H., Pervez, F., Shamshad, F., Ali, H. S., & Cambria, E. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10224-2 · Cited by 150
“A3C [16] Asynchronous gradient descent Consumes less resources; able to run on a standard multi-core CPU on-policy GA3C [17] Hybrid CPU/GPU-based A3C Achieves speed significantly higher than its CPU-based counterpart PAAC [18] Novel framework for asynchronous algorithms Computationally efficient & enables faster conve…”
BackgroundInfluentialDeep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Majid, A. Y., Saaybi, S., van Rietbergen, T., François-Lavet, V., Prasad, R. R. V., & Verhoeven, C. J. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264540 · Cited by 143
Deep Learning for Android Malware Defenses: A Systematic Literature Review
Liu, Y., Tantithamthavorn, C., Li, L., & Liu, Y. (2023) · ACM Computing Surveys · doi:10.1145/3544968 · Cited by 101
“DRL has demonstrated its significant performance in the fields of games, robotics, and self-driving cars [15, 47].”
BackgroundA Survey of Deep Graph Clustering: Taxonomy, Challenge, Application, and Open Resource
Liu, Y., Xia, J., Wu, B., Zhou, S., Yang, X., Liang, K., Fan, C.-C., Zhuang, Y., Yu, G., Li, S. Z., et al. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3669747 · Cited by 141
Data-driven science and machine learning methods in laser–plasma physics
Döpp, A., Eberle, C., Howard, S., Irshad, F., Lin, J., & Streeter, M. J. V. (2023) · High Power Laser Science and Engineering · doi:10.1017/hpl.2023.47 · Cited by 137
“[239] Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath.”
AI-Enhanced Cloud-Edge-Terminal Collaborative Network: Survey, Applications, and Future Directions
Gu, H., Zhao, L., Han, Z., Zheng, G., & Song, S. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3338153 · Cited by 147
Luo, J., Yu, F. R., Chen, Q., & Tang, L. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2955129 · Cited by 123
“See https://www.ieee.org/publications/rights/index.html for more information. integrated with deep neural network (DNN) [13], [14].”
BackgroundOsarogiagbon, A. U., Khan, F. I., Venkatesan, R., & Gillard, P. (2020) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2020.09.038 · Cited by 132
Secure Computation Offloading in Blockchain Based IoT Networks With Deep Reinforcement Learning
C. Nguyen, D., N. Pathirana, P., Ding, M., & Seneviratne, A. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3106956 · Cited by 106
“]. Nevertheless, in complex offloading problems with multi-user multi-IoT device scenarios, the dimension of state and action space can be extremely high that makes RL-based solutions inefficient [27], [28]. Fortunately, Deep Reinforcement Learning (DRL) methods [29] such as deep Q-network (DQN) have been introduced as…”
MethodMattar, M. G., & Lengyel, M. (2022) · Neuron · doi:10.1016/j.neuron.2021.12.018 · Cited by 108
“the neural bases of planning in a domain that remains challenging for even today’s AI algorithms (Arulkumaran et al., 2017).”
Biferale, L., Bonaccorso, F., Buzzicotti, M., Clark Di Leoni, P., & Gustavsson, K. (2019) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120370 · Cited by 121
CIoT-Net: a scalable cognitive IoT based smart city network architecture
Park, J.-H., Salim, M. M., Jo, J. H., Sicato, J. C. S., Rathore, S., & Park, J. H. (2019) · Human-centric Computing and Information Sciences · doi:10.1186/s13673-019-0190-9 · Cited by 121
Eckstein, M., Mamaev, I., Ditzen, B., & Sailer, U. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.555058 · Cited by 121
Local Energy Trading Behavior Modeling With Deep Reinforcement Learning
Chen, T., & Su, W. (2018) · IEEE Access · doi:10.1109/access.2018.2876652 · Cited by 121
Deep Reinforcement Learning Enabled Decision-Making for Autonomous Driving at Intersections
Li, G., Li, S., Li, S., Qin, Y., Cao, D., Qu, X., & Cheng, B. (2020) · Automotive Innovation · doi:10.1007/s42154-020-00113-1 · Cited by 121
A Comprehensive Survey on Local Differential Privacy toward Data Statistics and Analysis
Wang, T., Zhang, X., Feng, J., & Yang, X. (2020) · Sensors · doi:10.3390/s20247030 · Cited by 155
“Reinforcement learning plays an important role in AI [156].”
Background
Metadata
- Authors
- Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A.
- Affiliations
- University of Oxford (GB)
- Institution of Engineering and Technology (GB)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-11
- Volume
- 34
- Issue
- 6
- Pages
- 26-38
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W3100789280
- Cited by
- 4,733
- References
- 182
- Citations by year
- 2026: 444 · 2025: 855 · 2024: 806 · 2023: 749 · 2022: 590 · 2021: 511 · 2020: 388 · 2019: 260 · 2018: 122 · 2017: 6
- FWCI
- 126.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, value-based methods, deep Q-network, trust region policy optimization, asynchronous advantage actor-critic, visual understanding, autonomous systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.