Deep Reinforcement Learning: A Brief Survey
Open accessThe DOI matches its Crossref record; 177 of the 182 references it cites were found in a registry or an index.
Abstract
Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higher-level understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via RL. To conclude, we describe several current areas of research within the field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Foundational surveys on RL [7,72] provide broad overviews of classical and deep RL, respectively, but do not focus on human feedback or LLMs.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [7]
In the citing paper
“Classical approaches employ reinforcement learning (RL) [6, 7] and imitation learning [8-10] that operate on primitive actions, which could be challenging for systematic exploration [11-15], interpretability [16-18], and generalization [19-21].”
From Voyager: An Open-Ended Embodied Agent with Large Language Models (2023) · cites this paper as [7]
In the citing paper
“This can be frequently modeled as decision making and learning in a Markov decision process (MDP) [160].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [160]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 2×
“The workhorse of DRL, however, remains backpropagation [162, 111].”
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 10×
“[34] train locally linear models for use with the NAF algorithm— the continuous equivalent of the DQN [71]—to improve the algorithm’s sample complexity in the robotic domain where”
InfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“(temporal) credit assignment problem [115].”
InfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 1×
“[22] devised a method for calculating gradients to optimize stochastic policies by “reparameterizing” [30], [60] the stochasticity away from the network, thereby allowing standard gradients to be used (instead of the high-variance REINFORCE estimator [97]).”
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 7×
“dynamic programming over all possible actions (Figure 3, top left), sample trajectories for heuristic search (as was done by AlphaGo [108]), or even perform an exhaustive search (Figure 3, top right).”
InfluentialDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“technique known as distillation [40] has been utilised to both transfer policies learned by large DQNs to smaller DQNs, and transfer policies learned by several DQNs trained on separate games to one single DQN [85, 95].”
Cited by
Items 101–200 of 1,021
Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey
Li, T., Zhu, K., Luong, N. C., Niyato, D., Wu, Q., Zhang, Y., & Chen, B. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3160697 · Cited by 251
Artificial intelligence for aging and longevity research: Recent advances and perspectives
Zhavoronkov, A., Mamoshina, P., Vanhaelen, Q., Scheibye-Knudsen, M., Moskalev, A., & Aliper, A. (2019) · Ageing Research Reviews · doi:10.1016/j.arr.2018.11.003 · Cited by 246
“Reinforcement learning (RL) refers to goal-oriented algorithms, which learn how to attain a complex objective or maximize along a particular dimension over many steps (Arulkumaran et al., 2017; Kulkarni, 2017).”
Bao: Making Learned Query Optimization Practical
Marcus, R., Negi, P., Mao, H., Tatbul, N., Alizadeh, M., & Kraska, T. (2021) · Proceedings of the 2021 International Conference on Management of Data · doi:10.1145/3448016.3452838 · Cited by 241
“Such training can be viewed as off-policy reinforcement learning [12].”
Hu, H., Jia, X., He, Q., Fu, S., & Liu, K. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106749 · Cited by 241
A Survey on Resource Allocation in Vehicular Networks
Noor-A-Rahim, M., Liu, Z., Lee, H., Ali, G. G. M. N., Pesch, D., & Xiao, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3019322 · Cited by 238
A Survey of Deep RL and IL for Autonomous Driving Policy Learning
Zhu, Z., & Zhao, H. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3134702 · Cited by 237
“Below, we focus on policy learning for AD applications, while readers are suggested to refer [27], [29], [31] for comprehensive discussions on the general challenges of DRL/DIL.”
BackgroundOzkan-Okay, M., Akin, E., Aslan, Ö., Kosunalp, S., Iliev, T., Stoyanov, I., & Beloev, I. (2024) · IEEE Access · doi:10.1109/access.2024.3355547 · Cited by 236
“As seen in Figure 5, at the time t , the agent observes the state s t ∈ S concerning the reward r t ∈ R obtained from the previous experience, then takes action a t from the set of actions A and goes to a new state s t + 1 ∈ S with this action for converging or diverging to the target [29], [88].”
BackgroundRobust Distant Supervision Relation Extraction via Deep Reinforcement Learning
Qin, P., Xu, W., & Wang, W. Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1199 · Cited by 236
Machine Learning for Wireless Sensor Networks Security: An Overview of Challenges and Issues
Ahmad, R., Wazirali, R., & Abu-Ain, T. (2022) · Sensors · doi:10.3390/s22134730 · Cited by 235
Reinforcement learning in sustainable energy and electric systems: a survey
Yang, T., Zhao, L., Li, W., & Zomaya, A. Y. (2020) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2020.03.001 · Cited by 234
Why AI is harder than we think
Mitchell, M. (2021) · Proceedings of the Genetic and Evolutionary Computation Conference · doi:10.1145/3449639.3465421 · Cited by 129
Deep Reinforcement Learning for Sequence-to-Sequence Models
Keneshloo, Y., Shi, T., Ramakrishnan, N., & Reddy, C. K. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2929141 · Cited by 230
“Although, recently, there had been a couple of survey articles on the topic of deep RL [47], [48], these works heavily focused on the RL methods and their applications in robotics and vision while giving less emphasis to how these models could be used in a variety of other tasks.”
Deep deterministic policy gradient algorithm: A systematic review
Sumiea, E. H., Abdulkadir, S. J., Alhussian, H. S., Al-Selwi, S. M., Alqushaibi, A., Ragab, M. G., & Fati, S. M. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e30697 · Cited by 229
“DDPG can be extended to handle multi-objective optimization by using methods such as Pareto optimization or weighted sum optimization [1].”
BackgroundMethodInfluentialLearning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning
Hua, J., Zeng, L., Li, G., & Ju, Z. (2021) · Sensors · doi:10.3390/s21041278 · Cited by 229
“These two types of reinforcement learning algorithms can be divided into two categories based on the solution approach: the value-based learning method and the policy-based learning method [37].”
Deep Reinforcement Learning for Join Order Enumeration
Marcus, R., & Papaemmanouil, O. (2018) · Proceedings of the First International Workshop on Exploiting Artificial Intelligence Techniques for Data Management · doi:10.1145/3211954.3211957 · Cited by 227
Reinforcement learning for clinical decision support in critical care: comprehensive review
Liu, S., See, K. C., Ngiam, K. Y., Celi, L. A., Sun, X., & Feng, M. (2020) · Journal of Medical Internet Research · doi:10.2196/18477 · Cited by 225
An application of deep reinforcement learning to algorithmic trading
Théate, T., & Ernst, D. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114632 · Cited by 224
“al. (2015) and Goodfellow et al. (2016). For more information about RL, the reader can refer to the following textbooks and surveys: Sutton and Barto (2018), Szepesvari (2010), Busoniu et al. (2010), Arulkumaran et al. (2017) and Shao et al. (2019). 4.2. Articial trajectories generation In the scope of the algorithmic…”
BackgroundInfluentialAdvanced machine-learning techniques in drug discovery
Elbadawi, M., Gaisford, S., & Basit, A. W. (2021) · Drug Discovery Today · doi:10.1016/j.drudis.2020.12.003 · Cited by 223
“There are multiple versions of the reward function [31].”
Adversarial environment reinforcement learning algorithm for intrusion detection
Caminero, G., Lopez-Martin, M., & Carro, B. (2019) · Computer Networks · doi:10.1016/j.comnet.2019.05.013 · Cited by 222
“However, this is not the framework used to train a Deep Reinforcement Learning (DRL) algorithm [7], which is based on the interaction of an agent with an environment.”
BackgroundMethodArtificial intelligence in drug discovery: applications and techniques
Deng, J., Yang, Z., Ojima, I., Samaras, D., & Wang, F. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbab430 · Cited by 220
Evolutionary Machine Learning: A Survey
Telikani, A., Tahmassebi, A., Banzhaf, W., & Gandomi, A. H. (2022) · ACM Computing Surveys · doi:10.1145/3467477 · Cited by 219
Multi-vehicle routing problems with soft time windows: A multi-agent reinforcement learning approach
Zhang, K., He, F., Zhang, Z., Lin, X., & Li, M. (2020) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2020.102861 · Cited by 217
Battery Safety: Machine Learning-Based Prognostics
Zhao, J., Feng, X., Pang, Q., Fowler, M., Lian, Y., Ouyang, M., & Burke, A. F. (2024) · Progress in Energy and Combustion Science · doi:10.1016/j.pecs.2023.101142 · Cited by 216
The building blocks of a brain-inspired computer
Kendall, J. D., & Kumar, S. (2020) · Applied Physics Reviews · doi:10.1063/1.5129306 · Cited by 216
Volk, A. A., Epps, R. W., Yonemoto, D. T., Masters, B. S., Castellano, F. N., Reyes, K. G., & Abolhasani, M. (2023) · Nature Communications · doi:10.1038/s41467-023-37139-y · Cited by 215
Deep reinforcement learning in medical imaging: A literature review
Zhou, S. K., Le, H. N., Luu, K., V Nguyen, H., & Ayache, N. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102193 · Cited by 215
A Machine Learning-Oriented Survey on Tiny Machine Learning
Capogrosso, L., Cunico, F., Cheng, D. S., Fummi, F., & Cristani, M. (2024) · IEEE Access · doi:10.1109/access.2024.3365349 · Cited by 255
“This is useful because in real-world applications the space state is high-dimensional and the use of traditional RL algorithms is not effective [151].”
BackgroundVariable Impedance Control and Learning—A Review
Abu-Dakka, F. J., & Saveriano, M. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.590681 · Cited by 214
“(2020) reviews recent advancement in the field with a particular focus on data-efficient algorithms, while Arulkumaran et al. (2017) focuses on deep learning solutions.”
BackgroundInfluentialA Perspective on Using Machine Learning in 3D Bioprinting
Yu, C., & Jiang, J. (2020) · International Journal of Bioprinting · doi:10.18063/ijb.v6i1.253 · Cited by 213
“The most commonly used machine learning methods include supervised learning [33] , unsupervised learning [34] , and reinforcement learning [35] .”
MethodZheng, P., Xia, L., Li, C., Li, X., & Liu, B. (2021) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2021.08.002 · Cited by 213
Chen, Z., & Wang, X. (2020) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-020-01801-6 · Cited by 212
“In order to improve the model with adequate exploration of the state space, one major challenge in RL is the tradeoff between exploration and exploitation [42], which is even more difficult for learning in continuous action spaces.”
BackgroundLearning an approximate model predictive controller with guarantees
Hertneck, M. (2018) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-9722 · Cited by 210
Recent advancements of robotics in construction
Xiao, B., Chen, C., & Yin, X. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104591 · Cited by 204
Deep Reinforcement Learning for Smart Grid Operations: Algorithms, Applications, and Prospects
Li, Y., Yu, C., Shahidehpour, M., Yang, T., Zeng, Z., & Chai, T. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3303358 · Cited by 203
Nath, S., & Wu, J. (2020) · Intelligent and Converged Networks · doi:10.23919/icn.2020.0014 · Cited by 202
Hu, J., Zhang, H., Song, L., Schober, R., & Poor, H. V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3013599 · Cited by 201
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200
“For a more detailed description of DRL, including recent efforts and some applications, we refer the reader to [4], [63].”
BackgroundDiffusion-Based Reinforcement Learning for Edge-Enabled AI-Generated Content Services
Du, H., Li, Z., Niyato, D., Kang, J., Xiong, Z., Huang, H., & Mao, S. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3356178 · Cited by 200
Voyager: An Open-Ended Embodied Agent with Large Language Models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023) · arXiv · doi:10.48550/arxiv.2305.16291 · Cited by 222
“Classical approaches employ reinforcement learning (RL) [6, 7] and imitation learning [8–10] that operate on primitive actions, which could be challenging for systematic exploration [11–15], interpretability [16–18], and generalization [19–21].”
BackgroundDeep neural networks in the cloud: Review, applications, challenges and research directions
Chan, K. Y., Abu-Salih, B., Qaddoura, R., Al-Zoubi, A. M., Palade, V., Pham, D.-S., Ser, J. D., & Muhammad, K. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126327 · Cited by 199
Innovative Materials Science via Machine Learning
Gao, C., Min, X., Fang, M., Tao, T., Zheng, X., Liu, Y., Wu, X., & Huang, Z. (2022) · Advanced Functional Materials · doi:10.1002/adfm.202108044 · Cited by 198
Policy Iteration Reinforcement Learning-based control using a Grey Wolf Optimizer algorithm
Zamfirache, I. A., Precup, R., Roman, R., & Petriu, E. M. (2021) · Information Sciences · doi:10.1016/j.ins.2021.11.051 · Cited by 198
Ahmad, S., Shakeel, I., Mehfuz, S., & Ahmad, J. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2023.100568 · Cited by 197
Deep multiagent reinforcement learning: challenges and directions
Wong, A., Bäck, T., Kononova, A. V., & Plaat, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10299-x · Cited by 197
“Recent developments in RL research show a preference for policy-based strategies, even though value-based methods can capture the underlying structure of the environment (Arulkumaran et al., 2017).”
BackgroundDu, J., Cheng, W., Lu, G., Cao, H., Chu, X., Zhang, Z., & Wang, J. (2022) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3068340 · Cited by 197
A review on virtual power plant for energy management
Rouzbahani, H. M., Karimipour, H., & Lei, L. (2021) · Sustainable Energy Technologies and Assessments · doi:10.1016/j.seta.2021.101370 · Cited by 195
A Review of Practical AI for Remote Sensing in Earth Sciences
Janga, B., Asamani, G., Sun, Z., & Cristea, N. (2023) · Remote Sensing · doi:10.3390/rs15164112 · Cited by 194
“Deep Reinforcement Learning (DRL) offers advantages in remote sensing, such as learning from unlabeled data and improving decision-making processes [128].”
BackgroundNetwork Abnormal Traffic Detection Model Based on Semi-Supervised Deep Reinforcement Learning
Dong, S., Xia, Y., & Peng, T. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3120804 · Cited by 193
Hadi, M. U., tashi, Q. A., Qureshi, R., Shah, A., muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., et al. (2023) · doi:10.36227/techrxiv.23589741.v4 · Cited by 103
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2109.06668 · Cited by 192
“A few prior papers contain the investigation on exploration methods for DRL [8], [9], [10], [11] and MARL [12], [13], [14].”
MethodSurvey of deep learning for autonomous surface vehicles in marine environments
Qiao, Y., Yin, J., Wang, W., Duarte, F., Yang, J., & Ratti, C. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3235911 · Cited by 192
“systems learning the interaction between agent and environment [17], [18], [19].”
Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques, and Tools
Mayer, R., & Jacobsen, H.-A. (2021) · ACM Computing Surveys · doi:10.1145/3363554 · Cited by 192
“ally colorize grayscale images) [75], image captioning [68] (i.e., creating a description of an image), and as agents in reinforcement learning systems that map state-action pairs to expected rewards [10]. In DL, a network of mathematical operators is trained with classified or unclassified data sets until the weights…”
BackgroundA Survey of Blockchain and Artificial Intelligence for 6G Wireless Communications
Zuo, Y., Guo, J., Gao, N., Zhu, Y., Jin, S., & Li, X. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3315374 · Cited by 192
“For large-scale station-action pair, deep reinforcement learning (DRL) [115] is an end-to-end perception and control system with strong generality.”
BackgroundNeuro-symbolic artificial intelligence: a survey
Bhuyan, B. P., Ramdane-Cherif, A., Tomar, R., & Singh, T. P. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09960-z · Cited by 190
“In contrast, connectionist AI, particularly through deep learning, excels at learning representations from raw, unstructured data [184].”
Zappone, A., Di Renzo, M., Debbah, M., Lam, T. T., & Qian, X. (2019) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2019.2921627 · Cited by 190
“In addition, deep reinforcement-learning frameworks [15], which are able to learn optimal action policies in a fully data-driven fashion from direct interaction with the environment, offer another promising approach to explore and combine with transfer-learning methods.”
BackgroundDeep Learning Techniques for Future Intelligent Cross-Media Retrieval
Rehman, S. U., Waqas, M. M., Tu, S., Koubâa, A., Rehman, O. U., Ahmad, J. E., Hanif, M. I., & Han, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01191 · Cited by 190
“…of architectures, algorithms, and applications for deep learning [22] 2018 A survey on deep learning: Algorithms, techniques, and applications [23] 2017 Deep reinforcement learning: A brief survey [24] 2017 Imitation learning: A survey of learning methods (cid:106) [25] 2014 Big data deep…”
BackgroundExploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3236361 · Cited by 189
“A few prior papers contain the investigation on exploration methods for DRL [8], [9], [10], [11], [12] and MARL [13], [14], [15].”
MethodJamil, B., Ijaz, H., Shojafar, M., Munir, K., & Buyya, R. (2022) · ACM Computing Surveys · doi:10.1145/3513002 · Cited by 189
“The three main model-free methods to solve RL problems are value-based methods, policy-based methods, and actor-critic methods [15].”
MethodA Survey of Reinforcement Learning Informed by Natural Language
Luketina, J., Nardelli, N., Farquhar, G., Foerster, J., Andreas, J., Grefenstette, E., Whiteson, S., & Rocktäschel, T. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/880 · Cited by 188
Exploiting Cognitive Structure for Adaptive Learning
Liu, Q., Tong, S., Liu, C., Zhao, H., Chen, E., Ma, H., & Wang, S. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330922 · Cited by 239
“he dynamic specialty with infinite time horizons, makes it difficult to be described in the graph structure. Reinforcement Learning. Deep reinforcement learning, as one of state-of-the-art techniques [1], has shown superior abilities in many fields [36]. The main idea is to learn and refine model parameters according…”
BackgroundHa, D., & Schmidhuber, J. (2018) · Zenodo · doi:10.5281/zenodo.1207631 · Cited by 187
“Although there is a large body of research relating to model-based reinforcement learning, this article is not meant to be a review (Arulkumaran et al., 2017; Schmidhuber, 2015b) of the current state of the field.”
BackgroundActive Example Selection for In-Context Learning
Zhang, Y., Feng, S., & Tan, C. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.622 · Cited by 98
Baccour, E., Mhaisen, N., Abdellatif, A. A., Erbad, A., Mohamed, A., Hamdi, M., & Guizani, M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3200740 · Cited by 186
“Readers may refer to [153] for more details and comparison between these approaches.”
BackgroundDRL360: 360-degree Video Streaming with Deep Reinforcement Learning
Zhang, Y., Zhao, P., Bian, K., Liu, Y., Song, L., & Li, X. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737361 · Cited by 186
“, the problems with settings of high-dimensional state space and action space [7].”
BackgroundPerez-Cerrolaza, J., Abella, J., Borg, M., Donzella, C., Cerquides, J., Cazorla, F. J., Englund, C., Tauber, M., Nikolakopoulos, G., & Flores, J. L. (2024) · ACM Computing Surveys · doi:10.1145/3626314 · Cited by 185
“…al. 2019; Huang et al. 2020], veriication [Akella et al. 2020; Ehlers 2017; Huang et al. 2017]), generic AI solutions (e.g., reinforcement learning [Arulkumaran et al. 2017]) and safety adaptations (e.g., safe reinforcement learning [García and Fernández 2015]), with references to multiple…”
BackgroundMethodRahman, M. H., Sejan, M. A. S., Aziz, M. A., Tabassum, R., Baik, J.-I., & Song, H. (2024) · Remote Sensing · doi:10.3390/rs16050879 · Cited by 205
“This approach learns an optimal policy by trial and error to solve a problem in the real world [196].”
A Review on Data-Driven Constitutive Laws for Solids
Fuhg, J. N., Padmanabha, G. A., Bouklas, N., Bahmani, B., Sun, W., Vlassis, N. N., Flaschel, M., Carrara, P., & Lorenzis, L. D. (2024) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-024-10196-2 · Cited by 251
“To address this issue, an RL technique that hinges on the concept of dynamic programming is employed [52].”
Reinforcement learning for neural architecture search: A review
Jaafra, Y., Luc Laurent, J., Deruyver, A., & Saber Naceur, M. (2019) · Image and Vision Computing · doi:10.1016/j.imavis.2019.06.005 · Cited by 183
Strong mixed-integer programming formulations for trained neural networks
Anderson, R., Huchette, J., Ma, W., Tjandraatmadja, C., & Vielma, J. P. (2020) · Mathematical Programming · doi:10.1007/s10107-020-01474-5 · Cited by 182
“For example, this paradigm arises in deep reinforcement learning problems with high dimensional action spaces and where any of the cost-to-go function, immediate cost, or the state transition functions are learned by a neural network [6,23,56,62,64,80].”
BackgroundRecent Advancements in Fruit Detection and Classification Using Deep Learning Techniques
Ukwuoma, C. C., Zhiguang, Q., Bin Heyat, M. B., Ali, L., Almaspoor, Z., & Monday, H. N. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/9210947 · Cited by 182
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Fang, K., Toshev, A., Fei-Fei, L., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00063 · Cited by 178
Privacy-preserved task offloading in mobile blockchain with deep reinforcement learning
Nguyen, D. C., Pathirana, P. N., Ding, M., & Seneviratne, A. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3010967 · Cited by 178
“in high-dimensional offloading problems, the dimension of state and action space can be extremely high that makes RL-based solutions inefficient [15], [16].”
BackgroundDeep reinforcement learning for drone navigation using sensor data
Hodge, V. J., Hawkins, R., & Alexander, R. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05097-x · Cited by 178
Wang, L., Zhu, D., Pang, W., & Zhang, Y. (2023) · Ocean Engineering · doi:10.1016/j.oceaneng.2023.114393 · Cited by 178
Advances in trajectory optimization for space vehicle control
Malyuta, D., Yu, Y., Elango, P., & Açıkmeşe, B. (2021) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2021.04.013 · Cited by 176
Brain-inspired learning in artificial neural networks: A review
Schmidgall, S., Ziaei, R., Achterberg, J., Kirsch, L., Hajiseyedrazi, S. P., & Eshraghian, J. K. (2024) · APL Machine Learning · doi:10.1063/5.0186054 · Cited by 219
Multiagent DDPG-Based Deep Learning for Smart Ocean Federated Learning IoT Networks
Kwon, D., Jeon, J., Park, S., Kim, J., & Cho, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2988033 · Cited by 174
“Based on the interaction between an agent and its own environment, the reinforcement learning agent observes state transition and learns how to act good by updating its policy [26].”
BackgroundA Survey on Traffic Signal Control Methods
Wei, H., Zheng, G., Gayah, V. V., & Li, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.08117 · Cited by 172
“on representation, an RL method can be categorized as a tabular or approximation method respectively. 4.5.1 Model-based vs. model-free methods. Depending on the modeling philosophy of RL, literature [Arulkumaran et al. 2017; Kaelbling et al. 1996] divides current RL methods into two categories: model-based methods and…”
InfluentialSurvey of Deep Representation Learning for Speech Emotion Recognition
Latif, S., Rana, R., Khalifa, S., Jurdak, R., Qadir, J., & Schuller, B. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3114365 · Cited by 172
“to solve high-dimensional complex problems [187].”
Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics
Mosavi, A., Faghan, Y., Ghamisi, P., Duan, P., Ardabili, S. F., Salwana, E., & Band, S. S. (2020) · Mathematics · doi:10.3390/math8101640 · Cited by 172
Zamfirache, I. A., Precup, R.-E., Roman, R.-C., & Petriu, E. M. (2022) · Information Sciences · doi:10.1016/j.ins.2021.10.070 · Cited by 172
Tang, X., Chen, J., Liu, T., Qin, Y., & Cao, D. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3107734 · Cited by 171
“For Markov Decision Problems (MDPs), it can be simply described as the next state st+1 of the environmental system is only related to the current state st and the action at, and it can be described as follows [38]:”
BackgroundDing, Y., Ma, L., Ma, J., Suo, M., Tao, L., Cheng, Y., & Lu, C. (2019) · Advanced Engineering Informatics · doi:10.1016/j.aei.2019.100977 · Cited by 171
AI-Enabled Future Wireless Networks: Challenges, Opportunities, and Open Issues
Elsayed, M., & Erol-Kantarci, M. (2019) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2019.2919236 · Cited by 156
“Incorporating a deep neural network can enable reinforcement learning to scale to problems that were previously intractable [7].”
BackgroundDeep reinforcement learning based preventive maintenance policy for serial production lines
Huang, J., Chang, Q., & Arinez, J. F. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113701 · Cited by 171
Deep Reinforcement Learning for Resource Management on Network Slicing: A Survey
Sánchez, J. A. H., Casilimas, K., & Rendón, O. M. C. (2022) · Sensors · doi:10.3390/s22083031 · Cited by 204
“For further information about DRL and its algorithms, we refer the reader to [25,27,128].”
BackgroundMethodInfluentialHazarika, A., & Rahmati, M. (2023) · Sensors · doi:10.3390/s23073682 · Cited by 194
Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning
Ilahi, I., Usama, M., Qadir, J., Janjua, M. U., Al-Fuqaha, A., Hoang, D. T., & Niyato, D. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3111139 · Cited by 170
Ramu, K., Raju, S. V. S. R. K., Singh, S., Rachapudi, V., Mary, M. A., Roy, V., & Joshi, S. (2024) · SN Computer Science · doi:10.1007/s42979-024-03193-6 · Cited by 170
Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimization
Cappart, Q., Moisan, T., Rousseau, L.-M., Prémont-Schwarz, I., & Ciré, A. A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16484 · Cited by 225
Khoshafa, M. H., Maraqa, O., Moualeu, J. M., Aboagye, S., Ngatched, T. M. N., Ahmed, M. H., Gadallah, Y., & Renzo, M. D. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3487112 · Cited by 195
“DRL algorithms find application in robotics, enabling real-time learning from inputs, such as camera data, by leveraging the advantages of deep neural networks (DNNs) to enhance the learning speed of the DRL [331], [332].”
Wang, X., Wang, Y., Su, X., Wang, L., Lu, C., Peng, H., & Liu, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10620-2 · Cited by 169
“They exhibit super-human decision capabilities (Arulkumaran et al. 2017) in video games, poker, multiplayer games, autonomous driving and some complex board games, as shown in Fig.”
Elgendy, I. A., Zhang, W.-Z., He, H., Gupta, B. B., & Abd El-Latif, A. A. (2021) · Wireless Networks · doi:10.1007/s11276-021-02554-w · Cited by 169
“In particular, in recent works, Reinforcement Learning was utilized in dynamic multi-user MEC systems [9, 36] to cope empirically with the large-scale problems.”
MethodA Brief Introduction to Machine Learning for Engineers
Simeone, O. (2018) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000102 · Cited by 154
A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization
Stampa, G., Arias, M., Sánchez-Charles, D., Muntés-Mulero, V., & Cabellos‐Aparicio, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.07080 · Cited by 168
“By taking advantage of the recent breakthroughs of deep neural networks applied to reinforcement learning [6, 7] we design and train a DRL agent capable of optimizing routing according to a predefined target metric: network delay.”
Machine Learning into Metaheuristics: A Survey and Taxonomy
Talbi, E.-G. (2022) · ACM Computing Surveys · doi:10.1145/3459664 · Cited by 168
“In Reference [154], Convolution Neural Networks models learn the optimal solution for the traveling salesman problem as an image and extract the patterns (i.e., edges) to be included in a solution.”
MethodInfluentialDeep reinforcement learning with smooth policy update: Application to robotic cloth manipulation
Tsurumine, Y., Cui, Y., Uchibe, E., & Matsubara, T. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.11.004 · Cited by 168
“Although such a reward function design is not related to the features extracted from states by DRL and therefore has no conflict to the merit of DRL [63].”
BackgroundFerrag, M. A., Shu, L., Friha, O., & Yang, X. K. (2021) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2021.1004344 · Cited by 196
Woodman, R. J., & Mangoni, A. A. (2023) · Aging Clinical and Experimental Research · doi:10.1007/s40520-023-02552-2 · Cited by 129
Machine Learning in Real-Time Internet of Things (IoT) Systems: A Survey
Bian, J., Arafat, A. A., Xiong, H., Li, J., Li, L., Chen, H., Wang, J., Dou, D., & Guo, Z. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3161050 · Cited by 202
“However, due to its limitation on large-scale dynamic data-environment [21], [82], reinforcement learning intends to embrace the hug of deep learning in nowadays real-time IoT applications.”
Background
Metadata
- Authors
- Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A.
- Affiliations
- University of Oxford (GB)
- Institution of Engineering and Technology (GB)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-11
- Volume
- 34
- Issue
- 6
- Pages
- 26-38
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W3100789280
- Cited by
- 4,733
- References
- 182
- Citations by year
- 2026: 444 · 2025: 855 · 2024: 806 · 2023: 749 · 2022: 590 · 2021: 511 · 2020: 388 · 2019: 260 · 2018: 122 · 2017: 6
- FWCI
- 126.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, value-based methods, deep Q-network, trust region policy optimization, asynchronous advantage actor-critic, visual understanding, autonomous systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.