Deep Reinforcement Learning: A Brief Survey
Open accessThe DOI matches its Crossref record; 177 of the 182 references it cites were found in a registry or an index.
Abstract
Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higher-level understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via RL. To conclude, we describe several current areas of research within the field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Foundational surveys on RL [7,72] provide broad overviews of classical and deep RL, respectively, but do not focus on human feedback or LLMs.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [7]
In the citing paper
“Classical approaches employ reinforcement learning (RL) [6, 7] and imitation learning [8-10] that operate on primitive actions, which could be challenging for systematic exploration [11-15], interpretability [16-18], and generalization [19-21].”
From Voyager: An Open-Ended Embodied Agent with Large Language Models (2023) · cites this paper as [7]
In the citing paper
“This can be frequently modeled as decision making and learning in a Markov decision process (MDP) [160].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [160]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 2×
“The workhorse of DRL, however, remains backpropagation [162, 111].”
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 10×
“[34] train locally linear models for use with the NAF algorithm— the continuous equivalent of the DQN [71]—to improve the algorithm’s sample complexity in the robotic domain where”
InfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“(temporal) credit assignment problem [115].”
InfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 1×
“[22] devised a method for calculating gradients to optimize stochastic policies by “reparameterizing” [30], [60] the stochasticity away from the network, thereby allowing standard gradients to be used (instead of the high-variance REINFORCE estimator [97]).”
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 7×
“dynamic programming over all possible actions (Figure 3, top left), sample trajectories for heuristic search (as was done by AlphaGo [108]), or even perform an exhaustive search (Figure 3, top right).”
InfluentialDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“technique known as distillation [40] has been utilised to both transfer policies learned by large DQNs to smaller DQNs, and transfer policies learned by several DQNs trained on separate games to one single DQN [85, 95].”
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
Cuomo, S., Di Cola, V. S., Giampaolo, F., Rozza, G., Raissi, M., & Piccialli, F. (2022) · Journal of Scientific Computing · doi:10.1007/s10915-022-01939-z · Cited by 2,784
“RL enables agents to conduct experiments to comprehend their environment better, allowing them to acquire high-level causal links and reasoning about causes and effects (Arulkumaran et al, 2017).”
BackgroundSarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00815-1 · Cited by 2,678
“Deep reinforcement learning (DRL or deep RL) [9] integrates neural networks with a reinforcement learning architecture to allow the agents to learn the appropriate actions in a virtual environment, as shown in Fig.”
BackgroundFederated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
“to [54]–[59] for comprehensive discussions of DNN architectures and training strategies.”
BackgroundApplications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
“discuss the applications of DRL such as [7] and [8], but they are specifically for computer vision and natural language processing.”
Review of Deep Learning Algorithms and Architectures
Shrestha, A., & Mahmood, A. (2019) · IEEE Access · doi:10.1109/access.2019.2912200 · Cited by 1,883
“With DRL, problems that were intractable with reinforcement learning can now be solved with higher number of hidden layers of deep networks and reinforcement learning based Q-learning algorithm that maximizes the reward for actions taken by the agent [13].”
BackgroundDeep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620
Nguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306
“Deep RL is a broad term that indicates the combination between deep learning and RL to deal with high-dimensional environments [34]–[36].”
BackgroundAn Introduction to Deep Reinforcement Learning
Vincent, F.-L., Peter, H., Riashat, I., Marc, G. B., & Joelle, P. (2018) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000071 · Cited by 1,286
Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
Deng, S., Zhao, H., Fang, W., Yin, J., Dustdar, S., & Zomaya, A. Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2984887 · Cited by 1,147
Deep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079
Taye, M. M. (2023) · Computers · doi:10.3390/computers12050091 · Cited by 1,057
“In contrast, reinforcement learning gains information through direct interactions with the issue environment [41].”
BackgroundMachine learning & artificial intelligence in the quantum domain: a review of recent progress
Dunjko, V., & Briegel, H. J. (2018) · Reports on Progress in Physics · doi:10.1088/1361-6633/aab406 · Cited by 1,055
Deep Reinforcement Learning: A Survey
Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xu, X., Dai, B., & Miao, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3207346 · Cited by 972
“Previous survey papers summarized the development of DRL in various aspects [40], [41], [42], [43].”
BackgroundMulti-agent deep reinforcement learning: a survey
Gronauer, S., & Diepold, K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09996-w · Cited by 957
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941
Pham, Q.-V., Fang, F., Ha, V. N., Piran, M. J., Le, M., Le, L. B., Hwang, W.-J., & Ding, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3001277 · Cited by 870
“Moreover, some surveys and tutorials on ML [248], DL [249], (deep) RL [250], [251], as well as their applications in communications and networking [252], [253] have come out, and readers can refer to the aforementioned literature for more details.”
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.22520 · Cited by 849
Self-citationEdge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“This can be frequently modeled as decision making and learning in a Markov decision process (MDP) [160].”
BackgroundZhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774
“Recently, rapid progress has been made by incorporating deep learning in RL (i.e., DRL) [209].”
BackgroundSafe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-042920-020211 · Cited by 769
Xie, J., Yu, F. R., Huang, T., Xie, R., Liu, J., Wang, C., & Liu, Y. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2866942 · Cited by 700
“DRL [128]–[130].”
Deep Learning in Physical Layer Communications
Qin, Z., Ye, H., Li, G. Y., & Juang, B.-H. F. (2019) · IEEE Wireless Communications · doi:10.1109/mwc.2019.1800601 · Cited by 692
Deep Learning in Physical Layer Communications
Qin, Z., Ye, H., Li, G. Y., & Juang, B.-H. F. (2018) · arXiv · doi:10.48550/arxiv.1807.11713 · Cited by 692
Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?
Zappone, A., Di Renzo, M., & Debbah, M. (2019) · arXiv · doi:10.48550/arxiv.1902.02647 · Cited by 685
“This implementation of Q-learning is referred to as the Deep Q-Network approach [95], [96], which can be considered an algorithm belonging to the family of Q-learning methods, with the peculiarity that the approximate function Q̂(s, a, w) is specified through an ANN.”
BackgroundMethodWireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?
Zappone, A., Di Renzo, M., & Debbah, M. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2924010 · Cited by 684
“This implementation of Q-learning is referred to as the Deep Q-Network approach [95], [96], which can be considered an algorithm belonging to the family of Q-learning methods, with the peculiarity that the approximate function Q̂(s, a, w) is specified through an ANN.”
BackgroundMethodA survey and critique of multiagent deep reinforcement learning
Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 671
“We refer the reader to recent surveys on single-agent DRL [13,101,191] for a more detailed discussion of the literature.”
InfluentialAntonopoulos, I., Robu, V., Couraud, B., Kirli, D., Norbu, S., Kiprakis, A., Flynn, D., Elizondo-Gonzalez, S., & Wattam, S. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109899 · Cited by 656
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 656
A Survey of Deep Learning: Platforms, Applications and Emerging Research Trends
Hatcher, W. G., & Yu, W. (2018) · IEEE Access · doi:10.1109/access.2018.2830661 · Cited by 654
“An alternative to the quality function, the advantage function represents relative state-action values, as opposed to absolute state-action values [19].”
BackgroundMethodResultInfluentialJusup, M., Holme, P., Kanazawa, K., Takayasu, M., Romić, I., Wang, Z., Geček, S., Lipić, T., Podobnik, B., Wang, L., et al. (2022) · Physics Reports · doi:10.1016/j.physrep.2021.10.005 · Cited by 643
Q-Learning Algorithms: A Comprehensive Classification and Applications
Jang, B., Kim, M., Harerimana, G., & Kim, J. W. (2019) · IEEE Access · doi:10.1109/access.2019.2941229 · Cited by 635
Deep reinforcement learning for cyber security
Nguyen, T. T., & Reddi, V. J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3121870 · Cited by 620
“With the power of function approximation and representation learning, deep learning can learn a compact low-dimensional representation of raw high-dimensional data [58].”
BackgroundAn Introductory Review of Deep Learning for Prediction Models With Big Data
Emmert-Streib, F., Yang, Z., Feng, H., Tripathi, S., & Dehmer, M. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00004 · Cited by 604
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Liu, G., Tao, A., Kautz, J., & Catanzaro, B. (2018) · arXiv · doi:10.48550/arxiv.1808.06601 · Cited by 594
“For example, in model-based reinforcement learning [2,24], a video synthesis model finds use in approximating visual dynamics of the world for training the agent with less amount of real experience data.”
BackgroundTheissler, A., Pérez-Velázquez, J., Kettelgerdes, M., & Elger, G. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.107864 · Cited by 581
Fusing Blockchain and AI With Metaverse: A Survey
Yang, Q., Zhao, Y., Huang, H., Xiong, Z., Kang, J., & Zheng, Z. (2022) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2022.3188249 · Cited by 554
Exploration in deep reinforcement learning: A survey
Ladosz, P., Weng, L., Kim, M., & Oh, H. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.003 · Cited by 525
“A general overview of reinforcement learning can be found in [23].”
BackgroundInterpretable deep learning: interpretation, interpretability, trustworthiness, and beyond
Li, X., Xiong, H., Li, X., Wu, X., Zhang, X., Liu, J., Bian, J., & Dou, D. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01756-8 · Cited by 497
“DRL is also applicable and shows potentials of application in healthcare, finance and business management [13, 105], where human security and property safety issues should be considered, leading to the demands of explainable RL [134].”
BackgroundMethodCommunication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
“…has achieved breakthroughs in a wide range of applications, including speech processing [1], image classi-fication [2] and reinforcement learning [3], etc. AI is expected to affect significant segments of many vertical industries and our daily life, such as intelligent vehicles [4] and tactile…”
BackgroundReinforcement learning for control: Performance, stability, and deep approximators
Buşoniu, L., de Bruin, T., Tolić, D., Kober, J., & Palunko, I. (2018) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2018.09.005 · Cited by 463
“For a broader view of DRL we refer the reader to the recent reviews (Arulkumaran et al., 2017; Li, 2017).”
Crop Yield Prediction Using Deep Reinforcement Learning Model for Sustainable Agrarian Applications
Elavarasan, D., & Vincent, P. M. D. (2020) · IEEE Access · doi:10.1109/access.2020.2992480 · Cited by 449
“In the DRL framework, deep learning provides the agent with the ability to sense the environment and reinforcement learning provides the ability to learn the best strategy for real-time problems [48].”
BackgroundDeep Learning in Mining Biological Data
Mahmud, M., Kaiser, M. S., McGinnity, T. M., & Hussain, A. (2021) · Cognitive Computation · doi:10.1007/s12559-020-09773-x · Cited by 446
A Theoretical Analysis of Deep Q-LearningPreprint
Fan, J., Wang, Z., Xie, Y., & Yang, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00137 · Cited by 548
“We refer the reader to Arulkumaran et al. (2017) for a survey of the recent developments of DRL.”
InfluentialSharma, S. K., & Wang, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916177 · Cited by 427
“…rewards from the environment, resulting in the return of L l =0 γ l r l +1 , where γ ∈ [0 1] is the discount factor which provides more weights on the immediate rewards and L denotes the length of one episode, which denotes the time period after which the state is reset for an episodic MDP [45].”
BackgroundMethodInfluentialReinforcement Learning Based Routing in Networks: Review and Classification of Approaches
Mammeri, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2913776 · Cited by 427
“Recently, deep reinforcement learning has been proposed to enable RL to scale to complex problems [99], [100].”
BackgroundThe History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S. K. E., Sidike, P., Nasrin, M. S., Essen, B. C. V., Awwal, A. A. S., & Asari, V. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01164 · Cited by 426
“In addition, RNNs, such as LSTM and RL, are also used for unsupervised learning in many application domains [243].”
Deep learning: systematic review, models, challenges, and research directions
Talaei Khoei, T., Ould Slimane, H., & Kaabouch, N. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08957-4 · Cited by 411
Autonomous experimentation systems for materials development: A community perspective
Stach, E., DeCost, B., Kusne, A. G., Hattrick-Simpers, J., Brown, K. A., Reyes, K. G., Schrier, J., Billinge, S., Buonassisi, T., Foster, I., et al. (2021) · Matter · doi:10.1016/j.matt.2021.06.036 · Cited by 398
Hussain, F., Hassan, S. A., Hussain, R., & Hossain, E. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2964534 · Cited by 395
Reinforcement learning in robotic applications: a comprehensive survey
Singh, B., Kumar, R., & Singh, V. P. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09997-9 · Cited by 392
Niroui, F., Zhang, K., Kashino, Z., & Nejat, G. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2891991 · Cited by 385
“In DRL, state features are learned automatically, and dimensionality can be reduced through iterative interactions with the environment [22].”
BackgroundMethodInfluentialCheng, L., & Yu, T. (2019) · International Journal of Energy Research · doi:10.1002/er.4333 · Cited by 380
“117 For example, DRL algorithms are being applied to robotics, which allows control policies for robots to be learned directly from real ‐ world camera input.”
BackgroundMethodEfficient Large-Scale Fleet Management via Multi-Agent Deep Reinforcement Learning
Lin, K., Zhao, R., Xu, Z., & Zhou, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219993 · Cited by 379
“The RL framework is reward-driven, meaning that a sequence of actions from the policy is evaluated solely by the reward signal from environment [1].”
BackgroundBenti, N. E., Chaka, M. D., & Semie, A. G. (2023) · Sustainability · doi:10.3390/su15097087 · Cited by 378
“Reinforcement learning (RL) is a branch of ML in which an agent learns to make decisions in an environment to maximize a cumulative reward signal [84,85].”
BackgroundPath planning and obstacle avoidance for AUV: A review
Cheng, C., Sha, Q., He, B., & Li, G. (2021) · Ocean Engineering · doi:10.1016/j.oceaneng.2021.109355 · Cited by 378
Atitallah, S. B., Driss, M., Boulila, W., & Ghézala, H. B. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100303 · Cited by 375
IoT Connectivity Technologies and Applications: A Survey
Ding, J., Nemati, M., Ranaweera, C., & Choi, J. (2020) · IEEE Access · doi:10.1109/access.2020.2985932 · Cited by 374
A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
Figueiredo Prudencio, R., Maximo, M. R. O. A., & Colombini, E. L. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3250269 · Cited by 361
“These quantities are used throughout the RL field, which is conventionally subdivided into three classes of methods: dynamic programming, model free, and model based [31], [32].”
BackgroundApplication of deep reinforcement learning to intrusion detection for supervised problems
Lopez-Martin, M., Carro, B., & Sanchez-Esguevillas, A. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112963 · Cited by 360
Deep learning in drug discovery: opportunities, challenges and future prospects
Lavecchia, A. (2019) · Drug Discovery Today · doi:10.1016/j.drudis.2019.07.006 · Cited by 353
Federated Learning in Edge Computing: A Systematic Survey
Abreha, H. G., Hayajneh, M., & Serhani, M. A. (2022) · Sensors · doi:10.3390/s22020450 · Cited by 342
“DNN learning can not only be supervised, but also semisupervised [51], unsupervised [52], and use reinforcement [53] learning models.”
BackgroundProgress and Opportunities for Machine Learning in Materials and Processes of Additive Manufacturing
Ng, W. L., Goh, G. L., Goh, G. D., Ten, J. S. J., & Yeong, W. Y. (2024) · Advanced Materials · doi:10.1002/adma.202310006 · Cited by 339
Smart industrial robot control trends, challenges and opportunities within manufacturing
Arents, J., & Greitans, M. (2022) · Applied Sciences · doi:10.3390/app12020937 · Cited by 338
“Both types can be divided into two categories based on the learning approach: policy-based learning and value-based learning [69].”
BackgroundA review of deep learning techniques used in agriculture
Attri, I., Awasthi, L. K., Sharma, T. P., & Rathee, P. (2023) · Ecological Informatics · doi:10.1016/j.ecoinf.2023.102217 · Cited by 336
Revisiting Small Batch Training for Deep Neural Networks
Masters, D. A., & Luschi, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.07612 · Cited by 333
“…enabled significant advances in a number of applications, including computer vision, speech recognition and natural language processing, and in reinforcement learning for robotic control and game playing (LeCun et al., 2015; Schmidhuber, 2015; Goodfellow et al., 2016; Arulkumaran et al., 2017).”
BackgroundA Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers
Placed, J. A., Strader, J., Carrillo, H., Atanasov, N., Indelman, V., Carlone, L., & Castellanos, J. A. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3248510 · Cited by 332
“See [197] and [198] for a survey on the methods.”
MethodA survey on data‐efficient algorithms in big data era
Adadi, A. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00419-9 · Cited by 331
“[112] two main RL approaches can be distinguished: (i) methods based on value functions which are based on estimating the value (expected return) of being in a given state.”
A Comprehensive Review of Deep Learning: Architectures, Recent Advances, and Applications
Mienye, I. D., & Swart, T. G. (2024) · Information · doi:10.3390/info15120755 · Cited by 330
“…policy by updating its action-value function: where Q ( s , a ) represents the action-value function that calculates the expected return of taking action a in state s , r t is the immediate reward received, and γ is the discount factor that values future rewards relative to immediate rewards [95].”
Cyber security meets artificial intelligence: a survey
Li, J.-H. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1800573 · Cited by 329
Deep reinforcement learning in recommender systems: A survey and new perspectives
Chen, X., Yao, L., McAuley, J., Zhou, G., & Wang, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110335 · Cited by 323
Drone Deep Reinforcement Learning: A Review
Azar, A. T., Koubaa, A., Ali Mohamed, N., Ibrahim, H. A., Ibrahim, Z. F., Kazim, M., Ammar, A., Benjdira, B., Khamis, A. M., Hameed, I. A., et al. (2021) · Electronics · doi:10.3390/electronics10090999 · Cited by 314
“The main objective of any reinforcement learning algorithm is to let the agent learn quickly the optimal policy, which is represented by the symbol π, that achieves the targeted task accurately, and thus results in the highest reward value [20].”
BackgroundMethodInfluentialDeep Reinforcement Learning for Internet of Things: A Comprehensive Survey
Chen, W., Qiu, X., Cai, T., Dai, H.-N., Zheng, Z., & Zhang, Y. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3073036 · Cited by 311
“Another promising advent for future work is model-based DRL, which tends to be more data-efficient compared with model-free DRL [8].”
BackgroundMethodAI for UAV-Assisted IoT Applications: A Comprehensive Review
Cheng, N., Wu, S., Wang, X., Yin, Z., Li, C., Chen, W., & Chen, F. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3268316 · Cited by 307
“…(RNNs), and generative adversarial networks (GANs), as well as reinforcement learning (RL) and deep RL (DRL) algorithms that enable adaptation to dynamic environments and real-time decision making, such as Q -learning, deep Q -learning (DQN), deep deterministic policy gradient (DDPG), etc. [16].”
BackgroundMulti-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks
Wu, T., Zhou, P., Liu, K., Yuan, Y., Wang, X., Huang, H., & Wu, D. O. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2997896 · Cited by 307
“, Deep Reinforcement Learning (DRL) [17], [18].”
A review of reinforcement learning for autonomous building energy management
Mason, K., & Grijalva, S. (2019) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2019.07.019 · Cited by 306
Yang, Z., Yang, H., Tian, T., Deng, D., Hu, M., Ma, J., Gao, D., Zhang, J., Ma, S., Yang, L., et al. (2023) · Ultrasonics · doi:10.1016/j.ultras.2023.107014 · Cited by 302
Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., & Shen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988367 · Cited by 298
“Different from existing surveys on DRL [25], [26], we explain”
MethodDeep Cybersecurity: A Comprehensive Overview from Neural Network and Deep Learning Perspective
Sarker, I. H. (2021) · doi:10.20944/preprints202102.0340.v1 · Cited by 295
Miglani, A., & Kumar, N. (2019) · Vehicular Communications · doi:10.1016/j.vehcom.2019.100184 · Cited by 295
Deep Cybersecurity: A Comprehensive Overview from Neural Network and Deep Learning Perspective
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00535-6 · Cited by 289
A Survey of Deep Reinforcement Learning in Video Games
Shao, K., Tang, Z., Zhu, Y., Li, N., & Zhao, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10944 · Cited by 141
Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks
Liang, L., Ye, H., Yu, G., & Li, G. Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2957798 · Cited by 282
“[44] K.”
BackgroundLopez-Perez, D., De Domenico, A., Piovesan, N., Xinli, G., Bao, H., Qitao, S., & Debbah, M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3142532 · Cited by 282
Du, J., Yu, F. R., Lu, G., Wang, J., Jiang, J., & Chu, X. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3003449 · Cited by 278
“Asynchronous execution is a promising way to replace experience replay, where multiple agents work in parallel employing different exploration policies to learn from the environment, so asynchronous execution can decorrelate data since parallel agents will experience different states and thus can stabilize the trainin…”
BackgroundMethodInfluentialFinTech: a literature review of emerging financial technologies and applications
Kou, G., & Lu, Y. (2025) · Financial Innovation · doi:10.1186/s40854-024-00668-6 · Cited by 331
“For example, a virtual trader (agent) follows certain trading rules (actions) in a specific market (environment) to maximize its profits (reward) (Arulkumaran et al. 2017; François-Lavet et al. 2018).”
Neural Network-Based Uncertainty Quantification: A Survey of Methodologies and Applications
Kabir, H. M. D., Khosravi, A., Hosen, M. A., & Nahavandi, S. (2018) · IEEE Access · doi:10.1109/access.2018.2836917 · Cited by 272
“Such as, recently developed game strategic systems are applying the upper confidence bound (UCB) to optimize the exploration time [51]–[53].”
BackgroundDeep learning for insider threat detection: Review, challenges and opportunities
Yuan, S., & Wu, X. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102221 · Cited by 271
UAV Communications Based on Non-Orthogonal Multiple Access
Liu, Y., Qin, Z., Cai, Y., Gao, Y., Li, G. Y., & Nallanathan, A. (2019) · IEEE Wireless Communications · doi:10.1109/mwc.2018.1800196 · Cited by 263
Learning an Approximate Model Predictive Controller With Guarantees
Hertneck, M., Kohler, J., Trimpe, S., & Allgower, F. (2018) · IEEE Control Systems Letters · doi:10.1109/lcsys.2018.2843682 · Cited by 261
“Neural networks have recently been increasingly popular as control policies for complex tasks, [12].”
Deep reinforcement learning in computer vision: a comprehensive survey
Le, N., Rathour, V. S., Yamazaki, K., Luu, K., & Savvides, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10061-9 · Cited by 261
Adaptive Edge Association for Wireless Digital Twin Networks in 6G
Lu, Y., Maharjan, S., & Zhang, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3098508 · Cited by 258
“To fulfill the requirements of 6G communications, deep reinforcement learning [20] and digital twin [21], [22] will be two promising enabling technologies for MEC systems.”
Deep Learning for Video Game Playing
Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896986 · Cited by 257
“[2].”
A Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 257
“In effect, RL provide the agent with the ability to do experiments to get a better understanding of the environment, enabling the agent to learn even complex and high-level causal relationships [456].”
Edge Intelligence in Intelligent Transportation Systems: A Survey
Gong, T., Zhu, L., Yu, F. R., & Tang, T. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3275741 · Cited by 256
A Survey on Adaptive 360° Video Streaming: Solutions, Challenges and Opportunities
Yaqoob, A., Bi, T., & Muntean, G.-M. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3006999 · Cited by 254
“Deep learning enables RL to optimize an aggregated reward further using multi-faceted state and action spaces [138].”
BackgroundExplainable Artificial Intelligence for 6G: Improving Trust between Human and Machine
Guo, W. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.2000050 · Cited by 253
Medical deep learning—A systematic meta-review
Egger, J., Gsaxner, C., Pepe, A., Pomykala, K. L., Jonske, F., Kurz, M., Li, J., & Kleesiek, J. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106874 · Cited by 251
Metadata
- Authors
- Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A.
- Affiliations
- University of Oxford (GB)
- Institution of Engineering and Technology (GB)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-11
- Volume
- 34
- Issue
- 6
- Pages
- 26-38
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W3100789280
- Cited by
- 4,733
- References
- 182
- Citations by year
- 2026: 444 · 2025: 855 · 2024: 806 · 2023: 749 · 2022: 590 · 2021: 511 · 2020: 388 · 2019: 260 · 2018: 122 · 2017: 6
- FWCI
- 126.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, value-based methods, deep Q-network, trust region policy optimization, asynchronous advantage actor-critic, visual understanding, autonomous systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.