Deep Reinforcement Learning: A Brief Survey
Open accessThe DOI matches its Crossref record; 177 of the 182 references it cites were found in a registry or an index.
Abstract
Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higher-level understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via RL. To conclude, we describe several current areas of research within the field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Foundational surveys on RL [7,72] provide broad overviews of classical and deep RL, respectively, but do not focus on human feedback or LLMs.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [7]
In the citing paper
“Classical approaches employ reinforcement learning (RL) [6, 7] and imitation learning [8-10] that operate on primitive actions, which could be challenging for systematic exploration [11-15], interpretability [16-18], and generalization [19-21].”
From Voyager: An Open-Ended Embodied Agent with Large Language Models (2023) · cites this paper as [7]
In the citing paper
“This can be frequently modeled as decision making and learning in a Markov decision process (MDP) [160].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [160]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 2×
“The workhorse of DRL, however, remains backpropagation [162, 111].”
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 10×
“[34] train locally linear models for use with the NAF algorithm— the continuous equivalent of the DQN [71]—to improve the algorithm’s sample complexity in the robotic domain where”
InfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“(temporal) credit assignment problem [115].”
InfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 1×
“[22] devised a method for calculating gradients to optimize stochastic policies by “reparameterizing” [30], [60] the stochasticity away from the network, thereby allowing standard gradients to be used (instead of the high-variance REINFORCE estimator [97]).”
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 7×
“dynamic programming over all possible actions (Figure 3, top left), sample trajectories for heuristic search (as was done by AlphaGo [108]), or even perform an exhaustive search (Figure 3, top right).”
InfluentialDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“technique known as distillation [40] has been utilised to both transfer policies learned by large DQNs to smaller DQNs, and transfer policies learned by several DQNs trained on separate games to one single DQN [85, 95].”
Cited by
Items 301–400 of 1,021
A survey on human-aware robot navigation
Möller, R., Furnari, A., Battiato, S., Härmä, A., & Farinella, G. M. (2021) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103837 · Cited by 143
Al-Kharusi, G., Dunne, N., Little, S., & Levingstone, T. J. (2022) · Bioengineering · doi:10.3390/bioengineering9100561 · Cited by 127
“With the help of these two strategies, the model may gradually learn more about the environment and grasp the inputs that lead to favourable rewards, thus arriving at optimal answers [99].”
MethodMalBoT-DRL: Malware Botnet Detection Using Deep Reinforcement Learning in IoT Networks
Al-Fawa’reh, M., Abu-Khalaf, J., Szewczyk, P., & Kang, J. J. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3324053 · Cited by 144
“This RL approach uses two networks, online DQN, and target DQN, to stabilize the learning process [75], [89].”
MethodIntelligent integrated sensing and communication: a survey
Zhang, J., Lu, W., Xing, C., Zhao, N., Al‐Dhahir, N., Karagiannidis, G. K., & Yang, X. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4205-8 · Cited by 130
Toward a Smart Cloud: A Review of Fault-Tolerance Methods in Cloud Systems
Mukwevho, A. M., & Çelik, T. (2018) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2018.2816644 · Cited by 148
“The key strength of this approach is the ability to handle large data and therefore enable RL to be extended to problem areas that were previously intractable [96].”
BackgroundDeep learning in electron microscopy
Ede, J. M. (2021) · Machine Learning Science and Technology · doi:10.1088/2632-2153/abd614 · Cited by 144
Reinforcement learning in spacecraft control applications: Advances, prospects, and challenges
Tipaldi, M., Iervolino, R., & Massenio, P. R. (2022) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2022.07.004 · Cited by 141
Tang, X., Chen, J., Qin, Y., Liu, T., Yang, K., Khajepour, A., & Li, S. (2024) · Chinese Journal of Mechanical Engineering · doi:10.1186/s10033-024-01026-4 · Cited by 135
“When the convergence of the mean reward to its maximum, it means the acquisition of the optimal policy within the current environment [47].”
BackgroundA Comprehensive Survey of Incentive Mechanism for Federated LearningPreprint
Zeng, R., Zeng, C. L., Wang, X., Li, B., & Chu, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.15406 · Cited by 132
Machine Learning Methods for Data Association in Multi-Object Tracking
Emami, P., Pardalos, P. M., Elefteriadou, L., & Ranka, S. (2020) · ACM Computing Surveys · doi:10.1145/3394659 · Cited by 128
“rcement learning agent to select an action from an action space of size n!. Furthermore, if the agent’s policy is parameterized by a deep neural network, as is the case in deep reinforcement learning [3], the output of the policy network (if searching directly in the space of valid solutions) is a permutation matrix;…”
BackgroundMekrache, A., Bradai, A., Moulay, E., & Dawaliby, S. (2022) · Vehicular Communications · doi:10.1016/j.vehcom.2021.100398 · Cited by 110
Multi-Agent Deep Reinforcement Learning for UAVs Navigation in Unknown Complex Environment
Xue, Y., & Chen, W. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3298292 · Cited by 131
Ismail, A. A., Khalifa, N. E. M., & El-Khoribi, R. A. (2025) · Cluster Computing · doi:10.1007/s10586-024-04893-7 · Cited by 134
“Another success was the development of AlphaGo, a hybrid DRL system that defeated a human world champion in Go [37].”
Al-Saffar, M., & Musílek, P. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2972208 · Cited by 127
“Among machine learning mechanisms, Monte Carlo tree search (MCTS) based reinforcement learning (RL) [27], [28], [29] is particularly suitable for solving sequential decision-making problems.”
BackgroundReinforcement Learning-Based Routing Protocols for Vehicular Ad Hoc Networks: A Comparative Survey
Nazib, R. A., & Moh, S. (2021) · IEEE Access · doi:10.1109/access.2021.3058388 · Cited by 126
“RHR [49] uses next-hop neighbors as the available states which ensure the limited size of the Q-table.”
BackgroundMethodEdge Video Analytics: A Survey on Applications, Systems and Enabling Techniques
Xu, R., Razavi, S., & Zheng, R. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3323091 · Cited by 137
“Recently, deep reinforcement learning, a powerful sequential decision tool, has been applied in proactive configuration adaptation [25], [61], [184], [198], [206], [210], [212], [213], [214], [215], [216].”
MethodShen, G., Lei, L., Zhang, X., Li, Z., Cai, S., & Zhang, L. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3245120 · Cited by 124
Model-free reinforcement learning from expert demonstrations: a survey
Ramírez, J., Yu, W., & Perrusquía, A. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10085-1 · Cited by 135
“Different works survey the challenges for real-world applications in RL (Kober et al. 2013; Kormushev et al. 2013; Arulkumaran et al. 2017; Dulac-Arnold et al. 2020; Zhu et al. (2020) , while many others tackle these challenges separately.”
Backgrounddel Real, A. J., Andreiana, D. S., Roldán, Á. O., Bustos, A. H., & Acevedo, L. (2022) · Applied Sciences · doi:10.3390/app122312377 · Cited by 129
“Fusayasu et al. (2022) [146] present a novel application of DDPG in the control of multi-degree-of-freedom spherical actuators, characterised by their difficult control due to their strong non-linearities of torque.”
BackgroundA survey on deep reinforcement learning approaches for traffic signal control
Zhao, H., Dong, C., Cao, J., & Chen, Q. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108100 · Cited by 138
Assessing Generalization in Deep Reinforcement Learning
Packer, C., Gao, K., Kos, J., Krähenbühl, P., Koltun, V., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1810.12282 · Cited by 114
“Deep reinforcement learning (RL) has emerged as an important family of techniques that may support the development of intelligent systems that learn to accomplish goals in a variety of complex real-world environments (Mnih et al., 2015; Arulkumaran et al., 2017).”
BackgroundToward Packet Routing With Fully Distributed Multiagent Deep Reinforcement Learning
You, X., Li, X., Xu, Y., Feng, H., Zhao, J., & Yan, H. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3012832 · Cited by 128
Mahmoud, D., Magolon, M., Boer, J., Elbestawi, M. A., & Mohammadi, M. G. (2021) · Applied Sciences · doi:10.3390/app112411910 · Cited by 124
“Rewards or penalties are given to alter its behavior and improve the model accuracy [112].”
BackgroundTanveer, J., Haider, A., Ali, R., & Kim, A. (2022) · Applied Sciences · doi:10.3390/app12010426 · Cited by 131
Utilizing deep learning models in CSI-based human activity recognition
Shalaby, E., Elshennawy, N. M., & Sarhan, A. M. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06787-w · Cited by 123
“Several methods have been used for classification such as: RF [7], HMM [8], SVM [9] and KNN [10] CNN [17], RNN [18] including LSTM [19] and GRU [20], Auto-Encoder (AE) [21], Boltzmann machine (BM) [22], Restricted Boltzmann Machine (RBM) [23], Deep Belief Network (DBN) [24], Generative Adversarial Network (GAN) [25], a…”
MethodArrowsmith, S., Trugman, D. T., MacCarthy, J., Bergen, K. J., Lumley, D., & Magnani, M. B. (2022) · Reviews of Geophysics · doi:10.1029/2021rg000769 · Cited by 127
“If large volumes of labeled data are not available but labels can be generated on the fly, online learning, which learns on a single pass through data, or reinforcement learning (Arulkumaran et al., 2017), which learns sequentially through a trial-and-error process, offer solutions for big data analysis.”
BackgroundA Roadmap for Integrating Automation with Process Optimization for AI-powered Digital TransformationPreprint
Aldoseri, A., Al‐Khalifa, K. N., & Hamouda, A. M. S. (2023) · Preprints.org · doi:10.20944/preprints202310.1055.v1 · Cited by 156
Reinforcement and deep reinforcement learning for wireless Internet of Things: A survey
Frikha, M. S., Gammar, S. M., Lahmadi, A., & Andrey, L. (2021) · Computer Communications · doi:10.1016/j.comcom.2021.07.014 · Cited by 124
Shoeibi, A., Moridian, P., Khodatars, M., Ghassemi, N., Jafari, M., Alizadehsani, R., Kong, Y., Górriz, J. M., Ramírez, J., Khosravi, A., et al. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106053 · Cited by 136
Lu, J., Han, L., Wei, Q., Wang, X., Dai, X., & Wang, F. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3262132 · Cited by 117
“Deep reinforcement learning (DRL), as a showpiece technique of deep learning, has become an overarching technique for achieving human-level control and decision-making [7], [8], [9], [10] and has achieved abundant achievements in autonomous driving [11], [12], [13], [14].”
BackgroundGenerative AI for Secure Physical Layer Communications: A Survey
Zhao, C., Du, H., Niyato, D. T., Kang, J., Xiong, Z., Kim, D. I., Shen, X., & Letaief, K. B. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3438379 · Cited by 141
“Several studies have demonstrated that anti-jamming communications, enhanced by Deep Reinforcement Learning (DRL) [113], [114], can achieve near-optimal performance in dynamic and unpredictable environments [115], [116], [117].”
Learning Robotic Manipulation through Visual Planning and Acting
Wang, A., Kurutach, T., Liu, K., Abbeel, P., & Tamar, A. (2019) · Robotics: Science and Systems XV · doi:10.15607/rss.2019.xv.074 · Cited by 93
Liu, Y., Wei, Y., Yu, F. R., & Han, Z. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3188826 · Cited by 117
“Therefore, DRL combines the effective representation learning and function approximation properties of DNN, enabling RL to scale to solve problems with high-dimensional state–action spaces [26].”
MethodSafe Reinforcement Learning Using Robust Control Barrier Functions
Emam, Y., Notomista, G., Glotfelter, P., Kira, Z., & Egerstedt, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3216996 · Cited by 174
“REINFORCEMENT Learning (RL) has been used successfully in a number of robotics applications [1], [2].”
Abdulazeez, D. H., & Askar, S. K. (2023) · IEEE Access · doi:10.1109/access.2023.3241881 · Cited by 125
Al-Hamadani, M. N. A., Fadhel, M. A., Alzubaidi, L., & Harangi, B. (2024) · Sensors · doi:10.3390/s24082461 · Cited by 125
“The continuing task aims to maximize the expected cumulative reward gained over an infinite time horizon.”
BackgroundMohammed, M. Q., Kwek, L. C., & Chua, S. C. (2020) · IEEE Access · doi:10.1109/access.2020.3027923 · Cited by 128
Online Optimal Power Scheduling of a Microgrid via Imitation Learning
Gao, S., Xiang, C., Yu, M., Tan, K. T., & Lee, T. H. (2021) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2021.3122570 · Cited by 116
Intelligent problem-solving as integrated hierarchical reinforcement learning
Eppe, M., Gumbsch, C., Kerzel, M., Nguyen, P. D. H., Butz, M. V., & Wermter, S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-021-00433-9 · Cited by 143
Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
Latif, S., Rana, R. K., Khalifa, S., Jurdak, R., Qadir, J., & Schuller, B. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00378 · Cited by 66
“Methods such as deep Q-learning have significantly advanced the field [224].”
Special Issue “On Defining Artificial Intelligence”—Commentaries and Author’s Response
Monett, D., Lewis, C. W. P., Thórisson, K. R., Bach, J., Baldassarre, G., Granato, G., Berkeley, I. S. N., Chollet, F., Crosby, M., Shevlin, H., et al. (2020) · Journal of Artificial General Intelligence · doi:10.2478/jagi-2020-0003 · Cited by 84
Abiko, Y., Saito, T., Ikeda, D., Ohta, K., Mizuno, T., & Mineno, H. (2020) · IEEE Access · doi:10.1109/access.2020.2986050 · Cited by 122
“DRL is applied to various fields such as video games, robots, and autonomous cars to learn the optimal state control through repetitious trial and error [8].”
BackgroundMethodGao, Y., Yang, J., Yang, M. Y., & Li, Z. (2020) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2020.2986412 · Cited by 110
Bharati, S., & Podder, P. (2022) · Security and Communication Networks · doi:10.1155/2022/8951961 · Cited by 133
Alfaverh, F., Denai, M. A., & Sun, Y. (2022) · Electric Power Systems Research · doi:10.1016/j.epsr.2022.108949 · Cited by 118
Guan, Y., Zou, S., Peng, H., Ni, W., Sun, Y., & Gao, H. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3320796 · Cited by 116
“DRL can make multistep decisions to obtain optimal rewards in dynamic environments [11].”
BackgroundZhao, R., Li, Y., Fan, Y., Gao, F., Tsukada, M., & Gao, Z. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3452480 · Cited by 140
“…vehicle control, and end-to-end driving methods in autonomous vehicles (AVs) during the deep learning era, but with limited exploration of DRL. Surveys by Wang et al. [30] and Arulkumaran et al. [31] provide comprehensive reviews of general DRL methods but lack specific applications to AD.”
A Survey on Causal Reinforcement Learning
Zeng, Y., Cai, R., Sun, F., Huang, L., & Hao, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3403001 · Cited by 147
Xie, J., Ajagekar, A., & You, F. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121162 · Cited by 109
Huang, J., Huang, S., Moghaddam, S. K., Lu, Y., Wang, G., Yan, Y., & Shi, X. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3431095 · Cited by 109
A Review of Safe Reinforcement Learning Methods for Modern Power Systems
Su, T., Wu, T., Zhao, J., Scaglione, A., & Xie, L. (2025) · Proceedings of the IEEE · doi:10.1109/jproc.2025.3584656 · Cited by 136
“In general, these constraints can be expressed as follows: max where the control action must fulfill both the m equality and n inequality constraints.”
Model‐based vs data‐driven adaptive control: An overview
Benosman, M. (2018) · International Journal of Adaptive Control and Signal Processing · doi:10.1002/acs.2862 · Cited by 119
Optimizing and Updating LoRa Communication Parameters: A Machine Learning Approach
Sandoval, R. M., García-Sánchez, A.-J., & García-Haro, J. (2019) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2019.2927759 · Cited by 108
Li, K., Jiu, B., Wang, P., Liu, H., & Shi, Y. (2021) · Signal Processing · doi:10.1016/j.sigpro.2021.108130 · Cited by 116
An Energy and Carbon Footprint Analysis of Distributed and Federated Learning
Savazzi, S., Rampa, V., Kianoush, S., & Bennis, M. (2022) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2022.3186439 · Cited by 135
“On the other hand, in continual learning applications, devices produce new training data continuously [2], [21]: data thus needs to be moved while learning is in progress, therefore it is 1 < α ≤ n .”
BackgroundMethodInfluentialDistributed Massive MIMO for LEO Satellite Networks
Abdelsadek, M. Y., Kurt, G. K., & Yanıkömeroğlu, H. (2022) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2022.3219419 · Cited by 126
“This is in addition to the other drawbacks of using model-free algorithms as discussed in [57].”
MethodEngineering Semantic Communication: A Survey
Wheeler, D., & Natarajan, B. (2023) · IEEE Access · doi:10.1109/access.2023.3243065 · Cited by 121
“Another popular approach to ML is reinforcement learning (RL) [85].”
BackgroundAge of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning
Akbari, M., Abedi, M. R., Joda, R., Pourghasemian, M., Mokari, N., & Erol‐Kantarci, M. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087264 · Cited by 108
“Many recent works show that the machine-learning-based resource allocation outperforms conventional methods [5], [31], [32].”
BackgroundA DQN-based intelligent control method for heavy haul trains on long steep downhill section
Liu, W., Su, S., Tang, T., & Wang, X. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103249 · Cited by 107
Cost-aware job scheduling for cloud instances using deep reinforcement learning
Cheng, F., Huang, Y., Tanpure, B., Sawalani, P., Cheng, L., & Liu, C. (2021) · Cluster Computing · doi:10.1007/s10586-021-03436-8 · Cited by 107
Generative AI Empowered Network Digital Twins: Architecture, Technologies, and Applications
Li, T., Long, Q., Chai, H., Zhang, S., Jiang, F., Liu, H., Huang, W., Jin, D., & Li, Y. (2025) · ACM Computing Surveys · doi:10.1145/3711682 · Cited by 146
“Specifically, RL allows an agent to learn by interacting with its environment, using trial and error to gradually develop an optimal policy that guides decision-making to maximize long-term rewards [15].”
Neural Architecture Search Survey: A Hardware Perspective
Chitty-Venkata, K. T., & Somani, A. K. (2023) · ACM Computing Surveys · doi:10.1145/3524500 · Cited by 106
Likmeta, A., Metelli, A. M., Tirinzoni, A., Giol, R., Restelli, M., & Romano, D. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103568 · Cited by 106
Advances of Artificial Intelligence in Anti-Cancer Drug Design: A Review of the Past Decade
Wang, L., Song, Y., Wang, H., Zhang, X., Wang, M., He, J., Li, S., Zhang, L., Li, K., & Cao, L. (2023) · Pharmaceuticals · doi:10.3390/ph16020253 · Cited by 117
Review of machine learning in robotic grasping control in space application
Jahanshahi, H., & Zhu, Z. H. (2024) · Acta Astronautica · doi:10.1016/j.actaastro.2024.04.012 · Cited by 114
Chen, Y., Sun, Y., Wang, C., & Taleb, T. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3164441 · Cited by 111
“the policy gradient method, which mainly includes stochastic policy gradient and deterministic policy gradient [41].”
MethodAdvancing 3D bioprinting through machine learning and artificial intelligence
Ramesh, S., Deep, A., Tamayol, A., Kamaraj, A. B., Mahajan, C., & Madihally, S. V. (2024) · Bioprinting · doi:10.1016/j.bprint.2024.e00331 · Cited by 113
Deep Reinforcement Learning for Load-Balancing Aware Network Control in IoT Edge Systems
Liu, Q., Xia, T., Cheng, L., van Eijk, M., Özçelebi, T., & Mao, Y. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3116863 · Cited by 115
“As an emerging technology, DRL [4] is very suitable to cope with complex optimization problems with high-dimensional state spaces.”
Deep Reinforcement Learning for Anomaly Detection: A Systematic Review
Arshad, K., Ali, R. F., Muneer, A., Aziz, I. A., Naseer, S., Khan, N. S., & Taib, S. M. (2022) · IEEE Access · doi:10.1109/access.2022.3224023 · Cited by 121
Sustainable Robotic Joints 4D Printing with Variable Stiffness Using Reinforcement Learning
Mohammadi, M., Kouzani, A. Z., Bodaghi, M., Long, J. M., Khoo, S., Xiang, Y., & Zolfagharian, A. (2023) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2023.102636 · Cited by 107
Wang, Z., Xiao, F., Yi, R., Li, Y., & Xu, Y. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.123414 · Cited by 112
Sejan, M. A. S., Rahman, M. H., Shin, B.-S., Oh, J.-H., You, Y., & Song, H. (2022) · Sensors · doi:10.3390/s22145405 · Cited by 115
“Currently, RL is also applied to the field of robotics, allowing control policies for robots to be learned directly from camera inputs in the real world [59].”
BackgroundID-RDRL: a deep reinforcement learning-based feature selection intrusion detection model
Ren, K., Zeng, Y., Cao, Z., & Zhang, Y. (2022) · Scientific Reports · doi:10.1038/s41598-022-19366-3 · Cited by 103
A review of the applications of multi-agent reinforcement learning in smart factories
Bahrpeyma, F., & Reichelt, D. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.1027340 · Cited by 126
“Due to the limitations of this paper, here, we only brie fl y describe QL as the most popular RL technique and refer the reader to Arulkumaran et al., (2017) for other RL methods.”
MethodDistributional Reinforcement Learning
Bellemare, M. G., Dabney, W., & Rowland, M. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/14207.001.0001 · Cited by 272
Learning-based robotic grasping: A review
Xie, Z., Liang, X., & Canale, R. (2023) · Frontiers in Robotics and AI · doi:10.3389/frobt.2023.1038658 · Cited by 114
“…et al., 2017), deep deterministic policy gradient (DDPG) methods (LillicrapHunt et al., 2015), deep Q-learning (MnihKavukcuoglu et al., 2013), Monte Carlo (MC) policy evaluation (Xie and Zhong, 2016a; Arulkumaran et al., 2017), and more robust-corrected Monte Carlo methods (Betancourt, 2017).”
MethodTang, X., Chen, J., Yang, K., Toyoda, M., Liu, T., & Hu, X. (2022) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2022.3141780 · Cited by 102
Time Limits in Reinforcement LearningPreprint
Pardo, F., Tavakoli, A., Levdik, V., & Kormushev, P. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00378 · Cited by 198
“…range of novel and popular benchmark domains using tabular Q-learning and Proximal Policy Optimization (PPO), a modern deep reinforcement learning (Arulkumaran et al., 2017; Hen-derson et al., 2017) algorithm which has recently been used to achieve state-of-the-art performance in many domains…”
Yao, H., Mai, T., Xu, X., Zhang, P., Li, M., & Liu, Y. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2859480 · Cited by 106
Context-Aware Taxi Dispatching at City-Scale Using Deep Reinforcement Learning
Liu, Z., Li, J., & Wu, K. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3030252 · Cited by 112
“To tackle this tough problem, many research efforts have been already made [29], [49], while most of the recent advances [9], [18], [30], [40] primarily rely on deep reinforcement learning (DRL) [5] to directly learn the best dispatching policies rather than accurately modeling taxi demand/supply.”
BackgroundInfluentialYang, L., Li, M., Si, P., Yang, R., Sun, E., & Zhang, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3030646 · Cited by 108
Autonomic Robotic Ultrasound Imaging System Based on Reinforcement Learning
Ning, G., Zhang, X., & Liao, H. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3054413 · Cited by 101
Dynamic scheduling of tasks in cloud manufacturing with multi-agent reinforcement learning
Wang, X., Li, Z., Liu, Y., Li, F., Chen, Z., Zhao, C., & Bai, T. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.08.004 · Cited by 106
Quality-Oriented Hybrid Path Planning Based on A* and Q-Learning for Unmanned Aerial Vehicle
Li, D., Yin, W., Wong, W. E., Jian, M., & Chau, M. (2021) · IEEE Access · doi:10.1109/access.2021.3139534 · Cited by 100
Intelligent Reflecting Surface Configurations for Smart Radio Using Deep Reinforcement Learning
Wang, W., & Zhang, W. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3180787 · Cited by 104
“The coefficients w of Q ( s, a ; w ) are the weights of the DNN, and the DNN is termed as deep Q-network (DQN) [36], [37]. e The trajectory segment h S t = s, A t = a, R t +1 = R as → s ′ , S t +1 = s ′ in (12) constitutes an “experience sample” that will be used to train the DQN, and in accordance…”
MethodHuang, C., Zhang, H., Wang, L., Luo, X., & Song, Y. (2022) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2021.000394 · Cited by 100
Singh, V. K., Nanavati, B., Kar, A. K., & Gupta, A. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10314-0 · Cited by 116
“One possibility of extending this work could be using the deep Q network (Arulkumaran et al., 2017) that takes advertisement characteristics, user demographics, and history as attributes to build a more advanced RL model.”
BackgroundReinforcement Learning on Graphs: A Survey
Nie, M., Chen, D., & Wang, D. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3222545 · Cited by 113
“Deep Reinforcement Learning (DRL) is poised to revolutionize the field of artificial intelligence and represents a step toward building autonomous systems with a higher-level understanding of the visual world [40].”
BackgroundAI-based methodologies for exoskeleton-assisted rehabilitation of the lower limb: a review
Coser, O., Tamantini, C., Soda, P., & Zollo, L. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1341580 · Cited by 109
“Given the high quantity of algorithms available the choice strictly depends on the given problem (Arulkumaran et al., 2017).”
BackgroundLiu, Z., Du, H., Lin, J., Gao, Z., Huang, L., Hosseinalipour, S., & Niyato, D. T. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3486728 · Cited by 120
“Denoting the objective function in (43) as Ψ( F ( t )) , by calculating its corresponding second-order derivatives w.r.t. f rsu k ( t ) , we can obtain where Thus, ( P 3 ( t )) is a convex optimization problem and can be effi-ciently solved by Karush-Kuhn-Tucker (KKT) conditions [17] as below.”
Hu, Y., Miao, X., Zhang, J., Liu, J., & Pan, E. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.107056 · Cited by 99
openGauss: an autonomous database system
Li, G., Zhou, X., Sun, J., Yu, X., Han, Y., Jin, L., Li, W., Wang, T., & Li, S. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3476311.3476380 · Cited by 112
“DRL can achieve the highest performance, but needs to generate a well-trained model; on the other hand, the global search algorithm does not need to be trained in advance and can directly search knob configurations and tune the database.”
InfluentialYan, J., Huang, Y., Gupta, A. K., Gupta, A., Liu, C., Li, J., & Cheng, L. (2022) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2022.107688 · Cited by 99
Artificial intelligence for quantum computing
Alexeev, Y., Farag, M. H., Patti, T. L., Wolf, M. E., Ares, N., Aspuru‐Guzik, A., Benjamin, S. C., Cai, Z., Cao, S., Chamberland, C., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-65836-3 · Cited by 139
“DNN architectures are incredibly diverse and ever expanding, and we here overview but a handful of them that will prove germane in this review.”
A survey on visual navigation for artificial agents with deep reinforcement learning
Zeng, F., Wang, C., & Ge, S. S. (2020) · IEEE Access · doi:10.1109/access.2020.3011438 · Cited by 98
“Multiple surveys on visual navigation [4], [17], [18] have been published, and some surveys review a wide range of models and applications for DRL [19], [20].”
BackgroundDeep Learning in Lane Marking Detection: A Survey
Zhang, Y., Lu, Z., Zhang, X., Xue, J., & Liao, Q. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3070111 · Cited by 115
Grid-optimized UAV indoor path planning algorithms in a complex environment
Han, B., Qu, T., Tong, X., Jiang, J., Sisi Zlatanova, B., Wang, H., & Cheng, C. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102857 · Cited by 98
Manchella, K., Umrawal, A. K., & Aggarwal, V. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3048361 · Cited by 97
“2) Model-free Approaches for Transportation: Model-free approaches have surged in popularity across a variety of fields with the development of Deep Reinforcement Learning [15].”
BackgroundDeep Reinforcement Learning-Based Demand Response for Smart Facilities Energy Management
Lu, R., Bai, R., Luo, Z., Jiang, J., Sun, M., & Zhang, H. (2021) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2021.3104596 · Cited by 97
Predictive trajectory planning for autonomous vehicles at intersections using reinforcement learning
Zhang, E., Zhang, R., & Masoud, N. (2023) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2023.104063 · Cited by 108
Metadata
- Authors
- Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A.
- Affiliations
- University of Oxford (GB)
- Institution of Engineering and Technology (GB)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-11
- Volume
- 34
- Issue
- 6
- Pages
- 26-38
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W3100789280
- Cited by
- 4,733
- References
- 182
- Citations by year
- 2026: 444 · 2025: 855 · 2024: 806 · 2023: 749 · 2022: 590 · 2021: 511 · 2020: 388 · 2019: 260 · 2018: 122 · 2017: 6
- FWCI
- 126.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, value-based methods, deep Q-network, trust region policy optimization, asynchronous advantage actor-critic, visual understanding, autonomous systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.