Deep Reinforcement Learning: A Brief Survey
Open accessThe DOI matches its Crossref record; 177 of the 182 references it cites were found in a registry or an index.
Abstract
Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higher-level understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep RL, including the deep Q-network (DQN), trust region policy optimization (TRPO), and asynchronous advantage actor critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via RL. To conclude, we describe several current areas of research within the field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Foundational surveys on RL [7,72] provide broad overviews of classical and deep RL, respectively, but do not focus on human feedback or LLMs.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [7]
In the citing paper
“Classical approaches employ reinforcement learning (RL) [6, 7] and imitation learning [8-10] that operate on primitive actions, which could be challenging for systematic exploration [11-15], interpretability [16-18], and generalization [19-21].”
From Voyager: An Open-Ended Embodied Agent with Large Language Models (2023) · cites this paper as [7]
In the citing paper
“This can be frequently modeled as decision making and learning in a Markov decision process (MDP) [160].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [160]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 2×
“The workhorse of DRL, however, remains backpropagation [162, 111].”
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 10×
“[34] train locally linear models for use with the NAF algorithm— the continuous equivalent of the DQN [71]—to improve the algorithm’s sample complexity in the robotic domain where”
InfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 7×
“(temporal) credit assignment problem [115].”
InfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 1×
“[22] devised a method for calculating gradients to optimize stochastic policies by “reparameterizing” [30], [60] the stochasticity away from the network, thereby allowing standard gradients to be used (instead of the high-variance REINFORCE estimator [97]).”
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 7×
“dynamic programming over all possible actions (Figure 3, top left), sample trajectories for heuristic search (as was done by AlphaGo [108]), or even perform an exhaustive search (Figure 3, top right).”
InfluentialDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“technique known as distillation [40] has been utilised to both transfer policies learned by large DQNs to smaller DQNs, and transfer policies learned by several DQNs trained on separate games to one single DQN [85, 95].”
Cited by
Items 1,001–1,021 of 1,021
Collaborative Edge Intelligence Service Provision in Blockchain Empowered Urban Rail Transit Systems
Liang, H., Zhu, L., & Yu, F. R. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3294400 · Cited by 35
“With the superiority of solving complex sequential decision-making problems, the reinforcement learning (RL) method has achieved great success in service scheduling optimization via computation offloading [13], [14], [15].”
MethodDeep Reinforcement Learning for Intelligent Penetration Testing Path Design
Yi, J., & Liu, X. (2023) · Applied Sciences · doi:10.3390/app13169467 · Cited by 41
Discriminative Particle Filter Reinforcement Learning for Complex Partial ObservationsPreprint
Ma, X., Karkus, P., Hsu, D., Lee, W. S., & Ye, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09884 · Cited by 50
“However, natural real-world environments remain challenging for current DRL methods (Arulkumaran et al., 2017), in part because they require (i) reasoning in a partially observable environment and (ii) reasoning with complex observations, such as visually rich images.”
Far, A. Z., Far, M. Z., Gharibzadeh, S., Naeini, H. K., Amini, L., Zangeneh, S., Rahimi, M., & Asadi, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.16444 · Cited by 45
“Their study explores the application areas of RL. Resources management and other networking-related fields have a number of applications for RL. [86] A discussion of the applications of Open Radio Access Networks (O-RAN) in supporting next-generation smart world IoT systems, providing a problem…”
Large-Scale Inventory Optimization: A Recurrent Neural Networks–Inspired Simulation Approach
Wang, T., & Hong, L. J. (2022) · INFORMS journal on computing · doi:10.1287/ijoc.2022.1253 · Cited by 43
ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation
Xia, F., Li, C., Martín-Martín, R., Litany, O., Toshev, A., & Savarese, S. (2020) · arXiv · doi:10.48550/arxiv.2008.07792 · Cited by 33
“Moreover, MGs excel at answering “how” to move to a point, but not “where” to move to achieve the task in hand based on current observations, where deep reinforcement learning (deep RL [3, 4]) has shown strong results.”
BackgroundDeep Active Inference for Partially Observable MDPs
van der Himst, O., & Lanillos, P. (2020) · arXiv · doi:10.48550/arxiv.2009.03622 · Cited by 27
Das, S. K., Mudi, R., Rahman, M. S., Rabie, K. M., & Li, X. (2024) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2024.3466858 · Cited by 27
“MARL is often designed as a decision-making and training model within a discrete-time stochastic control process, such as the MDP [65].”
BackgroundZheng, J., Cao, H., Chen, D., Ansari, R., Chu, K.-C., & Huang, M.-C. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.2986768 · Cited by 17
Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system
Khater, A. A., Fekry, M., El-Bardini, M., & El-Nagar, A. M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10741-x · Cited by 17
“Random samples are selected from the M as experienced samples during each specific sample time to train a Q - network [24].”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2018) · arXiv · doi:10.48550/arxiv.1810.07862 · Cited by 17
“discuss the applications of DRL such as [7] and [8], but they are specifically for computer vision and natural language processing.”
Sim-to-Real in Reinforcement Learning for Everyone
Vacaro, J., Marques, G., Oliveira, B., Paz, G., Paula, T., Staehler, W., & Murphy, D. (2019) · 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE) · doi:10.1109/lars-sbr-wre48964.2019.00060 · Cited by 13
Vessel Classification through Convolutional Neural Networks using Passive Sonar Spectrogram Images
Cinelli, L., Chaves, G., & Lima, M. (2018) · Anais de XXXVI Simpósio Brasileiro de Telecomunicações e Processamento de Sinais · doi:10.14209/sbrt.2018.340 · Cited by 13
Survey of Computerized Adaptive Testing: A Machine Learning Perspective
Zhuang, Y., Liu, Q., Bi, H., Huang, Z., Huang, W., Li, J., Yu, J., Liu, Z., Hu, Z., Hong, Y., et al. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3672850 · Cited by 9
“It has been successfully applied in various domains, including robotics, autonomous vehicles, education, and healthcare [136], [137].”
A multi-agent deep reinforcement learning based energy management for behind-the-meter resources
Wilk, P., Wang, N., & Li, J. (2022) · The Electricity Journal · doi:10.1016/j.tej.2022.107129 · Cited by 9
Colas, J. T., O’Doherty, J. P., & Grafton, S. T. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1011950 · Cited by 6
Thompson sampling via local uncertainty
Wang, Z., & Zhou, M. (2019) · arXiv · doi:10.48550/arxiv.1910.13673 · Cited by 5
“There has been significant recent interest in employing deep neural networks to better solve sequential decision-making problems, such as these in reinforcement learning (Mnih et al., 2013; 2015; 2016; Arulkumaran et al., 2017; FrançoisLavet et al., 2018) and contextual bandits (Riquelme et al.”
BackgroundWang, B., Gong, X., Wang, Y., Xu, R., & Huang, H. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3497185 · Cited by 3
“However, the time horizon of data processing is one of the key parameters for feasibility and convergence in the machine learning [24].”
A Machine with Short-Term, Episodic, and Semantic Memory Systems
Kim, T., Cochez, M., Francois-Lavet, V., Neerincx, M., & Vossen, P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25075 · Cited by 3
Dizon-Paradis, O., Wormald, S., Capecci, D., Bhandarkar, A., & Woodard, D. (2023) · doi:10.36227/techrxiv.23739099 · Cited by 1
Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities
Bei, Y., Zhang, W., Wang, S., Chen, W., Zhou, S., Chen, H., Li, Y., Bu, J., Pan, S., Yu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.18019
“The paradigm focuses on sequential decision making, that is, optimizing a chain of actions for long-term return [28], [29].”
Metadata
- Authors
- Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A.
- Affiliations
- University of Oxford (GB)
- Institution of Engineering and Technology (GB)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2017-11
- Volume
- 34
- Issue
- 6
- Pages
- 26-38
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W3100789280
- Cited by
- 4,733
- References
- 182
- Citations by year
- 2026: 444 · 2025: 855 · 2024: 806 · 2023: 749 · 2022: 590 · 2021: 511 · 2020: 388 · 2019: 260 · 2018: 122 · 2017: 6
- FWCI
- 126.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep reinforcement learning, value-based methods, deep Q-network, trust region policy optimization, asynchronous advantage actor-critic, visual understanding, autonomous systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.