Human-level control through deep reinforcement learning
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-networ…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In previous studies, the agents are assumed to act based on simple and heuristic policy functions, and learned in isolated and restricted environments [1][2][3][4][5][6].”
From A survey on large language model based autonomous agents (2024) · cites this paper as [1]
In the citing paper
“This technique was developed in 2013 with Google Deep Mind [55].”
From Review of deep learning: concepts, CNN architectures, challenges, applications, future directions (2021) · cites this paper as [55]
In the citing paper
“Whereas previous reinforcement-learning algorithms often rely only on task-specific learning signals9,10, Dreamer rests predominantly on the unsupervised objective of its world model.”
From Mastering diverse control tasks through world models (2025) · cites this paper as [9]
In the citing paper
“Traditional RL such as AlphaGo Zero and the DQN series (Silver et al., 2017; Mnih et al., 2015; Yue et al., 2023) can continuously improve the performance of a policy in environments like Go and Atari games without an explicit upper bound .”
From Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (2025) · cites this paper as Mnih et al., 2015
In the citing paper
“Unlike previous reinforcement learning algorithms that often rely only on task-specific learning signals 7,8 , Dreamer rests predominantly on the unsupervised objective of its world model.”
From Mastering Diverse Domains through World Models (2023) · cites this paper as [7]
24 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“We use one particularly successful architecture, the deep convolutional network 17 , which uses hierarchical layers of tiled convolutional filters tomimictheeffectsofreceptivefields—inspiredbyHubelandWiesel’s seminalworkonfeedforwardprocessinginearlyvisualcortex 18 —thereby…”
Method(2008) · Journal of Machine Learning Research · Cited by 33,051
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
(2010) · International Conference on Machine Learning · Cited by 13,995
A neural substrate of prediction and reward
Schultz, W., Dayan, P., & Montague, P. R. (1997) · Science · doi:10.1126/science.275.5306.1593 · Cited by 9,388 · cited 1×
“…4,5 , the former evidenced by a wealth of neural data revealingnotableparallelsbetweenthephasicsignalsemittedbydopa-minergicneuronsandtemporaldifferencereinforcementlearning algorithms 3 .”
BackgroundLearning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239 · cited 2×
“Notably, recent advances in deep neural networks 9–11 , in which several layers of nodes areused to buildup progressivelymore abstractrepresentations of the data, have made it possible for artificial neural networks to learn concepts such as object categories directly from raw sensory data.”
BackgroundWatkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Cited by
Items 1,101–1,200 of 1,418
Distributed Deep Reinforcement Learning for Intelligent Load Scheduling in Residential Smart Grids
Chung, H.-M., Maharjan, S., Zhang, Y., & Eliassen, F. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3007167 · Cited by 120
“To further address the discretization issue associated with Q-learning [18]–[23], DQN [24]–[26], and policy gradient [27], [28], the employed model-free approach is similar to model-based methods that provide a continuous value as the solution without discretizing the action space.”
BackgroundMethodInfluentialToward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
“The above examples suggest that at least some such mechanisms could be understood from the perspective of optimizing cost functions.”
BackgroundInfluentialCharacterizing the deployment of deep neural networks on commercial edge devices
Hadidi, R., Cao, J., Xie, Y., Asgari, B., Krishna, T., & Kim, H. (2019) · 2019 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc47752.2019.9041955 · Cited by 118
Deep Reinforcement Adversarial Learning Against Botnet Evasion Attacks
Apruzzese, G., Andreolini, M., Marchetti, M., Venturi, A., & Colajanni, M. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3031843 · Cited by 117
Adversarially Robust Policy Learning: Active construction of physically-plausible perturbations
Mandlekar, A., Zhu, Y., Garg, A., Fei-Fei, L., & Savarese, S. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206245 · Cited by 116
Deep active inference as variational policy gradients
Millidge, B. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102348 · Cited by 115
Assessing Generalization in Deep Reinforcement Learning
Packer, C., Gao, K., Kos, J., Krähenbühl, P., Koltun, V., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1810.12282 · Cited by 114
“Deep reinforcement learning (RL) has emerged as an important family of techniques that may support the development of intelligent systems that learn to accomplish goals in a variety of complex real-world environments (Mnih et al., 2015; Arulkumaran et al., 2017).”
BackgroundAffordances from Human Videos as a Versatile Representation for Robotics
Bahl, S., Mendonca, R., Chen, L., Jain, U., & Pathak, D. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01324 · Cited by 119
“• takuseno/d3rlpy [116] for DQN [76] implementation.”
MethodIteratively Questioning and Answering for Interpretable Legal Judgment Prediction
Zhong, H., Wang, Y., Tu, C., Zhang, T., Liu, Z., & Sun, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5479 · Cited by 113
15 challenges for AI: Or what AI (currently) can’t do
Hagendorff, T., & Wezel, K. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00886-y · Cited by 113
Deep predictive policy training using reinforcement learning
Ghadirzadeh, A., Maki, A., Kragic, D., & Bjorkman, M. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206046 · Cited by 113
Optimization for Reinforcement Learning: From a single agent to cooperative agents
Lee, D., He, N., Kamalaruban, P., & Cevher, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2976000 · Cited by 113
Network planning with deep reinforcement learning
Zhu, H., Gupta, V., Ahuja, S. S., Tian, Y., Zhang, Y., & Jin, X. (2021) · Proceedings of the 2021 ACM SIGCOMM 2021 Conference · doi:10.1145/3452296.3472902 · Cited by 112
Predictive maps in rats and humans for spatial navigation
de Cothi, W., Nyberg, N., Griesbauer, E.-M., Ghanamé, C., Zisch, F., Lefort, J. M., Fletcher, L., Newton, C., Renaudineau, S., Bendor, D., et al. (2022) · Current Biology · doi:10.1016/j.cub.2022.06.090 · Cited by 109
Xie, J., Ajagekar, A., & You, F. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121162 · Cited by 109
A multi-agent reinforcement learning model of common-pool resource appropriation
Perolat, J., Leibo, J. Z., Zambaldi, V., Beattie, C., Tuyls, K., & Graepel, T. (2017) · arXiv · doi:10.48550/arxiv.1707.06600 · Cited by 109
“To that end, we simulated N independent agents, each simultaneously learning via the deep reinforcement learning algorithm of Mnih et al. (2015) [24].”
MethodSelf-citationVR-Goggles for Robots: Real-to-Sim Domain Adaptation for Visual Control
Zhang, J., Tai, L., Yun, P., Xiong, Y., Liu, M., Boedecker, J., & Burgard, W. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2894216 · Cited by 109
Machine Learning for Security in Vehicular Networks: A Comprehensive Survey
Talpur, A., & Gurusamy, M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3129079 · Cited by 107
A Deep Learning-Based Algorithm Identifies Glaucomatous Discs Using Monoscopic Fundus Photographs
Liu, S., Graham, S. L., Schulz, A., Kalloniatis, M., Zangerl, B., Cai, W., Gao, Y., Chua, B., Arvind, H., Grigg, J., et al. (2018) · Ophthalmology Glaucoma · doi:10.1016/j.ogla.2018.04.002 · Cited by 107
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106
“Experience replay has also been widely used in reinforcement learning (Mnih et al., 2015, 2013; Van Hasselt, Guez, & Silver, 2016; Lillicrap et al., 2016; Lin, 1992; Adam, Busoniu, & Babuska, 2011; Foerster et al., 2017; Kapturowski, Ostrovski, Dabney, Quan, & Munos, 2019; Atkinson et al., 2021).”
Likmeta, A., Metelli, A. M., Tirinzoni, A., Giol, R., Restelli, M., & Romano, D. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103568 · Cited by 106
Deep learning generalizes because the parameter-function map is biased towards simple functions
Valle-Pérez, G., Camargo, C. Q., & Louis, A. A. (2018) · arXiv · doi:10.48550/arxiv.1805.08522 · Cited by 105
“These models have been remarkably successful in domains ranging from image recognition and synthesis, to natural language processing, and reinforcement learning (Mnih et al. (2015); LeCun et al. (2015); Radford et al. (2015); Schmidhuber (2015)).”
MethodOptimizing risk-based breast cancer screening policies with reinforcement learning
Yala, A., Mikhael, P. G., Lehman, C., Lin, G., Strand, F., Wan, Y.-L., Hughes, K., Satuluru, S., Kim, T., Banerjee, I., et al. (2022) · Nature Medicine · doi:10.1038/s41591-021-01599-w · Cited by 103
“RL algorithms are commonly used in game playing [17, 18] and robotics [19-23].”
MethodOrganizing memories for generalization in complementary learning systems
Sun, W., Advani, M., Spruston, N., Saxe, A., & Fitzgerald, J. E. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01382-9 · Cited by 103
RobotGPT: Robot Manipulation Learning From ChatGPT
Jin, Y., Li, D., A, Y., Shi, J., Hao, P., Sun, F., Zhang, J., & Fang, B. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3357432 · Cited by 103
“The reason for the subjective result of bin_packing task leading to simple results is due to”
BackgroundOn the Binding Problem in Artificial Neural Networks
Greff, K., van Steenkiste, S., & Schmidhuber, J. (2020) · arXiv · doi:10.48550/arxiv.2012.05208 · Cited by 102
“In reinforcement learning, where the use of neural networks has led to superhuman performance in gameplay (Mnih et al., 2015; Silver et al., 2017; Berner et al., 2019), it is found that agents are fragile under distributional shift (Kansky et al.”
BackgroundMaintaining cooperation in complex social dilemmas using deep reinforcement learning
Lerer, A., & Peysakhovich, A. (2017) · arXiv · doi:10.48550/arxiv.1707.01068 · Cited by 101
“Atari, (Mnih et al., 2015)) then we can also construct agents that solve social dilemmas in that environment.”
BackgroundMorales, E. F., Murrieta-Cid, R., Becerra, I., & Esquivel-Basaldua, M. A. (2021) · Intelligent Service Robotics · doi:10.1007/s11370-021-00398-z · Cited by 101
Olfactory Sensing and Navigation in Turbulent Environments
Reddy, G., Murthy, V. N., & Vergassola, M. (2022) · Annual Review of Condensed Matter Physics · doi:10.1146/annurev-conmatphys-031720-032754 · Cited by 100
System Design Perspective for Human-Level Agents Using Deep Reinforcement Learning: A Survey
Nguyen, N. D., Nguyen, T., & Nahavandi, S. (2017) · IEEE Access · doi:10.1109/access.2017.2777827 · Cited by 100
“incorporated deep learning with RL to create a human-level agent that is virtually unbeatable in a series of 49 video Atari games [20], [21].”
BackgroundMethodA survey on visual navigation for artificial agents with deep reinforcement learning
Zeng, F., Wang, C., & Ge, S. S. (2020) · IEEE Access · doi:10.1109/access.2020.3011438 · Cited by 98
Meta reinforcement learning for sim-to-real domain adaptation
Arndt, K., Hazara, M., Ghadirzadeh, A., & Kyrki, V. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9196540 · Cited by 97
“Reinforcement learning, however, requires vast amounts of training data, which can be relatively costly to provide in robotics [1], in contrast to applications like computer games [2], [3].”
MethodPocketFlow is a data-and-knowledge-driven structure-based molecular generative model
Jiang, Y., Zhang, G., You, J., Zhang, H., Yao, R., Xie, H., Zhang, L., Xia, Z., Dai, M., Wu, Y., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00808-8 · Cited by 97
Ueltzhöffer, K. (2018) · Biological Cybernetics · doi:10.1007/s00422-018-0785-7 · Cited by 98
Blockchain-based distributed software-defined vehicular networks: a dueling deep q-learning approach
Zhang, D., Yu, F. R., & Yang, R. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2944399 · Cited by 96
“A deep Q-learning (DQL) concept introduced by [23] aims to solve the instability of traditional Q-network [24], [25].”
Mastering Diverse Domains through World Models
Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2023) · arXiv · doi:10.48550/arxiv.2301.04104 · Cited by 124
“…predominantly rests on the unsupervised reconstruction loss of its world model, unlike most prior algorithms that rely predominantly on reward and value prediction gradients 7,5,8 . c , The performance of Dreamer increases monotonically with larger model sizes, ranging from 12M to 400M parameters.”
BackgroundMethodInfluentialA Survey of Deep Learning Applications to Autonomous Vehicle Control
Kuutti, S., Bowden, R., Jin, Y., Barber, P., & Fallah, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2962338 · Cited by 96
“deep learning enables developers to describe the desired behaviour and teach the system to perform well and generalise to new environments through learning [43]–[47].”
BackgroundChen, S., & Wu, S. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01695-4 · Cited by 96
“Deep reinforcement learning [12, 13] combines the decision-making ability of reinforcement learning with the perception ability of deep learning.”
BackgroundOnline Model Distillation for Efficient Video Inference
Mullapudi, R. T., Chen, S., Zhang, K., Ramanan, D., & Fatahalian, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00367 · Cited by 96
Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs
Millidge, B., Tschantz, A., & Buckley, C. L. (2022) · Neural Computation · doi:10.1162/neco_a_01497 · Cited by 95
“L G ] 11 J un 2020 Vaswani et al., 2017), and computer game playing (Mnih et al., 2015; Schrittwieser et al., 2019; Silver et al., 2017; Vinyals et al., 2019).”
BackgroundEX2: Exploration with Exemplar Models for Deep Reinforcement Learning
Fu, J., Co-Reyes, J. D., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01260 · Cited by 95
Smart Musical Instruments: Vision, Design Principles, and Future Directions
Turchet, L. (2019) · IEEE Access · doi:10.1109/access.2018.2876891 · Cited by 94
“For this purpose, one could employ deep reinforcement-learning techniques [75], where a musician can interactively update a model based on his/her personal behavior.”
BackgroundDynamic robotic tracking of underwater targets using reinforcement learning
Masmitja, I., Martin, M., O’Reilly, T., Kieft, B., Palomeras, N., Navarro, J., & Katija, K. (2023) · Science Robotics · doi:10.1126/scirobotics.ade7811 · Cited by 96
“The use of deep Q-learning and off-policy algorithms in continuous state and action space presents a major challenge for stability and convergence ( 38 ).”
MethodMineRL: A Large-Scale Dataset of Minecraft Demonstrations
Guss, W. H., Houghton, B., Topin, N., Wang, P., Codel, C., Veloso, M., & Salakhutdinov, R. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/339 · Cited by 94
Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach
Kolat, M., Kővári, B., Bécsi, T., & Aradi, S. (2023) · Sustainability · doi:10.3390/su15043479 · Cited by 94
“Since then, RL has been applied in many applications, from robotics [25] through video games [23] to navigation systems [26].”
MethodNokkala, J., Martínez-Peña, R., Giorgi, G. L., Parigi, V., Soriano, M. C., & Zambrini, R. (2020) · arXiv · doi:10.48550/arxiv.2006.04821 · Cited by 94
Towards Generalization and Simplicity in Continuous Control
Rajeswaran, A., Lowrey, K., Todorov, E., & Kakade, S. (2017) · arXiv · doi:10.48550/arxiv.1703.02660 · Cited by 94
Open Problems in Cooperative AI
Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., & Graepel, T. (2020) · arXiv · doi:10.48550/arxiv.2012.08630 · Cited by 93
Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning
Juliani, A., Khalifa, A., Berges, V.-P., Harper, J., Teng, E., Henry, H., Crespi, A., Togelius, J., & Lange, D. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/373 · Cited by 93
Quantum Algorithms for Fixed Qubit Architectures
Farhi, E., Goldstone, J., Gutmann, S., & Neven, H. (2017) · arXiv · doi:10.48550/arxiv.1703.06199 · Cited by 93
What can AI do for me?: evaluating machine learning interpretations in cooperative play
Feng, S., & Boyd-Graber, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302265 · Cited by 92
Symbol Emergence in Cognitive Developmental Systems: A Survey
Taniguchi, T., Piater, J., Worgotter, F., Ugur, E., Hoffmann, M., Jamone, L., Nagai, T., Rosman, B., Matsuka, T., Iwahashi, N., et al. (2019) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2867772 · Cited by 92
“behavior learning against discovered, higher-level representations of sensory information [95].”
BackgroundGeneralization and Regularization in DQN
Farebrother, J., Machado, M. C., & Bowling, M. (2018) · arXiv · doi:10.48550/arxiv.1810.00123 · Cited by 92
“In all experiments performed in this paper we utilize the neural network architecture used by Mnih et al. (2015).”
BackgroundMethodInfluentialA review of deep learning-based deformable medical image registration
Zou, J., Gao, B., Song, Y., & Qin, J. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.1047215 · Cited by 126
“Since the initial publishing from Mnih et al. (68) and Silver et al. (69), Reinforcement Learning (RL) has gained a lot of attention and has been used for diverse applications including robotics, video games, and healthcare.”
BackgroundLearning to dress: Synthesizing human dressing motion via deep reinforcement learning
Clegg, A., Yu, W., Tan, J., Liu, C. K., & Turk, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275048 · Cited by 89
“…learning largely removes the need for dimension reduction and feature selection of the state and action spaces and has led to several notable successes in recent years [Heess et al. 2017; Liu and Hodgins 2017; Mnih et al. 2015; Peng et al. 2018; Silver et al. 2016, 2017; Yu et al. 2018].”
BackgroundZou, J., Lou, J., Wang, B., & Liu, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122801 · Cited by 89
(2018) · Neural Information Processing Systems · Cited by 89
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3737257 · Cited by 90
On-edge multi-task transfer learning: model and practice with data-driven task allocation
Chen, Q., Zheng, Z., Hu, C., Wang, D., & Liu, F. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2962435 · Cited by 88
From motor control to team play in simulated humanoid football
Liu, S., Lever, G., Wang, Z., Merel, J., Eslami, S. M. A., Hennes, D., Czarnecki, W. M., Tassa, Y., Omidshafiei, S., Abdolmaleki, A., et al. (2022) · Science Robotics · doi:10.1126/scirobotics.abo0235 · Cited by 88
Deep Reinforcement Learning Approach for Trading Automation in the Stock Market
Kabbani, T., & Duman, E. (2022) · IEEE Access · doi:10.1109/access.2022.3203697 · Cited by 88
“He proved that DRL had outperformed all existing algorithms in 2015 [17].”
MethodA Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
Zhang, A., Ballas, N., & Pineau, J. (2018) · arXiv · doi:10.48550/arxiv.1806.07937 · Cited by 88
Multimodal image registration with deep context reinforcement learning
Ma, K., Wang, J., Singh, V., Tamersoy, B., Chang, Y.-J., Wimmer, A., & Chen, T. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_28 · Cited by 87
Research community dynamics behind popular AI benchmarks
Martínez-Plumed, F., Barredo, P., hÉigeartaigh, S. Ó., & Hernández-Orallo, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00339-6 · Cited by 86
A Survey on Reinforcement Learning in Aviation Applications
Razzaghi, P., Tabrizian, A., Guo, W., Chen, S., Taye, A., Thompson, E., Bregeon, A., Baheri, A., & Wei, P. (2022) · arXiv · doi:10.48550/arxiv.2211.02147 · Cited by 86
Target-Driven Visual Navigation by Using Causal Intervention
Zhao, X., Wang, T., Li, Y., Zhang, B., Liu, K., Liu, D., Wang, C., & Snoussi, H. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3288810 · Cited by 86
Inferring neural activity before plasticity as a foundation for learning beyond backpropagation
Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01514-1 · Cited by 85
“…include online learning, where weights must be updated after each experience (rather than a batch of train ing examples) 29 , continual learning with multiple tasks 30 , learning in changing environments 31 , learning with a limited amount of training examples and reinforcement learning 4 .”
BackgroundA survey on hybrid human-artificial intelligence for autonomous driving
Ning, H., Yin, R., Ullah, A., & Shi, F. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3074695 · Cited by 84
“Besides, in recent years, with the proposition of reinforcement learning (RL) [137], scholars deem that RL may be an appropriate way to decide driving priority [138].”
BackgroundAlgorithms at War: The Promise, Peril, and Limits of Artificial Intelligence
Jensen, B. M., Whyte, C., & Cuomo, S. (2020) · International Studies Review · doi:10.1093/isr/viz025 · Cited by 84
Intelligent Edge-Assisted Crowdcast with Deep Reinforcement Learning for Personalized QoE
Wang, F., Zhang, C., wang, F., Liu, J., Zhu, Y., Pang, H., & Sun, L. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737456 · Cited by 87
Reinforcement Learning for Satellite Communications: From LEO to Deep Space Operations
Ferreira, P. V. R., Paffenroth, R., Wyglinski, A. M., Hackett, T. M., Bilen, S. G., Reinhart, R. C., & Mortensen, D. J. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1800796 · Cited by 82
Enhancing scientific discoveries in molecular biology with deep generative models
Lopez, R., Gayoso, A., & Yosef, N. (2020) · Molecular Systems Biology · doi:10.15252/msb.20199198 · Cited by 82
“This risk can be mitigated by careful assumptions as well as experimental validation.”
BackgroundMeta reinforcement learning as task inference
Humplik, J., Galashov, A., Hasenclever, L., Ortega, P. A., Teh, Y. W., & Heess, N. (2019) · arXiv · doi:10.48550/arxiv.1905.06424 · Cited by 82
Everitt, T., Hutter, M., Kumar, R., & Krakovna, V. (2021) · Synthese · doi:10.1007/s11229-021-03141-4 · Cited by 80
Application of deep reinforcement learning on automated stock trading
Chen, L., & Gao, Q. (2019) · 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS) · doi:10.1109/icsess47205.2019.9040728 · Cited by 78
“So we apply the two famous approaches[2] to make the DQN more stable.”
BackgroundMethodInfluentialAn empirical investigation of the challenges of real-world reinforcement learning
Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., & Hester, T. (2020) · arXiv · doi:10.48550/arxiv.2003.11881 · Cited by 78
“Zahavy et al. (2018) propose an Action Elimination Deep Q Network (AE-DQN) that uses a contextual bandit to eliminate irrelevant actions.”
BackgroundMethodResultInfluentialVerkhivker, G. M., Agajanian, S., Hu, G., & Tao, P. (2020) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2020.00136 · Cited by 76
Nguyen, H. T., Cong Luong, N., Zhao, J., Yuen, C., & Niyato, D. (2020) · 2020 IEEE 6th World Forum on Internet of Things (WF-IoT) · doi:10.1109/wf-iot48130.2020.9221089 · Cited by 75
“Therefore, the Deep Q-Learning (DQL) algorithm [9], which is a combination of deep neural network (DNN) and QL, is adopted to find the optimal policy for the model owner.”
BackgroundMethodInfluentialOptimizing Agent Behavior over Long Time Scales by Transporting Value
Hung, C.-C., Lillicrap, T., Abramson, J., Wu, Y., Mirza, M., Carnevale, F., Ahuja, A., & Wayne, G. (2018) · arXiv · doi:10.48550/arxiv.1810.06721 · Cited by 75
“Thus, we have a negative conditional log-likelihood loss contribution at each time step of − log p(ot, Rt|zt) ≡αimageLimage + αvalueLvalue + αrewLrew + αactLact, (8) where”
BackgroundSome Considerations on Learning to Explore via Meta-Reinforcement Learning
Stadie, B. C., Yang, G., Houthooft, R., Chen, X., Duan, Y., Wu, Y., Abbeel, P., & Sutskever, I. (2018) · arXiv · doi:10.48550/arxiv.1803.01118 · Cited by 74
“This resulted in most deep RL work utilizing epsilon greedy exploration [15], or perhaps a simple scheme like Boltzmann exploration [6].”
BackgroundMethod(2018) · arXiv (Cornell University) · Cited by 74
Target localization using multi-agent deep reinforcement learning with proximal policy optimization
Alagha, A., Singh, S., Mizouni, R., Bentahar, J., & Otrok, H. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.06.015 · Cited by 72
Collective intelligence for deep learning: A survey of recent developments
Ha, D., & Tang, Y. (2022) · Collective Intelligence · doi:10.1177/26339137221114874 · Cited by 71
“Deep learning has also been incorporated into reinforcement learning (RL) to tackle vision-based computer games such as Doom (Ha and Schmidhuber, 2018) and Atari (Mnih et al. 2015), and play games with large search spaces such as Go (Silver et al.”
BackgroundLearning to run a power network challenge for training topology controllers
Marot, A., Donnot, B., Romero, C., Donon, B., Lerousseau, M., Veyrin-Forrer, L., & Guyon, I. (2020) · Electric Power Systems Research · doi:10.1016/j.epsr.2020.106635 · Cited by 70
Digital transformation in water organizations
Boyle, C., Ryan, G., Bhandari, P., Law, K. M. Y., Gong, J., & Creighton, D. (2022) · Journal of Water Resources Planning and Management · doi:10.1061/(asce)wr.1943-5452.0001555 · Cited by 70
Uncertainty-Aware Action Advising for Deep Reinforcement Learning Agents
Da Silva, F. L., Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6036 · Cited by 69
Maximum Entropy RL (Provably) Solves Some Robust RL Problems
Eysenbach, B., & Levine, S. (2021) · arXiv · doi:10.48550/arxiv.2103.06257 · Cited by 69
“Unlike many other RL algorithms, such as DQN (Mnih et al., 2015) and DDPG (Lillicrap et al.”
MethodRisk Averse Robust Adversarial Reinforcement Learning
Pan, X., Seita, D., Gao, Y., & Canny, J. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794293 · Cited by 69
Influence-Based Multi-Agent Exploration
Wang, T., Wang, J., Wu, Y., & Zhang, C. (2019) · arXiv · doi:10.48550/arxiv.1910.05512 · Cited by 69
“…learning rely on a dense shaped reward function, such as distance to the goal (Mirowski et al., 2016; Wu et al., 2018), scores in games (Mnih et al., 2015) or expert-designed rewards (Wu & Tian, 2016; OpenAI, 2018), while tend to struggle in many real-world scenarios with sparse…”
BackgroundMethodInfluentialQuantum enhancements for deep reinforcement learning in large spaces
Jerbi, S., Trenkwalder, L. M., Nautrup, H. P., Briegel, H. J., & Dunjko, V. (2019) · arXiv · doi:10.48550/arxiv.1910.12760 · Cited by 69
“In their seminal work [7] [7] and the DEBN we consider.”
BackgroundMethodInfluentialCase study: verifying the safety of an autonomous racing car with a neural network controller
Ivanov, R., Carpenter, T. J., Weimer, J., Alur, R., Pappas, G. J., & Lee, I. (2020) · Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control · doi:10.1145/3365365.3382216 · Cited by 68
Mastering Atari Games with Limited Data
Ye, W., Liu, S., Kurutach, T., Abbeel, P., & Gao, Y. (2021) · arXiv · doi:10.48550/arxiv.2111.00210 · Cited by 68
“Temporal difference learning [24, 38, 40, 16] and policy gradient based methods [25, 23, 29, 31] are two types of popular reinforcement learning algorithms.”
BackgroundMethodInfluentialEmergent behaviour and neural dynamics in artificial agents tracking odour plumes
Singh, S. H., van Breugel, F., Rao, R. P. N., & Brunton, B. W. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00599-w · Cited by 61
“6), simulated by appending historical sensory observations onto instantaneous network inputs 98 .”
BackgroundProtein design via deep learning
Ding, W., Nakai, K., & Gong, H. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac102 · Cited by 66
Bio-Inspired Collision Avoidance in Swarm Systems via Deep Reinforcement Learning
Na, S., Niu, H., Lennox, B., & Arvin, F. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3145346 · Cited by 64
“The approach of using deep neural networks for reinforcement learning problem is referred to as DRL [40].”
BackgroundMethodInfluentialReview on Recent Strategies for Integrating Energy Storage Systems in Microgrids
Kandari, R., Neeraj, N., & Micallef, A. (2022) · Energies · doi:10.3390/en16010317 · Cited by 64
Deep active inference agents using Monte-Carlo methods
Fountas, Z., Sajid, N., Mediano, P. A. M., & Friston, K. (2020) · arXiv · doi:10.48550/arxiv.2006.04176 · Cited by 64
Hu, Q., Liu, Y., Cai, Y., Yu, G., & Ding, Z. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087233 · Cited by 64
“In addition, deep reinforcement learning (DRL) [27] has also been widely applied to solve the high-dimensional non-convex optimization problems.”
MethodRadulescu, A., Shin, Y. S., & Niv, Y. (2021) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092920-120559 · Cited by 63
Machine learning in materials design: Algorithm and application*
Song, Z., Chen, X., Meng, F., Cheng, G., Wang, C., Sun, Z., & Yin, W.-J. (2020) · Chinese Physics B · doi:10.1088/1674-1056/abc0e3 · Cited by 63
Metadata
- Authors
- Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., & Hassabis, D.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Corresponding author
- Kavukcuoglu, Koray
- Hassabis, Demis
- ORCID
- Venue
- Nature
- Published
- 2015-02-25
- Volume
- 518
- Issue
- 7540
- Pages
- 529-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2145339207
- Semantic Scholar
- 340f48901f72278f6bf78a04ee5b01df208cc508
- Cited by
- 30,357
- References
- 37
- Citations by year
- 2026: 2851 · 2025: 3619 · 2024: 3335 · 2023: 3592 · 2022: 3367 · 2021: 3971 · 2020: 3466 · 2019: 2907 · 2018: 1776 · 2017: 912 · 2016: 416 · 2015: 132
- FWCI
- 1057.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep Q-network, deep reinforcement learning, game-playing agents, end-to-end learning, deep neural network, temporal difference learning, dopaminergic neurons, human-level performance, DeepMind, AlphaGo, AlphaZero
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Models, Psychological, Reinforcement, Psychology, Reward, Neural Networks, Computer, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.