The DOI matches its DataCite record; 5 of the 5 references it cites were found in a registry or an index.
Asynchronous Methods for Deep Reinforcement Learning
Open accessAbstract
We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of four standard reinforcement learning algorithms and show that parallel actor-learners have a stabilizing effect on training allowing all four methods to successfully train neural network controllers. The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU. Furthermore, we show that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“the former underestimates state uncertainty at future time steps [119], thus possibly leading to a decrease in performance”
From Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning (2021) · cites this paper as [119]
In the citing paper
“The ensemble strategy [51] combines three DRL algorithms (PPO [42], A2C [32] and DDPG [26]) to improve the robustness.”
From FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance (2021) · cites this paper as [32]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Self-citationADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Model-Free reinforcement learning with continuous action in practice
Degris, T., Pilarski, P. M., & Sutton, R. S. (2012) · 2012 American Control Conference (ACC) · doi:10.1109/acc.2012.6315022 · Cited by 239
Cited by
Sorted by citations
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Self-citationRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
“Here, we take the standard batched A2C [52] agent as base model and add CPC as an auxiliary loss.”
MethodCARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
Self-citationAn Introduction to Deep Learning for the Physical Layer
O'Shea, T., & Hoydis, J. (2017) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2017.2758370 · Cited by 2,980
Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning
Elfwing, S., Uchibe, E., & Doya, K. (2018) · Neural Networks · doi:10.1016/j.neunet.2017.12.012 · Cited by 2,119
Soft Actor-Critic Algorithms and Applications
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994
Rainbow: Combining Improvements in Deep Reinforcement Learning
Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., & Silver, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11796 · Cited by 1,846
Self-citationDeep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Hitaj, B., Ateniese, G., & Perez-Cruz, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134012 · Cited by 1,377
Resource Management with Deep Reinforcement Learning
Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016) · Proceedings of the 15th ACM Workshop on Hot Topics in Networks · doi:10.1145/3005745.3005750 · Cited by 1,353
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
Chu, T., Wang, J., Codeca, L., & Li, Z. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2901791 · Cited by 1,046
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1703.03864 · Cited by 1,027
“For example, we tried both the larger network and smaller network used in A3C [Mnih et al., 2016] for learning Atari 2600 games, and on average obtained better results using the larger network.”
MethodResultInfluentialTransfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
Overcoming Exploration in Reinforcement Learning with Demonstrations
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8463162 · Cited by 712
Kendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.-M., Lam, V.-D., Bewley, A., & Shah, A. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793742 · Cited by 694
“Many variants of actor-critic methods exist, see e.g. [28], [29].”
BackgroundHuman-level performance in 3D multiplayer games with population-based reinforcement learning
Jaderberg, M., Czarnecki, W. M., Dunning, I., Marris, L., Lever, G., Castañeda, A. G., Beattie, C., Rabinowitz, N. C., Morcos, A. S., Ruderman, A., et al. (2019) · Science · doi:10.1126/science.aau6249 · Cited by 662
Self-citationRegularizing Neural Networks by Penalizing Confident Output Distributions
Pereyra, G., Tucker, G., Chorowski, J., Kaiser, Ł., & Hinton, G. (2017) · arXiv · doi:10.48550/arxiv.1701.06548 · Cited by 655
Learning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579
Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review
Cao, D., Hu, W., Zhao, J., Zhang, G., Zhang, B., Liu, Z., Chen, Z., & Blaabjerg, F. (2020) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2020.000552 · Cited by 460
James, S., Wohlhart, P., Kalakrishnan, M., Kalashnikov, D., Irpan, A., Ibarz, J., Levine, S., Hadsell, R., & Bousmalis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01291 · Cited by 420
Single and multi-agent deep reinforcement learning for AI-enabled wireless networks: A tutorial
Feriani, A., & Hossain, E. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3063822 · Cited by 410
Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning
Henderson, P., Hu, J., Romoff, J., Brunskill, E., Jurafsky, D., & Pineau, J. (2020) · arXiv · doi:10.48550/arxiv.2002.05651 · Cited by 393
Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., & Bowling, M. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5699 · Cited by 359
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363
Towards Better Decoding and Language Model Integration in Sequence to Sequence Models
Chorowski, J., & Jaitly, N. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-343 · Cited by 358
Shi, Z., Yao, W., Li, Z., Zeng, L., Zhao, Y., Zhang, R., Tang, Y., & Wen, J. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115733 · Cited by 327
Reinforcement learning for intelligent healthcare applications: A survey
Coronato, A., Naeem, M., De Pietro, G., & Paragliola, G. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101964 · Cited by 320
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
Tan, H., Yu, L., & Bansal, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1268 · Cited by 293
Virtual to Real Reinforcement Learning for Autonomous Driving
Pan, X., You, Y., Wang, Z., & Lu, C. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.11 · Cited by 326
“that of [18] where the actor network is a 4-layer convolutional network (shown in figure 3) with ReLU activation functions in-between.”
BackgroundMethodInfluentialLiu, B., Wang, L., & Liu, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2931179 · Cited by 242
VLN↻BERT: A Recurrent Vision-and-Language BERT for Navigation
Hong, Y., Wu, Q., Qi, Y., Rodriguez-Opazo, C., & Gould, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00169 · Cited by 264
Preparing for the unknown: Learning a universal policy with online system identification
Yu, W., Tan, J., Karen Liu, C., & Turk, G. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.048 · Cited by 237
Deep Reinforcement Learning for Trading
Zhang, Z., Zohren, S., & Roberts, S. (2020) · The Journal of Financial Data Science · doi:10.3905/jfds.2020.1.030 · Cited by 227
RoboTHOR: An Open Simulation-to-Real Embodied AI Platform
Deitke, M., Han, W., Herrasti, A., Kembhavi, A., Kolve, E., Mottaghi, R., Salvador, J., Schwenk, D., VanderBilt, E., Wallingford, M., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00323 · Cited by 220
Decision Making in Multiagent Systems: A Survey
Rizk, Y., Awad, M., & Tunstel, E. W. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2840971 · Cited by 219
Reinforcement learning for iot security: A comprehensive survey
Uprety, A., & Rawat, D. B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3040957 · Cited by 214
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200
Voyager: An Open-Ended Embodied Agent with Large Language Models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023) · arXiv · doi:10.48550/arxiv.2305.16291 · Cited by 222
“[48] Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P.”
BackgroundStabilizing Transformers for Reinforcement Learning
Parisotto, E., Song, H. F., Rae, J. W., Pascanu, R., Gulcehre, C., Jayakumar, S. M., Jaderberg, M., Kaufman, R. L., Clark, A., Noury, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.06764 · Cited by 199
Ahmad, S., Shakeel, I., Mehfuz, S., & Ahmad, J. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2023.100568 · Cited by 197
Deep reinforcement learning for automated stock trading: an ensemble strategy
Yang, H., Liu, X.-Y., Zhong, S., & Walid, A. (2020) · Proceedings of the First ACM International Conference on AI in Finance · doi:10.1145/3383455.3422540 · Cited by 196
“A2C [32] is a typical actor-critic algorithm and we use it a component in the ensemble strategy.”
BackgroundMethodExploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2109.06668 · Cited by 192
Lehman, J., Clune, J., Misevic, D., Adami, C., Altenberg, L., Beaulieu, J., Bentley, P. J., Bernard, S., Beslon, G., Bryson, D. M., et al. (2020) · Artificial Life · doi:10.1162/artl_a_00319 · Cited by 181
Neural Map: Structured Memory for Deep Reinforcement Learning
Parisotto, E., & Salakhutdinov, R. (2017) · arXiv · doi:10.48550/arxiv.1702.08360 · Cited by 178
Survey on reinforcement learning for language processing
Uc-Cetina, V., Navarro-Guerrero, N., Martin-Gonzalez, A., Weber, C., & Wermter, S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10205-5 · Cited by 177
AWAC: Accelerating Online Reinforcement Learning with Offline Datasets
Nair, A., Gupta, A., Dalal, M., & Levine, S. (2020) · arXiv · doi:10.48550/arxiv.2006.09359 · Cited by 175
“deling of the behavior policy, AWAC avoids overly conservative updates. 5Related Work Off-policy RL algorithms are designed to reuse off-policy data during training, and have been studied extensively [19, 5, 25, 11, 7, 4, 32, 48, 44, 3]. While standard off-policy methods are able to benefit from including data seen dur…”
BackgroundMethodInfluentialLoss of plasticity in deep continual learning
Dohare, S., Hernandez-Garcia, J. F., Lan, Q., Rahman, P., Mahmood, A. R., & Sutton, R. S. (2024) · Nature · doi:10.1038/s41586-024-07711-7 · Cited by 172
“Altogether, these results, along with other extensive results in Methods, constitute substantial evidence of plasticity loss.”
Deep Reinforcement Learning for Mobile Edge Caching: Review, New Features, and Open Issues
Zhu, H., Cao, Y., Wang, W., Jiang, T., & Jin, S. (2018) · IEEE Network · doi:10.1109/mnet.2018.1800109 · Cited by 153
PCGRL: Procedural Content Generation via Reinforcement Learning
Khalifa, A., Bontrager, P., Earle, S., & Togelius, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v16i1.7416 · Cited by 156
Reinforcement learning for HVAC control in intelligent buildings: A technical and conceptual review
Al Sayed, K., Boodi, A., Sadeghian Broujeny, R., & Beddiar, K. (2024) · Journal of Building Engineering · doi:10.1016/j.jobe.2024.110085 · Cited by 140
Futures of artificial intelligence through technology readiness levels
Martínez-Plumed, F., Gómez, E., & Hernández-Orallo, J. (2021) · Telematics and Informatics · doi:10.1016/j.tele.2020.101525 · Cited by 130
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010009 · Cited by 129
Deep Reinforcement Adversarial Learning Against Botnet Evasion Attacks
Apruzzese, G., Andreolini, M., Marchetti, M., Venturi, A., & Colajanni, M. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3031843 · Cited by 117
Assessing Generalization in Deep Reinforcement Learning
Packer, C., Gao, K., Kos, J., Krähenbühl, P., Koltun, V., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1810.12282 · Cited by 114
Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback
Nguyen, K., Daumé III, H., & Boyd-Graber, J. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1153 · Cited by 110
VR-Goggles for Robots: Real-to-Sim Domain Adaptation for Visual Control
Zhang, J., Tai, L., Yun, P., Xiong, Y., Liu, M., Boedecker, J., & Burgard, W. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2894216 · Cited by 109
Maintaining cooperation in complex social dilemmas using deep reinforcement learning
Lerer, A., & Peysakhovich, A. (2017) · arXiv · doi:10.48550/arxiv.1707.01068 · Cited by 101
Toward Communication-Efficient Federated Learning in the Internet of Things With Edge Computing
Sun, H., Li, S., Yu, F. R., Qi, Q., Wang, J., & Liao, J. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2994596 · Cited by 95
MineRL: A Large-Scale Dataset of Minecraft Demonstrations
Guss, W. H., Houghton, B., Topin, N., Wang, P., Codel, C., Veloso, M., & Salakhutdinov, R. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/339 · Cited by 94
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3737257 · Cited by 90
Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
Yarats, D., Fergus, R., Lazaric, A., & Pinto, L. (2021) · arXiv · doi:10.48550/arxiv.2107.09645 · Cited by 89
From motor control to team play in simulated humanoid football
Liu, S., Lever, G., Wang, Z., Merel, J., Eslami, S. M. A., Hennes, D., Czarnecki, W. M., Tassa, Y., Omidshafiei, S., Abdolmaleki, A., et al. (2022) · Science Robotics · doi:10.1126/scirobotics.abo0235 · Cited by 88
Vision-Language Navigation with Random Environmental Mixup
Liu, C., Zhu, F., Chang, X., Liang, X., Ge, Z., & Shen, Y.-D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00167 · Cited by 80
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Castro da Silva, B. (2026) · ACM Computing Surveys · doi:10.1145/3743127 · Cited by 92
A Survey of Deep Learning Applications to Autonomous Vehicle Control
Kuutti, S., Bowden, R., Jin, Y., Barber, P., & Fallah, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2962338 · Cited by 96
“The third class, actor-critic algorithms (e.g. A3C [67]), are hybrid methods which combine the use of a value function with a parametrised policy function.”
MethodActive visual information gathering for vision-language navigation
Wang, H., Wang, W., Shu, T., Liang, W., & Shen, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58542-6_19 · Cited by 74
Some Considerations on Learning to Explore via Meta-Reinforcement Learning
Stadie, B. C., Yang, G., Houthooft, R., Chen, X., Duan, Y., Wu, Y., Abbeel, P., & Sutskever, I. (2018) · arXiv · doi:10.48550/arxiv.1803.01118 · Cited by 74
Hua, M., Qi, X., Chen, D., Jiang, K., Liu, Z. E., Sun, H., Zhou, Q., & Xu, H. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3574280 · Cited by 88
“…Q ( s , a ) to select actions via maximization; Policy-based methods (e.g., DDPG): optimize a parameterized policy π θ directly using gradient ascent; Actor-critic methods (e.g., Advantage Actor-Critic (A2C) [35]): combine policy and value function learning to improve stability and reduce variance.”
Learning to run a power network challenge for training topology controllers
Marot, A., Donnot, B., Romero, C., Donon, B., Lerousseau, M., Veyrin-Forrer, L., & Guyon, I. (2020) · Electric Power Systems Research · doi:10.1016/j.epsr.2020.106635 · Cited by 70
SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation
Gordon, D., Kadian, A., Parikh, D., Hoffman, J., & Batra, D. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00111 · Cited by 57
MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores
Kao, S.-C., & Krishna, T. (2022) · 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca53966.2022.00065 · Cited by 58
Robustifying the Deployment of tinyML Models for Autonomous mini-vehicles
de Prado, M., Rusci, M., Capotondi, A., Donze, R., Benini, L., & Pazos, N. (2021) · Sensors · doi:10.3390/s21041339 · Cited by 57
Using Spatial Reinforcement Learning to Build Forest Wildfire Dynamics Models From Satellite Images
Ganapathi Subramanian, S., & Crowley, M. (2018) · Frontiers in ICT · doi:10.3389/fict.2018.00006 · Cited by 56
Deep Reinforcement Learning for Syntactic Error Repair in Student Programs
Gupta, R., Kanade, A., & Shevade, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301930 · Cited by 54
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Chen, Q., Zhang, R., Yang, L., Xiao, B., & Wang, C. (2025) · doi:10.2139/ssrn.3737859 · Cited by 53
“In the middle is the agent layer that provides fine-tuned standard DRL algorithms (DQN [29][34], DDPG [29], Adaptive DDPG [27], Multi-Agent DDPG [30], PPO [40], SAC [18], A2C [33] and TD3 [11], etc.), commonly used reward functions and standard evaluation baselines to alleviate the debugging workloads and promote the r…”
InfluentialThe MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priors
Guss, W. H., Codel, C., Hofmann, K., Houghton, B., Kuno, N., Milani, S., Mohanty, S., Liebana, D. P., Salakhutdinov, R., Topin, N., et al. (2019) · arXiv · doi:10.48550/arxiv.1904.10079 · Cited by 46
Discovering cognitive strategies with tiny recurrent neural networks
Ji-An, L., Benna, M. K., & Mattar, M. G. (2025) · Nature · doi:10.1038/s41586-025-09142-4 · Cited by 64
FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance
Liu, X.-Y., Yang, H., Gao, J., & Wang, C. (2021) · doi:10.2139/ssrn.3955949 · Cited by 46
Minimalistic Attacks: How Little It Takes to Fool Deep Reinforcement Learning Policies
Qu, X., Sun, Z., Ong, Y.-S., Gupta, A., & Wei, P. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.2974509 · Cited by 40
Wang, H., Tan, A. H., & Nejat, G. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3412638 · Cited by 40
Algorithmic and Ethical Aspects of Recommender Systems in e-Commerce
Paraschakis, D. (2018) · doi:10.24834/2043/24268 · Cited by 35
Reinforcement Learning with an Ensemble of Binary Action Deep Q-Networks
M. Hafiz, A., Hassaballah, M., Alqahtani, A., Alsubai, S., & Abdel Hameed, M. (2023) · Computer Systems Science and Engineering · doi:10.32604/csse.2023.031720 · Cited by 33
Safe Reinforcement Learning via Probabilistic Logic Shields
Yang, W.-C., Marra, G., Rens, G., & De Raedt, L. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/637 · Cited by 34
Social Diversity and Social Preferences in Mixed-Motive Reinforcement Learning
McKee, K. R., Gemp, I., McWilliams, B., Duèñez-Guzmán, E. A., Hughes, E., & Leibo, J. Z. (2020) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/dnuf7773 · Cited by 33
Ghaffari, A., & Savaria, Y. (2020) · Electronics · doi:10.3390/electronics9122200 · Cited by 31
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 27
Cyber-physical defense in the quantum Era
Barbeau, M., & Garcia-Alfaro, J. (2022) · Scientific Reports · doi:10.1038/s41598-022-05690-1 · Cited by 22
FinMem: A Performance-Enhanced LLM Trading Agent With Layered Memory and Character Design
Yu, Y., Li, H., Chen, Z., Jiang, Y., Li, Y., Suchow, J. W., Zhang, D., & Khashanah, K. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3593370 · Cited by 23
Learning Reciprocity in Complex Sequential Social Dilemmas
Eccles, T., Hughes, E., Kramár, J., Wheelwright, S., & Leibo, J. Z. (2019) · arXiv · doi:10.48550/arxiv.1903.08082 · Cited by 19
Understanding the Effects of RLHF on LLM Generalisation and Diversity
Kirk, R., Mediratta, I., Nalmpantis, C., Luketina, J., Hambro, E., Grefenstette, E., & Raileanu, R. (2023) · arXiv · doi:10.48550/arxiv.2310.06452 · Cited by 17
“Most previous work on RLHF uses on-policy RL algorithms such as Proximal Policy Optimization (PPO) (Schulman et al., 2017) or Advantage Actor Critic (A2C) (Mnih et al., 2016), but offline RL methods have also been proposed (Snell et al., 2022).”
MethodSelf-Net: Lifelong Learning via Continual Self-Modeling
Mandivarapu, J. K., Camp, B., & Estrada, R. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00019 · Cited by 14
In-context Reinforcement Learning with Algorithm Distillation
Laskin, M., Wang, L., Oh, J., Parisotto, E., Spencer, S., Steigerwald, R., Strouse, D., Hansen, S., Filos, A., Brooks, E., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.14215 · Cited by 14
“In the Dark Room and Dark Key-to-Door environments we collect 2000 learning histories with an Asynchronous Advantage Actor-Critic (A3C) (Mnih et al., 2016) with 100 actors, while in DMLab Watermaze we collect 4000 learning histories with a distributed DQN with 16 parallel actors (see Appendix F for asymptotic learning…”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1602.01783
- OpenAlex
- W2260756217
- Cited by
- 7,132
- References
- 5
- Citations by year
- 2026: 18 · 2025: 375 · 2024: 795 · 2023: 914 · 2022: 926 · 2021: 1188 · 2020: 1108 · 2019: 981 · 2018: 545 · 2017: 238 · 2016: 39
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Keywords
- deep reinforcement learning, actor-critic, neural network controller, Atari, visual input, training stability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
