Memory-based control with recurrent neural networks
Open accessAbstract
Partially observed control problems are a challenging aspect of reinforcement learning. We extend two related, model-free algorithms for continuous control -- deterministic policy gradient and stochastic value gradient -- to solve partially observed domains using recurrent neural networks trained with backpropagation through time. We demonstrate that this approach, coupled with long-short term memory is able to solve a variety of physical control problems exhibiting an assortment of memory requirements. These include the short-term integration of information from noisy sensors and the identification of system parameters, as well as long-term memory problems that require preserving information over many time steps. We also demonstrate success on a combined exploration and memory problem in the form of a simplified version of the well-known Morris water maze task. Finally, we show that our approach can deal with high-dimensional observations by learning directly from pixels. We find that recurrent deterministic and stochastic policies are able to learn similarly good solutions to these tasks, including the water maze where the agent must learn effective search strategies.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
PILCO allows avoiding the reality gap by training a simulator with few real interactions and using it for the trial and error procedure.
“This solution allows to avoid the RG in the rst place, by training a simulator with few real interactions and using it for the trial and error procedure.”
From Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning (2021) · cites this paper as [118]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Self-citationAuto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Self-citationThrun, S. (2002) · CACM · doi:10.1145/504729.504754 · Cited by 7,941 · see citing works
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · see citing works
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578 · see citing works
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · see citing works
Planning and acting in partially observable stochastic domains
Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998) · Artificial Intelligence · doi:10.1016/s0004-3702(98)00023-x · Cited by 4,149 · see citing works
End-to-End Training of Deep Visuomotor PoliciesPreprint
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2015) · Journal of machine learning research · doi:10.48550/arxiv.1504.00702 · Cited by 4,011 · see citing works
Stochastic Backpropagation and Approximate Inference in Deep Generative ModelsPreprint
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4082 · Cited by 3,679 · see citing works
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275 · see citing works
Self-citationApplications of the Morris water maze in the study of learning and memory
D’Hooge, R., & Deyn, P. P. D. (2001) · Brain Research Reviews · doi:10.1016/s0165-0173(01)00067-4 · Cited by 2,035 · see citing works
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,569 · see citing works
Deep Recurrent Q-Learning for Partially Observable MDPsPreprint
Hausknecht, M., & Stone, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06527 · Cited by 1,224 · see citing works
A survey of point-based POMDP solvers
Shani, G., Pineau, J., & Kaplow, R. (2012) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-012-9200-2 · Cited by 427 · see citing works
Learning Continuous Control Policies by Stochastic Value GradientsPreprint
Heess, N., Wayne, G., Silver, D., Lillicrap, T., Tassa, Y., & Erez, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.09142 · Cited by 359 · see citing works
Self-citationLearning Without State-Estimation in Partially Observable Markovian Decision Processes
Singh, S. P., Jaakkola, T., & Jordan, M. I. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50042-8 · Cited by 335 · see citing works
Reinforcement learning in feedback control
Hafner, R., & Riedmiller, M. (2011) · Machine Learning · doi:10.1007/s10994-011-5235-x · Cited by 219 · see citing works
Reinforcement Learning with Long Short-Term Memory
Bakker, B. (2001) · Cited by 209 · see citing works
Solving Deep Memory POMDPs with Recurrent Policy Gradients
Wierstra, D., Foerster, A., Peters, J., & Schmidhuber, J. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74690-4_71 · Cited by 143 · see citing works
Reinforcement Learning With Hidden States
Lin, L.-J., & Mitchell, T. M. (1993) · The MIT Press eBooks · doi:10.7551/mitpress/3116.003.0038 · Cited by 106 · see citing works
Learning Policies with External MemoryPreprint
Peshkin, L., Meuleau, N., & Kaelbling, L. P. (2001) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0103003 · Cited by 92 · see citing works
Bias in Natural Actor-Critic Algorithms
Thomas, P. S. (2014) · Cited by 83 · see citing works
Reinforcement learning for factored markov decision processes
Hinton, G. E., & Sallans, B. (2002) · Cited by 41 · see citing works
Compatible Value Gradients for Reinforcement Learning of Continuous Deep PoliciesPreprint
Balduzzi, D., & Ghifary, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.03005 · Cited by 23 · see citing works
Utsunomiya, H., & Shibata, K. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-03040-6_118 · Cited by 18 · see citing works
Wierstra, D., & Schmidhuber, J. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74958-5_43 · Cited by 14 · see citing works
Policy Learning with Continuous Memory States for Partially Observed Robotic Control.
Zhang, M., Levine, S., McCarthy, Z., Finn, C., & Abbeel, P. (2015) · Cited by 11 · see citing works
Flagged first, then by citations
2 of the 33 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Self-citationDeep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · see citing works
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460528 · Cited by 1,309 · see citing works
Nguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306 · see citing works
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114 · see citing works
Cognitive Mapping and Planning for Visual Navigation
Gupta, S., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.769 · Cited by 521 · see citing works
Wang, C., Wang, J., Shen, Y., & Zhang, X. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890773 · Cited by 451 · see citing works
Imitating driver behavior with generative adversarial networks
Kuefler, A., Morton, J., Wheeler, T. A., & Kochenderfer, M. J. (2017) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2017.7995721 · Cited by 436 · see citing works
Navigating Occluded Intersections with Autonomous Vehicles Using Deep Reinforcement Learning
Isele, D., Rahimi, R., Cosgun, A., Subramanian, K., & Fujimura, K. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461233 · Cited by 379 · see citing works
Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks
Wu, T., Zhou, P., Liu, K., Yuan, Y., Wang, X., Huang, H., & Wu, D. O. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2997896 · Cited by 315 · see citing works
Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., & Shen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988367 · Cited by 307 · see citing works
Study on deep reinforcement learning techniques for building energy consumption forecasting
Liu, T., Tan, Z., Xu, C., Chen, H., & Li, Z. (2019) · Energy and Buildings · doi:10.1016/j.enbuild.2019.109675 · Cited by 278 · see citing works
Cognitive Mapping and Planning for Visual Navigation
Gupta, S., Tolani, V., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01236-7 · Cited by 275 · see citing works
Federated reinforcement learning: techniques, applications, and open challenges
Qi, J., Zhou, Q., Lei, L., & Zheng, K. (2021) · doi:10.20517/ir.2021.02 · Cited by 241 · see citing works
Di, X. S., & Shi, R. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103008 · Cited by 240 · see citing works
Preparing for the unknown: Learning a universal policy with online system identification
Yu, W., Tan, J., Karen Liu, C., & Turk, G. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.048 · Cited by 237 · see citing works
A Deep Reinforcement Learning Based Offloading Game in Edge Computing
Zhan, Y., Guo, S., Li, P., & Zhang, J. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2969148 · Cited by 221 · see citing works
Venon, A., Dupuis, Y., Vasseur, P., & Merriaux, P. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3167733 · Cited by 212 · see citing works
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200 · see citing works
UAV navigation in high dynamic environments: A deep reinforcement learning approach
Guo, T., Jiang, N., Li, B., Zhu, X., Wang, Y., & Du, W. (2020) · Chinese Journal of Aeronautics · doi:10.1016/j.cja.2020.05.011 · Cited by 197 · see citing works
Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning
Siekmann, J., Green, K., Warila, J., Fern, A., & Hurst, J. (2021) · Robotics: Science and Systems XVII · doi:10.15607/rss.2021.xvii.061 · Cited by 193 · see citing works
Reinforcement learning with temporal logic rewards
Li, X., Vasile, C.-I., & Belta, C. A. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206234 · Cited by 171 · see citing works
Wang, Y., Xu, T., Niu, X., Tan, C., Chen, E., & Xiong, H. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3033782 · Cited by 169 · see citing works
Autonomous Unmanned Aerial Vehicle navigation using Reinforcement Learning: A systematic review
AlMahamid, F., & Grolinger, K. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105321 · Cited by 162 · see citing works
Motion Planning for Mobile Robots—Focusing on Deep Reinforcement Learning: A Systematic Review
Sun, H., Zhang, W., Yu, R., & Zhang, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3076530 · Cited by 143 · see citing works
Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach
Liu, Y., Liu, Q., Zhao, H., Pan, Z., & Liu, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5587 · Cited by 141 · see citing works
Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control
Sadeghi, F., Toshev, A., Jang, E., & Levine, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00493 · Cited by 133 · see citing works
Deep reinforcement learning in transportation research: A review
Farazi, N. P., Zou, B., Ahamed, T., & Barua, L. (2021) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2021.100425 · Cited by 128 · see citing works
Dynamic Energy Dispatch Based on Deep Reinforcement Learning in IoT-Driven Smart Isolated Microgrids
Lei, L., Tan, Y., Dahlenburg, G., Xiang, W., & Zheng, K. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3042007 · Cited by 119 · see citing works
Guided Deep Reinforcement Learning for Swarm Systems
Hüttenrauch, M., Šošić, A., & Neumann, G. M. (2017) · TUbilio (Technical University of Darmstadt) · doi:10.48550/arxiv.1709.06011 · Cited by 118 · see citing works
Wang, C., Wang, J., Zhang, X., & Zhang, X. (2017) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2017.8309082 · Cited by 110 · see citing works
Model-free reinforcement learning from expert demonstrations: a survey
Ramírez, J., Yu, W., & Perrusquía, A. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10085-1 · Cited by 110 · see citing works
Memory-based Deep Reinforcement Learning for POMDPs
Meng, L., Gorbet, R., & Kulić, D. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636140 · Cited by 110 · see citing works
Deep learning in electron microscopy
Ede, J. M. (2020) · Machine Learning Science and Technology · doi:10.1088/2632-2153/abd614 · Cited by 105 · see citing works
Emergent Coordination Through CompetitionPreprint
Liu, S., Lever, G., Merel, J., Tunyasuvunakool, S., Heess, N., & Graepel, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07151 · Cited by 103 · see citing works
Self-citationVisual Navigation for Biped Humanoid Robots Using Deep Reinforcement Learning
Lobos-Tsunekawa, K., Leiva, F., & Ruíz-del-Solar, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2851148 · Cited by 99 · see citing works
Deep Learning for Time Series Forecasting: Review and Applications in Geotechnics and Geosciences
Mojtahedi, F. F., Yousefpour, N., Chow, S. H., & Cassidy, M. J. (2025) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-025-10244-5 · Cited by 96 · see citing works
Model-based Deep Reinforcement Learning for Dynamic Portfolio OptimizationPreprint
Yu, P., Sern, L. J., Kulyatin, I., Shi, Z., & Dasgupta, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08740 · Cited by 82 · see citing works
Reinforcement Mechanism Design for e-commerce
Cai, Q., Filos-Ratsikas, A., Tang, P., & Zhang, Y. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3178876.3186039 · Cited by 81 · see citing works
Learning Memory-Based Control for Human-Scale Bipedal Locomotion
Siekmann, J., Valluri, S., Dao, J., Bermillo, L., Duan, H., Fern, A., & HURST, J. W. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.031 · Cited by 78 · see citing works
Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement LearningPreprint
Zhou, M., Liu, Z., Sui, P., Li, Y., & Chung, Y. Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02529 · Cited by 71 · see citing works
Plume Tracing via Model-Free Reinforcement Learning Method
Hu, H., Song, S., & Chen, C. L. P. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2885374 · Cited by 69 · see citing works
A UAV Navigation Approach Based on Deep Reinforcement Learning in Large Cluttered 3D Environments
Xue, Y., & Chen, W. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3218855 · Cited by 64 · see citing works
AI-Based Resource Allocation in End-to-End Network Slicing Under Demand and CSI Uncertainties
Gharehgoli, A., Nouruzi, A., Mokari, N., Azmi, P., Javan, M. R., & Jorswieck, E. A. (2023) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2023.3243837 · Cited by 64 · see citing works
Sun, Y., Cheng, J., Zhang, G., & Xu, H. (2019) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-019-01004-2 · Cited by 63 · see citing works
Li, X., Tiong, A. M. H., Cao, L., Lai, W., Phan, P. T., & Phee, S. J. (2019) · International Journal of Mechanical Sciences · doi:10.1016/j.ijmecsci.2019.105129 · Cited by 58 · see citing works
Zhu, Z., Gupta, S., Gupta, A., & Canova, M. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3318552 · Cited by 51 · see citing works
Model-free control of dynamical systems with deep reservoir computing
Canaday, D., Pomerance, A., & Gauthier, D. J. (2021) · Journal of Physics Complexity · doi:10.1088/2632-072x/ac24f3 · Cited by 50 · see citing works
Wang, H., Huang, S.-C., Huang, P.-J., Wang, K.-L., Teng, Y.-C., Ko, Y.-T., Jeon, D., & Wu, I. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3251193 · Cited by 47 · see citing works
The Free Energy Principle for Perception and Action: A Deep Learning Perspective
Mazzaglia, P., Verbelen, T., Çatal, O., & Dhoedt, B. (2022) · Entropy · doi:10.3390/e24020301 · Cited by 47 · see citing works
Hybrid tensor decomposition in neural network compression
Wu, B., Wang, D., Zhao, G., Deng, L., & Li, G. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.09.006 · Cited by 46 · see citing works
Deep Reinforcement Learning-Based Air-to-Air Combat Maneuver Generation in a Realistic Environment
Bae, J. H., Jung, H., Kim, S., Kim, S., & Kim, Y.-D. (2023) · IEEE Access · doi:10.1109/access.2023.3257849 · Cited by 45 · see citing works
Ye, F., Ge, J., Zhang, S., Wu, J., & Luo, B. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3266226 · Cited by 44 · see citing works
Combining Motion Planner and Deep Reinforcement Learning for UAV Navigation in Unknown Environment
Xue, Y., & Chen, W. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3334978 · Cited by 44 · see citing works
Subramanian, J., Sinha, A. K., Seraj, R., & Mahajan, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08843 · Cited by 44 · see citing works
Machine learning meets advanced robotic manipulation
Nahavandi, S., Alizadehsani, R., Nahavandi, D., Lim, C. P., Kelly, K., & Bello, F. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102221 · Cited by 42 · see citing works
Meta-Q-LearningPreprint
Fakoor, R., Chaudhari, P., Soatto, S., & Smola, A. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00125 · Cited by 42 · see citing works
A Survey of Exploration Methods in Reinforcement Learning
Amin, S., Gomrokchi, M., Satija, H., van Hoof, H., & Precup, D. (2021) · arXiv · doi:10.48550/arxiv.2109.00157 · Cited by 41 · see citing works
The Evolution of Reinforcement Learning in Quantitative Finance: A Survey
Pippas, N., Ludvig, E. A., & Turkay, Ç. (2025) · ACM Computing Surveys · doi:10.1145/3733714 · Cited by 39 · see citing works
Jagannath, J., Jagannath, A., Furman, S., & Gwin, T. (2021) · Studies in computational intelligence · doi:10.1007/978-3-030-77939-9_2 · Cited by 38 · see citing works
Occlusion-Aware Search for Object Retrieval in Clutter
Bejjani, W., Agboh, W. C., Doğar, M. R., & Leonetti, M. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636230 · Cited by 38 · see citing works
Recurrent MADDPG for Object Detection and Assignment in Combat Tasks
Wei, X., Yang, L., Cao, G., Lü, T., & Wang, B. (2020) · IEEE Access · doi:10.1109/access.2020.3022638 · Cited by 37 · see citing works
An Optimized Path Planning Method for Coastal Ships Based on Improved DDPG and DP
Du, Y., Zhang, X., Cao, Z., Wang, S., Liang, J., Zhang, F., & Tang, J. (2021) · Journal of Advanced Transportation · doi:10.1155/2021/7765130 · Cited by 37 · see citing works
Deep Reinforcement Learning Based Real-Time Renewable Energy Bidding With Battery Control
Jeong, J., Kim, S. W., & Kim, H. (2023) · IEEE Transactions on Energy Markets Policy and Regulation · doi:10.1109/tempr.2023.3258409 · Cited by 36 · see citing works
DeepComp: Deep reinforcement learning based renewable energy error compensable forecasting
Jeong, J., & Kim, H. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.116970 · Cited by 36 · see citing works
Recurrent Deterministic Policy Gradient Method for Bipedal Locomotion on Rough Terrain Challenge
Song, D. R., Yang, C., McGreavy, C., & Li, Z. (2018) · International Conference on Control, Automation, Robotics and Vision (ICARCV) · doi:10.1109/icarcv.2018.8581309 · Cited by 36 · see citing works
Navigating Intersections with Autonomous Vehicles using Deep Reinforcement LearningPreprint
Isele, D., Cosgun, A., Subramanian, K. A., & Fujimura, K. (2017) · arXiv (Cornell University) · Cited by 36 · see citing works
Haklıdır, M., & Temeltaş, H. (2021) · IEEE Access · doi:10.1109/access.2021.3131772 · Cited by 34 · see citing works
Learning Link Schedules in Self-Backhauled Millimeter Wave Cellular Networks
Gupta, M., Rao, A., Visotsky, E., Ghosh, A., & Andrews, J. G. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3018955 · Cited by 34 · see citing works
Development of reinforced learning based non-linear controller for unmanned aerial vehicle
Din, A. F. U., Mir, I., Gul, F., & Akhtar, S. (2022) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-022-04467-8 · Cited by 34 · see citing works
Li, L., Zhao, W., Wang, C., Fotouhi, A., & Liu, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121458 · Cited by 33 · see citing works
Guo, H., Keyvan‐Ekbatani, M., & Xie, K. (2024) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2024.104703 · Cited by 33 · see citing works
A Multistage Hybrid Model ICEEMDAN-SE-VMD-RDPG for a Multivariate Solar Irradiance Forecasting
Sibtain, M., Li, X., Saleem, S., Qurat‐ul‐Ain, Asad, M. S., Tahir, T., & Apaydın, H. (2021) · IEEE Access · doi:10.1109/access.2021.3062764 · Cited by 33 · see citing works
Policy Gradients for Contextual Recommendations
Pan, F., Cai, Q., Tang, P., Zhuang, F., & He, Q. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313616 · Cited by 33 · see citing works
Rangel-Martínez, D., & Ricardez‐Sandoval, L. (2024) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2024.108748 · Cited by 31 · see citing works
Yoon, S., Cho, J.-H., Kim, D. S., Moore, T. J., Nelson, F. F., & Lim, H. (2021) · IEEE Access · doi:10.1109/access.2021.3076599 · Cited by 31 · see citing works
Unsupervised Visual Attention and Invariance for Reinforcement Learning
Wang, X., Lian, L., & Yu, S. X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00661 · Cited by 31 · see citing works
Kulkarni, P., Kober, J., Babuška, R., & Santina, C. D. (2021) · Advanced Intelligent Systems · doi:10.1002/aisy.202100095 · Cited by 31 · see citing works
Variational Recurrent Models for Solving Partially Observable Control TasksPreprint
Han, D., Doya, K., & Tani, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10703 · Cited by 30 · see citing works
Self-optimizing adaptive optics control with reinforcement learning for high-contrast imaging
Landman, R., Haffert, S. Y., Radhakrishnan, V. M., & Keller, C. U. (2021) · Journal of Astronomical Telescopes Instruments and Systems · doi:10.1117/1.jatis.7.3.039002 · Cited by 29 · see citing works
Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learning
Qi, J., Lei, L., Zheng, K., Yang, S. X., & Shen, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3267625 · Cited by 28 · see citing works
Deep reinforcement learning navigation via decision transformer in autonomous driving
Ge, L., Zhou, X., Li, Y., & Wang, Y. (2024) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1338189 · Cited by 28 · see citing works
Advancing Additive Manufacturing Through Machine Learning Techniques: A State-of-the-Art Review
Xiao, S., Li, J., Wang, Z., Chen, Y., & Tofighi, S. (2024) · Future Internet · doi:10.3390/fi16110419 · Cited by 28 · see citing works
Akbari, M., Hassanzadeh, E., Asl, Y. D., & Moghanian, A. (2025) · Journal of Advanced Joining Processes · doi:10.1016/j.jajp.2025.100316 · Cited by 28 · see citing works
Gu, F., Yin, H., Ghaoui, L. E., Arcak, M., Seiler, P. J., & Jin, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20476 · Cited by 28 · see citing works
Huang, J.-T., Lu, C.-L., Chang, P.-K., Huang, C.-I., Hsu, C.-C., Ewe, Z. L., Huang, P.-J., & Wang, H. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3062011 · Cited by 27 · see citing works
Xu, C., Zhang, X., Yang, H. H., Wang, X., Παππάς, Ν., Niyato, D. T., & Quek, T. Q. S. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3329170 · Cited by 27 · see citing works
Consequentialist conditional cooperation in social dilemmas with imperfect informationPreprint
Peysakhovich, A., & Lerer, A. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06975 · Cited by 27 · see citing works
A Multi-Critic Reinforcement Learning Method: An Application to Multi-Tank Water Systems
Martinez-Piazuelo, J., Ochoa, D. E., Quijano, N., & Giraldo, L. F. (2020) · IEEE Access · doi:10.1109/access.2020.3025194 · Cited by 26 · see citing works
Regret Minimization for Partially Observable Deep Reinforcement LearningPreprint
Jin, P., Keutzer, K., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.11424 · Cited by 26 · see citing works
Spatial–temporal recurrent reinforcement learning for autonomous ships
Waltz, M., & Okhrin, O. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.06.015 · Cited by 25 · see citing works
Control of neural systems at multiple scales using model-free, deep reinforcement learning
Mitchell, B., & Petzold, L. R. (2018) · Scientific Reports · doi:10.1038/s41598-018-29134-x · Cited by 24 · see citing works
Kim, G., Kang, J., & Sohn, K. (2022) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12924 · Cited by 24 · see citing works
Dynamic Control of a Fiber Manufacturing Process Using Deep Reinforcement Learning
Kim, S., Kim, D. D., & Anthony, B. (2021) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2021.3070973 · Cited by 24 · see citing works
Deep Reinforcement Learning With Modulated Hebbian Plus Q-Network Architecture
Ładosz, P., Ben-Iwhiwhu, E., Dick, J., Ketz, N., Kolouri, S., Krichmar, J. L., Pilly, P. K., & Soltoggio, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3110281 · Cited by 24 · see citing works
Approximate information state for partially observed systems
Subramanian, J., & Mahajan, A. (2019) · 2019 IEEE 58th Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029898 · Cited by 23 · see citing works
Task-Aware Verifiable RNN-Based Policies for Partially Observable Markov Decision Processes
Carr, S., Jansen, N., & Topcu, U. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12963 · Cited by 23 · see citing works
Sheikhzadeh, S., Pourghasemian, M., Javan, M. R., Mokari, N., & Jorswieck, E. A. (2021) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2021.3135479 · Cited by 23 · see citing works
Review of State-of-the-Art in Deep Learning Artificial Intelligence
Shakirov, V. V., Solovyeva, K., & Dunin-Barkowski, W. L. (2018) · Optical Memory and Neural Networks · doi:10.3103/s1060992x18020066 · Cited by 22 · see citing works
Mei, J., Wang, X., & Zheng, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3109878 · Cited by 22 · see citing works
- Authors
- Heess, N., Hunt, J. J., Lillicrap, T. P., & Silver, D.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1512.04455
- OpenAlex
- W2291973609
- Cited by
- 211
- References
- 33
- Citations by year
- 2025: 8 · 2024: 18 · 2023: 28 · 2022: 16 · 2021: 48 · 2020: 39 · 2019: 23 · 2018: 15 · 2017: 12 · 2016: 3 · 2015: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Adaptive Dynamic Programming Control
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- memory-based control, recurrent neural networks, deterministic policy gradient, long short-term memory, backpropagation through time, continuous control, physical control, system parameter identification, Morris water maze, high-dimensional observations, exploration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review