The DOI matches its Crossref record; 153 of the 153 references it cites were found in a registry or an index.
Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
Open accessAbstract
The Internet of Things (IoT) extends the Internet connectivity into billions of IoT devices around the world, where the IoT devices collect and share information to reflect status of the physical world. The Autonomous Control System (ACS), on the other hand, performs control functions on the physical systems without external intervention over an extended period of time. The integration of IoT and ACS results in a new concept - autonomous IoT (AIoT). The sensors collect information on the system status, based on which the intelligent agents in the IoT devices as well as the Edge/Fog/Cloud servers make control decisions for the actuators to react. In order to achieve autonomy, a promising method is for the intelligent agents to leverage the techniques in the field of artificial intelligence, especially reinforcement learning (RL) and deep reinforcement learning (DRL) for decision making. In this paper, we first provide a tutorial of DRL, and then propose a general model for the applications of RL/DRL in AIoT. Next, a comprehensive survey of the state-of-art research on DRL for AIoT is presented, where the existing works are classified and summarized under the umbrella of the proposed general DRL model. Finally, the challenges and open issues for future research are identified.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Natural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
A Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 2,202
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Counterfactual Multi-Agent Policy Gradients
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11794 · Cited by 1,873
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540 · cited 1×
“Cloud/Fog/Edge Computing [21], [22] [23] [24]”
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2844341 · Cited by 1,471
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989385 · Cited by 1,424
An Introduction to Deep Reinforcement Learning
Vincent, F.-L., Peter, H., Riashat, I., Marc, G. B., & Joelle, P. (2018) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000071 · Cited by 1,286
Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies
(2009) · A Field Guide to Dynamical Recurrent Networks · doi:10.1109/9780470544037.ch14 · Cited by 1,246
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
Chen, M., Challita, U., Saad, W., Yin, C., & Debbah, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2926625 · Cited by 1,117
Huang, L., Bi, S., & Zhang, Y.-J. A. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2928811 · Cited by 1,040
Machine Learning for Internet of Things Data Analysis: A Survey
Mahdavinejad, M. S., Rezvan, M., Barekatain, M., Adibi, P., Barnaghi, P. M., & Sheth, A. (2017) · Surrey Research Insight Open Access (The University of Surrey) · Cited by 1,008
Machine learning in wireless sensor networks: Algorithms, strategies, and applications
Alsheikh, M. A., Lin, S., Niyato, D., & Tan, H.-P. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2014.2320099 · Cited by 965
Machine learning for internet of things data analysis: a survey
Mahdavinejad, M. S., Rezvan, M., Barekatain, M., Adibi, P., Barnaghi, P., & Sheth, A. P. (2018) · Digital Communications and Networks · doi:10.1016/j.dcan.2017.10.002 · Cited by 921
End-to-End Learning of Driving Models from Large-Scale Video Datasets
Xu, H., Gao, Y., Yu, F., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.376 · Cited by 884
Space/Aerial-Assisted Computing Offloading for IoT Applications: A Learning-Based Approach
Cheng, X., Lyu, F., Quan, W., Zhou, C., He, H., Shi, W., & Shen, X. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2906789 · Cited by 867
Self-citationDeep Learning for Intelligent Wireless Networks: A Comprehensive Survey
Mao, Q., Hu, F., & Hao, Q. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2846401 · Cited by 789
Kakade, S. M. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 775
Review of Internet of Things (IoT) in Electric Power and Energy Systems
Bedi, G., Venayagamoorthy, G. K., Singh, R., Brooks, R. R., & Wang, K.-C. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2802704 · Cited by 755
QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.10293 · Cited by 697
Kendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.-M., Lam, V.-D., Bewley, A., & Shah, A. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793742 · Cited by 694 · cited 1×
“The authors in [126] apply a continuous, model-free DRL algorithm for autonomous driving.”
MethodWireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?
Zappone, A., Di Renzo, M., & Debbah, M. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2924010 · Cited by 684
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2876279 · Cited by 680
Deep Recurrent Q-Learning for Partially Observable MDPs
Hausknecht, M., & Stone, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06527 · Cited by 667
On-Line Building Energy Optimization Using Deep Reinforcement Learning
Mocanu, E., Mocanu, D. C., Nguyen, P. H., Liotta, A., Webber, M. E., Gibescu, M., & Slootweg, J. G. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2834219 · Cited by 649
A Review of Machine Learning and IoT in Smart Transportation
Zantalis, F., Koulouras, G., Karabetsos, S., & Kandris, D. (2019) · Future Internet · doi:10.3390/fi11040094 · Cited by 625
Liu, Y., Yu, H., Xie, S., & Zhang, Y. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2935450 · Cited by 570
He, Y., Zhao, N., & Yin, H. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2760281 · Cited by 569
Review on the research and practice of deep learning and reinforcement learning in smart grids
China Electric Power Research Institute, Zhang, D., Han, X., Taiyuan University of Technology, Deng, C., & China Electric Power Research Institute (2018) · CSEE Journal of Power and Energy Systems · doi:10.17775/cseejpes.2018.00520 · Cited by 561
Learning-Based Computation Offloading for IoT Devices With Energy Harvesting
Min, M., Xiao, L., Chen, Y., Cheng, P., Wu, D., & Zhuang, W. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890685 · Cited by 560
An introduction to deep reinforcement learning
François-Lavet, V., Henderson, P., Islam, R., Bellemare, M. G., & Pineau, J. (2018) · doi:10.1561/9781680835397 · Cited by 544
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
Long, P., Fanl, T., Liao, X., Liu, W., Zhang, H., & Pan, J. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461113 · Cited by 538
Hossain, E., Khan, I., Un-Noor, F., Sikander, S. S., & Sunny, M. S. H. (2019) · IEEE Access · doi:10.1109/access.2019.2894819 · Cited by 513
Context-Aware Computing, Learning, and Big Data in Internet of Things: A Survey
Sezer, O. B., Dogdu, E., & Ozbayoglu, A. M. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2773600 · Cited by 453
Playing FPS Games with Deep Reinforcement Learning
Lample, G., & Chaplot, D. S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10827 · Cited by 452
Smart Resource Allocation for Mobile Edge Computing: A Deep Reinforcement Learning Approach
Wang, J., Zhao, L., Liu, J., & Kato, N. (2021) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2019.2902661 · Cited by 451
Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach
Challita, U., Saad, W., & Bettstetter, C. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2900035 · Cited by 440
Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
Wu, Y., Mansimov, E., Liao, S., Grosse, R., & Ba, J. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05144 · Cited by 432
A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach
Lu, R., Hong, S. H., & Zhang, X. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.072 · Cited by 430
Sharma, S. K., & Wang, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916177 · Cited by 427
Vehicular Fog Computing: Enabling Real-Time Traffic Management for Smart Cities
Ning, Z., Huang, J., & Wang, X. (2019) · IEEE Wireless Communications · doi:10.1109/mwc.2019.1700441 · Cited by 425
A survey of point-based POMDP solvers
Shani, G., Pineau, J., & Kaplow, R. (2013) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-012-9200-2 · Cited by 422
Reinforcement Learning in the Multi-Robot Domain
Matarić, M. J. (1997) · Autonomous Robots · doi:10.1023/a:1008819414322 · Cited by 408
Counterfactual Multi-Agent Policy Gradients
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08926 · Cited by 401
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
Foerster, J., Nardelli, N., Farquhar, G., Afouras, T., Torr, P. H. S., Kohli, P., & Whiteson, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08887 · Cited by 397
Incentive-based demand response for smart grid with reinforcement learning and deep neural network
Lu, R., & Hong, S. H. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.12.061 · Cited by 394
Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications
Qiu, C., Hu, Y., Chen, Y., & Zeng, B. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2921159 · Cited by 383
Dynamic Energy Management System for a Smart Microgrid
Venayagamoorthy, G. K., Sharma, R. K., Gautam, P. K., & Ahmadi, A. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2514358 · Cited by 328
Wei, Y., Yu, F. R., Song, M., & Han, Z. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2878435 · Cited by 321
Distributed Distributional Deterministic Policy Gradients
Barth-Maron, G., Hoffman, M. W., Budden, D., Dabney, W., Horgan, D., Tb, D., Muldal, A., Heess, N., & Lillicrap, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.08617 · Cited by 300
Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning
Kim, B.-G., Zhang, Y., van der Schaar, M., & Lee, J.-W. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2495145 · Cited by 299
A New Deep-Q-Learning-Based Transmission Scheduling Mechanism for the Cognitive Internet of Things
Zhu, J., Song, Y., Jiang, D., & Song, H. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2759728 · Cited by 296
Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid
Foruzan, E., Soh, L.-K., & Asgarpoor, S. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2018.2823641 · Cited by 294
A survey on application of machine learning for Internet of Things
Cui, L., Yang, S., Chen, F., Ming, Z., Lu, N., & Qin, J. (2018) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-018-0834-5 · Cited by 293
Qiu, X., Liu, L., Chen, W., Hong, Z., & Zheng, Z. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2924015 · Cited by 287
Residential Demand Response Using Reinforcement Learning
O'Neill, D., Levorato, M., Goldsmith, A., & Mitra, U. (2010) · 2010 First IEEE International Conference on Smart Grid Communications · doi:10.1109/smartgrid.2010.5622078 · Cited by 281
Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning
Ji, Y., Wang, J., Xu, J., Fang, X., & Zhang, H. (2019) · Energies · doi:10.3390/en12122291 · Cited by 275
Qi, Q., Wang, J., Ma, Z., Sun, H., Cao, Y., Zhang, L., & Liao, J. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2894437 · Cited by 269
Optimal Demand Response Using Device-Based Reinforcement Learning
Wen, Z., O'Neill, D., & Maei, H. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2396993 · Cited by 266
Huang, L., Feng, X., Zhang, C., Qian, L., & Wu, Y. (2019) · Digital Communications and Networks · doi:10.1016/j.dcan.2018.10.003 · Cited by 261
Leveraging Machine Learning and Big Data for Smart Buildings: A Comprehensive Survey
Qolomany, B., Al-Fuqaha, A., Gupta, A., Benhaddou, D., Alwajidi, S., Qadir, J., & Fong, A. C. (2019) · IEEE Access · doi:10.1109/access.2019.2926642 · Cited by 246
Rodrigues, T. K., Suto, K., Nishiyama, H., Liu, J., & Kato, N. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2943405 · Cited by 245
Liu, B., Wang, L., & Liu, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2931179 · Cited by 242
Toward Intelligent Vehicular Networks: A Machine Learning Framework
Liang, L., Ye, H., & Li, G. Y. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2872122 · Cited by 240
From Cloud Down to Things: An Overview of Machine Learning in Internet of Things
Samie, F., Bauer, L., & Henkel, J. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2893866 · Cited by 240
An SMDP-Based Resource Allocation in Vehicular Cloud Computing Systems
Zheng, K., Meng, H., Chatzimisios, P., Lei, L., & Shen, X. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2015.2482119 · Cited by 239
Self-citationZeng, P., Li, H., He, H., & Li, S. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2859821 · Cited by 232
iRAF: A Deep Reinforcement Learning Approach for Collaborative Mobile Edge Computing IoT Networks
Chen, J., Chen, S., Wang, Q., Cao, B., Feng, G., & Hu, J. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2913162 · Cited by 225
Sun, Y., Peng, M., & Mao, S. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2871020 · Cited by 219
A Review of Fog Computing and Machine Learning: Concepts, Applications, Challenges, and Open Issues
Abdulkareem, K. H., Mohammed, M. A., Gunasekaran, S. S., Al-Mhiqani, M. N., Mutlag, A. A., Mostafa, S. A., Ali, N. S., & Ibrahim, D. A. (2019) · IEEE Access · doi:10.1109/access.2019.2947542 · Cited by 216
Memory-based control with recurrent neural networks
Heess, N., Hunt, J. J., Lillicrap, T. P., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1512.04455 · Cited by 211
Unsupervised Predictive Memory in a Goal-Directed AgentPreprint
Wayne, G., Hung, C.-C., Amos, D., Mirza, M., Ahuja, A., Grabska‐Barwińska, A., Rae, J. W., Mirowski, P., Leibo, J. Z., Santoro, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10760 · Cited by 207
Khamis, M. A., & Gomaa, W. (2014) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2014.01.007 · Cited by 204
Mirchevska, B., Pek, C., Werling, M., Althoff, M., & Boedecker, J. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569448 · Cited by 198
Hierarchical Deep Reinforcement Learning for Continuous Action Control
Yang, Z., Merrick, K., Jin, L., & Abbass, H. A. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2805379 · Cited by 197
Ren, J., Wang, H., Hou, T., Zheng, S., & Tang, C. (2019) · IEEE Access · doi:10.1109/access.2019.2919736 · Cited by 185
Q-PrOP: Sample-efficient policy gradient with an off-policy critic
Gu, S., Lillicrap, T., Ghahramani, Z., Turner, R. E., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.17863/cam.21294 · Cited by 174
Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practice
Ruelens, F., Claessens, B. J., Quaiyum, S., De Schutter, B., Babuska, R., & Belmans, R. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2640184 · Cited by 173
Su, Y., Lu, X., Zhao, Y., Huang, L., & Du, X. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2925719 · Cited by 172
Deep reinforcement learning with smooth policy update: Application to robotic cloth manipulation
Tsurumine, Y., Cui, Y., Uchibe, E., & Matsubara, T. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.11.004 · Cited by 168
A Survey on an Emerging Area: Deep Learning for Smart City Data
Chen, Q., Wang, W., Wu, F., De, S., Wang, R., Zhang, B., & Huang, X. (2019) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2019.2907718 · Cited by 167
Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems
Leong, A. S., Ramaswamy, A., Quevedo, D. E., Karl, H., & Shi, L. (2020) · Automatica · doi:10.1016/j.automatica.2019.108759 · Cited by 161
Claessens, B., Vrancx, P., & Ruelens, F. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2629450 · Cited by 160
Playing FPS Games with Deep Reinforcement LearningPreprint
Lample, G., & Chaplot, D. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.05521 · Cited by 160
Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-NetworksPreprint
Foerster, J., Assael, Y., de Freitas, N., & Whiteson, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02672 · Cited by 159
Formulation of deep reinforcement learning architecture toward autonomous driving for on-ramp merge
Wang, P., & Chan, C. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317735 · Cited by 158
Deep Reinforcement Learning based Resource Allocation in Low Latency Edge Computing Networks
Yang, T., Hu, Y., Gursoy, M. C., Schmeink, A., & Mathar, R. (2018) · 2018 15th International Symposium on Wireless Communication Systems (ISWCS) · doi:10.1109/iswcs.2018.8491089 · Cited by 156
Soft-defined heterogeneous vehicular network: architecture and challenges
Zheng, K., Hou, L., Meng, H., Zheng, Q., Lu, N., & Lei, L. (2016) · IEEE Network · doi:10.1109/mnet.2016.7513867 · Cited by 154
Self-citationDeep Reinforcement Learning for Mobile Edge Caching: Review, New Features, and Open Issues
Zhu, H., Cao, Y., Wang, W., Jiang, T., & Jin, S. (2018) · IEEE Network · doi:10.1109/mnet.2018.1800109 · Cited by 153 · cited 1×
“Cloud/Fog/Edge Computing [21], [22] [23] [24]”
Chu, M., Li, H., Liao, X., & Cui, S. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2872440 · Cited by 153
Caching Transient Data for Internet of Things: A Deep Reinforcement Learning Approach
Zhu, H., Cao, Y., Wei, X., Wang, W., Jiang, T., & Jin, S. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2882583 · Cited by 150
Flow: Architecture and Benchmarking for Reinforcement Learning in Traffic Control.Preprint
Wu, C., Kreidieh, A., Parvate, K., Vinitsky, E., & Bayen, A. M. (2017) · arXiv (Cornell University) · Cited by 149
On Improving Deep Reinforcement Learning for POMDPsPreprint
Zhu, P., Li, X., Poupart, P., & Miao, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.07978 · Cited by 146
Solving Deep Memory POMDPs with Recurrent Policy Gradients
Wierstra, D., Foerster, A., Peters, J., & Schmidhuber, J. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74690-4_71 · Cited by 141
Vehicular fog computing: Vision and challenges
Xiao, Y., & Zhu, C. (2017) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2017.7917508 · Cited by 138
A Comprehensive Survey of Recent Trends in Cloud Robotics Architectures and Applications
Saha, O., & Dasgupta, P. (2018) · Robotics · doi:10.3390/robotics7030047 · Cited by 137
Markov decision processes with delays and asynchronous cost collection
Katsikopoulos, K. V., & Engelbrecht, S. E. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.809799 · Cited by 129
Deep Reinforcement Learning Solutions for Energy Microgrids Management
François-Lavet, V., Taralla, D., Ernst, D., & Raphael Fonteneau, D. (2016) · ORBi (University of Liège) · Cited by 128
Application of deep reinforcement learning in mobile robot path planning
Xin, J. L., Zhao, H. Q., Liu, D., & Li, M. (2017) · Chinese Automation Congress · doi:10.1109/cac.2017.8244061 · Cited by 128
Learning How to Communicate in the Internet of Things: Finite Resources and Heterogeneity
Park, T., Abuzainab, N., & Saad, W. (2016) · IEEE Access · doi:10.1109/access.2016.2615643 · Cited by 121
Resource Allocation for Edge Computing in IoT Networks via Reinforcement Learning
Liu, X., Qin, Z., & Gao, Y. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761385 · Cited by 118
POMDP-based control of workflows for crowdsourcing
Dai, P., Lin, C. H., Mausam, M., & Weld, D. S. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2013.06.002 · Cited by 117
Distributed Reinforcement Learning for Multi-robot Decentralized Collective Construction
Sartoretti, G., Wu, Y., Paivine, W., Kumar, T. K. S., Koenig, S., & Choset, H. (2019) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-05816-6_3 · Cited by 91
Vehicular software-defined networking and fog computing: Integration and design principles
Nobre, J. C., Souza, A. M. D., Rosário, D., Both, C. B., Villas, L. A., Cerqueira, E., Braun, T. I., & Gerla, M. (2018) · Ad Hoc Networks · doi:10.1016/j.adhoc.2018.07.016 · Cited by 90
Hou, L., Lei, L., Zheng, K., & Wang, X. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2883762 · Cited by 89
Self-citationOptimal Reliability in Energy Harvesting Industrial Wireless Sensor Networks
Lei, L., Kuang, Y., Shen, X., Yang, K., Qiao, J., & Zhong, Z. D. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2558146 · Cited by 88
Self-citationRuelens, F., Claessens, B., Vandael, S., Iacovella, S., Vingerhoets, P., & Belmans, R. J. M. (2014) · Power Systems Computation Conference · doi:10.1109/pscc.2014.7038106 · Cited by 86
A study on vision-based mobile robot learning by deep Q-network
Sasaki, H., Horiuchi, T., & Kato, S. (2017) · Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE) · doi:10.23919/sice.2017.8105597 · Cited by 86
Heterogeneous Energy Storage Optimization for Microgrids
Qiu, X., Nguyen, T. A., & Crow, M. L. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2461134 · Cited by 82
Scheduling the Operation of a Connected Vehicular Network Using Deep Reinforcement Learning
Atallah, R. F., Assi, C. M., & Khabbaz, M. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2832219 · Cited by 82
Multiuser Resource Control With Deep Reinforcement Learning in IoT Edge Computing
Lei, L., Xu, H., Xiong, X., Zheng, K., Xiang, W., & Wang, X. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2935543 · Cited by 77
Self-citationExploring Applications of Deep Reinforcement Learning for Real-world Autonomous Driving Systems
Talpaert, V., Sobh, I., Kiran, B. R., Mannion, P., Yogamani, S. K., El-Sallab, A., & Pérez, P. (2019) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0007520305640572 · Cited by 76
Reinforcement Learning in the Multi-Robot Domain
Matarić, M. J. (1997) · Springer eBooks · doi:10.1007/978-1-4757-6451-2_4 · Cited by 76
Learning and planning in environments with delayed feedback
Walsh, T. J., Nouri, A., Li, L., & Littman, M. L. (2008) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-008-9056-7 · Cited by 72
Highway Traffic Modeling and Decision Making for Autonomous Vehicle Using Reinforcement Learning
You, C., Lǚ, J., Filev, D., & Tsiotras, P. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500675 · Cited by 62
Strategy Learning for Autonomous Agents in Smart Grid Markets
Reddy, P. P., & Veloso, M. (2011) · Cited by 61
Control delay in Reinforcement Learning for real-time dynamic systems: A memoryless approach
Schuitema, E., Busoniu, L., Babuska, R., & Jonker, P. (2010) · 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2010.5650345 · Cited by 59
Rana, M., Li, L., & Su, S. W. (2017) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2017.2675542 · Cited by 58
Intelligent IoT Connectivity: Deep Reinforcement Learning Approach
Kwon, M., Lee, J., & Park, H. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2949997 · Cited by 56
A Reinforcement Learning-Based Resource Allocation Scheme for Cloud Robotics
Liu, H., Liu, S., & Zheng, K. (2018) · IEEE Access · doi:10.1109/access.2018.2814606 · Cited by 50
Self-citationOda, T., Obukata, R., Ikeda, M., Barolli, L., & Takizawa, M. (2017) · International Conference on Advanced Information Networking and Applications Workshops (WAINA) · doi:10.1109/waina.2017.67 · Cited by 46
Machine Learning for Vehicular NetworksPreprint
Ye, H., Liang, L., Li, G. Y., Kim, J., Lu, L., & Wu, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.07143 · Cited by 43
Deep Reinforcement Learning based Distributed Resource Allocation for V2V Broadcasting
Ye, H., & Li, G. Y. (2018) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc.2018.8450518 · Cited by 36
Reinforcement Learning-based Energy Trading for MicrogridsPreprint
Xiao, L., Xiao, X., Dai, C., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06285 · Cited by 32
Liu, B., Wang, L., & Liu, M. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967908 · Cited by 32
Chen, J., Shu, T., Li, T., & de Silva, C. W. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2920390 · Cited by 30
IoT Enabled Indoor Autonomous Mobile Robot using CNN and Q-Learning
Saravanan, M., Kumar, K. S., & Sharma, A. K. (2019) · IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT) · doi:10.1109/iciaict.2019.8784847 · Cited by 26
Tongloy, T., Chuwongin, S., Jaksukam, K., Chousangsuntorn, C., & Boonsang, S. (2017) · International Conference on Robotics and Automation Engineering (ICRAE) · doi:10.1109/icrae.2017.8291355 · Cited by 23
A survey on multiagent reinforcement learning towards multi-robot systems
Yang, E., & Gu, D. (2005) · Cited by 21
Cooperative collision avoidance for multi-vehicle systems using reinforcement learning
Wang, Q., & Phillips, C. I. (2013) · International Conference on Methods and Models in Automation and Robotics · doi:10.1109/mmar.2013.6669888 · Cited by 18
Vehicular Edge Computing via Deep Reinforcement Learning
Qi, Q., & Ma, Z. (2019) · arXiv · doi:10.48550/arxiv.1901.04290 · Cited by 17
Reinforcement Learning-Based Computing and Transmission Scheduling for LTE-U-Enabled IoT
He, H., Shan, H., Huang, A., Ye, Q. J., & Zhuang, W. (2018) · Globecom · doi:10.1109/glocom.2018.8647178 · Cited by 16
Path Planning for Mobile Robot's Continuous Action Space Based on Deep Reinforcement Learning
Yan, T., Zhang, Y., & Wang, B. (2018) · IEEE International Conference on Big Data and Artificial Intelligence (BDAI) · doi:10.1109/bdai.2018.8546675 · Cited by 11
Coordinated Sensing Coverage in Sensor Networks using Distributed Reinforcement Learning
Renaud, J.-C., & Tham, C. (2006) · The ... IEEE International Conference on Networks/The IEEE International Conference on Networks · doi:10.1109/icon.2006.302580 · Cited by 10
Cooperative multi-robot reinforcement learning: A framework in hybrid state space
Sun, X., Mao, T., Kralik, J. D., & Ray, L. E. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354406 · Cited by 10
Partially Observable Double DQN Based IoT Scheduling for Energy Harvesting
Li, D., Xu, S., & Zhao, J. (2019) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2019.8756797 · Cited by 8
Cited by
Cellular, Wide-Area, and Non-Terrestrial IoT: A Survey on 5G Advances and the Road Toward 6G
Vaezi, M., Azari, A., Khosravirad, S. R., Shirvanimoghaddam, M., Azari, M. M., Chasaki, D., & Popovski, P. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3151028 · Cited by 631
A Review of Deep Reinforcement Learning for Smart Building Energy Management
Yu, L., Qin, S., Zhang, M., Shen, C., Jiang, T., & Guan, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3078462 · Cited by 476
Single and multi-agent deep reinforcement learning for AI-enabled wireless networks: A tutorial
Feriani, A., & Hossain, E. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3063822 · Cited by 410
Toward Autonomous Multi-UAV Wireless Network: A Survey of Reinforcement Learning-Based Approaches
Bai, Y., Zhao, H., Zhang, X., Chang, Z., Jäntti, R., & Yang, K. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3323344 · Cited by 295
Deep reinforcement learning in production systems: a systematic literature review
Panzer, M., & Bender, B. (2022) · International Journal of Production Research · doi:10.1080/00207543.2021.1973138 · Cited by 282
Metadata
- Authors
- Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., & Shen, X.
- Venue
- IEEE Communications Surveys & Tutorials
- Published
- 2020
- Volume
- 22
- Issue
- 3
- Pages
- 1722-1760
- ISSN
- 1553-877X (ISSN-L)
- 2373-745X
- OpenAlex
- W3017275226
- Cited by
- 298
- References
- 153
- FWCI
- 21.57
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.