The DOI matches its Crossref record; 166 of the 166 references it cites were found in a registry or an index.
Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey
Abstract
The incumbent Internet of Things suffers from poor scalability and elasticity exhibiting in communication, computing, caching and control (4Cs) problems. The recent advances in deep reinforcement learning (DRL) algorithms can potentially address the above problems of IoT systems. In this context, this paper provides a comprehensive survey that overviews DRL algorithms and discusses DRL-enabled IoT applications. In particular, we first briefly review the state-of-the-art DRL algorithms and present a comprehensive analysis on their advantages and challenges. We then discuss on applying DRL algorithms to a wide variety of IoT applications including smart grid, intelligent transportation systems, industrial IoT applications, mobile crowdsensing, and blockchain-empowered IoT. Meanwhile, the discussion of each IoT application domain is accompanied by an in-depth summary and comparison of DRL algorithms. Moreover, we highlight emerging challenges and outline future research directions in driving the further success of DRL in IoT applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Mohamed Shakeel, P., Baskar, S., Sarma Dhulipala, V. R., Mishra, S., & Jaber, M. M. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-1045-z · Cited by 195
RetractedHuman-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications
Al-Fuqaha, A., Guizani, M., Mohammadi, M., Aledhari, M., & Ayyash, M. (2015) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2444095 · Cited by 8,630 · cited 1×
“For examples, Al-Fuqaha et al. [1] presented a survey of IoT with an emphasis on enabling protocols, technologies, and application issues while the studies of [6] and [7] presented the reviews on using deep learning techniques to IoT data analytics.”
BackgroundPower Generation, Operation, and Control
Wood, A. J., & Wollenberg, B. F. (1984) · Cited by 8,303
COUNTERSPECULATION, AUCTIONS, AND COMPETITIVE SEALED TENDERS
Vickrey, W. (1961) · The Journal of Finance · doi:10.1111/j.1540-6261.1961.tb02789.x · Cited by 7,722
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Internet of Things in industries: A survey
Xu, L. D., He, W., & Li, S. (2014) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2014.2300753 · Cited by 5,172 · cited 1×
“Although industrial wireless networks share many common features with traditional IoT network, they are more keen on the system reliability, performance and energy-efficiency [105], which will be highlighted in the following.”
BackgroundDeep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Blockchains and smart contracts for the internet of things
Christidis, K., & Devetsikiotis, M. (2016) · IEEE Access · doi:10.1109/access.2016.2566339 · Cited by 4,552
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
A Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970
Bellman, R. (1957) · Indiana University Mathematics Journal · doi:10.1512/iumj.1957.6.56038 · Cited by 2,408
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 2,202
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
On-line Q-learning Using Connectionist Systems
Rummery, G. A. (1994) · Cited by 1,477
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2844341 · Cited by 1,471
Resource Management with Deep Reinforcement Learning
Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016) · Proceedings of the 15th ACM Workshop on Hot Topics in Networks · doi:10.1145/3005745.3005750 · Cited by 1,353 · cited 1×
“With one step further, Mao et al. in [130] considered dispatching tasks with multiple resource demands and proposed a novel policy gradient method that promotes the monotonic improvement of policy iteration.”
BackgroundNguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306
(2015) · Journal of Machine Learning Research · Cited by 1,119
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
Chu, T., Wang, J., Codeca, L., & Li, Z. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2901791 · Cited by 1,046
Huang, L., Bi, S., & Zhang, Y.-J. A. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2928811 · Cited by 1,040
Self-citationIn-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning
Wang, X., Han, Y., Wang, C., Zhao, Q., Chen, X., & Chen, M. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800286 · Cited by 1,030 · cited 2×
“In addition, the computation offloading problem in blockchain-empowered IoT can also be found in [166] and [167].”
BackgroundDeep Reinforcement Learning Based Resource Allocation for V2V Communications
Ye, H., Li, G. Y., & Juang, B.-H. F. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2897134 · Cited by 870
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
Kulkarni, T. D., Narasimhan, K., Saeedi, A., & Tenenbaum, J. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06057 · Cited by 770
Zyzzyva: speculative byzantine fault tolerance
Kotla, R., Alvisi, L., Dahlin, M., Clement, A., & Wong, E. (2007) · Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles · doi:10.1145/1294261.1294267 · Cited by 638
Liu, C. H., Chen, Z., Tang, J., Xu, J., & Piao, C. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2864373 · Cited by 633
He, Y., Zhao, N., & Yin, H. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2760281 · Cited by 569
Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks
Wang, S., Liu, H., Gomes, P. H., & Krishnamachari, B. (2018) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2018.2809722 · Cited by 477
Smart Resource Allocation for Mobile Edge Computing: A Deep Reinforcement Learning Approach
Wang, J., Zhao, L., Liu, J., & Kato, N. (2021) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2019.2902661 · Cited by 451
Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach
Challita, U., Saad, W., & Bettstetter, C. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2900035 · Cited by 440
Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations
Duan, J., Shi, D., Diao, R., Li, H., Wang, Z., Zhang, B., Bian, D., & Yi, Z. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2019.2941134 · Cited by 434
A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach
Lu, R., Hong, S. H., & Zhang, X. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.072 · Cited by 430
Permissioned Blockchain and Edge Computing Empowered Privacy-Preserving Smart Grid Networks
Gai, K., Wu, Y., Zhu, L., Xu, L., & Zhang, Y. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2904303 · Cited by 407
Self-citationIncentive-based demand response for smart grid with reinforcement learning and deep neural network
Lu, R., & Hong, S. H. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.12.061 · Cited by 394
Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications
Qiu, C., Hu, Y., Chen, Y., & Zeng, B. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2921159 · Cited by 383
Liu, M., Yu, F. R., Teng, Y., Leung, V. C. M., & Song, M. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2897805 · Cited by 365
He, Y., Yu, F. R., Zhao, N., Leung, V. C. M., & Yin, H. (2017) · IEEE Communications Magazine · doi:10.1109/mcom.2017.1700246 · Cited by 358
Blockchain and Deep Reinforcement Learning Empowered Intelligent 5G Beyond
Dai, Y., Xu, D., Maharjan, S., Chen, Z., He, Q., & Zhang, Y. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800376 · Cited by 354
Self-citationBlockchain-Enabled Data Collection and Sharing for Industrial IoT With Deep Reinforcement Learning
Liu, C. H., Lin, Q., & Wen, S. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2890203 · Cited by 341
Online Vehicle Routing With Neural Combinatorial Optimization and Deep Reinforcement Learning
Yu, J. J. Q., Yu, W., & Gu, J. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2909109 · Cited by 338
Vehicular communication ad hoc routing protocols: A survey
Sharef, B. T., Alsaqour, R. A., & Ismail, M. (2014) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2013.09.008 · Cited by 332
Wei, Y., Yu, F. R., Song, M., & Han, Z. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2878435 · Cited by 321
From IoT to 5G I-IoT: The Next Generation IoT-Based Intelligent Algorithms and 5G Technologies
Wang, D., Chen, D., Song, B., Guizani, N., Yu, X., & Du, X. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1701310 · Cited by 306
Liu, C. H., Ma, X., Gao, X., & Tang, J. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2908171 · Cited by 301
Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning
Kim, B.-G., Zhang, Y., van der Schaar, M., & Lee, J.-W. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2495145 · Cited by 299
Self-citationDeep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., & Shen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988367 · Cited by 298
UAV Relay in VANETs Against Smart Jamming With Reinforcement Learning
Xiao, L., Lu, X., Xu, D., Tang, Y., Wang, L., & Zhuang, W. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2789466 · Cited by 297
A New Deep-Q-Learning-Based Transmission Scheduling Mechanism for the Cognitive Internet of Things
Zhu, J., Song, Y., Jiang, D., & Song, H. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2759728 · Cited by 296
Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid
Foruzan, E., Soh, L.-K., & Asgarpoor, S. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2018.2823641 · Cited by 294
Reinforcement Learning in Large Discrete Action Spaces.Preprint
Dulac-Arnold, G., Evans, R., Sunehag, P., & Coppin, B. (2015) · arXiv (Cornell University) · Cited by 292
Deep Reinforcement Learning in Large Discrete Action Spaces
Dulac-Arnold, G., Evans, R., van Hasselt, H., Sunehag, P., Lillicrap, T., Hunt, J., Mann, T., Weber, T., Degris, T., & Coppin, B. (2015) · arXiv · doi:10.48550/arxiv.1512.07679 · Cited by 290
Qiu, X., Liu, L., Chen, W., Hong, Z., & Zheng, Z. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2924015 · Cited by 287
Self-citationMean Field Multi-Agent Reinforcement Learning
Yang, Y., Luo, R., Li, M., Zhou, M. Q., Zhang, W., & Wang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05438 · Cited by 286
DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning
Gao, G., Li, J., & Wen, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2992117 · Cited by 283
Q-Learning-Based Vulnerability Analysis of Smart Grid Against Sequential Topology Attacks
Yan, J., He, H., Zhong, X., & Tang, Y. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2607701 · Cited by 264
Multi-Hop Cooperative Computation Offloading for Industrial IoT–Edge–Cloud Computing Environments
Hong, Z., Chen, W., Huang, H., Guo, S., & Zheng, Z. (2019) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2926979 · Cited by 263
Self-citationA deep reinforcement learning based framework for power-efficient resource allocation in cloud RANs
Xu, Z., Wang, Y., Tang, J., Wang, J., & Gursoy, M. C. (2017) · 2017 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2017.7997286 · Cited by 262
Robust Mobile Crowd Sensing: When Deep Learning Meets Edge Computing
Zhou, Z., Liao, H., Gu, B., Huq, K. M. S., Mumtaz, S., & Rodriguez, J. (2018) · IEEE Network · doi:10.1109/mnet.2018.1700442 · Cited by 260
Distributed Economic Dispatch in Microgrids Based on Cooperative Reinforcement Learning
Liu, W., Zhuang, P., Liang, H., Peng, J., & Huang, Z. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2801880 · Cited by 247
Resource Trading in Blockchain-Based Industrial Internet of Things
Yao, H., Mai, T., Wang, J., Ji, Z., Jiang, C., & Qian, Y. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2902563 · Cited by 244
Smart grids security challenges: Classification by sources of threats
Otuoze, A. O., Mustafa, M. W., & Larik, R. M. (2018) · Journal of Electrical Systems and Information Technology · doi:10.1016/j.jesit.2018.01.001 · Cited by 239
Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach
Kurt, M. N., Ogundijo, O., Li, C., & Wang, X. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2878570 · Cited by 238
Learn What Not to Learn: Action Elimination with Deep Reinforcement LearningPreprint
Zahavy, T., Haroush, M., Merlis, N., Mankowitz, D. J., & Mannor, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02121 · Cited by 235
Evaluation of Reinforcement Learning-Based False Data Injection Attack to Automatic Voltage Control
Chen, Y., Huang, S., Liu, F., Wang, Z., & Sun, X. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2790704 · Cited by 227
iRAF: A Deep Reinforcement Learning Approach for Collaborative Mobile Edge Computing IoT Networks
Chen, J., Chen, S., Wang, Q., Cao, B., Feng, G., & Hu, J. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2913162 · Cited by 225
Sun, Y., Peng, M., & Mao, S. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2871020 · Cited by 219
Cellular Network Traffic Scheduling With Deep Reinforcement Learning
Chinchali, S., Hu, P., Chu, T., Sharma, M., Bansal, M., Misra, R., Pavone, M., & Katti, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11339 · Cited by 218
A Multistage Game in Smart Grid Security: A Reinforcement Learning Solution
Ni, Z., & Paul, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2885530 · Cited by 212
Li, Z., Liu, P., Xu, C., Duan, H., & Wang, W. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2687620 · Cited by 205
Aublin, P.-L., Guerraoui, R., Knežević, N., Quéma, V., & Vukolić, M. (2015) · ACM Transactions on Computer Systems · doi:10.1145/2658994 · Cited by 204
Mirchevska, B., Pek, C., Werling, M., Althoff, M., & Boedecker, J. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569448 · Cited by 198
Qiu, C., Yu, F. R., Yao, H., Jiang, C., Xu, F., & Zhao, C. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2871394 · Cited by 195
Dynamic Access Control Policy based on Blockchain and Machine Learning for the Internet of Things
OUTCHAKOUCHT, A., ES-SAMAALI, H., & Philippe, J. (2017) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2017.080757 · Cited by 185
Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RLPreprint
Nagabandi, A., Finn, C., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07671 · Cited by 184
Zyzzyva: speculative byzantine fault tolerance
Kotla, R., Alvisi, L., Dahlin, M., Clement, A., & Wong, E. L. (2007) · ACM SIGOPS Operating Systems Review · doi:10.1145/1323293.1294267 · Cited by 183
Yang, H., Xie, X., & Kadoch, M. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890686 · Cited by 182
A Secure Mobile Crowdsensing Game With Deep Reinforcement Learning
Xiao, L., Li, Y., Han, G., Dai, H., & Poor, H. V. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2737968 · Cited by 180
Self-citationDeep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards
Schoettler, G., Nair, A., Luo, J., Bahl, S., Aparicio Ojea, J., Solowjow, E., & Levine, S. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341714 · Cited by 179
Su, Y., Lu, X., Zhao, Y., Huang, L., & Du, X. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2925719 · Cited by 172
Reinforcement Learning Based Energy Management Algorithm for Smart Energy Buildings
Kim, S., & Lim, H. (2018) · Energies · doi:10.3390/en11082010 · Cited by 169
Lee, S.-S., & Lee, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2996213 · Cited by 169
A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization
Stampa, G., Arias, M., Sánchez-Charles, D., Muntés-Mulero, V., & Cabellos‐Aparicio, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.07080 · Cited by 168
A Survey on an Emerging Area: Deep Learning for Smart City Data
Chen, Q., Wang, W., Wu, F., De, S., Wang, R., Zhang, B., & Huang, X. (2019) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2019.2907718 · Cited by 167
Learning-Based Energy-Efficient Data Collection by Unmanned Vehicles in Smart Cities
Zhang, B., Liu, C. H., Tang, J., Xu, Z., Ma, J., & Wang, W. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2783439 · Cited by 166
User-Centric View of Unmanned Aerial Vehicle Transmission Against Smart Attacks
Xiao, L., Xie, C., Min, M., & Zhuang, W. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2785414 · Cited by 166
A Trusted Routing Scheme Using Blockchain and Reinforcement Learning for Wireless Sensor Networks
Yang, J., He, S., Xu, Y., Chen, L., & Ren, J. (2019) · Sensors · doi:10.3390/s19040970 · Cited by 161
Deep learning for signal authentication and security in massive internet-of-things systems
Ferdowsi, A., & Saad, W. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2878025 · Cited by 160
Performance Optimization in Mobile-Edge Computing via Deep Reinforcement Learning
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2018) · 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall) · doi:10.1109/vtcfall.2018.8690980 · Cited by 158
Design of a Multi–Unit Double Auction E–Market
Huang, P., Scheller-Wolf, A. A., & Sycara, K. P. (2002) · Computational Intelligence · doi:10.1111/1467-8640.t01-1-00206 · Cited by 148
RUDDER: Return Decomposition for Delayed RewardsPreprint
Arjona-Medina, J. A., Gillhofer, M., Widrich, M., Unterthiner, T., Brandstetter, J., & Hochreiter, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07857 · Cited by 146
Robust Deep Reinforcement Learning for Security and Safety in Autonomous Vehicle Systems
Ferdowsi, A., Challita, U., Saad, W., & Mandayam, N. B. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569635 · Cited by 144
Solving large scale disassembly line balancing problem with uncertainty using reinforcement learning
Tüncel, E., Zeid, A., & Kamarthi, S. V. (2012) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-012-0711-0 · Cited by 143
Energy-Efficient Distributed Mobile Crowd Sensing: A Deep Learning Approach
Liu, C. H., Chen, Z., & Zhan, Y. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904353 · Cited by 143
Two-Dimensional Antijamming Mobile Communication Based on Reinforcement Learning
Xiao, L., Jiang, D., Xu, D., Zhu, H., Zhang, Y., & Poor, H. V. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2856854 · Cited by 139
Self-citationLiu, C. H., Dai, Z., Zhao, Y., Crowcroft, J., Wu, D. O., & Leung, K. K. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2938509 · Cited by 136
Dynamic Service Function Chain Embedding for NFV-Enabled IoT: A Deep Reinforcement Learning Approach
Fu, X., Yu, F. R., Wang, J., Qi, Q., & Liao, J. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2946797 · Cited by 133
SpanEdge: Towards Unifying Stream Processing over Central and Near-the-Edge Data Centers
Sajjad, H. P., Danniswara, K., Al-Shishtawy, A., & Vlassov, V. (2016) · IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec.2016.17 · Cited by 131
Mobile Crowdsensing Games in Vehicular Networks
Xiao, L., Chen, T., Xie, C., Dai, H., & Poor, H. V. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2647624 · Cited by 131
Self-citationLightweight Reinforcement Learning for Energy Efficient Communications in Wireless Sensor Networks
Savaglio, C., Pace, P., Aloi, G., Liotta, A., & Fortino, G. (2019) · IEEE Access · doi:10.1109/access.2019.2902371 · Cited by 127
A Reinforcement Learning and Blockchain-Based Trust Mechanism for Edge Networks
Xiao, L., Ding, Y., Jiang, D., Huang, J., Wang, D., Li, J., & Poor, H. V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.2995371 · Cited by 125
Yu, T., Wang, H. Z., Zhou, B., Chan, K. W., & Tang, J. (2014) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2014.2357079 · Cited by 120
Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity
Xu, H., Sun, H., Nikovski, D. N., Kitamura, S., Mori, K., & Hashimoto, H. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2903756 · Cited by 120
Deep Reinforcement Scheduling for Mobile Crowdsensing in Fog Computing
Li, H., Ota, K., & Dong, M. (2019) · ACM Transactions on Internet Technology · doi:10.1145/3234463 · Cited by 120
Deep Echo State Network (DeepESN): A Brief SurveyPreprint
Gallicchio, C., & Micheli, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.04323 · Cited by 119
Li, F., Qin, J., & Zheng, W. X. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2921475 · Cited by 118
Mobile crowd sensing and computing: when participatory sensing meets participatory social media
Guo, B., Chen, C., Zhang, D., Yu, Z., & Chin, A. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7402272 · Cited by 108
Chen, H., Li, X., & Zhao, F. (2016) · IEEE Sensors Journal · doi:10.1109/jsen.2016.2517084 · Cited by 105
Intelligent traffic light control using distributed multi-agent Q learning
Liu, Y., Liu, L., & Chen, W. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317730 · Cited by 103
Self-citationZhan, Y., Liu, C. H., Zhao, Y., Zhang, J., & Tang, J. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2927314 · Cited by 100
Hierarchical Routing for Vehicular Ad Hoc Networks via Reinforcement Learning
Li, F., Song, X., Chen, H., Li, X., & Wang, Y. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2887282 · Cited by 99
Atallah, R. F., Assi, C. M., & Yu, J. Y. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2622180 · Cited by 85
Motion Planning for Industrial Robots using Reinforcement Learning
Meyes, R., Tercan, H., Roggendorf, S., Thiele, T., Büscher, C., Obdenbusch, M., Brecher, C., Jeschke, S., & Meisen, T. (2017) · Procedia CIRP · doi:10.1016/j.procir.2017.03.095 · Cited by 84
Wang, X., Wang, H., & Qi, C. (2014) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-013-0864-5 · Cited by 80
Deep Learning-Based Dynamic Watermarking for Secure Signal Authentication in the Internet of Things
Ferdowsi, A., & Saad, W. (2018) · International Conference on Communications · doi:10.1109/icc.2018.8422728 · Cited by 78
Deep Reinforcement Learning Based Performance Optimization in Blockchain-Enabled Internet of Vehicle
Liu, M., Teng, Y., Yu, F. R., Leung, V. C. M., & Song, M. N. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761206 · Cited by 78
Wang, C., Zhang, J., Xu, L., Li, L., & Ran, B. (2019) · IEEE Access · doi:10.1109/access.2019.2904619 · Cited by 76
Autonomous Planning and Control for Intelligent Vehicles in Traffic
You, C., Lǚ, J., Filev, D., & Tsiotras, P. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2918071 · Cited by 76
Smart Multi-RAT Access Based on Multiagent Reinforcement Learning
Yan, M., Feng, G., Zhou, J., & Qin, S. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2793186 · Cited by 74
When Edge Computing Meets Microgrid: A Deep Reinforcement Learning Approach
Munir, M. S., Abedin, S. F., Tran, N. H. K., & Hong, C. S. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2899673 · Cited by 74
Enabling Cooperative Behavior for Building Demand Response Based on Extended Joint Action Learning
Hurtado, L. A., Mocanu, E., Nguyen, P. H., Gibescu, M., & Kamphuis, R. I. G. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2753408 · Cited by 73
Multi-Agent Common Knowledge Reinforcement LearningPreprint
de Witt, C. A. S., Foerster, J., Farquhar, G., Torr, P. H. S., Böhmer, W., & Whiteson, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11702 · Cited by 68
Approximate Policy-Based Accelerated Deep Reinforcement Learning
Wang, X., Gu, Y., Cheng, Y., Liu, A., & Chen, C. L. P. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2927227 · Cited by 67
Service Function Chain Embedding for NFV-Enabled IoT Based on Deep Reinforcement Learning
Fu, X., Yu, F. R., Wang, J., Qi, Q., & Liao, J. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900097 · Cited by 67
QoE-Driven Content-Centric Caching With Deep Reinforcement Learning in Edge-Enabled IoT
He, X., Wang, K., & Xu, W. (2019) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2019.2937608 · Cited by 65
A Context-Aware Multiarmed Bandit Incentive Mechanism for Mobile Crowd Sensing Systems
Wu, Y., Li, F., Ma, L., Xie, Y., Li, T., & Wang, Y. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2903197 · Cited by 60
Broad Reinforcement Learning for Supporting Fast Autonomous IoT
Wei, X., Zhao, J., Zhou, L., & Qian, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2980198 · Cited by 60
Caching in Dynamic IoT Networks by Deep Reinforcement Learning
Yao, J., & Ansari, N. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3004394 · Cited by 49
Scaling Up Robust MDPs by Reinforcement LearningPreprint
Tamar, A., Xu, H., & Mannor, S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.6189 · Cited by 48
Cell Selection with Deep Reinforcement Learning in Sparse Mobile Crowdsensing
Wang, L., Liu, W., Zhang, D., Wang, Y., Wang, E., & Yang, Y. (2018) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2018.00164 · Cited by 44
Energy-efficiency opportunistic spectrum allocation in cognitive wireless sensor network
Wu, C., Wang, Y., & Yin, Z. (2018) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-017-1018-9 · Cited by 43
Learning Based Security for VANET with Blockchain
Dai, C., Xiao, X., Ding, Y., Xiao, L., Tang, Y., & Zhou, S. (2018) · IEEE International Conference on Communication Systems (ICCS) · doi:10.1109/iccs.2018.8689228 · Cited by 42
Sen, T., & Shen, H. (2019) · IEEE International Conference on Fog and Edge Computing (ICFEC) · doi:10.1109/cfec.2019.8733153 · Cited by 42
Deep Q-Learning for Self-Organizing Networks Fault Management and Radio Performance Improvement
Mismar, F. B., & Evans, B. L. (2018) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2018.8645083 · Cited by 40
IntelligentCrowd: Mobile Crowdsensing via Multi-Agent Reinforcement Learning
Chen, Y., & Wang, H. (2020) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2020.3042244 · Cited by 37
Reinforcement Learning-based Energy Trading for MicrogridsPreprint
Xiao, L., Xiao, X., Dai, C., Pengy, M., Wang, L., & Poor, H. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06285 · Cited by 36
Li, Y., Zhao, X., & Liang, H. P. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3002936 · Cited by 34
Reinforcement Learning Based Pricing for Demand Response
Ghasemkhani, A., & Yang, L. (2018) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2018.8403783 · Cited by 31
Luong, N. C., Anh, T. T., Bình, H. T. T., Niyato, D. T., Kim, D. I., & Liang, Y. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683228 · Cited by 31
Oyewobi, S. S., Hancke, G. P., Abu-Mahfouz, A. M. I., & Onumanyi, A. J. (2019) · Sensors · doi:10.3390/s19061395 · Cited by 27
Multi-Dimensional Urban Sensing in Sparse Mobile Crowdsensing
Liu, W., Yang, Y., Wang, E., Wang, L., Zeghlache, D., & Zhang, D. (2019) · IEEE Access · doi:10.1109/access.2019.2924184 · Cited by 26
Manufacturing Dispatching Using Reinforcement and Transfer Learning
Zheng, S., Gupta, C., & Serita, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46133-1_39 · Cited by 23
Machine Preventive Replacement Policy for Serial Production Lines Based on Reinforcement Learning
Huang, J., Chang, Q., & Chakraborty, N. (2019) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2019.8843338 · Cited by 21
An energy-efficient offloading framework with predictable temporal correctness
Dong, Z., Liu, Y., Zhou, H., Xiao, X., Gu, Y. J., Zhang, L., & Liu, C. (2017) · ACM/IEEE Symposium on Edge Computing (SEC) · doi:10.1145/3132211.3134448 · Cited by 20
Cooperative Learning for Spectrum Management in Railway Cognitive Radio Network
Wu, C., Wang, C. Q., Sheng, J., & Wang, Y. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2912673 · Cited by 19
Effective crowdsensing and routing algorithms for next generation vehicular networks
Kim, S. (2017) · Wireless Networks · doi:10.1007/s11276-017-1632-9 · Cited by 18
Schwung, D., Csaplar, F., Schwung, A., & Ding, S. X. (2017) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin.2017.8104770 · Cited by 18
The wireless control plane: An overview and directions for future research
Mazied, E. A., ElNainay, M. Y., Abdel‐Rahman, M. J., Midkiff, S. F., Rizk, M. R. M., Rakha, H. A., & MacKenzie, A. B. (2018) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2018.09.017 · Cited by 17
Balancing Energy in the Smart Grid Using Distributed Value Function (DVF)
Shirzeh, H., Naghdy, F., Ciufo, P., & Ros, M. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2363844 · Cited by 16
Learning to Cooperate in Decentralized Multi-robot Exploration of Dynamic Environments
Geng, M., Zhou, X., Ding, B., Wang, H., & Zhang, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04239-4_4 · Cited by 16
Reinforcement Learning-Based Computing and Transmission Scheduling for LTE-U-Enabled IoT
He, H., Shan, H., Huang, A., Ye, Q. J., & Zhuang, W. (2018) · Globecom · doi:10.1109/glocom.2018.8647178 · Cited by 16
RL-BAGS: A Tool for Smart Grid Risk Assessment
Wadhawan, Y., & Neuman, C. (2018) · International Conference on Smart Grid and Clean Energy Technologies · doi:10.1109/icsgce.2018.8556775 · Cited by 12
A deep reinforcement learning approach to preserve connectivity for multi-robot systems
Huang, W., Wang, Y., & Yi, X. (2017) · International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI) · doi:10.1109/cisp-bmei.2017.8302157 · Cited by 10
Adaptive Passive Mobile Sensing Using Reinforcement Learning
Cai, L., Boukhechba, M., Kaur, N., Wu, C., Barnes, L. E., & Gerber, M. S. (2019) · IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM) · doi:10.1109/wowmom.2019.8792967 · Cited by 10
Partially Observable Reinforcement Learning for Intelligent Transportation Systems.Preprint
Zhang, R., Ishikawa, A., Wang, W., et al. (2018) · arXiv (Cornell University) · Cited by 8
Crowdsensing Game with Demand Uncertainties: A Deep Reinforcement Learning ApproachPreprint
Zhan, Y., Xia, Y., Zhang, J., Li, T., & Wang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00733 · Cited by 7
Recurrent Deep Multiagent Q-Learning for Autonomous Agents in Future Smart Grid
Yang, Y., Hao, J., Wang, Z., Sun, M., & Štrbac, G. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/wuim1798 · Cited by 5
Cited by
Deep reinforcement learning in smart manufacturing: A review and prospects
Li, C., Zheng, P., Yin, Y., Wang, B., & Wang, L. (2023) · CIRP Journal of Manufacturing Science and Technology · doi:10.1016/j.cirpj.2022.11.003 · Cited by 327
Wu, Y., Dai, H.-N., Wang, H., Xiong, Z., & Guo, S. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3158270 · Cited by 323
Self-citationA Survey on Intelligent Internet of Things: Applications, Security, Privacy, and Future Directions
Aouedi, O., Vu, T.-H., Sacco, A., Nguyen, D. C., Piamrat, K., Marchetto, G., & Pham, Q.-V. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3430368 · Cited by 277
Zhang, B., Wang, X., Xie, R., Li, C., Zhang, H., & Jiang, F. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2022.09.010 · Cited by 43
A multi-agent deep reinforcement learning based energy management for behind-the-meter resources
Wilk, P., Wang, N., & Li, J. (2022) · The Electricity Journal · doi:10.1016/j.tej.2022.107129 · Cited by 9
Metadata
- Authors
- Chen, W., Qiu, X., Cai, T., Dai, H.-N., Zheng, Z., & Zhang, Y.
- Venue
- IEEE Communications Surveys & Tutorials
- Published
- 2021
- Volume
- 23
- Issue
- 3
- Pages
- 1659-1692
- ISSN
- 1553-877X (ISSN-L)
- 2373-745X
- OpenAlex
- W3156032009
- Cited by
- 311
- References
- 166
- FWCI
- 22.36
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.