The DOI matches its Crossref record; 160 of the 160 references it cites were found in a registry or an index.
Toward Autonomous Multi-UAV Wireless Network: A Survey of Reinforcement Learning-Based Approaches
Open accessAbstract
Unmanned aerial vehicle (UAV)-based wireless networks have received increasing research interest in recent years and are gradually being utilized in various aspects of our society. The growing complexity of UAV applications such as disaster management, plant protection, and environment monitoring, has resulted in escalating and stringent requirements for UAV networks that a single UAV cannot fulfill. To address this, multi-UAV wireless networks (MUWNs) have emerged, offering enhanced resource-carrying capacity and enabling collaborative mission completion by multiple UAVs. However, the effective operation of MUWNs necessitates a higher level of autonomy and intelligence, particularly in decision-making and multi-objective optimization under diverse environmental conditions. Reinforcement Learning (RL), an intelligent and goal-oriented decision-making approach, has emerged as a promising solution for addressing the intricate tasks associated with MUWNs. As one may notice, the literature still lacks a comprehensive survey of recent advancements in RL-based MUWNs. Thus, this paper aims to bridge this gap by providing a comprehensive review of RL-based approaches in the context of autonomous MUWNs. We present an informative overview of RL and demonstrate its application within the framework of MUWNs. Specifically, we summarize various applications of RL in MUWNs, including data access, sensing and collection, resource allocation for wireless connectivity, UAV-assisted mobile edge computing, localization, trajectory planning, and network security. Furthermore, we identify and discuss several open challenges based on the insights gained from our review.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · cited 1×
“TheSoft Actor-Critic (SAC) algorithm [51], another variant of AC, introduces entropy regularization to the optimization process.”
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Markov games as a framework for multi-agent reinforcement learning
Littman, M. L. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50027-1 · Cited by 2,614
Real-time status: How often should one update?
Kaul, S., Yates, R., & Gruteser, M. (2012) · 2012 Proceedings IEEE INFOCOM · doi:10.1109/infcom.2012.6195689 · Cited by 2,360
Survey of Important Issues in UAV Communication Networks
Gupta, L., Jain, R., & Vaszkun, G. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2495297 · Cited by 2,343
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
Energy-Efficient UAV Communication With Trajectory Optimization
Zeng, Y., & Zhang, R. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2688328 · Cited by 2,164
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Counterfactual Multi-Agent Policy Gradients
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11794 · Cited by 1,873
Reciprocal n-Body Collision Avoidance
van den Berg, J., Guy, S. J., Lin, M., & Manocha, D. (2011) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-642-19457-3_1 · Cited by 1,858
Survey on Unmanned Aerial Vehicle Networks for Civil Applications: A Communications Viewpoint
Hayat, S., Yanmaz, E., & Muzaffar, R. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2016.2560343 · Cited by 1,386
Throughput Maximization for UAV-Enabled Mobile Relaying Systems
Zeng, Y., Zhang, R., & Lim, T. J. (2016) · IEEE Transactions on Communications · doi:10.1109/tcomm.2016.2611512 · Cited by 1,315
A Review on UAV-Based Applications for Precision Agriculture
Tsouros, D. C., Bibi, S., & Sarigiannidis, P. G. (2019) · Information · doi:10.3390/info10110349 · Cited by 1,223
Fotouhi, A., Qiang, H., Ding, M., Hassan, M., Giordano, L. G., Garcia-Rodriguez, A., & Yuan, J. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2906228 · Cited by 1,115
A Concise Introduction to Decentralized POMDPs
Oliehoek, F. A., & Amato, C. (2016) · SpringerBriefs in Intelligent Systems · doi:10.1007/978-3-319-28929-8 · Cited by 968
Hossein Motlagh, N., Taleb, T., & Arouk, O. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2612119 · Cited by 944
Optimal and Autonomous Control Using Reinforcement Learning: A Survey
Kiumarsi, B., Vamvoudakis, K. G., Modares, H., & Lewis, F. L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2773458 · Cited by 891
Space/Aerial-Assisted Computing Offloading for IoT Applications: A Learning-Based Approach
Cheng, X., Lyu, F., Quan, W., Zhou, C., He, H., Shi, W., & Shen, X. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2906789 · Cited by 867
Clerckx, B., Zhang, R., Schober, R., Ng, D. W. K., Kim, D. I., & Poor, H. V. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2872615 · Cited by 754
Monahan, G. E. (1982) · Management Science · doi:10.1287/mnsc.28.1.1 · Cited by 745
Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Bohez, S., & Vanhoucke, V. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.010 · Cited by 735
Self-citationA review on deep convolutional neural networks
Aloysius, N., & Geetha, M. (2017) · 2017 International Conference on Communication and Signal Processing (ICCSP) · doi:10.1109/iccsp.2017.8286426 · Cited by 636
Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks
Cui, J., Liu, Y., & Nallanathan, A. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2935201 · Cited by 593
What Will the Future of UAV Cellular Communications Be? A Flight From 5G to 6G
Geraci, G., Garcia-Rodriguez, A., Azari, M. M., Lozano, A., Mezzavilla, M., Chatzinotas, S., Chen, Y., Rangan, S., & Renzo, M. D. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3171135 · Cited by 535
Peng, H., & Shen, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036962 · Cited by 521
Multi-Agent Deep Reinforcement Learning for Task Offloading in UAV-Assisted Mobile Edge Computing
Zhao, N., Ye, Z., Pei, Y., Liang, Y., & Niyato, D. T. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3153316 · Cited by 520
DroNet: Learning to Fly by Driving
Loquercio, A., Maqueda, A. I., del-Blanco, C. R., & Scaramuzza, D. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2795643 · Cited by 514
Wang, L., Wang, K., Pan, C., Xu, W., Aslam, N., & Hanzo, L. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.3027695 · Cited by 510
Learning a Generic Value-Selection Heuristic Inside a Constraint Programming Solver
Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W. M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J. Z., Tuyls, K., et al. (2017) · arXiv (Cornell University) · doi:10.4230/lipics.cp.2023.25 · Cited by 491
UAV Trajectory Planning for Data Collection from Time-Constrained IoT Devices
Samir, M., Sharafeddine, S., Assi, C. M., Nguyen, T. M., & Ghrayeb, A. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2940447 · Cited by 429
Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
Rashid, T., Samvelyan, M., de Witt, C. S., Farquhar, G., Foerster, J., & Whiteson, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08839 · Cited by 418
Single and multi-agent deep reinforcement learning for AI-enabled wireless networks: A tutorial
Feriani, A., & Hossain, E. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3063822 · Cited by 410
Multipurpose UAV for search and rescue operations in mountain avalanche events
Silvagni, M., Tonoli, A., Zenerino, E., & Chiaberge, M. (2017) · Geomatics, Natural Hazards and Risk · doi:10.1080/19475705.2016.1238852 · Cited by 407
Reinforcement Learning in Multiple-UAV Networks: Deployment and Movement Design
Liu, X., Liu, Y., & Chen, Y. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2922849 · Cited by 342
Ding, R., Gao, F., & Shen, X. S. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3016024 · Cited by 331
Deep Reinforcement Learning Based Dynamic Trajectory Control for UAV-Assisted Mobile Edge Computing
Wang, L., Wang, K., Pan, C., Xu, W., Aslam, N., & Nallanathan, A. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3059691 · Cited by 324
Learning in the Air: Secure Federated Learning for UAV-Assisted Crowdsensing
Wang, Y., Su, Z., Zhang, N., & Benslimane, A. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2020.3014385 · Cited by 323
Singla, A., Padakandla, S., & Bhatnagar, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2954952 · Cited by 317
Aerospace Integrated Networks Innovation for Empowering 6G: A Survey and Future Challenges
Zhou, D., Sheng, M., Li, J., & Han, Z. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3245614 · Cited by 309
A Tutorial on UAVs for Wireless Networks: Applications, Challenges, and Open Problems
Mozaffari, M., Saad, W., Bennis, M., Nam, Y.-H., & Debbah, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2902862 · Cited by 299
Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., & Shen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988367 · Cited by 298
Shakeri, R., Al-Garadi, M. A., Badawy, A., Mohamed, A., Khattab, T., Al-Ali, A. K., Harras, K. A., & Guizani, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2924143 · Cited by 295
Multi-Agent DRL for Task Offloading and Resource Allocation in Multi-UAV Enabled IoT Edge Network
Seid, A. M., Boateng, G. O., Mareri, B., Sun, G., & Jiang, W. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3096673 · Cited by 285
UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning
Zhang, Y., Mou, Z., Gao, F., Jiang, J., Ding, R., & Han, Z. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3014788 · Cited by 276
Du, W., & Ding, S. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09938-y · Cited by 270
Mobile Edge Computing-Enabled Internet of Vehicles: Toward Energy-Efficient Scheduling
Ning, Z., Huang, J., Wang, X., Rodrigues, J. J. P. C., & Guo, L. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800309 · Cited by 260
Yun, W. J., Park, S., Kim, J., Shin, M., Jung, S., Mohaisen, D. A., & Kim, J.-H. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3143175 · Cited by 252
Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey
Li, T., Zhu, K., Luong, N. C., Niyato, D., Wu, Q., Zhang, Y., & Chen, B. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3160697 · Cited by 251
Fast, Reliable, and Secure Drone Communication: A Comprehensive Survey
Hassija, V., Chamola, V., Agrawal, A., Goyal, A., Luong, N. C., Niyato, D., Yu, F. R., & Guizani, M. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3097916 · Cited by 248
Multi-Agent Reinforcement Learning Aided Intelligent UAV Swarm for Target Tracking
Xia, Z., Du, J., Wang, J., Jiang, C., Ren, Y., Li, G., & Han, Z. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3129504 · Cited by 248
Yu, Y., Tang, J., Huang, J., Zhang, X., So, D. K. C., & Wong, K.-K. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3089476 · Cited by 244
Self-citationA review of artificial intelligence applied to path planning in UAV swarms
Puente-Castro, A., Rivero, D., Pazos, A., & Fernandez-Blanco, E. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06569-4 · Cited by 236
Security and privacy issues of UAV: A survey
Zhi, Y., Fu, Z., Sun, X., & Yu, J. (2020) · Mobile Networks and Applications · doi:10.1007/s11036-018-1193-x · Cited by 226 · cited 1×
“MUWNs are susceptible to attacks due to their openness and multi-connectivity [9], [180], [181].”
Survey on computation offloading in UAV-Enabled mobile edge computing
Huda, S. M. A., & Moh, S. (2022) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2022.103341 · Cited by 224
Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards
Wang, C., Wang, J., Wang, J., & Zhang, X. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2973193 · Cited by 217
Self-citationZhang, R., Zong, Q., Zhang, X., Dou, L., & Tian, B. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3146976 · Cited by 215
Self-citationZhu, B., Bedeer, E., Nguyen, H. H., Barton, R., & Henry, J. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3102161 · Cited by 214
Hu, J., Zhang, H., Song, L., Schober, R., & Poor, H. V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3013599 · Cited by 201
Samir, M., Assi, C., Sharafeddine, S., Ebrahimi, D., & Ghrayeb, A. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3023861 · Cited by 199
A Q-Learning Approach to Flocking With UAVs in a Stochastic Environment
Hung, S.-M., & Givigi, S. N. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2509646 · Cited by 196
Development of UAV-Based Target Tracking and Recognition Systems
Wang, S., Jiang, F., Zhang, B., Ma, R., & Hao, Q. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2927838 · Cited by 192
Autonomous Unmanned Aerial Vehicle navigation using Reinforcement Learning: A systematic review
AlMahamid, F., & Grolinger, K. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105321 · Cited by 192
Liu, Y., Xie, S., & Zhang, Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3016840 · Cited by 191
Energy Efficiency of Downlink Networks With Caching at Base Stations
Liu, D., & Yang, C. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2549398 · Cited by 190
Li, C. (2019) · Reinforcement Learning for Cyber-Physical Systems · doi:10.1201/9781351006620-6 · Cited by 188
Tang, F., Hofner, H., Kato, N., Kaneko, K., Yamashita, Y., & Hangai, M. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126073 · Cited by 180
A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
Hernández-Leal, P., Kaisers, M., Baarslag, T., & de Cote, E. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09183 · Cited by 179
Cooperative UAV Cluster-Assisted Terrestrial Cellular Networks for Ubiquitous Coverage
Wu, H., Tao, X., Zhang, N., & Shen, X. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2864418 · Cited by 178
Al-Hilo, A., Samir, M., Assi, C., Sharafeddine, S., & Ebrahimi, D. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3020220 · Cited by 167
Overcoming Endurance Issue: UAV-Enabled Communications With Proactive Caching
Xu, X., Zeng, Y., Guan, Y. L., & Zhang, R. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2844979 · Cited by 166
A Survey of Wireless Networks for Future Aerial Communications (FACOM)
Baltaci, A., Dinc, E., Ozger, M., Alabbasi, A., Cavdar, C., & Schupke, D. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3103044 · Cited by 165
Deep Reinforcement Learning for UAV Navigation Through Massive MIMO Technique
Huang, H., Yang, Y., Wang, H., Ding, Z., Sari, H., & Adachi, F. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2952549 · Cited by 163
Leveraging Deep Reinforcement Learning for Traffic Engineering: A Survey
Xiao, Y., Liu, J., Wu, J., & Ansari, N. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3102580 · Cited by 162
Autonomous Tracking Using a Swarm of UAVs: A Constrained Multi-Agent Reinforcement Learning Approach
Chen, Y.-J., Chang, D.-K., & Zhang, C. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3023733 · Cited by 161
Energy Efficient UAV-Enabled Mobile Edge Computing for IoT Devices: A Review
Abrar, M., Ajmal, U., Almohaimeed, Z. M., Gui, X., Akram, R., & Masroor, R. (2021) · IEEE Access · doi:10.1109/access.2021.3112104 · Cited by 159
Nie, Y., Zhao, J., Gao, F., & Yu, F. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3118446 · Cited by 159
Deep Reinforcement Learning Multi-UAV Trajectory Control for Target Tracking
Moon, J., Papaioannou, S., Laoudias, C., Kolios, P., & Kim, S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3073973 · Cited by 152
Fu, S., Tang, Y., Wu, Y., Zhang, N., Gu, H., Chen, C., & Liu, M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3051370 · Cited by 149
Nguyen, K. K., Masaracchia, A., Sharma, V., Poor, H. V., & Duong, T. Q. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3172587 · Cited by 147
Wang, D., Fan, T., Han, T., & Pan, J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974648 · Cited by 146
Liu, L., Xiong, K., Cao, J., Lu, Y., Fan, P., & Letaief, K. B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3110138 · Cited by 146
Learning-Based UAV Path Planning for Data Collection With Integrated Collision Avoidance
Wang, X., Gursoy, M. C., Erpek, T., & Sagduyu, Y. E. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3153585 · Cited by 145
Energy-Efficient Distributed Mobile Crowd Sensing: A Deep Learning Approach
Liu, C. H., Chen, Z., & Zhan, Y. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904353 · Cited by 143
Abedin, S. F., Munir, M. S., Tran, N. H. K., Han, Z., & Hong, C. S. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3039617 · Cited by 142
Deep Reinforcement Learning for Flocking Motion of Multi-UAV Systems: Learn From a Digital Twin
Shen, G., Lei, L., Li, Z., Cai, S., Zhang, L., Cao, P., & Liu, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3127873 · Cited by 142
Hierarchical Deep Reinforcement Learning for Backscattering Data Collection With Multiple UAVs
Zhang, Y., Mou, Z., Gao, F., Xing, L., Jiang, J., & Han, Z. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3024666 · Cited by 138
Leveraging UAVs for Coverage in Cell-Free Vehicular Networks: A Deep Reinforcement Learning Approach
Samir, M., Ebrahimi, D., Assi, C. M., Sharafeddine, S., & Ghrayeb, A. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.2991326 · Cited by 137
Hsu, Y.-H., & Gau, R.-H. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3003639 · Cited by 137
Multi-Agent Reinforcement Learning in NOMA-Aided UAV Networks for Cellular Offloading
Zhong, R., Liu, X., Liu, Y., & Chen, Y. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3104633 · Cited by 137
A Survey of Game Theory in Unmanned Aerial Vehicles Communications
Mkiramweni, M. E., Yang, C., Li, J., & Zhang, W. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2919613 · Cited by 136
Asheralieva, A., & Niyato, D. T. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2961958 · Cited by 135
Ren, T., Niu, J., Dai, B., Liu, X., Hu, Z., Xu, M., & Guizani, M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3071531 · Cited by 123
UAV-Assisted Wireless Energy and Data Transfer With Deep Reinforcement Learning
Xiong, Z., Zhang, Y., Lim, W. Y. B., Kang, J., Niyato, D. T., Leung, C., & Miao, C. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.3027696 · Cited by 120
Deep Reinforcement Learning-Based Resource Allocation in Cooperative UAV-Assisted Wireless Networks
Luong, P., Gagnon, F., Tran, L., & Labeau, F. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3086503 · Cited by 120
Energy-Efficient Online Path Planning of Multiple Drones Using Reinforcement Learning
Hong, D., Lee, S., Cho, Y. H., Baek, D., Kim, J., & Chang, N. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3102589 · Cited by 119
Yin, S., & Yu, F. R. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3094651 · Cited by 117
Cooperative Trajectory Design of Multiple UAV Base Stations With Heterogeneous Graph Neural Networks
Zhang, X., Zhao, H., Wei, J., Yan, C., Xiong, J., & Liu, X. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3204794 · Cited by 116
Self-citationLearning-Based Computation Offloading Approaches in UAVs-Assisted Edge Computing
Zhu, S., Gui, L., Zhao, D., Cheng, N., Zhang, Q., & Lang, X. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3048938 · Cited by 114
Guo, S., & Zhao, X. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3222460 · Cited by 114
UAVs as an Intelligent Service: Boosting Edge Intelligence for Air-Ground Integrated Networks
Dong, C., Shen, Y. C., Qu, Y., Wang, K., Zheng, J., Wu, Q., & Wu, F. (2021) · IEEE Network · doi:10.1109/mnet.011.2000651 · Cited by 113
Nguyen, K. K., Khosravirad, S. R., da Costa, D. B., Nguyen, L. D., & Duong, T. Q. (2021) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2021.3134162 · Cited by 110
Anwar, A., & Raychowdhury, A. (2020) · IEEE Access · doi:10.1109/access.2020.2971172 · Cited by 109
Liu, R., Liu, A., Qu, Z., & Xiong, N. N. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3122567 · Cited by 108 · cited 1×
“The vehicular network has created diversified novel applications with extremely multiple service requirements, including ultra-high reliability, delay sensitivity, high bandwidth requirements, and computing-intensive applications [111], [112].”
On-Board Deep Q-Network for UAV-Assisted Online Power Transfer and Data Collection
Li, K., Ni, W., Tovar, E., & Jamalipour, A. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2945037 · Cited by 108
Jiang, F., Wang, K., Dong, L., Pan, C., Xu, W., & Yang, K. (2020) · IEEE Network · doi:10.1109/mnet.011.2000440 · Cited by 108
Self-citationDesign of a 5G Network Slice Extension With MEC UAVs Managed With Reinforcement Learning
Faraci, G., Grasso, C., & Schembra, G. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3000416 · Cited by 106
Yan, C., Wang, C., Xiang, X., Lan, Z., & Jiang, Y. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3094207 · Cited by 105
Network Slicing with MEC and Deep Reinforcement Learning for the Internet of Vehicles
Mlika, Z., & Cherkaoui, S. (2021) · IEEE Network · doi:10.1109/mnet.011.2000591 · Cited by 104
Intelligent Offloading and Resource Allocation in Heterogeneous Aerial Access IoT Networks
Lakew, D. S., Tran, A.-T., Dao, N., & Cho, S. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3161571 · Cited by 104
Optimal UAV Base Station Trajectories Using Flow-Level Models for Reinforcement Learning
Saxena, V., Jaldén, J., & Klessig, H. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2948324 · Cited by 100
Wu, F., Zhang, H., Wu, J., Han, Z., Poor, H. V., & Song, L. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3065135 · Cited by 99
Joint deployment and trajectory optimization in UAV-assisted vehicular edge computing networks
Wu, Z., Yang, Z., Yang, C., Lin, J., Liu, Y., & Chen, X. (2021) · Journal of Communications and Networks · doi:10.23919/jcn.2021.000026 · Cited by 99
Game Combined Multi-Agent Reinforcement Learning Approach for UAV Assisted Offloading
Gao, A., Wang, Q., Liang, W. J., & Ding, Z. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3121281 · Cited by 99
Distributed Reinforcement Learning for Flexible and Efficient UAV Swarm Control
Venturini, F., Mason, F., Pase, F., Chiariotti, F., Testolin, A., Zanella, A., & Zorzi, M. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2021.3063170 · Cited by 98
Delay-Sensitive Energy-Efficient UAV Crowdsensing by Deep Reinforcement Learning
Dai, Z., Liu, C. H., Han, R., Wang, G., Leung, K. K., & Tang, J. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3113052 · Cited by 98
Sustainable Task Offloading in UAV Networks via Multi-Agent Reinforcement Learning
Sacco, A., Esposito, F., Marchetto, G., & Montuschi, P. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3074304 · Cited by 96
Li, T., Liu, W., Zeng, Z., & Xiong, N. N. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3067904 · Cited by 94
Caching Placement and Resource Allocation for Cache-Enabling UAV NOMA Networks
Zhang, T., Wang, Z., Liu, Y., Xu, W., & Nallanathan, A. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3015578 · Cited by 93
A Novel Cooperative NOMA for Designing UAV-Assisted Wireless Backhaul Networks
Nguyen, T. M., Ajib, W., & Assi, C. M. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2874136 · Cited by 85
Deep-Ensemble-Learning-Based GPS Spoofing Detection for Cellular-Connected UAVs
Dang, Y., Benzaïd, C., Yang, B., Taleb, T., & Shen, Y. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3195320 · Cited by 85
Three-Dimension Trajectory Design for Multi-UAV Wireless Network With Deep Reinforcement Learning
Zhang, W., Wang, Q., Liu, X., Liu, Y., & Chen, Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3047800 · Cited by 83
Qin, Z., Liu, Z., Han, G., Lin, C., Guo, L., & Xie, L. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3117792 · Cited by 80
Deep-Reinforcement-Learning-Based Intrusion Detection in Aerial Computing Networks
Tao, J., Han, T., & Li, R. (2021) · IEEE Network · doi:10.1109/mnet.011.2100068 · Cited by 76
UAV-Aided Wireless Power Transfer and Data Collection in Rician Fading
Liu, Y., Xiong, K., Lu, Y., Ni, Q., Fan, P., & Letaief, K. B. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3088693 · Cited by 75
Asheralieva, A., & Niyato, D. T. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2934027 · Cited by 69
Aerial Refueling: Scheduling Wireless Energy Charging for UAV Enabled Data Collection
Zhu, K., Yang, J., Zhang, Y., Nie, J., Lim, W. Y. B., Zhang, H., & Xiong, Z. (2022) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2022.3164602 · Cited by 69
Ji, J., Zhu, K., & Cai, L. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3181308 · Cited by 69
Deep Reinforcement Learning Approach for Joint Trajectory Design in Multi-UAV IoT Networks
Xu, S., Zhang, X., Li, C., Wang, D., & Yang, L. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3144277 · Cited by 65
Self-citationParvaresh, N., Kulhandjian, M., Kulhandjian, H., D’Amours, C., & Kantarcı, B. (2022) · Vehicular Communications · doi:10.1016/j.vehcom.2022.100474 · Cited by 63
Li, X., Cheng, S., Ding, H., Pan, M., & Zhao, N. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3198665 · Cited by 61
Cheng, Z., Liwang, M., Chen, N., Huang, L., Du, X., & Guizani, M. (2022) · Computer Communications · doi:10.1016/j.comcom.2022.06.017 · Cited by 55
Li, Y., Zhang, H., & Long, K. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3088623 · Cited by 53
Parvaresh, N., & Kantarcı, B. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3251297 · Cited by 53
Zhang, T., Wang, Z., Liu, Y., Xu, W., & Nallanathan, A. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3121584 · Cited by 52
Anokye, S., Ayepah-Mensah, D., Seid, A. M., Boateng, G. O., & Sun, G. (2021) · IEEE Systems Journal · doi:10.1109/jsyst.2021.3082837 · Cited by 51
Ke, H., Wang, H., Sun, W., & Sun, H. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3118696 · Cited by 51
Chen, Y., Liao, K.-M., Ku, M., Tso, F. P., & Chen, G. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3094273 · Cited by 48
Liu, Y., Yan, J., & Zhao, X. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3142269 · Cited by 45
Islam, S., Badsha, S., Khalil, I., Atiquzzaman, M., & Konstantinou, C. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3172936 · Cited by 38
UAV-Aided Two-Way Multi-User Relaying
Sheng, Z., Tuan, H. D., Duong, T. Q., & Hanzo, L. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3030679 · Cited by 34
Yuan, T., Rothenberg, C. E., Obraczka, K., Barakat, C., & Turletti, T. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3122505 · Cited by 33
A Self-Learning Strategy for Task Offloading in UAV Networks
Sacco, A., Esposito, F., Marchetto, G., & Montuschi, P. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3144654 · Cited by 32
Single- and Multiagent Actor–Critic for Initial UAV’s Deployment and 3-D Trajectory Design
Nasr-Azadani, M., Abouei, J., & Plataniotis, K. N. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3150184 · Cited by 29
Joint Optimization in Cached-Enabled Heterogeneous Network for Efficient Industrial IoT
Yang, J., Ma, C., Jiang, B., Ding, G., Zheng, G., & Wang, H. H. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.2980907 · Cited by 28
Wen, C., Fang, Y., & Qiu, L. (2022) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc51071.2022.9771555 · Cited by 27
Trajectory Planning of UAV in Wireless Powered IoT System Based on Deep Reinforcement Learning
Zhang, J., Yu, Y., Wang, Z., Ao, S., Tang, J., Zhang, X. Y., & Wong, K. (2020) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/iccc49849.2020.9238842 · Cited by 24
Self-citationContext-Aware Caching Distribution and UAV Deployment: A Game-Theoretic Approach
Fang, T., Tian, H., Zhang, X., Chen, X., Shao, X., & Zhang, Y. (2018) · Applied Sciences · doi:10.3390/app8101959 · Cited by 20
Self-citationA Decoupled Access Scheme With Reinforcement Learning Power Control for Cellular-Enabled UAVs
Shi, Y., Hamdan, M. Q., Alsusa, E. A., Hamdi, K. A., & Baidas, M. W. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3078188 · Cited by 19
RF Aerially Charging Scheduling for UAV Fleet : A Q-Learning Approach
Xu, J., Zhu, K., & Wang, R. (2019) · International Conference on Mobile Ad-Hoc and Sensor Networks (MSN) · doi:10.1109/msn48538.2019.00046 · Cited by 14
Metadata
- Authors
- Bai, Y., Zhao, H., Zhang, X., Chang, Z., Jäntti, R., & Yang, K.
- Venue
- IEEE Communications Surveys & Tutorials
- Published
- 2023
- Volume
- 25
- Issue
- 4
- Pages
- 3038-3067
- ISSN
- 1553-877X (ISSN-L)
- 2373-745X
- OpenAlex
- W4387587706
- Cited by
- 295
- References
- 160
- FWCI
- 36.22
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.