The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Distributed Energy-Efficient Multi-UAV Navigation for Long-Term Communication Coverage by Deep Reinforcement Learning
Abstract
In this paper, we aim to design a fully-distributed control solution to navigate a group of unmanned aerial vehicles (UAVs), as the mobile Base Stations (BSs) to fly around a target area, to provide long-term communication coverage for the ground mobile users. Different from existing solutions that mainly solve the problem from optimization perspectives, we proposed a decentralized deep reinforcement learning (DRL) based framework to control each UAV in a distributed manner. Our goal is to maximize the temporal average coverage score achieved by all UAVs in a task, maximize the geographical fairness of all considered point-of-interests (PoIs), and minimize the total energy consumptions, while keeping them connected and not flying out of the area border. We designed the state, observation, action space, and reward in an explicit manner, and model each UAV by deep neural networks (DNNs). We conducted extensive simulations and found the appropriate set of hyperparameters, including experience replay buffer size, number of neural units for two fully-connected hidden layers of actor, critic, and their target networks, and the discount factor for remembering the future reward. The simulation results justified the superiority of the proposed model over the state-of-the-art DRL-EC3 approach based on deep deterministic policy gradient (DDPG), and three other baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
(1999) · Cited by 5,147
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Scenarios for 5G mobile and wireless communications: the vision of the METIS project
Osseiran, A., Boccardi, F., Braun, V., Kusume, K., Marsch, P., Maternia, M., Queseth, O., Schellmann, M., Schotten, H., Taoka, H., et al. (2014) · IEEE Communications Magazine · doi:10.1109/mcom.2014.6815890 · Cited by 2,006
Rainbow: Combining Improvements in Deep Reinforcement Learning
Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., & Silver, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11796 · Cited by 1,846
Efficient Deployment of Multiple Unmanned Aerial Vehicles for Optimal Wireless Coverage
Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2016) · IEEE Communications Letters · doi:10.1109/lcomm.2016.2578312 · Cited by 1,095
Mobile Unmanned Aerial Vehicles (UAVs) for Energy-Efficient Internet of Things Communications
Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2751045 · Cited by 1,093
Alzenad, M., El-Keyi, A., Lagum, F., & Yanikomeroglu, H. (2017) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2017.2700840 · Cited by 949
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · Machine Learning · doi:10.1023/a:1022672621406 · Cited by 817
Chen, M., Mozaffari, M., Saad, W., Yin, C., Debbah, M., & Hong, C. S. (2017) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2017.2680898 · Cited by 742
Liu, C. H., Chen, Z., Tang, J., Xu, J., & Piao, C. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2864373 · Cited by 633
Self-citationNoisy Networks for Exploration
Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., Pietquin, O., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.10295 · Cited by 543
UAV assisted heterogeneous networks for public safety communications
Merwaday, A., & Guvenc, I. (2015) · 2015 IEEE Wireless Communications and Networking Conference Workshops (WCNCW) · doi:10.1109/wcncw.2015.7122576 · Cited by 455
Ultra Reliable UAV Communication Using Altitude and Cooperation Diversity
Azari, M. M., Rosas, F., Chen, K.-C., & Pollin, S. (2018) · IEEE Transactions on Communications · doi:10.1109/tcomm.2017.2746105 · Cited by 448
Wang, C., Wang, J., Shen, Y., & Zhang, X. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890773 · Cited by 443
Rainbow: Combining Improvements in Deep Reinforcement Learning
Hessel, M., Modayil, J., van Hasselt, H. P., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M. G., & Silver, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.02298 · Cited by 369
Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. A. (2017) · HAL · Cited by 362
Energy-Aware Coverage Path Planning of UAVs
Di Franco, C., & Buttazzo, G. (2015) · 2015 IEEE International Conference on Autonomous Robot Systems and Competitions · doi:10.1109/icarsc.2015.17 · Cited by 360
Deployment Algorithms for UAV Airborne Networks Toward On-Demand Coverage
Zhao, H., Wang, H., Wu, W., & Wei, J. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2864376 · Cited by 307
Decentralized model predictive control of cooperating UAVs
Richards, A., & How, J. (2004) · 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601) · doi:10.1109/cdc.2004.1429425 · Cited by 227
Fast Deployment of UAV Networks for Optimal Wireless Coverage
Zhang, X., & Duan, L. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2018.2840143 · Cited by 196
Cooperative UAV Cluster-Assisted Terrestrial Cellular Networks for Ubiquitous Coverage
Wu, H., Tao, X., Zhang, N., & Shen, X. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2864418 · Cited by 178
Q-PrOP: Sample-efficient policy gradient with an off-policy critic
Gu, S., Lillicrap, T., Ghahramani, Z., Turner, R. E., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.17863/cam.21294 · Cited by 174
Coarse-to-Fine UAV Target Tracking With Deep Reinforcement Learning
Zhang, W., Song, K., Rong, X., & Li, Y. (2018) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2018.2877499 · Cited by 152
Efficient 3D placement of a UAV using particle swarm optimization
Shakhatreh, H., Khreishah, A. A., Alsarhan, A., Khalil, I., Sawalmeh, A., & Othman, N. S. (2017) · International Conference on Information and Communication Systems (ICICS) · doi:10.1109/iacs.2017.7921981 · Cited by 140
Communication architectures and protocols for networking unmanned aerial vehicles
Li, J., Zhou, Y., & Lamont, L. (2013) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2013.6825193 · Cited by 129
Toward QoI and Energy-Efficiency in Internet-of-Things Sensory Environments
Liu, C. H., Fan, J., Branch, J. W., & Leung, K. K. (2014) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2014.2364915 · Cited by 106
Self-citationDrone Path Planning for Secure Positioning and Secure Position Verification
Perazzo, P., Sorbelli, F. B., Conti, M., et al. (2016) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2016.2627552 · Cited by 75
Park, S.-Y., Shin, C. S., Jeong, D., & Lee, H. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2870397 · Cited by 70
Coverage Probability of 3-D Mobile UAV Networks
Sharma, P. K., & Kim, D. I. (2018) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2018.2859923 · Cited by 65
Indoor Mobile Coverage Problem Using UAVs
Shakhatreh, H., Khreishah, A. A., & Khalil, I. M. (2018) · IEEE Systems Journal · doi:10.1109/jsyst.2018.2824802 · Cited by 44
A fault-tolerant self-organizing flocking approach for UAV aerial survey
Benedetti, M. D., D’Urso, F., Fortino, G., Messina, F., Pappalardo, G., & Santoro, C. (2017) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2017.08.004 · Cited by 44
A Mission-Oriented Coordination Framework for Teams of Mobile Aerial and Terrestrial Smart Objects
Pace, P., Aloi, G., Caliciuri, G., & Fortino, G. (2016) · Mobile Networks and Applications · doi:10.1007/s11036-016-0726-4 · Cited by 33
Energy-Efficient Event Detection by Participatory Sensing Under Budget Constraints
Liu, C. H., Zhao, J., Zhang, H., et al. (2016) · IEEE Systems Journal · doi:10.1109/jsyst.2016.2533538 · Cited by 27
Self-citation
Cited by
Applications of artificial intelligence and machine learning in smart cities
Ullah, Z., Al-Turjman, F., Mostarda, L., & Gagliardi, R. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.02.069 · Cited by 756
Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey
Azari, M. M., Solanki, S., Chatzinotas, S., Kodheli, O., Sallouha, H., Colpaert, A., Mendoza Montoya, J. F., Pollin, S., Haqiqatnejad, A., Mostaani, A., et al. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3199901 · Cited by 668
Wu, Q., Xu, J., Zeng, Y., Ng, D. W. K., Al-Dhahir, N., Schober, R., & Swindlehurst, A. L. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3088681 · Cited by 562
A Survey on Machine-Learning Techniques for UAV-Based Communications
Bithas, P. S., Michailidis, E. T., Nomikos, N., Vouyioukas, D., & Kanatas, A. G. (2019) · Sensors · doi:10.3390/s19235170 · Cited by 353
Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey
Chen, W., Qiu, X., Cai, T., Dai, H.-N., Zheng, Z., & Zhang, Y. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3073036 · Cited by 311
Metadata
- Authors
- Liu, C. H., Ma, X., Gao, X., & Tang, J.
- Venue
- IEEE Transactions on Mobile Computing
- Published
- 2020-06-01
- Volume
- 19
- Issue
- 6
- Pages
- 1274-1285
- ISSN
- 1536-1233 (ISSN-L)
- 1558-0660
- 2161-9875
- OpenAlex
- W2925554975
- Cited by
- 301
- References
- 40
- FWCI
- 21.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.