The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach
Open accessAbstract
In this paper, an interference-aware path planning scheme for a network of cellular-connected unmanned aerial vehicles (UAVs) is proposed. In particular, each UAV aims at achieving a tradeoff between maximizing energy efficiency and minimizing both wireless latency and the interference caused on the ground network along its path. The problem is cast as a dynamic game among UAVs. To solve this game, a deep reinforcement learning algorithm, based on echo state network (ESN) cells, is proposed. The introduced deep ESN architecture is trained to allow each UAV to map each observation of the network state to an action, with the goal of minimizing a sequence of time-dependent utility functions. Each UAV uses the ESN to learn its optimal path, transmission power, and cell association vector at different locations along its path. The proposed algorithm is shown to reach a subgame perfect Nash equilibrium upon convergence. Moreover, an upper bound and a lower bound for the altitude of the UAVs are derived thus reducing the computational complexity of the proposed algorithm. The simulation results show that the proposed scheme achieves better wireless latency per UAV and rate per ground user (UE) while requiring a number of steps that are comparable to a heuristic baseline that considers moving via the shortest distance toward the corresponding destinations. The results also show that the optimal altitude of the UAVs varies based on the ground network density and the UE data rate requirements and plays a vital role in minimizing the interference level on the ground UEs as well as the wireless transmission delay of the UAV.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Bertsekas, D. P., & Gallager, R. G. (1987) · Cited by 5,561
Energy-Efficient UAV Communication With Trajectory Optimization
Zeng, Y., & Zhang, R. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2688328 · Cited by 2,164
An Introduction to Game Theory
(2014) · Applying the Strategic Perspective: Problems and Models · doi:10.4135/9781452275567.n5 · Cited by 1,493
Throughput Maximization for UAV-Enabled Mobile Relaying Systems
Zeng, Y., Zhang, R., & Lim, T. J. (2016) · IEEE Transactions on Communications · doi:10.1109/tcomm.2016.2611512 · Cited by 1,315
Synchronization Techniques for Digital Receivers
Mengali, U., & D’Andrea, A. N. (1997) · Springer eBooks · doi:10.1007/978-1-4899-1807-9 · Cited by 1,276
Modeling air-to-ground path loss for low altitude platforms in urban environments
Al-Hourani, A., Kandeepan, S., & Jamalipour, A. (2014) · 2014 IEEE Global Communications Conference · doi:10.1109/glocom.2014.7037248 · Cited by 1,182
Unmanned Aerial Vehicle With Underlaid Device-to-Device Communications: Performance and Tradeoffs
Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2531652 · Cited by 1,126
Self-citationOptimization and applications of echo state networks with leaky- integrator neurons
Jaeger, H., Lukoševičius, M., Popovici, D., & Siewert, U. (2007) · Neural Networks · doi:10.1016/j.neunet.2007.04.016 · Cited by 948 · cited 1×
“In particular, we consider leaky integrator reservoir units [29] for updating the state transition functions x where ω ( n ) j ∈ [0 , 1] is the leaking parameter at layer n for UAV j which relates to the speed of the reservoir dynamics in response to the input, with larger values of ω resulting in…”
Background
Cited by
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Accessing From the Sky: A Tutorial on UAV Communications for 5G and Beyond
Zeng, Y., Wu, Q., & Zhang, R. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2952892 · Cited by 1,589
Applications of artificial intelligence and machine learning in smart cities
Ullah, Z., Al-Turjman, F., Mostarda, L., & Gagliardi, R. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.02.069 · Cited by 756
Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues
Sun, Y., Peng, M., Zhou, Y., Huang, Y., & Mao, S. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2924243 · Cited by 678
Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey
Azari, M. M., Solanki, S., Chatzinotas, S., Kodheli, O., Sallouha, H., Colpaert, A., Mendoza Montoya, J. F., Pollin, S., Haqiqatnejad, A., Mostaani, A., et al. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3199901 · Cited by 668
Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks
Cui, J., Liu, Y., & Nallanathan, A. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2935201 · Cited by 593
6G Wireless Communications Networks: A Comprehensive Survey
Alsabah, M., Naser, M. A., Mahmmod, B. M., Abdulhussain, S. H., Eissa, M. R., Al-Baidhani, A., Noordin, N. K., Sait, S. M., Al-Utaibi, K. A., & Hashim, F. (2021) · IEEE Access · doi:10.1109/access.2021.3124812 · Cited by 575
What Will the Future of UAV Cellular Communications Be? A Flight From 5G to 6G
Geraci, G., Garcia-Rodriguez, A., Azari, M. M., Lozano, A., Mezzavilla, M., Chatzinotas, S., Chen, Y., Rangan, S., & Renzo, M. D. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3171135 · Cited by 535
A Survey on the Convergence of Edge Computing and AI for UAVs: Opportunities and Challenges
McEnroe, P., Wang, S., & Liyanage, M. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3176400 · Cited by 482
Cellular-Enabled UAV Communication: A Connectivity-Constrained Trajectory Optimization Perspective
Zhang, S., Zeng, Y., & Zhang, R. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2880468 · Cited by 426
A Prospective Look: Key Enabling Technologies, Applications and Open Research Topics in 6G Networks
Bariah, L., Mohjazi, L., Muhaidat, S., Sofotasios, P. C., Kurt, G. K., Yanikomeroglu, H., & Dobre, O. A. (2020) · IEEE Access · doi:10.1109/access.2020.3019590 · Cited by 354
A Survey on Machine-Learning Techniques for UAV-Based Communications
Bithas, P. S., Michailidis, E. T., Nomikos, N., Vouyioukas, D., & Kanatas, A. G. (2019) · Sensors · doi:10.3390/s19235170 · Cited by 353
Communication and Control in Collaborative UAVs: Recent Advances and Future Trends
Javaid, S., Saeed, N., Qadir, Z., Fahim, H., He, B., Song, H., & Bilal, M. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3248841 · Cited by 345
Reinforcement Learning in Multiple-UAV Networks: Deployment and Movement Design
Liu, X., Liu, Y., & Chen, Y. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2922849 · Cited by 342
Drone Deep Reinforcement Learning: A Review
Azar, A. T., Koubaa, A., Ali Mohamed, N., Ibrahim, H. A., Ibrahim, Z. F., Kazim, M., Ammar, A., Benjdira, B., Khamis, A. M., Hameed, I. A., et al. (2021) · Electronics · doi:10.3390/electronics10090999 · Cited by 314
“It discusses an interference-aware path planning scheme that allows minimization interference with a ground map and wireless transmission latency caused while sharing online mission-related data [46].”
BackgroundMethodDeep Reinforcement Learning for Internet of Things: A Comprehensive Survey
Chen, W., Qiu, X., Cai, T., Dai, H.-N., Zheng, Z., & Zhang, Y. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3073036 · Cited by 311
A Tutorial on UAVs for Wireless Networks: Applications, Challenges, and Open Problems
Mozaffari, M., Saad, W., Bennis, M., Nam, Y.-H., & Debbah, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2902862 · Cited by 299
Self-citationDeep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., & Shen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988367 · Cited by 298
UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning
Zhang, Y., Mou, Z., Gao, F., Jiang, J., Ding, R., & Han, Z. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3014788 · Cited by 276
Zeng, Y., Xu, X., Jin, S., & Zhang, R. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3056573 · Cited by 264
5G support for Industrial IoT Applications— Challenges, Solutions, and Research gaps
Varga, P., Peto, J., Franko, A., Balla, D., Haja, D., Janky, F., Soos, G., Ficzere, D., Maliosz, M., & Toka, L. (2020) · Sensors · doi:10.3390/s20030828 · Cited by 261
Yang, T., Jiang, Z., Sun, R., Cheng, N., & Feng, H. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.2974047 · Cited by 261
A Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 257
Cognition in UAV-Aided 5G and Beyond Communications: A Survey
Ullah, Z., Al-Turjman, F., & Mostarda, L. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.2968311 · Cited by 226
Deep learning for signal authentication and security in massive internet-of-things systems
Ferdowsi, A., & Saad, W. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2878025 · Cited by 160
Self-citationDistributed Multi-Agent Meta Learning for Trajectory Design in Wireless Drone Networks
Hu, Y., Chen, M., Saad, W., Poor, H. V., & Cui, S. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3088689 · Cited by 152
Self-citationDrones’ Edge Intelligence Over Smart Environments in B5G: Blockchain and Federated Learning Synergy
Alsamhi, S. H., Almalki, F. A., Afghah, F., Hawbani, A., Shvetsov, A. V., Lee, B., & Song, H. (2022) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2021.3132561 · Cited by 139
Edge Computing Assisted Autonomous Flight for UAV: Synergies between Vision and Communications
Chen, Q., Zhu, H., Yang, L., Chen, X., Pollin, S., & Vinogradov, E. (2021) · IEEE Communications Magazine · doi:10.1109/mcom.001.2000501 · Cited by 62
Zhang, D., Yu, F. R., & Yang, R. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3110591 · Cited by 45
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2018) · arXiv · doi:10.48550/arxiv.1810.07862 · Cited by 17
Metadata
- Authors
- Challita, U., Saad, W., & Bettstetter, C.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2019-04
- Volume
- 18
- Issue
- 4
- Pages
- 2125-2140
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W3106530718
- Cited by
- 440
- References
- 32
- FWCI
- 47.59
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.