Federated Reinforcement Learning for Wireless Networks: Fundamentals, Challenges and Future Research Trends
Open accessPublished version of a preprint
Abstract
The increasing popularity of Internet of Things (IoT)-based wireless services highlights the urgent need to upgrade fifth-generation (5G) wireless networks and beyond to accommodate these services. Although 5G networks currently support a variety of wireless services, they might not fully meet the high computational and communication resource demands of new applications. Issues such as latency, energy consumption, network congestion, signaling overhead, and potential privacy breaches contribute to this limitation. Machine learning (ML) frequently offers solutions to these problems. As a result, sixth-generation (6G) wireless technologies are being developed to address the deficiencies of 5G networks. Traditional ML methods are generally centralized. However, the vast amount of wireless data generated, growing privacy concerns, and the increasing computational capabilities of edge devices have led to a shift towards optimizing system performance in a distributed manner. This paper provides a thorough analysis of distributed learning techniques, including federated learning (FL), multi-agent reinforcement learning (MARL), and the multi-agent federated reinforcement learning (FRL) framework. It explains how these techniques can be effectively and efficiently implemented in wireless networks. These methods offer potential solutions to the challenges faced by current wireless networks, promising to create a more robust, capable, and versatile network that meets the growing demands of IoT and other emerging applications. Implementing the FRL framework can significantly improve the learning efficiency of wireless networks. To tackle the challenges posed by rapidly changing radio channels, we propose a robust FRL framework that enables local users to perform distributed power allocation, bandwidth allocation, interference mitigation, and communication mode selection. Finally, the paper outlines several future research directions aimed at effectively integrating the FRL framework into wireless networks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Federated Learning With Differential Privacy: Algorithms and Performance Analysis
Wei, K., Li, J., Ding, M., Ma, C., Yang, H. H., Farokhi, F., Jin, S., Quek, T. Q. S., & Vincent Poor, H. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2988575 · Cited by 2,400 · see citing works
Dynamic computation offloading for mobile-edge computing with energy harvesting devices
Mao, Y., Zhang, J., & Letaief, K. B. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2611964 · Cited by 1,705 · see citing works
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · see citing works
Deep Learning Based Communication Over the Air
Dorner, S., Cammerer, S., Hoydis, J., & Brink, S. T. (2018) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2784180 · Cited by 859 · see citing works
Learning Differentially Private Recurrent Language Models
McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L. (2017) · arXiv · doi:10.48550/arxiv.1710.06963 · Cited by 849 · see citing works
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829 · see citing works
Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications
Aledhari, M., Razzak, R., Parizi, R. M., & Saeed, F. (2020) · IEEE Access · doi:10.1109/access.2020.3013541 · Cited by 733 · see citing works
Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges
Martínez Beltrán, E. T., Pérez, M. Q., Sánchez, P. M. S., Bernal, S. L., Bovet, G., Pérez, M. G., Pérez, G. M., & Celdrán, A. H. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3315746 · Cited by 625 · cited 1× · see citing works
“7 [77], [78], [79], [80], [81], [82], [83]. that account for asynchronous cooperation and predictive techniques is essential for dynamic wireless networks.”
BackgroundA Learning-Based Incentive Mechanism for Federated Learning
Zhan, Y., Li, P., Qu, Z., Zeng, D., & Guo, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2967772 · Cited by 602 · see citing works
Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues
Du, Z., Wu, C., Yoshinaga, T., Yau, K.-L. A., Ji, Y., & Li, J. (2020) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2020.2992630 · Cited by 393 · see citing works
Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning
Shi, W., Zhou, S., Niu, Z., Jiang, M., & Geng, L. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3025446 · Cited by 381 · see citing works
A Crowdsourcing Framework for On-Device Federated Learning
Pandey, S. R., Tran, N. H., Bennis, M., Tun, Y. K., Manzoor, A., & Hong, C. S. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2971981 · Cited by 318 · see citing works
Zhang, X., Peng, M., Yan, S., & Sun, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2962715 · Cited by 282 · see citing works
Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach
Kang, J., Xiong, Z., Niyato, D., Yu, H., Liang, Y.-C., & Kim, D. I. (2019) · 2019 IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS) · doi:10.1109/vts-apwcs.2019.8851649 · Cited by 255 · see citing works
Privacy for Free: Wireless Federated Learning via Uncoded Transmission With Adaptive Power Control
Liu, D., & Simeone, O. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036948 · Cited by 234 · see citing works
Federated Learning for Industrial Internet of Things in Future Industries
Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., Niyato, D., & Poor, H. V. (2021) · IEEE Wireless Communications · doi:10.1109/mwc.001.2100102 · Cited by 166 · see citing works
A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access
Zhong, C., Lu, Z., Gursoy, M. C., & Velipasalar, S. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2952909 · Cited by 123 · see citing works
Flagged first, then by citations
- Authors
- Das, S. K., Mudi, R., Rahman, M. S., Rabie, K. M., & Li, X.
- Affiliations
- Military Institute of Science and Technology (BD)
- Jahangirnagar University (BD)
- Universiti Teknologi Petronas (MY)
- Manchester Metropolitan University (GB)
- Henan Polytechnic University (CN)
- Venue
- IEEE Open Journal of Vehicular Technology
- Published
- 2024
- Volume
- 5
- Pages
- 1400-1440
- ISSN
- 2644-1330 (ISSN-L)
- OpenAlex
- W4402809349
- Cited by
- 27
- References
- 17
- Citations by year
- 2026: 12 · 2025: 14 · 2024: 1
- FWCI
- 3.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Open Journal of Vehicular Technology (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- e-space (Manchester Metropolitan University) (submittedVersion)
- Figshare (submittedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Network Optimization
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Communication Technologies
- Keywords
- federated reinforcement learning, wireless networks, multi-agent reinforcement learning, distributed learning, 6G, Internet of Things, power allocation, bandwidth allocation, interference mitigation, communication mode selection, privacy preservation, edge computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
