Convergence Time Optimization for Federated Learning Over Wireless Networks
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection po…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A probabilistic device scheduling policy was further proposed in [90], [243] to minimize the overall training time in wireless FL.”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [90]
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems
Saad, W., Bennis, M., & Chen, M. (2020) · IEEE Network · doi:10.1109/mnet.001.1900287 · Cited by 4,985 · cited 1×
“However, in wireless networks, it is impractical for wireless users to transmit their training data samples to such a central controller due to privacy concerns and the nature of limited wireless resources [2].”
BackgroundSelf-citationFederated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023 · cited 1×
“distribution [20] or statistical heterogeneity [21].”
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · cited 1×
“The update of the global FL model at iteration µ is given by [23]: where g ( a µ ) is the global FL model at iteration µ , and a µ = [ a 1 ,µ , . . . , a U,µ ] is a user association vector with a i,µ ∈ { 0 , 1 } , a i,µ = 1 indicates that user i connects to the BS and user i must send its local FL…”
BackgroundTowards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232 · cited 1×
“The works in [6] and [7] optimized the FL performance with communication constraints such as limited spectrum and computational resources.”
BackgroundFederated Optimization: Distributed Machine Learning for On-Device Intelligence
Konečný, J., McMahan, H. B., Ramage, D., & Richtárik, P. (2016) · arXiv · doi:10.48550/arxiv.1610.02527 · Cited by 1,563 · cited 3×
“For comparison purposes, we use two baselines: a) an FL algorithm that uses the proposed user association policy without the prediction of users’ local FL models at each given learning step and b) a standard FL algorithm in [20] that randomly determines user selection and resource allocation without using FNNs to esti…”
MethodA Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2019) · arXiv · doi:10.48550/arxiv.1909.07972 · Cited by 1,547 · cited 1×
“The works in [6] and [7] optimized the FL performance with communication constraints such as limited spectrum and computational resources.”
BackgroundSelf-citationA Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547 · cited 1×
“The works in [6] and [7] optimized the FL performance with communication constraints such as limited spectrum and computational resources.”
BackgroundSelf-citationUser Association for Load Balancing in Heterogeneous Cellular Networks
Ye, Q., Rong, B., Chen, Y., Al-Shalash, M., Caramanis, C., & Andrews, J. G. (2013) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2013.040413.120676 · Cited by 1,228 · cited 2×
“Similarly, the downlink data rate of the BS when transmitting the global FL model parameters to each user i is given by [22]”
BackgroundMethodEnergy Efficient Federated Learning Over Wireless Communication Networks
Yang, Z., Chen, M., Saad, W., Hong, C. S., & Shikh-Bahaei, M. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3037554 · Cited by 1,153 · cited 2×
“In [8] and [9], the authors investigated the tradeoff between the FL convergence time and users’ energy consumption.”
BackgroundSelf-citationParallelized Stochastic Gradient Descent
Zinkevich, M., Weimer, M., Li, L., & Smola, A. J. (2010) · Cited by 1,124
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
Chen, M., Challita, U., Saad, W., Yin, C., & Debbah, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2926625 · Cited by 1,117 · cited 1×
“For different learning tasks, the definition of the loss functions can be different [22].”
BackgroundSelf-citationFederated Learning over Wireless Networks: Optimization Model Design and Analysis
Tran, N. H., Bao, W., Zomaya, A., Nguyen, M. N. H., & Hong, C. S. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737464 · Cited by 1,013 · cited 2×
“In [8] and [9], the authors investigated the tradeoff between the FL convergence time and users’ energy consumption.”
BackgroundBlockchained On-Device Federated Learning
Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2020) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2921755 · Cited by 887 · cited 1×
“The authors in [5] proposed a blockchain-based FL scheme that enables edge devices to train FL without sending FL model parameters to a central controller.”
BackgroundBlockchained On-Device Federated Learning
Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2018) · arXiv · doi:10.48550/arxiv.1808.03949 · Cited by 882 · cited 1×
“The authors in [5] proposed a blockchain-based FL scheme that enables edge devices to train FL without sending FL model parameters to a central controller.”
BackgroundScheduling Policies for Federated Learning in Wireless Networks
Yang, H. H., Liu, Z., Quek, T. Q. S., & Poor, H. V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2944169 · Cited by 640 · cited 1×
“The work in [13] derived an analytical model to characterize the training loss of FL in wireless networks and evaluated the effectiveness of different scheduling policies.”
MethodSelf-citationToward an Intelligent Edge: Wireless Communication Meets Machine Learning
Zhu, G., Liu, D., Du, Y., You, C., Zhang, J., & Huang, K. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900103 · Cited by 606
LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed LearningPreprint
Chen, T., Giannakis, G. B., Sun, T., & Yin, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09965 · Cited by 329 · cited 3×
“However, the LAG algorithm in [16] is a deterministic user selection scheme [16] in which some users may not get an opportunity to send their local FL models to the BS thus increasing the training loss.”
BackgroundMethodInfluentialSolving large-scale linear programs by interior-point methods under the Matlab∗Environment†
Zhang, Y. (1998) · Optimization Methods and Software · doi:10.1080/10556789808805699 · Cited by 320 · cited 1×
“Problem (14) is equivalent to (13) and is an integer linear programming problem, which can be solved by known optimization algorithms such as interior-point methods [24].”
BackgroundHybrid deterministic-stochastic methods for data fitting
Friedlander, M. P., & Schmidt, M. (2011) · Cited by 314
Hybrid Deterministic-Stochastic Methods for Data Fitting
Friedlander, M. P., & Schmidt, M. (2012) · SIAM Journal on Scientific Computing · doi:10.1137/110830629 · Cited by 254 · cited 1×
“…µ , x ik , y ik is strongly convex and twice-continuously differentiable, we have the following observations: • ∇ F ( g µ ) is uniformly Lipschitz continuous with respect to g µ [31] and, hence, we can find a positive constant L , such that: where (cid:107) g µ +1 − g µ is the norm of g µ +1 − g µ .”
BackgroundAccelerating DNN Training in Wireless Federated Edge Learning Systems
Ren, J., Yu, G., & Ding, G. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036971 · Cited by 226 · cited 1×
“In [15], the authors define a metric to evaluate the FL performance and optimized the batchsize selection and communication resource allocation for the acceleration of the FL training process.”
BackgroundCell-Free Massive MIMO for Wireless Federated Learning
Vu, T. T., Ngo, D. T., Tran, N. H., Ngo, H. Q., Dao, M. N., & Middleton, R. H. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3002988 · Cited by 194
Convergence analysis of online gradient method for BP neural networks
Wu, W., Wang, J., Cheng, M., & Li, Z. (2011) · Neural Networks · doi:10.1016/j.neunet.2010.09.007 · Cited by 158 · cited 1×
“To enable the FNN to predict each user’s local FL model, the FNN must be trained by the online gradient descent method [24].”
MethodJoint Service Pricing and Cooperative Relay Communication for Federated Learning
Feng, S., Niyato, D., Wang, P., Kim, D. I., & Liang, Y.-C. (2019) · 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) · doi:10.1109/ithings/greencom/cpscom/smartdata.2019.00148 · Cited by 123 · cited 1×
“The authors in [14] studied the use of a relay network to construct a cooperative communication platform for supporting FL model transmission.”
BackgroundFederated Echo State Learning for Minimizing Breaks in Presence in Wireless Virtual Reality Networks
Chen, M., Semiari, O., Saad, W., Liu, X., & Yin, C. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2942929 · Cited by 111 · cited 1×
“The authors in [10] developed an echo state network based FL to predict locations and orientations of wireless virtual reality users so as to minimize their breaks in presence.”
BackgroundSelf-citationMachine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Amiri, M. M., & Gunduz, D. (2019) · 2019 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849334 · Cited by 109
Wireless Network Intelligence at the Edge
Park, J., Samarakoon, S., Bennis, M., & Debbah, M. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2941458 · Cited by 92 · cited 1×
“However, implementing FL over wireless networks faces several challenges [4]–[6], such as selection of users for FL, energy efficiency of FL implementation, computational resource allocation for training FL models at edge devices, spectrum resource allocation for FL parameter transmission, and design of communication-…”
BackgroundConvergence Time Minimization of Federated Learning over Wireless Networks
Chen, M., Poor, H. V., Saad, W., & Cui, S. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9148815 · Cited by 74 · cited 1×
“The work in [14] developed a cell-free massive multiple-input multiple-output (MIMO) system A preliminary version of this work [1] was submitted to IEEE ICC 2020. for the implementation of FL over wireless networks.”
MethodSelf-citationFederated Neuromorphic Learning of Spiking Neural Networks for Low-Power Edge Intelligence
Skatchkovsky, N., Jang, H., & Simeone, O. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053861 · Cited by 35 · cited 5×
“However, most of these existing works [4]–[15] whose goal is to minimize the FL convergence time must sacrifice the training loss of the FL algorithm.”
BackgroundInfluential
Research works printed but not linked: 3
- 2020, Classify MNIST Digits Using a Feedforward Neural Network With MATLAB
- lecun, 2020, The MNIST Database of Handwritten Digits
- 2020, Function Fitting Neural Networks
Cited by
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
“In contrast, the authors in [125] propose a neural network based approach to estimate the local models of FL partici-”
BackgroundA Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2019) · arXiv · doi:10.48550/arxiv.1909.07972 · Cited by 1,547
Self-citationA Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547
Self-citationEnergy Efficient Federated Learning Over Wireless Communication Networks
Yang, Z., Chen, M., Saad, W., Hong, C. S., & Shikh-Bahaei, M. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3037554 · Cited by 1,153
Self-citationEdge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“Specifically, for edge training systems, we need to jointly optimize the subcarrier and bandwidth allocation [241], [88], [90], transmit power and receive beamforming [31], [48], [178], passive beamforming at RIS [48], [178], device selection [31], [242] and activity detection [76], local updates…”
Background
Metadata
- Authors
- Chen, M., Poor, H. V., Saad, W., & Cui, S.
- Affiliations
- Princeton University (US)
- Virginia Tech (US)
- Shenzhen Research Institute of Big Data (CN)
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2021-04
- Volume
- 20
- Issue
- 4
- Pages
- 2457-2471
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W3111182214
- Cited by
- 403
- References
- 28
- Citations by year
- 2026: 28 · 2025: 50 · 2024: 80 · 2023: 69 · 2022: 71 · 2021: 71
- FWCI
- 24.08
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Wireless Communications (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key Research and Development Program of China (2018YFB1800800)
- Key Area R&D Program of Guangdong Province (2018B030338001)
- Shenzhen Outstanding Talents Training Fund
- Guangdong Research Project (2017ZT07X152)
- U.S. National Science Foundation (CCF-1908308, CCF-0939370, CCF-1513915)
- Office of Naval Research (N00014-19-1-2621)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Network Optimization
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- federated learning, convergence time, wireless networks, user selection, uplink resource allocation, resource blocks, artificial neural networks, training loss, base station, handwritten digit recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.