Design and Analysis of Uplink and Downlink Communications for Federated Learning
The DOI matches its Crossref record; 58 of the 61 references it cites were found in a registry or an index.
Abstract
Communication has been known to be one of the primary bottlenecks of federated learning (FL), and yet existing studies have not addressed the efficient communication design, particularly in wireless FL where both uplink and downlink communications have to be considered. In this paper, we focus on the design and analysis of physical layer quantization and transmission methods for wireless FL. We answer the question of what and how to communicate between clients and the parameter server and evaluate the impact of the various quantization and transmission options of the updated model on the learning performance. We provide new convergence analysis of the well-known FED AVG under non-i.i.d. dataset distributions, partial clients participation, and finite-precision quantization in uplink and downlink communications. These analyses reveal that, in order to achieve anO(1/T) convergence rate with quantization, transmitting the weight requires increasing the quantization level at a logarithmic rate, while transmitting the weight differential can keep a constant quantization level. Comprehensive numerical evaluation on various real-world datasets reveals that the benefit of a FL-tailored uplink and downlink communication design is enormous - a carefully designed quantization and transmission achieves more than 98% of the floating-point baseline accuracy with fewer than 10% of the baseline bandwidth, for majority of the experiments on both i.i.d. and non-i.i.d. datasets. In particular,…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The convergence speeds of FL algorithms were characterized in [245] by considering non-identical dataset distributions, partial edge devices participation, and quantized model updates in both uplink and downlink communications.”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [245]
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Fundamentals of Wireless Communication
Tse, D., & Viswanath, P. (2005) · doi:10.1017/cbo9780511807213 · Cited by 13,162
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
Towards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232
Federated Learning with Non-IID Data
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018) · arXiv · doi:10.48550/arxiv.1806.00582 · Cited by 2,077
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · International Conference on Machine Learning · Cited by 1,590
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Konečný, J., McMahan, H. B., Ramage, D., & Richtárik, P. (2016) · arXiv · doi:10.48550/arxiv.1610.02527 · Cited by 1,563
A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547
A Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2019) · arXiv · doi:10.48550/arxiv.1909.07972 · Cited by 1,547
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364
Energy Efficient Federated Learning Over Wireless Communication Networks
Yang, Z., Chen, M., Saad, W., Hong, C. S., & Shikh-Bahaei, M. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3037554 · Cited by 1,153
A Hybrid Approach to Privacy-Preserving Federated Learning
Truex, S., Baracaldo, N., Anwar, A., Steinke, T., Ludwig, H., Zhang, R., & Zhou, Y. (2019) · Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3338501.3357370 · Cited by 1,029
LEAF: A Benchmark for Federated Settings
Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Konečný, J., McMahan, H. B., Smith, V., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.01097 · Cited by 894
A Survey on Distributed Machine Learning
Verbraeken, J., Wolting, M., Katzy, J., Kloppenburg, J., Verbelen, T., & Rellermeyer, J. S. (2021) · ACM Computing Surveys · doi:10.1145/3377454 · Cited by 872
QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
Alistarh, D., Grubic, D., Li, J., Tomioka, R., & Vojnović, M. (2016) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · doi:10.48550/arxiv.1610.02132 · Cited by 859
Adaptive Federated Optimization
Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečný, J., Kumar, S., & McMahan, H. B. (2020) · arXiv · doi:10.48550/arxiv.2003.00295 · Cited by 840
Federated Learning for Wireless Communications: Motivation, Opportunities, and Challenges
Niknam, S., Dhillon, H. S., & Reed, J. H. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900461 · Cited by 820
Broadband Analog Aggregation for Low-Latency Federated Edge Learning
Zhu, G., Wang, Y., & Huang, K. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2946245 · Cited by 761
Fixed Point Quantization of Deep Convolutional Networks
Lin, D. D., Talathi, S. S., & Annapureddy, V. S. (2015) · arXiv · doi:10.48550/arxiv.1511.06393 · Cited by 684
Federated Learning Over Wireless Fading Channels
Amiri, M. M., & Gunduz, D. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2974748 · Cited by 625
Toward an Intelligent Edge: Wireless Communication Meets Machine Learning
Zhu, G., Liu, D., Du, Y., You, C., Zhang, J., & Huang, K. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900103 · Cited by 606
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1909.13014 · Cited by 587
signSGD: Compressed Optimisation for Non-Convex Problems
Bernstein, J., Wang, Y.-X., Azizzadenesheli, K., & Anandkumar, A. (2018) · arXiv · doi:10.48550/arxiv.1802.04434 · Cited by 541
Fair Resource Allocation in Federated Learning
Li, T., Sanjabi, M., Beirami, A., & Smith, V. (2020) · International Conference on Learning Representations · Cited by 512
Fair Resource Allocation in Federated Learning
Li, T., Sanjabi, M., Beirami, A., & Smith, V. (2019) · arXiv · doi:10.48550/arxiv.1905.10497 · Cited by 475
Xu, J., & Wang, H. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3031503 · Cited by 466
Convergence Time Optimization for Federated Learning Over Wireless Networks
Chen, M., Poor, H. V., Saad, W., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3042530 · Cited by 403
Zhu, G., Du, Y., Gunduz, D., & Huang, K. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3039309 · Cited by 376
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Caldas, S., Konečny, J., McMahan, H. B., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.07210 · Cited by 312
Energy-Efficient Radio Resource Allocation for Federated Edge Learning
Zeng, Q., Du, Y., Huang, K., & Leung, K. K. (2020) · 2020 IEEE International Conference on Communications Workshops (ICC Workshops) · doi:10.1109/iccworkshops49005.2020.9145118 · Cited by 286
Protection Against Reconstruction and Its Applications in Private Federated Learning
Bhowmick, A., Duchi, J. C., Freudiger, J., Kapoor, G., & Rogers, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00984 · Cited by 240
Device Scheduling with Fast Convergence for Wireless Federated Learning
Shi, W., Zhou, S., & Niu, Z. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9149138 · Cited by 216
CoCoA: A General Framework for Communication-Efficient Distributed Optimization
Smith, V., Forte, S., Ma, C., Takáč, M., Jordan, M. I., & Jaggi, M. (2016) · arXiv (Cornell University) · doi:10.3929/ethz-b-000282738 · Cited by 193
Local SGD Converges Fast and Communicates Little
Stich, S. U. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09767 · Cited by 191
Federated Learning With Quantized Global Model UpdatesPreprint
Amiri, M. M., Gündüz, D., Kulkarni, S. R., & Poor, H. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10672 · Cited by 184
William Shakespeare: The Complete Works
Wilson, J. D., Shakespeare, W., & Sisson, C. J. (1956) · The Modern Language Review · doi:10.2307/3718454 · Cited by 184
Energy-Efficient Federated Edge Learning with Joint Communication and Computation Design
Mo, X., & Xu, J. (2021) · Journal of Communications and Information Networks · doi:10.23919/jcin.2021.9475121 · Cited by 164
Wang, J., & Joshi, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07576 · Cited by 146
An Exact Quantized Decentralized Gradient Descent Algorithm
Reisizadeh, A., Mokhtari, A., Hassani, H., & Pedarsani, R. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2932876 · Cited by 138
High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning
Du, Y., Yang, S., & Huang, K. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2983166 · Cited by 116
Federated Learning with Quantization Constraints
Shlezinger, N., Chen, M., Eldar, Y. C., Poor, H. V., & Cui, S. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054168 · Cited by 115
Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization
Haddadpour, F., Kamani, M. M., Mahdavi, M., & Cadambe, V. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.13598 · Cited by 98
Design of fixed-point iterative decoders for concatenated codes with interleavers
Montorsi, G., & Benedetto, S. (2001) · IEEE Journal on Selected Areas in Communications · doi:10.1109/49.924871 · Cited by 89
Federated Learning with Autotuned Communication-Efficient Secure Aggregation
Bonawitz, K., Salehi, F., Konečný, J., McMahan, B., & Gruteser, M. (2019) · 2019 53rd Asilomar Conference on Signals, Systems, and Computers · doi:10.1109/ieeeconf44664.2019.9049066 · Cited by 76
SLSGD: Secure and Efficient Distributed On-device Machine Learning
Xie, C., Koyejo, S., & Gupta, I. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46147-8_13 · Cited by 70
Zeno++: Robust Fully Asynchronous SGD
Xie, C., Koyejo, S., & Gupta, I. (2019) · arXiv · doi:10.48550/arxiv.1903.07020 · Cited by 60
SLSGD: Secure and Efficient Distributed On-device Machine Learning
Xie, C., Koyejo, S., & Gupta, I. (2019) · arXiv · doi:10.48550/arxiv.1903.06996 · Cited by 46
Secure Federated Submodel Learning
Niu, C., Wu, F., Tang, S., Hua, L., Jia, R., Lv, C., Wu, Z., & Chen, G. (2019) · arXiv · doi:10.48550/arxiv.1911.02254 · Cited by 28
(2020) · International Conference on Learning Representations
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges
Chen, M., Gunduz, D., Huang, K., Saad, W., Bennis, M., Feljan, A. V., & Poor, H. V. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118346 · Cited by 548
Federated Learning for 6G: Applications, Challenges, and Opportunities
Yang, Z., Chen, M., Wong, K.-K., Poor, H. V., & Cui, S. (2022) · Engineering · doi:10.1016/j.eng.2021.12.002 · Cited by 374
Rauniyar, A., Hagos, D. H., Jha, D., Håkegård, J. E., Bagci, U., Rawat, D. B., & Vlassov, V. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3329061 · Cited by 315
Machine Learning for Large-Scale Optimization in 6G Wireless Networks
Shi, Y., Lian, L., Shi, Y., Wang, Z., Zhou, Y., Fu, L., Bai, L., Zhang, J., & Zhang, W. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3300664 · Cited by 205
Metadata
- Authors
- Zheng, S., Shen, C., & Chen, X.
- Affiliations
- Sun Yat-sen University (CN)
- University of Virginia (US)
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2021-07
- Volume
- 39
- Issue
- 7
- Pages
- 2150-2167
- ISSN
- 0733-8716 (ISSN-L)
- 1558-0008
- OpenAlex
- W3108383389
- Cited by
- 191
- References
- 61
- Citations by year
- 2026: 17 · 2025: 46 · 2024: 40 · 2023: 49 · 2022: 24 · 2021: 14
- FWCI
- 8.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Journal on Selected Areas in Communications (publishedVersion)
- Funders
- State’s Key Project of Research and Development Plan (2019YFE0196400)
- Guangdong Research and Development Project in Key Areas (2019B010158001)
- Guangdong Provincial Special Fund For Modern Agriculture Industry Technology Innovation Teams (2020KJ122)
- Research Grant from Kneron Inc
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Sensor Networks and Detection Algorithms
- Keywords
- federated learning, wireless federated learning, convergence analysis, non-IID data, partial client participation, 1-bit quantization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.