One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis
Open accessThe DOI matches its Crossref record; 35 of the 37 references it cites were found in a registry or an index.
Abstract
Federated edge learning (FEEL) is a popular framework for model training at an edge server using data distributed at edge devices (e.g., smart-phones and sensors) without compromising their privacy. In the FEEL framework, edge devices periodically transmit high-dimensional stochastic gradients to the edge server, where these gradients are aggregated and used to update a global model. When the edge devices share the same communication medium, the multiple access channel (MAC) from the devices to the edge server induces a communication bottleneck. To overcome this bottleneck, an efficient broadband analog transmission scheme has been recently proposed, featuring the aggregation of analog modulated gradients (or local models) via the waveform-superposition property of the wireless medium. However, the assumed linear analog modulation makes it difficult to deploy this technique in modern wireless systems that exclusively use digital modulation. To address this issue, we propose in this work a novel digital version of broadband over-the-air aggregation, called one-bit broadband digital aggregation (OBDA). The new scheme features one-bit gradient quantization followed by digital quadrature amplitude modulation (QAM) at edge devices and over-the-air majority-voting based decoding at edge server. We provide a comprehensive analysis of the effects of wireless channel hostilities (channel noise, fading, and channel estimation errors) on the convergence rate of the proposed FEEL scheme. The analysis shows that the hostilities slow down the convergence of the learning process by introducing a scaling factor and a bias term into the gradient norm. However, we show that all the negative effects vanish as the number of participating devices grows, but at a different rate for each type of channel hostility.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607 · cited 1×
“(Aggregated quantized local gradient estimates) g̃ = [ g̃[1] , g̃[2] , .”
Self-citationFederated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766
A Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2019) · arXiv · doi:10.48550/arxiv.1909.07972 · Cited by 1,547 · cited 1×
“In [21], a novel bandwidth allocation strategy is proposed for minimizing the model training loss of FEEL via convergence analysis accounting for packet transmission errors.”
MethodA Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547 · cited 1×
“In [21], a novel bandwidth allocation strategy is proposed for minimizing the model training loss of FEEL via convergence analysis accounting for packet transmission errors.”
MethodDeep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
Lin, Y., Han, S., Mao, H., Wang, Y., & Dally, W. J. (2017) · arXiv · doi:10.48550/arxiv.1712.01887 · Cited by 1,134 · cited 1×
“The last approach focuses on update compression by exploiting the sparsity of gradient updates [12], [13] and low-resolution gradient/model-parameter quantization [14]–[16].”
BackgroundFederated Learning via Over-the-Air Computation
Yang, K., Jiang, T., Shi, Y., & Ding, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2961673 · Cited by 1,084 · cited 4×
“Existing works on over-the-air aggregation, as well as its application in the context of FEEL [6], [7], [17]–[19] consider analog modulation assuming that the transmitter can modulate the carrier waveform as desired, freely choosing the I/Q coefficients as arbitrary real number.”
BackgroundMethodInfluentialQSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
Alistarh, D., Grubic, D., Li, J., Tomioka, R., & Vojnović, M. (2016) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · doi:10.48550/arxiv.1610.02132 · Cited by 859
Broadband Analog Aggregation for Low-Latency Federated Edge Learning
Zhu, G., Wang, Y., & Huang, K. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2946245 · Cited by 761 · cited 8×
“A performance comparison between digital transmission using OFDMA, BAA developed in [6], and the proposed OBDA is presented in Fig.”
BackgroundMethodInfluentialSelf-citationComputation Over Multiple-Access Channels
Nazer, B., & Gastpar, M. (2007) · IEEE Transactions on Information Theory · doi:10.1109/tit.2007.904785 · Cited by 753 · cited 1×
“In [23], function computation over a MAC is studied from a fundamental information theoretic point of view, assuming independent and identically distributed (i.”
BackgroundScheduling Policies for Federated Learning in Wireless Networks
Yang, H. H., Liu, Z., Quek, T. Q. S., & Poor, H. V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2944169 · Cited by 640
Federated Learning Over Wireless Fading Channels
Amiri, M. M., & Gunduz, D. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2974748 · Cited by 625 · cited 3×
“Existing works on over-the-air aggregation, as well as its application in the context of FEEL [6], [7], [17]–[19] consider analog modulation assuming that the transmitter can modulate the carrier waveform as desired, freely choosing the I/Q coefficients as arbitrary real number.”
BackgroundToward an Intelligent Edge: Wireless Communication Meets Machine Learning
Zhu, G., Liu, D., Du, Y., You, C., Zhang, J., & Huang, K. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900103 · Cited by 606
Self-citationFedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1909.13014 · Cited by 587 · cited 2×
“Finally, it is worth emphasizing that the current work differs from the closely related work [15], [16] in that our focus is on designing customized wireless communication techniques for implementing FEEL over a wireless network rather than developing new federated learning algorithms.”
BackgroundMachine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Mohammadi Amiri, M., & Gunduz, D. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2981904 · Cited by 548
Revisiting Distributed Synchronous SGD
Pan, X., Chen, J., Monga, R., Bengio, S., & Józefowicz, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05800 · Cited by 548 · cited 1×
“The first is to discard the updates from slow-responding edge devices (stragglers) for fast update synchronization [8], [9].”
BackgroundsignSGD: Compressed Optimisation for Non-Convex Problems
Bernstein, J., Wang, Y.-X., Azizzadenesheli, K., & Anandkumar, A. (2018) · arXiv · doi:10.48550/arxiv.1802.04434 · Cited by 541
Gradient Coding: Avoiding Stragglers in Distributed Learning
Tandon, R., Lei, Q., Dimakis, A. G., & Karampatziakis, N. (2017) · International Conference on Machine Learning · Cited by 431
Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks
Abad, M. S. H., Ozfatura, E., GUndUz, D., & Ercetin, O. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054634 · Cited by 424 · cited 2×
“In [20], a hierarchical FEEL framework is introduced in a cellular network.”
BackgroundConvergence Time Optimization for Federated Learning Over Wireless Networks
Chen, M., Poor, H. V., Saad, W., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3042530 · Cited by 403
LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed LearningPreprint
Chen, T., Giannakis, G. B., Sun, T., & Yin, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09965 · Cited by 329
Linear Coherent Decentralized Estimation
Xiao, J.-J., Cui, S., Luo, Z.-Q., & Goldsmith, A. J. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.906762 · Cited by 277 · cited 1×
“From an implementation perspective, techniques for distributed power control and robust design against channel estimation errors are proposed in [26] and [27], respectively.”
BackgroundMIMO Over-the-Air Computation for High-Mobility Multimodal Sensing
Zhu, G., & Huang, K. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2871070 · Cited by 273 · cited 2×
“More recently, AirComp techniques are designed for multiple-input-multiple-output (MIMO) systems for enabling spatially multiplexed vector-valued function computation in [27]–[29].”
MethodSelf-citationSparse Binary Compression: Towards Distributed Deep Learning with minimal Communication
Sattler, F., Wiedemann, S., Muller, K.-R., & Samek, W. (2019) · 2019 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852172 · Cited by 240 · cited 1×
“The last approach focuses on update compression by exploiting the sparsity of gradient updates [12], [13] and low-resolution gradient/model-parameter quantization [14]–[16].”
BackgroundWhen Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8486403 · Cited by 209
Robust Analog Function Computation via Wireless Multiple-Access Channels
Goldenbaum, M., & Stanczak, S. (2013) · IEEE Transactions on Communications · doi:10.1109/tcomm.2013.072913.120815 · Cited by 187 · cited 1×
“in [24] to wireless MACs and a more general class of non-linear functions.”
BackgroundEfficient Decentralized Deep Learning by Dynamic Model Averaging
Kamp, M., Adilova, L., Sicking, J., Hüger, F., Schlicht, P., Wirtz, T., & Wrobel, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10925-7_24 · Cited by 146
Li, X., Zhu, G., Gong, Y., & Huang, K. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2914046 · Cited by 139 · cited 9×
“This novel design will allow implementing AirComp across devices that are endowed with digital modulation chips, without requiring significant changes in the hardware or the communication architecture.”
MethodInfluentialSelf-citationOver-the-air Function Computation in Sensor Networks
Abari, O., Rahul, H., & Katabi, D. (2016) · arXiv · doi:10.48550/arxiv.1612.02307 · Cited by 114 · cited 1×
“A practical implementation is presented in [25].”
Wen, D., Zhu, G., & Huang, K. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2934956 · Cited by 92 · cited 2×
“The framework is extended to wirelessly-powered AirComp [29] and massive MIMO AirComp systems [30].”
MethodSelf-citationSignal Processing and Optimal Resource Allocation for the Interference Channel
Hong, M., & Luo, Z.-Q. T. (2014) · Academic press library in signal processing · doi:10.1016/b978-0-12-396500-4.00008-9 · Cited by 88
On the Channel Estimation Effort for Analog Computation over Wireless Multiple-Access Channels
Goldenbaum, M., & Stanczak, S. (2014) · IEEE Wireless Communications Letters · doi:10.1109/wcl.2014.022314.140005 · Cited by 64 · cited 1×
“From an implementation perspective, techniques for distributed power control and robust design against channel estimation errors are proposed in [26] and [27], respectively.”
BackgroundCollaborative Machine Learning at the Wireless Edge with Blind Transmitters
Amiri, M. M., Duman, T. M., & Gunduz, D. (2019) · 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP) · doi:10.1109/globalsip45357.2019.8969185 · Cited by 58 · cited 1×
“Subsequently, channel-state information (CSI) requirement for overthe-air aggregation is relaxed by exploiting multiple antennas at the edge server in [18].”
BackgroundNatasha 2: Faster Non-Convex Optimization Than SGD
Allen-Zhu, Z. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.08694 · Cited by 50 · cited 2×
“Assumptions 2 and 3 below, on the Lipschitz smoothness and bounded variance, respectively, are standard in the stochastic optimization literature [34].”
Background
Research works printed but not linked: 2
- arunabha, 2010, Fundamentals of LTE
- 2010, Timing Advance (TA) in LTE
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“, vanilla gradient method [174], quantized gradient method [175], sparsified gradient method [173], and operator splitting method [87].”
MethodFederated learning for smart healthcare: A survey
Nguyen, D. C., Pham, Q.-V., Pathirana, P. N., Ding, M., Seneviratne, A., Lin, Z., Dobre, O., & Hwang, W.-J. (2023) · ACM Computing Surveys · doi:10.1145/3501296 · Cited by 827
“A recent promising solution for model aggregation is called over-the-air aggregation, which exploits the properties of wireless channels to wirelessly aggregate local updates [59].”
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Gunduz, D., Qin, Z., Aguerri, I. E., Dhillon, H. S., Yang, Z., Yener, A., Wong, K. K., & Chae, C.-B. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3223408 · Cited by 776
Xu, W., Yang, Z., Ng, D. W. K., Levorato, M., Eldar, Y. C., & Debbah, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3239189 · Cited by 660
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges
Chen, M., Gunduz, D., Huang, K., Saad, W., Bennis, M., Feljan, A. V., & Poor, H. V. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118346 · Cited by 548
Self-citation
Metadata
- Authors
- Zhu, G., Du, Y., Gunduz, D., & Huang, K.
- Affiliations
- Shenzhen Research Institute of Big Data (CN)
- University of Hong Kong (HK)
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2021-03
- Volume
- 20
- Issue
- 3
- Pages
- 2120-2135
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W3108590825
- Cited by
- 376
- References
- 37
- Citations by year
- 2026: 30 · 2025: 45 · 2024: 61 · 2023: 93 · 2022: 68 · 2021: 58
- FWCI
- 23.57
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Wireless Communications (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62001310)
- National Key Research and Development Program of China (2018YFB1800800)
- Guangdong Province Key Area Research and Development Program (2018B030338001)
- Shenzhen Peacock Plan (KQTD2015033114415450)
- European Research Council (ERC) through the Starting Grant BEACON (677854)
- U.K. EPSRC (EP/T023600/1)
- Guangdong Basic and Applied Basic Research Foundation (2019B1515130003)
- Hong Kong Research Grants Council (17208319, 17209917)
- Innovation and Technology Fund (GHP/016/18GD)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Sensor Networks and Detection Algorithms
- Keywords
- federated edge learning, over-the-air aggregation, 1-bit quantization, convergence analysis, channel estimation error, fading, channel noise, communication bottleneck, edge devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.