Broadband Analog Aggregation for Low-Latency Federated Edge Learning
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
To leverage rich data distributed at the network edge, a new machine-learning paradigm, called edge learning, has emerged where learning algorithms are deployed at the edge for providing intelligent services to mobile users. While computing speeds are advancing rapidly, the communication latency is becoming the bottleneck of fast edge learning. To address this issue, this work is focused on designing a low-latency multi-access scheme for edge learning. To this end, we consider a popular privacy-preserving framework, federated edge learning (FEEL), where a global AI-model at an edge-server is updated by aggregating (averaging) local models trained at edge devices. It is proposed that the updates simultaneously transmitted by devices over broadband channels should be analog aggregated “over-the-air” by exploiting the waveform-superposition property of a multi-access channel. Such broadband analog aggregation (BAA) results in dramatical communication-latency reduction compared with the conventional orthogonal access (i.e., OFDMA). In this work, the effects of BAA on learning performance are quantified targeting a single-cell random network. First, we derive two tradeoffs between communication-and-learning metrics, which are useful for network planning and optimization. The power control (“truncated channel inversion”) required for BAA results in a tradeoff between the update-reliability [as measured by the receive signal-to-noise ratio (SNR)] and the expected update-truncation ratio. Consider the scheduling of cell-interior devices to constrain path loss. This gives rise to the other tradeoff between the receive SNR and fraction of data exploited in learning. Next, the latency-reduction ratio of the proposed BAA with respect to the traditional OFDMA scheme is proved to scale almost linearly with the device population. Experiments based on a neural network and a real dataset are conducted for corroborating the theoretical results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607 · cited 1×
“Since the path-loss exponent α ∈ [3, 4] in practice, the requirement of K > α2 for the above result can be easily satisfied by having the number of edge devices K > 2.”
BackgroundSelf-citationA survey of mobility models for ad hoc network research
Camp, T., Boleng, J., & Davies, V. (2002) · Wireless Communications and Mobile Computing · doi:10.1002/wcm.72 · Cited by 4,621
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766
Mobility increases the capacity of ad hoc wireless networks
Grossglauser, M., & Tse, D. N. C. (2002) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2002.801403 · Cited by 2,839
Variable-rate variable-power MQAM for fading channels
Goldsmith, A. J., & Chua, S.-G. (1997) · IEEE Transactions on Communications · doi:10.1109/26.634685 · Cited by 2,296
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
Lin, Y., Han, S., Mao, H., Wang, Y., & Dally, W. J. (2017) · arXiv · doi:10.48550/arxiv.1712.01887 · Cited by 1,134
Self-citationComputation Over Multiple-Access Channels
Nazer, B., & Gastpar, M. (2007) · IEEE Transactions on Information Theory · doi:10.1109/tit.2007.904785 · Cited by 753
Sparse Communication for Distributed Gradient Descent
Aji, A. F., & Heafield, K. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1045 · Cited by 585
Narrow Band Internet of Things
Chen, M., Miao, Y., Hao, Y., & Hwang, K. (2017) · IEEE Access · doi:10.1109/access.2017.2751586 · Cited by 458
Communication-Efficient Distributed Statistical Inference
Jordan, M. I., Lee, J. D., & Yang, Y. (2019) · Journal of the American Statistical Association · doi:10.1080/01621459.2018.1429274 · Cited by 442
Research works printed but not linked: 14
- 2010, Timing Advance (TA) in LTE
- goodfellow, 2016, Deep Learning
- yang, 2018, Federated learning via over-the-air computation, arXiv 1812 11750
- 10.1109/TWC.2013.022113.120506 · DOI
- srivastava, 2014, Dropout: A simple way to prevent neural networks from overfitting, J Mach Learn Res, 15, 1929
- arunabha, 2010, Fundamentals of LTE
- kamp, 2018, Efficient decentralized deep learning by dynamic model averaging, arXiv 1807 03210
- chen, 2018, LAG: Lazily aggregated gradient for communication-efficient distributed learning, Proc Conf Neural Inf Process Syst (NIPS), 5055
- mcmahan, 2017, Communication-efficient learning of deep networks from decentralized data, Proc Int Conf Artif Intell Statist (AISTATS), 1
- zhu, 2018, Broadband analog aggregation for low-latency federated edge learning (extended version), arXiv 1812 11494
- chen, 2017, Revisiting distributed synchronous SGD, arXiv 1604 00981
- zhu, 2018, Towards an intelligent edge: Wireless communication meets machine learning, arXiv 1809 00343
- tandon, 2017, Gradient coding: Avoiding stragglers in distributed learning, Proc Int Conf Mach Learn (ICML), 3368
- poggi, 2017, 3 Key Internet of Things Trends to Keep Your Eye on in 2017
Cited by
Federated Learning via Over-the-Air Computation
Yang, K., Jiang, T., Shi, Y., & Ding, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2961673 · Cited by 1,084
“This tradeoff between learning and aggregation is also considered in the recent parallel work [30], which quantifies the device population by excluding the devices with weak channel coefficients under deep channel fading and assuming IID located devices.”
BackgroundEdge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“To tackle the channel fading perturbation, a channel inversion method was proposed in [45], [173], [44] by multiplying the inverse of channel gain for the transmit signal, which may however not satisfy the power constraint at edge devices.”
MethodXu, W., Yang, Z., Ng, D. W. K., Levorato, M., Eldar, Y. C., & Debbah, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3239189 · Cited by 660
Toward an Intelligent Edge: Wireless Communication Meets Machine Learning
Zhu, G., Liu, D., Du, Y., You, C., Zhang, J., & Huang, K. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900103 · Cited by 606
Self-citationWu, Q., Xu, J., Zeng, Y., Ng, D. W. K., Al-Dhahir, N., Schober, R., & Swindlehurst, A. L. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3088681 · Cited by 562
Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Mohammadi Amiri, M., & Gunduz, D. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2981904 · Cited by 548
“We also remark that both [32] and [33] consider transmitting model updates, while we focus on gradient transmission, which is more energy-efficient as each device transmits only the innovation obtained through gradient descent at that particular iteration (together with error accumulation), whereas model transmission…”
BackgroundDistributed Learning in Wireless Networks: Recent Progress and Future Challenges
Chen, M., Gunduz, D., Huang, K., Saad, W., Bennis, M., Feljan, A. V., & Poor, H. V. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118346 · Cited by 548
Self-citationCommunication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
Xu, J., & Wang, H. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3031503 · Cited by 466
She, C., Sun, C., Gu, Z., Li, Y., Yang, C., Poor, H. V., & Vucetic, B. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3053601 · Cited by 405
Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues
Du, Z., Wu, C., Yoshinaga, T., Yau, K.-L. A., Ji, Y., & Li, J. (2020) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2020.2992630 · Cited by 393
“In [59], Zhu et al. present a broadband analog aggregation framework for low latency FL in wireless networks.”
BackgroundJoint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning
Shi, W., Zhou, S., Niu, Z., Jiang, M., & Geng, L. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3025446 · Cited by 381
Zhu, G., Du, Y., Gunduz, D., & Huang, K. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3039309 · Cited by 376
“A performance comparison between digital transmission using OFDMA, BAA developed in [6], and the proposed OBDA is presented in Fig.”
BackgroundMethodInfluentialSelf-citationFederated Learning for 6G: Applications, Challenges, and Opportunities
Yang, Z., Chen, M., Wong, K.-K., Poor, H. V., & Cui, S. (2022) · Engineering · doi:10.1016/j.eng.2021.12.002 · Cited by 374
Multi-Armed Bandit-Based Client Scheduling for Federated Learning
Xia, W., Quek, T. Q. S., Guo, K., Wen, W., Yang, H. H., & Zhu, H. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3008091 · Cited by 313
Energy-Efficient Radio Resource Allocation for Federated Edge Learning
Zeng, Q., Du, Y., Huang, K., & Leung, K. K. (2020) · 2020 IEEE International Conference on Communications Workshops (ICC Workshops) · doi:10.1109/iccworkshops49005.2020.9145118 · Cited by 286
Self-citationBlockchain-enabled Federated Learning: A Survey
Qu, Y., Uddin, M. P., Gan, C., Xiang, Y., Gao, L., & Yearwood, J. (2023) · ACM Computing Surveys · doi:10.1145/3524104 · Cited by 286
A survey of federated learning for edge computing: Research problems and solutions
Xia, Q., Ye, W., Tao, Z., Wu, J., & Li, Q. (2021) · High-Confidence Computing · doi:10.1016/j.hcc.2021.100008 · Cited by 279
What is Semantic Communication? A View on Conveying Meaning in the Era of Machine Intelligence
Lan, Q., Wen, D., Zhang, Z., Zeng, Q., Chen, X., Popovski, P., & Huang, K. (2021) · Journal of Communications and Information Networks · doi:10.23919/jcin.2021.9663101 · Cited by 255
Self-citationPrivacy for Free: Wireless Federated Learning via Uncoded Transmission With Adaptive Power Control
Liu, D., & Simeone, O. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036948 · Cited by 234
Device Scheduling with Fast Convergence for Wireless Federated Learning
Shi, W., Zhou, S., & Niu, Z. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9149138 · Cited by 216
Federated Learning via Intelligent Reflecting Surface
Wang, Z., Qiu, J., Zhou, Y., Shi, Y., Fu, L., Chen, W., & Letaief, K. B. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3099505 · Cited by 205
Design and Analysis of Uplink and Downlink Communications for Federated Learning
Zheng, S., Shen, C., & Chen, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041388 · Cited by 191
Federated Machine Learning for Intelligent IoT via Reconfigurable Intelligent Surface
Yang, K., Shi, Y., Zhou, Y., Yang, Z., Fu, L., & Chen, W. (2020) · IEEE Network · doi:10.1109/mnet.011.2000045 · Cited by 184
“Over-the-air computation (AirComp) provides a novel simultaneous access technique to support fast local model aggregation for federated machine learning via exploiting the signal superposition property of multi-access channels [6–8].”
MethodLo, S. K., Lu, Q., Wang, C., Paik, H.-Y., & Zhu, L. (2022) · ACM Computing Surveys · doi:10.1145/3450288 · Cited by 202
Enabling AI in Future Wireless Networks: A Data Life Cycle Perspective
Nguyen, D. C., Cheng, P., Ding, M., Lopez-Perez, D., Pathirana, P. N., Li, J., Seneviratne, A., Li, Y., & Poor, H. V. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3024783 · Cited by 160
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147
Communication-efficient distributed learning: An overview
Cao, X., Başar, T., Diggavi, S., Eldar, Y. C., Letaief, K. B., Poor, H. V., & Zhang, J. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3242710 · Cited by 139
Decentralized Wireless Federated Learning With Differential Privacy
Chen, S., Yu, D., Zou, Y., Yu, J., & Cheng, X. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3145010 · Cited by 100
Toward Communication-Efficient Federated Learning in the Internet of Things With Edge Computing
Sun, H., Li, S., Yu, F. R., Qi, Q., Wang, J., & Liao, J. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2994596 · Cited by 95
“In [32], the framework of BAA is presented that exploited the wave from superposition property of a multiaccess channel for communication efficiency.”
Hybrid Beamforming for Massive MIMO Over-the-Air Computation
Zhai, X., Chen, X., Xu, J., & Kwan Ng, D. W. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3051397 · Cited by 74
Bao, W., Wu, C., Guleng, S., Zhang, J., Yau, K.-L. A., & Ji, Y. (2021) · China Communications · doi:10.23919/jcc.2021.06.004 · Cited by 71
Blind Over-the-Air Computation and Data Fusion via Provable Wirtinger Flow
Dong, J., Shi, Y., & Ding, Z. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2970338 · Cited by 65
Zheng, J., Tian, H., Ni, W., Ni, W., & Zhang, P. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3188502 · Cited by 47
On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing
Chellapandi, V. P., Upadhyay, A., Hashemi, A., & Żak, S. H. (2023) · IEEE Control Systems Letters · doi:10.1109/lcsys.2023.3290470 · Cited by 38
“Other works including [17], [18], [19], [20], [21] study the impact of noise in server-assisted FL.”
BackgroundLi, L., Yang, L., Guo, X., Shi, Y., Wang, H., Chen, W., & Letaief, K. B. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118431 · Cited by 37
Fast Convergence Algorithm for Analog Federated Learning
Xia, S., Zhu, J., Yang, Y., Zhou, Y., Shi, Y., & Chen, W. (2021) · ICC 2021 - IEEE International Conference on Communications · doi:10.1109/icc42927.2021.9500875 · Cited by 33
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications
Park, J., Samarakoon, S., Elgabli, A., Kim, J., Bennis, M., Kim, S.-L., & Debbah, M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3055679 · Cited by 25
Multi-Level Over-the-Air Aggregation of Mobile Edge Computing Over D2D Wireless Networks
Wang, F., & Lau, V. K. N. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3165658 · Cited by 21
Optimal Adaptive Power Control for Over-The-Air Federated Edge Learning Under Fading Channels
Yu, X., Xiao, B., Ni, W., & Wang, X. (2023) · IEEE Transactions on Communications · doi:10.1109/tcomm.2023.3283790 · Cited by 19
Harnessing wireless channels for scalable and privacy-preserving federated learning
Elgabli, A., Park, J., Issaid, C. B., & Bennis, M. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3078783 · Cited by 14
Federated Learning over Noisy Channels
Wei, X., & Shen, C. (2021) · ICC 2021 - IEEE International Conference on Communications · doi:10.1109/icc42927.2021.9500833 · Cited by 14
Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis
Zhang, Z., Wang, R., Li, T., Sun, L., Lau, V. K. N., & Huang, K. (2021) · 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc51858.2021.9593170 · Cited by 5
Self-citationOptimal Power Control for Over-The-Air Federated Edge Learning Using Statistical Channel Knowledge
Yu, X., Xiao, B., Ni, W., & Wang, X. (2022) · 2022 14th International Conference on Wireless Communications and Signal Processing (WCSP) · doi:10.1109/wcsp55476.2022.10039388 · Cited by 4
“• Truncated Channel Inversion [4]: This is a sub-optimal policy typically used in OTA-FEEL systems, where the edge devices amplify the gradients adaptively so that they decrease the scaling factor with the increase of the channel gains in good channel conditions and withhold transmissions in bad…”
Background
Metadata
- Authors
- Zhu, G., Wang, Y., & Huang, K.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2020-01
- Volume
- 19
- Issue
- 1
- Pages
- 491-506
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W2944099577
- Cited by
- 761
- References
- 25
- FWCI
- 45.58
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Funders
- Hong Kong Research Grants Council (17208319, 17209917, 17259416)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.