On Analog Gradient Descent Learning Over Multiple Access Fading Channels
Open accessThe DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.
Abstract
We consider a distributed learning problem over multiple access channel (MAC) using a large wireless network. The computation is made by the network edge and is based on received data from a large number of distributed nodes which transmit over a noisy fading MAC. The objective function is a sum of the nodes' local loss functions. This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning. We develop a novel Gradient-Based Multiple Access (GBMA) algorithm to solve the distributed learning problem over MAC. Specifically, the nodes transmit an analog function of the local gradient using common shaping waveforms and the network edge receives a superposition of the analog transmitted signals used for updating the estimate. GBMA does not require power control or beamforming to cancel the fading effect as in other algorithms, and operates directly with noisy distorted gradients. We analyze the performance of GBMA theoretically, and prove that it can approach the convergence rate of the centralized gradient descent (GD) algorithm in large networks. Specifically, we establish a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient. Furthermore, we provide energy scaling laws for approaching the centralized convergence rate as the number of nodes increases. Finally, experimental results support the theoretical findings, and demonstrate strong performance of GBMA using synthetic and real data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The convergence behavior demonstrates that the noisy iterates typically introduce non-negligible optimality gap in various FL algorithms, e.g., vanilla gradient method [174], quantized gradient method [175], sparsified gradient method [173], and operator splitting method [87].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [174]
Citations
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
Goldsmith, A. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511841224 · Cited by 10,658
Mathematical Analysis of Random Noise
Rice, S. O. (1944) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1944.tb00874.x · Cited by 5,480
Mathematical Analysis of Random Noise
Rice, S. O. (1945) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1945.tb00453.x · Cited by 2,898
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232
Robust Stochastic Approximation Approach to Stochastic Programming
Nemirovski, A., Juditsky, A., Lan, G., & Shapiro, A. (2009) · SIAM Journal on Optimization · doi:10.1137/070704277 · Cited by 1,945
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Konečný, J., McMahan, H. B., Ramage, D., & Richtárik, P. (2016) · arXiv · doi:10.48550/arxiv.1610.02527 · Cited by 1,563
Pegasos: primal estimated sub-gradient solver for SVM
Shalev-Shwartz, S., Singer, Y., Srebro, N., & Cotter, A. (2011) · Mathematical Programming · doi:10.1007/s10107-010-0420-4 · Cited by 1,477
Federated Learning via Over-the-Air Computation
Yang, K., Jiang, T., Shi, Y., & Ding, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2961673 · Cited by 1,084 · cited 1×
“In [15], [16], multiple antennas were used at the receiver, where beamforming was used to maximize the number of devices scheduled for transmission.”
MethodBertin-Mahieux, T., Ellis, D. P. W., Whitman, B., & Lamere, P. (2011) · Columbia University · doi:10.7916/d8nz8j07 · Cited by 944 · cited 1×
“We used the dataset available by UCI Machine Learning Repository [58], extracted from the Million Song Dataset collaborative project between The Echo Nest and LabROSA [59].”
MethodDistributed optimization in sensor networks
Rabbat, M., & Nowak, R. (2004) · Proceedings of the 3rd international symposium on Information processing in sensor networks · doi:10.1145/984622.984626 · Cited by 901 · cited 1×
“Source Localization using a Wireless Sensor Network In this section we consider a parameter estimation problem of localizing a source that emits acoustic waves using wireless sensor networks, as in [1], [47], [48].”
MethodIncremental Adaptive Strategies Over Distributed Networks
Lopes, C. G., & Sayed, A. H. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.896034 · Cited by 732
Federated Learning Over Wireless Fading Channels
Amiri, M. M., & Gunduz, D. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2974748 · Cited by 625 · cited 3×
“In the case of fading channels [10], each node uses power control to cancel the channel effect at the receiver, where nodes that experience deep fading do not transmit.”
BackgroundMachine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Mohammadi Amiri, M., & Gunduz, D. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2981904 · Cited by 548
Fading dispersive communication channels
Kennedy, R. S. (1969) · Medical Entomology and Zoology · Cited by 494
Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks
Abad, M. S. H., Ozfatura, E., GUndUz, D., & Ercetin, O. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054634 · Cited by 424
Beyond the regret minimization barrier
Hazan, E., & Kale, S. (2014) · Journal of Machine Learning Research · doi:10.5555/2627435.2670328 · Cited by 334
LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed LearningPreprint
Chen, T., Giannakis, G. B., Sun, T., & Yin, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09965 · Cited by 329
A Convergent Incremental Gradient Method with a Constant Step Size
Blatt, D., Hero, A. O., & Gauchman, H. (2007) · SIAM Journal on Optimization · doi:10.1137/040615961 · Cited by 306
Lacoste-Julien, S., Schmidt, M., & Bach, F. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1212.2002 · Cited by 292
Energy-Efficient Radio Resource Allocation for Federated Edge Learning
Zeng, Q., Du, Y., Huang, K., & Leung, K. K. (2020) · 2020 IEEE International Conference on Communications Workshops (ICC Workshops) · doi:10.1109/iccworkshops49005.2020.9145118 · Cited by 286
Incremental Stochastic Subgradient Algorithms for Convex Optimization
Ram, S. S., Nedić, A., & Veeravalli, V. V. (2009) · SIAM Journal on Optimization · doi:10.1137/080726380 · Cited by 239
Wireless Federated Learning with Local Differential Privacy
Seif, M., Tandon, R., & Li, M. (2020) · 2020 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9174426 · Cited by 200
Type based estimation over multiaccess channels
Mergen, G., & Tong, L. (2006) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2005.861896 · Cited by 185
Decentralized Detection With Censoring Sensors
Appadwedula, S., Veeravalli, V. V., & Jones, D. L. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.909355 · Cited by 185
Energy Efficient Signal Detection in Sensor Networks Using Ordered Transmissions
Blum, R. S., & Sadler, B. M. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.919101 · Cited by 134
Decentralized source localization and tracking [wireless sensor networks]
Rabbat, M., & Nowak, R. D. (2004) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2004.1326696 · Cited by 131
Type-Based Decentralized Detection in Wireless Sensor Networks
Liu, K., & Sayeed, A. M. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2006.890821 · Cited by 122
Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Data
Ahn, J.-H., Simeone, O., & Kang, J. (2019) · 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc.2019.8904164 · Cited by 116
Energy-Aware Analog Aggregation for Federated Learning with Redundant Data
Sun, Y., Zhou, S., & Gunduz, D. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9148853 · Cited by 111
Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Amiri, M. M., & Gunduz, D. (2019) · 2019 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849334 · Cited by 109
Asymptotic Detection Performance of Type-Based Multiple Access Over Multiaccess Fading Channels
Mergen, G., Naware, V., & Tong, L. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2006.887564 · Cited by 104
On Stochastic Subgradient Mirror-Descent Algorithm with Weighted Averaging
Nedić, A., & Lee, S. (2014) · SIAM Journal on Optimization · doi:10.1137/120894464 · Cited by 98
Competing with the Empirical Risk Minimizer in a Single PassPreprint
Frostig, R., Ge, R., Kakade, S. M., & Sidford, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6606 · Cited by 78
Type-Based Random Access for Distributed Detection Over Multiaccess Fading Channels
Anandkumar, A., & Tong, L. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.896302 · Cited by 74
Stochastic Incremental Gradient Descent for Estimation in Sensor Networks
Ram, S. S., Nedic, A., & Veeravalli, V. V. (2007) · 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers · doi:10.1109/acssc.2007.4487280 · Cited by 66
Nevat, I., Peters, G. W., & Collings, I. B. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2293970 · Cited by 65
Over-the-Air Machine Learning at the Wireless Edge
Amiri, M. M., & Gündüz, D. (2019) · 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2019.8815402 · Cited by 60
Federated Learning Based on Over-the-Air Computation
Yang, K., Jiang, T., Shi, Y., & Ding, Z. J. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761429 · Cited by 58
Collaborative Machine Learning at the Wireless Edge with Blind Transmitters
Amiri, M. M., Duman, T. M., & Gunduz, D. (2019) · 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP) · doi:10.1109/globalsip45357.2019.8969185 · Cited by 58
Decision Fusion Over Noncoherent Fading Multiaccess Channels
Li, F., Evans, J., & Dey, S. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2157501 · Cited by 56
Cohen, K., Nedić, A., & Srikant, R. (2017) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2017.2705559 · Cited by 54
Self-citationCohen, K., & Leshem, A. (2011) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2011.110916 · Cited by 45
Self-citationLow-Latency Broadband Analog Aggregation for Federated Edge Learning.Preprint
Zhu, G., Wang, Y., & Huang, K. (2018) · arXiv (Cornell University) · Cited by 43
Communication Efficient Federated Learning over Multiple Access Channels
Chang, W.-T., & Tandon, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.08737 · Cited by 43
Single-Transmission Distributed Detection via Order Statistics
Braca, P., Maranò, S., & Matta, V. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2180720 · Cited by 39
Ordering for Estimation and Optimization in Energy Efficient Sensor Networks
Blum, R. S. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2116015 · Cited by 38
Energy-Efficient Decision Fusion for Distributed Detection in Wireless Sensor Networks
Sriranga, N., Nagananda, K. G., Blum, R. S., Saucan, A.-A., & Varshney, P. K. (2018) · International Conference on Information Fusion (FUSION) · doi:10.23919/icif.2018.8454976 · Cited by 37
A Likelihood-Based Multiple Access for Estimation in Sensor Networks
Maranò, S., Matta, V., Tong, L., & Willett, P. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.899391 · Cited by 36
Wireless Compressive Sensing Over Fading Channels With Distributed Sparse Random Projections
Wimalajeewa, T., & Varshney, P. K. (2015) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2015.2442156 · Cited by 29
Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)
Zhu, G., Wang, Y., & Huang, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.11494 · Cited by 29
Performance Analysis of Likelihood-Based Multiple Access for Detection Over Fading Channels
Cohen, K., & Leshem, A. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2012.2232348 · Cited by 27
Self-citationHierarchical censoring sensors for change detection
Patwari, N., Hero, A. O., & Sadler, B. M. (2004) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2003.1289329 · Cited by 24
Optimal Resource Allocation for Detection of a Gaussian Process Using a MAC in WSNs
Maya, J. A., Vega, L. J. R., & Galarza, C. G. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2407323 · Cited by 18
Optimizing Pipelined Computation and Communication for Latency-Constrained Edge Learning
Skatchkovsky, N., & Simeone, O. (2019) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2922658 · Cited by 18
Spectrum and Energy Efficient Multiple Access for Detection in Wireless Sensor Networks
Cohen, K., & Leshem, A. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2873512 · Cited by 15
Self-citationOn-Device Federated Learning via Second-Order Optimization with Over-the-Air Computation
Hua, S., Yang, K., & Shi, Y. (2019) · IEEE Vehicular Technology Conference · doi:10.1109/vtcfall.2019.8891310 · Cited by 15
Zhang, J., Chen, Z., Blum, R. S., Lu, X., & Xu, W. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2655492 · Cited by 13
Event Detection in Sensor Networks With Non-Linear Amplifiers via Mixture Series Expansion
Zhang, P., Nevat, I., Peters, G. W., & Clavier, L. (2016) · IEEE Sensors Journal · doi:10.1109/jsen.2016.2592103 · Cited by 10
Asymptotically Consistent One-Bit Detection In Large Sensor Networks
Braca, P., Maranò, S., & Matta, V. (2011) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.42394 · Cited by 9
Exploiting Spatial Correlation in Energy Constrained Distributed DetectionPreprint
Maya, J. A., Galarza, C. G., & Vega, L. J. R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.04119 · Cited by 8
Cohen, K., Nedić, A., & Srikant, R. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472090 · Cited by 6
Self-citationAn Incremental Clustered Gradient Method for Wireless Sensor Networks
Mahmud, A., Adnan, M. A., & Shopon, M. (2018) · Saudi Computer Society National Computer Conference (NCC) · doi:10.1109/ncg.2018.8593074 · Cited by 2
Research works printed but not linked: 2
- 1998, Introductory lectures on convex programming volume i: Basic course, Lecture Notes Series, 3, 51
- bertin-mahieux, 0, Proc Int Conf Music Inf Retrieval
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“The convergence behavior demonstrates that the noisy iterates typically introduce non-negligible optimality gap in various FL algorithms, e.g., vanilla gradient method [174], quantized gradient method [175], sparsified gradient method [173], and operator splitting method [87].”
MethodA Survey on Integrated Sensing, Communication, and Computation
Wen, D., Zhou, Y., Li, X., Shi, Y., Huang, K., & Letaief, K. B. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3521498 · Cited by 363
Over-the-Air Computing for Wireless Data Aggregation in Massive IoT
Zhu, G., Xu, J., Huang, K., & Cui, S. (2021) · IEEE Wireless Communications · doi:10.1109/mwc.011.2000467 · Cited by 235
Privacy for Free: Wireless Federated Learning via Uncoded Transmission With Adaptive Power Control
Liu, D., & Simeone, O. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036948 · Cited by 234
Over-the-Air Federated Learning From Heterogeneous Data
Sery, T., Shlezinger, N., Cohen, K., & Eldar, Y. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3090323 · Cited by 219
Self-citation
Metadata
- Authors
- Sery, T., & Cohen, K.
- Affiliations
- Ben-Gurion University of the Negev (IL)
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2020
- Volume
- 68
- Pages
- 2897-2911
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W2969343398
- Cited by
- 172
- References
- 64
- Citations by year
- 2026: 8 · 2025: 24 · 2024: 29 · 2023: 29 · 2022: 28 · 2021: 37 · 2020: 15
- FWCI
- 12.81
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Signal Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- United States-Israel Binational Science Foundation (2017723)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Sensor Networks and Detection Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- multiple access channel, fading channels, distributed learning, federated learning, noisy gradients, convergence rate, energy scaling law, finite-sample error bounds, large-scale wireless networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.