FedCau: A Proactive Stop Policy for Communication and Computation Efficient Federated Learning
Open accessAbstract
This paper investigates efficient distributed training of a Federated Learning (FL) model over a wireless network of wireless devices. The communication iterations of the distributed training algorithm may be substantially deteriorated or even blocked by the effects of the devices’ background traffic, packet losses, congestion, or latency. We abstract the communication-computation impacts as an ‘iteration cost’ and propose a cost-aware causal FL algorithm (FedCau) to tackle this problem. We propose an iteration-termination method that trade-offs the training performance and networking costs. We apply our approach when workers use the slotted-ALOHA, carrier-sense multiple access with collision avoidance (CSMA/CA), and orthogonal frequency-division multiple access (OFDMA) protocols. We show that, given a total cost budget, the training performance degrades as either the background communication traffic or the dimension of the training problem increases. Our results demonstrate the importance of proactively designing optimal cost-efficient stopping criteria to avoid unnecessary communication-computation costs to achieve a marginal FL training improvement. We validate our method by training and testing FL over the MNIST and CIFAR-10 dataset. Finally, we apply our approach to existing communication efficient FL methods from the literature, achieving further efficiency. We conclude that cost-efficient stopping criteria are essential for the success of practical FL over wireless networks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When frequency-division multiple-access (FDMA) protocols, including OFDMA, are considered, each selected client would enjoy much narrower bandwidths, which would prolong their model uploading time at the cost of their model training time.
“On the other hand, when frequency-division multiple-access (FDMA) protocols, including orthogonal frequency-division multiple-access (OFDMA) protocols, are considered, each selected client would enjoy much narrower bandwidths, which would prolong their model uploading time at the cost of their model training time [180].”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [180]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601 · see citing works
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · see citing works
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,398 · see citing works
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · see citing works
Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data
Sattler, F., Wiedemann, S., Muller, K.-R., & Samek, W. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944481 · Cited by 1,725 · see citing works
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575 · see citing works
A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547 · see citing works
Energy Efficient Federated Learning Over Wireless Communication Networks
Yang, Z., Chen, M., Saad, W., Hong, C. S., & Shikh-Bahaei, M. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3037554 · Cited by 1,153 · see citing works
Casini, E., Gaudenzi, R. D., & Herrero, O. R. (2007) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2007.348337 · Cited by 707 · see citing works
Yu, H., Yang, S., & Zhu, S.-H. (2018) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015693 · Cited by 696 · see citing works
Linear and Nonlinear Optimization
Griva, I., Nash, S. G., & Sofer, A. (2009) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898717730 · Cited by 639 · see citing works
CSMA/CA performance under high traffic conditions: throughput and delay analysis
Ziouva, E., & Antonakopoulos, T. A. (2002) · Computer Communications · doi:10.1016/s0140-3664(01)00369-3 · Cited by 553 · see citing works
A Very Brief Introduction to Machine Learning With Applications to Communication Systems
Simeone, O. (2018) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2018.2881442 · Cited by 516 · see citing works
Luo, S., Chen, X., Wu, Q., Zhou, Z., & Yu, S. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3003744 · Cited by 456 · see citing works
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Caldas, S., Konečny, J., McMahan, H. B., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.07210 · Cited by 312 · see citing works
Deep Unfolding for Communications Systems: A Survey and Some New Directions
Balatsoukas‐Stimming, A., & Studer, C. (2019) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips47522.2019.9020494 · Cited by 262 · see citing works
Application Aware Workload Allocation for Edge Computing-Based IoT
Fan, Q., & Ansari, N. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2826006 · Cited by 224 · see citing works
Delay distributions of slotted ALOHA and CSMA
Yang, Y., & Yum, T.-S. P. (2003) · IEEE Transactions on Communications · doi:10.1109/tcomm.2003.819201 · Cited by 152 · see citing works
Deep Learning's Diminishing Returns: The Cost of Improvement is Becoming Unsustainable
Thompson, N. C., Greenewald, K., Lee, K., & Manso, G. F. (2021) · IEEE Spectrum · doi:10.1109/mspec.2021.9563954 · Cited by 139 · see citing works
Efficient Federated Learning Algorithm for Resource Allocation in Wireless IoT Networks
Nguyen, V., Sharma, S. K., Vu, T. X., Chatzinotas, S., & Ottersten, B. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3022534 · Cited by 136 · see citing works
Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning
Sun, J., Chen, T., Giannakis, G. B., Yang, Q., & Yang, Z. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3033286 · Cited by 102 · see citing works
OFDMA Uplink Scheduling in IEEE 802.11ax Networks
Bankov, D., Didenko, A., Khorov, E., et al. (2018) · International Conference on Communications · doi:10.1109/icc.2018.8422767 · Cited by 84 · see citing works
Communication-Efficient Distributed Learning via Lazily Aggregated Quantized GradientsPreprint
Sun, J., Chen, T., Giannakis, G. B., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.07588 · Cited by 60 · see citing works
A discrete approach to the concept of derivative
Weigand, H.-G. (2014) · ZDM · doi:10.1007/s11858-014-0595-x · Cited by 31 · see citing works
Efficient Lossy Compression for Scientific Data Based on Pointwise Relative Error Bound
Di, S., Tao, D., Liang, X., et al. (2018) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2018.2859932 · Cited by 25 · see citing works
Communication-Efficient Decentralized Event Monitoring in Wireless Sensor Networks
Yuan, K., Ling, Q. H., & Tian, Z. (2014) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2014.2350474 · Cited by 17 · see citing works
Cost-efficient Distributed optimization In Machine Learning Over Wireless Networks
Mahmoudi, A., Shokri‐Ghadikolaei, H., & Fischione, C. (2020) · International Conference on Communications · doi:10.1109/icc40277.2020.9149216 · Cited by 14 · see citing works
Self-citationJoint Prioritized Scheduling and Resource Allocation for OFDMA-Based Wireless Networks
Hoang, T. D., & Le, L. B. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2765309 · Cited by 12 · see citing works
Efficient Learning on High-dimensional Operational Data
Samani, F. S., Zhang, H., & Stadler, R. (2019) · International Conference on Network and Service Management (CNSM) · doi:10.23919/cnsm46954.2019.9012741 · Cited by 11 · see citing works
Machine Learning over Networks: Co-design of Distributed Optimization and Communications
Mahmoudi, A., Shokri‐Ghadikolaei, H., & Fischione, C. (2020) · IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc48557.2020.9154264 · Cited by 4 · see citing works
Self-citationYu, H., Yang, S., & Zhu, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06629 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 12
- J. Park, S. Samarakoon, M. Bennis, and M. Debbah, "Wireless network intelligence at the edge," Proc. IEEE, vol. 107, no. 11, pp. 2204-2239, Nov. 2019.
- S. U Stich, J.-B. Cordonnier, and M. Jaggi, "Sparsified SGD with memory," in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 4447-4458.
- J. Wangni, J. Wang, J. Liu, and T. Zhang, "Gradient sparsification for communication-efficient distributed optimization," in Proc. Int. Conf. Adv. Neural Inf. Process. Syst., 2018, pp. 1299-1309.
- T. Chen, G. Giannakis, T. Sun, and W. Yin, "LAG: Lazily aggregated gradient for communication-efficient distributed learning," in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 5050-5060.
- K. Hsieh et al., "GAIA: Geo-distributed machine learning approaching $LAN $ speeds," in Proc. 14th {USENIX} Symp. Netw. Syst. Design Implement. (NSDI), 2017, pp. 629-647.
- A. Defazio et al., "SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives," in Proc. Adv. Neural Inf. Process. Syst., vol. 27. Red Hook, NY, USA: Curran Associates, 2014, pp. 1-9.
- A. Mahmoudi, H. S. Ghadikolaei, J. M. B. Da Silva Júnior, and C. Fischione, "FedCau: A proactive stop policy for communication and computation efficient federated learning," 2022, arXiv:2204.07773. · arXiv
- D. Bertsekas et al., Data Networks, vol. 2, 2nd ed. Upper Saddle River, NJ, USA: Prentice-Hall, 2004.
- K. Koh et al., "An interior-point method for large-scale ℓ 1 -regularized logistic regression," J. Mach. Learn. Res., vol. 8, no. Jul, pp. 1519-1555, 2007.
- Afsaneh Mahmoudi (Graduate Student Member, IEEE) received the bachelor's degree from the Sharif University of Technology, Tehran, Iran, in 2015, and the master's degree in electrical engi- neering from the University of Tehran, Tehran, in 2018. She is currently pursuing the Ph.D. degree with the Department of Electrical Engineering and Computer Science, KTH Royal Institute of Technol- ogy, Stockholm, Sweden. Her research area focuses on federated learning, cell-free massive MIMO, machine learning, and optimization.
- Hossein S. Ghadikolaei received the B.Sc. degree in electrical engineering from the Iran University of Science and Technology in 2009, the M.Sc. degree in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2011, and the Ph.D. degree in electrical engineering and computer science from the KTH Royal Institute of Technology, Stockholm, Sweden, in 2018. He is cur- rently with Ericsson Research, Kista, Sweden. His research interests include distributed optimization and machine learning, with applications in commu- nication networks. He was a recipient of the IEEE Communications Society
- Stephen O. Rice Prize, in 2018, the Premium Award for the Best Paper in IET Communications, in 2014, and the Program of Excellence Award from KTH, in 2013, among others. He was selected as an Exemplary Reviewer for the IEEE TRANSACTIONS ON COMMUNICATIONS, in 2017 and 2018, and served as an Associate Editor of IEEE COMMUNICATIONS LETTERS.
Cited by
Sorted by citations
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
“Fairness measures, such as proportional fairness and max-min fairness, could also be considered when jointly designing the backdoor defense and WFL operations.”
Background
- Authors
- Mahmoudi, A., Ghadikolaei, H. S., Barros Da Silva, J. M., & Fischione, C.
- Affiliations
- KTH Royal Institute of Technology (SE)
- Ericsson, Stockholm, Sweden
- Department of Information Technology, Uppsala University, Uppsala, Sweden
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2024-09
- Volume
- 23
- Issue
- 9
- Pages
- 11076-11093
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W4393170838
- Cited by
- 16
- References
- 33
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 7 · 2023: 1 · 2022: 1
- FWCI
- 2.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Wireless Communications (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- Uppsala University Publications (Uppsala University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- European Union’s Horizon Europe Research and Innovation Program under the Marie Skłodowska-Curie Project FLASH (101067652)
- Ericsson Research Foundation
- Hans Werthén Foundation
- Digital Futures and VR
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Sensor Networks and Detection Algorithms
- Keywords
- federated learning, communication efficiency, stopping rules, wireless networks, background traffic, packet loss, network congestion, slotted ALOHA, CSMA/CA, OFDMA, computation-communication trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
