On Model Transmission Strategies in Federated Learning With Lossy Communications
Abstract
Recently, federated learning (FL) has received tremendous attention in both academia and industry, in which decentralized clients collaboratively complete model training by exchanging model updates with a parameter server through the Internet. Its distributed nature well utilizes the localized data and preserves clients’ privacy, but also incurs heavy communication overhead. Existing studies on model update have mostly focused on the bandwidth constraint of the communication channels. Today's Internet however is highly unreliable. Simply using Transmission Control Protocol (TCP) would lead to low network utilization under frequent losses. In this paper, we closely examine the optimal transmission strategies in FL over the realistic lossy Internet. We systematically integrate model compression, forward error correction (FEC) and retransmission towards Federated Learning with Lossy Communications (FedLC). We derive the convergence rate of FedLC under non-convex loss with the optimal transmission. We then decompose this non-convex problem and present effective practical solutions. Public datasets are exploited for performance evaluation by varying the packet loss rate from 10% to 50%. In a fixed training time budget, FedLC can improve model accuracy by 3.91% on average or reduce the communication traffic by 34.27%-47.57% in comparison with state-of-the-art baselines.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
TCP can ensure the reliable delivery of data across a network but requires one or multiple retransmissions in each communication cycle.
“For instance, the TCP can ensure the reliable delivery of data across a network but requires one or multiple retransmissions in each communication cycle [200].”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [200]
Cites
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · see citing works
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232 · see citing works
Padhye, J. D., Firoiu, V., Towsley, D., & Kurose, J. (1998) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/285237.285291 · Cited by 2,107 · see citing works
Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data
Sattler, F., Wiedemann, S., Muller, K.-R., & Samek, W. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944481 · Cited by 1,725 · see citing works
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575 · cited 1× · see citing works
“It has been proved that the convergence of FedAvg can be guaranteed under both strongly convex loss [18] and non-convex loss [19].”
BackgroundThe macroscopic behavior of the TCP congestion avoidance algorithm
Mathis, M., Semke, J., Mahdavi, J., & Ott, T. J. (1997) · ACM SIGCOMM Computer Communication Review · doi:10.1145/263932.264023 · Cited by 1,496 · see citing works
Client-Edge-Cloud Hierarchical Federated Learning
Liu, L., Zhang, J., Song, S. H., & Letaief, K. B. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9148862 · Cited by 1,033 · see citing works
Federated Learning over Wireless Networks: Optimization Model Design and Analysis
Tran, N. H., Bao, W., Zomaya, A. Y., Nguyen, M. N. H., & Hong, C. (2019) · IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737464 · Cited by 943 · see citing works
Stich, S. U., Cordonnier, J.-B., & Jaggi, M. (2018) · arXiv · doi:10.48550/arxiv.1809.07599 · Cited by 508 · see citing works
Federated learning over wireless networks Convergence analysis and resource allocation
Dinh, C. T., Tran, N. H., Nguyen, M. N. H., Hong, C. S., Bao, W., Zomaya, A. Y., & Gramoli, V. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2020.3035770 · Cited by 483 · see citing works
Shen, M., Tang, X., Zhu, L., Du, X., & Guizani, M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2901840 · Cited by 443 · see citing works
Wu, Y., Dai, H., Wang, H., Xiong, Z., & Guo, S. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3158270 · Cited by 324 · see citing works
Lim, W. Y. B., Ng, J. S., Xiong, Z., Jin, J., Zhang, Y., Niyato, D. T., Leung, C., & Miao, C. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3096076 · Cited by 320 · see citing works
Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computing
Mills, J., Hu, J. W., & Min, G. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3098467 · Cited by 303 · see citing works
Communication-Efficient Federated Learning With Compensated Overlap-FedAvg
Zhou, Y., Ye, Q., & Lv, J. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3090331 · Cited by 237 · see citing works
Self-citationTackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling
Luo, B., Xiao, W., Wang, S., Huang, J., & Tassiulas, L. (2022) · IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · doi:10.1109/infocom48880.2022.9796935 · Cited by 229 · see citing works
AUCTION: Automated and Quality-Aware Client Selection Framework for Efficient Federated Learning
Deng, Y., Lyu, F., Ren, J., Wu, H., Zhou, Y., Zhang, Y., & Shen, X. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3134647 · Cited by 220 · see citing works
Self-citationTang, F., Zhou, Y., & Kato, N. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3005495 · Cited by 216 · see citing works
Self-citationTCP Friendly Rate Control (TFRC): Protocol Specification
Floyd, S., Handley, M. J., Padhye, J., et al. (2008) · doi:10.17487/rfc5348 · Cited by 204 · see citing works
Li, L., Shi, D., Hou, R.-H., Li, H., Pan, M., & Han, Z. (2020) · IEEE Conference on Computer Communications · doi:10.1109/infocom42981.2021.9488839 · Cited by 198 · see citing works
Design and Analysis of Uplink and Downlink Communications for Federated Learning
Zheng, S., Shen, C., & Chen, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041388 · Cited by 191 · see citing works
Decentralized Federated Learning With Unreliable Communications
Ye, H., Liang, L., & Li, G. Y. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3152445 · Cited by 184 · see citing works
FDC: A Secure Federated Deep Learning Mechanism for Data Collaborations in the Internet of Things
Yin, B., Yin, H., Wu, Y., & Jiang, Z. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2966778 · Cited by 149 · see citing works
Flexible Clustered Federated Learning for Client-Level Data Distribution Shift
Duan, M., Liu, D., Ji, X., Wu, Y., Liang, L., Chen, X., Tan, Y., & Ren, A. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3134263 · Cited by 129 · see citing works
Diversified Technologies in Internet of Vehicles Under Intelligent Edge Computing
Lv, Z., Chen, D., & Wang, Q. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3019756 · Cited by 122 · see citing works
Packet Loss Characterization in WiFi-Based Long Distance Networks
Sheth, A. N., Nedevschi, S., Patra, R. K., et al. (2007) · doi:10.1109/infcom.2007.44 · Cited by 118 · see citing works
Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization
Perazzone, J. B., Wang, S., Ji, M., & Chan, K. (2022) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom48880.2022.9796818 · Cited by 90 · see citing works
Secure and Efficient Blockchain-Assisted Authentication for Edge-Integrated Internet-of-Vehicles
Shen, M., Lu, H., Wang, F., Liu, H., & Zhu, L. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3194008 · Cited by 90 · see citing works
P-FedAvg: Parallelizing Federated Learning with Theoretical Guarantees
Zhong, Z., Zhou, Y., Wu, D., Chen, X., Chen, M., Li, C., & Sheng, Q. Z. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488877 · Cited by 72 · see citing works
Self-citationConvergence Analysis and System Design for Federated Learning Over Wireless Networks
Wan, S., Lu, J., Fan, P., Shao, Y., Peng, C., & Letaief, K. B. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118351 · Cited by 65 · see citing works
Optimal Rate Adaption in Federated Learning with Compressed Communications
Cui, L., Su, X., Zhou, Y., & Liu, J. (2022) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom48880.2022.9796982 · Cited by 63 · see citing works
Self-citationOn the Convergence of Communication-Efficient Local SGD for Federated Learning
Gao, H., Xu, A. P., & Huang, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16920 · Cited by 59 · see citing works
Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates
Cui, L., Su, X., Zhou, Y., & Pan, Y. F. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087262 · Cited by 53 · see citing works
Self-citationAn Efficient Statistical-based Gradient Compression Technique for Distributed Training SystemsPreprint
Abdelmoniem, A. M., Elzanaty, A., Alouini, M., & Canini, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.10761 · Cited by 48 · see citing works
Yu, X., Modestino, J. W., Kurceren, R., et al. (2008) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2007.900416 · Cited by 46 · see citing works
Performance evaluation of Forward Error Correction in ATM networks
Biersack, E. W. (1992) · ACM SIGCOMM Computer Communication Review · doi:10.1145/144191.144294 · Cited by 42 · see citing works
DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated LearningPreprint
Hoenig, R., Zhao, Y., & Mullins, R. (2021) · arXiv (Cornell University) · doi:10.17863/cam.104286 · Cited by 41 · see citing works
Anomaly Detection using Distributed Log Data: A Lightweight Federated Learning Approach
Guo, Y., Wu, Y., Zhu, Y., Yang, B., & Han, C. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533294 · Cited by 37 · see citing works
Ellis, M., Pezaros, D. P., Kypraios, T., et al. (2014) · Computer Networks · doi:10.1016/j.comnet.2014.05.013 · Cited by 36 · see citing works
Modeling Best-Effort and FEC Streaming of Scalable Video in Lossy Network Channels
Kang, S.-R., & Loguinov, D. (2007) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2006.890110 · Cited by 35 · see citing works
DC2: Delay-aware Compression Control for Distributed Machine Learning
Abdelmoniem, A. M., & Canini, M. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488810 · Cited by 33 · see citing works
Intra-flow loss recovery and control for VoIP
Sanneck, H., Le, N. T. L., Wolisz, A., et al. (2001) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/500141.500206 · Cited by 28 · see citing works
CFedAvg: Achieving Efficient Communication and Fast Convergence in Non-IID Federated Learning
Yang, H., Liu, J., & Bentley, E. S. (2021) · International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt) · doi:10.23919/wiopt52861.2021.9589061 · Cited by 21 · see citing works
Self-citationEffects of MAC retransmission on TCP performance in IEEE 802.11-based ad-hoc networks
Wang, Y., Yu, K., Liu, Y., et al. (2005) · IEEE Vehicular Technology Conference (VTC-Spring) · doi:10.1109/vetecs.2004.1390665 · Cited by 14 · see citing works
Interaction Between Hidden Node Collisions and Congestions in Multihop Wireless Ad-hoc Networks
Khalifé, H., & Malouch, N. (2006) · International Conference on Communications · doi:10.1109/icc.2006.255698 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
“For instance, the TCP can ensure the reliable delivery of data across a network but requires one or multiple retransmissions in each communication cycle [200].”
Background
- Authors
- Su, X., Zhou, Y., Cui, L., & Liu, J.
- Affiliations
- Shenzhen University (CN)
- Macquarie University (AU)
- Simon Fraser University (CA)
- Venue
- IEEE Transactions on Parallel and Distributed Systems
- Published
- 2023
- Pages
- 1-14
- ISSN
- 1045-9219 (ISSN-L)
- 1558-2183
- 2161-9883
- OpenAlex
- W4319865686
- Cited by
- 47
- References
- 46
- Citations by year
- 2026: 8 · 2025: 11 · 2024: 21 · 2023: 7
- FWCI
- 4.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Parallel and Distributed Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- federated learning, lossy communication, packet loss, transmission strategies, model compression, forward error correction, retransmission, convergence rate, non-convex optimization, communication overhead, unreliable networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
