FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Open accessAbstract
Federated learning is a distributed framework according to which a model is trained over a set of devices, while keeping data localized. This framework faces several systems-oriented challenges which include (i) communication bottleneck since a large number of devices upload their local updates to a parameter server, and (ii) scalability as the federated network consists of millions of devices. Due to these systems challenges as well as issues related to statistical heterogeneity of data and privacy concerns, designing a provably efficient federated learning method is of significant importance yet it remains challenging. In this paper, we present FedPAQ, a communication-efficient Federated Learning method with Periodic Averaging and Quantization. FedPAQ relies on three key features: (1) periodic averaging where models are updated locally at devices and only periodically averaged at the server; (2) partial device participation where only a fraction of devices participate in each round of the training; and (3) quantized message-passing where the edge nodes quantize their updates before uploading to the parameter server. These features address the communications and scalability challenges in federated learning. We also show that FedPAQ achieves near-optimal theoretical guarantees for strongly convex and non-convex loss functions and empirically demonstrate the communication-computation tradeoff provided by our method.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MASM can reduce energy cost and delay while guaranteeing DL inference accuracy by optimizing workload assignment weights and computing capacities of VMs.
“by optimizing the workload assignment weights and computing capacities of VMs, MASM [173] can reduce the energy cost and delay while guaranteeing the DL inference accuracy.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [173]
Cites
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · see citing works
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · see citing works
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575 · see citing works
On First-Order Meta-Learning Algorithms
Nichol, A., Achiam, J., & Schulman, J. (2018) · arXiv · doi:10.48550/arxiv.1803.02999 · Cited by 1,438 · see citing works
The Tradeoffs of Large-Scale Learning
Bottou, L., & Bousquet, O. (2011) · The MIT Press eBooks · doi:10.7551/mitpress/8996.003.0015 · Cited by 1,201 · see citing works
QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
Alistarh, D., Grubic, D., Li, J. Z., Tomioka, R., & Vojnović, M. (2016) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · doi:10.48550/arxiv.1610.02132 · Cited by 1,189 · see citing works
Seide, F., Fu, H., Droppo, J., Li, G., & Yu, D. (2014) · Interspeech · doi:10.21437/interspeech.2014-274 · Cited by 1,172 · see citing works
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
Yin, D., Chen, Y., Ramchandran, K., & Bartlett, P. (2018) · arXiv · doi:10.48550/arxiv.1803.01498 · Cited by 1,130 · see citing works
Federated Learning of Deep Networks using Model AveragingPreprint
McMahan, H. B., Moore, E., Ramage, D., & y Arcas, B. A. (2016) · arXiv (Cornell University) · Cited by 923 · see citing works
Learning Differentially Private Recurrent Language Models
McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L. (2017) · arXiv · doi:10.48550/arxiv.1710.06963 · Cited by 849 · see citing works
Speeding Up Distributed Machine Learning Using Codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2736066 · Cited by 829 · see citing works
Self-citationLocal SGD Converges Fast and Communicates LittlePreprint
Stich, S. U. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09767 · Cited by 679 · see citing works
Reptile: a Scalable Metalearning AlgorithmPreprint
Nichol, A., & Schulman, J. (2018) · arXiv (Cornell University) · Cited by 548 · see citing works
Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
Samarakoon, S., Bennis, M., Saad, W., & Debbah, M. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2956472 · Cited by 459 · see citing works
Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix MultiplicationPreprint
Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10464 · Cited by 439 · see citing works
“Short-Dot”: Computing Large Linear Transforms Distributedly Using Coded Short Dot Products
Dutta, S., Cadambe, V. R., & Grover, P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2927558 · Cited by 362 · see citing works
Wang, J., & Joshi, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07576 · Cited by 302 · see citing works
Don't Use Large Mini-Batches, Use Local SGDPreprint
Lin, T., Stich, S. U., Patel, K. K., & Jaggi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07217 · Cited by 265 · see citing works
Duchi, J. C., Jordan, M. I., & Wainwright, M. J. (2014) · Journal of the ACM · doi:10.1145/2666468 · Cited by 238 · see citing works
On the Convergence of Federated Optimization in Heterogeneous Networks.Preprint
Sahu, A. K., Li, T., Sanjabi, M., Zaheer, M., Talwalkar, A., & Smith, V. (2018) · arXiv (Cornell University) · Cited by 237 · see citing works
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
Chen, Y., Su, L., & Xu, J. (2018) · Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3219617.3219655 · Cited by 228 · see citing works
The landscape of empirical risk for nonconvex losses
Mei, S., Bai, Y. D., & Andrea Montanari, A. (2018) · The Annals of Statistics · doi:10.1214/17-aos1637 · Cited by 216 · see citing works
Robust Federated Learning in a Heterogeneous EnvironmentPreprint
Ghosh, A., Hong, J., Yin, D., & Ramchandran, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.06629 · Cited by 204 · see citing works
CoCoA: A General Framework for Communication-Efficient Distributed OptimizationPreprint
Smith, V., Forte, S., Ma, C., Takáč, M., Jordan, M. I., & Jaggi, M. (2016) · arXiv (Cornell University) · doi:10.3929/ethz-b-000282738 · Cited by 204 · see citing works
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed CommunicationPreprint
Koloskova, A., Stich, S. U., & Jaggi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00340 · Cited by 177 · see citing works
MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling
Wang, J., Sahu, A. K., Yang, Z., Joshi, G., & Kar, S. (2019) · 2019 Sixth Indian Control Conference (ICC) · doi:10.1109/icc47138.2019.9123209 · Cited by 149 · see citing works
Peer-to-peer Federated Learning on Graphs
Lalitha, A., Kilinc, O. C., Javidi, T., & Koushanfar, F. (2019) · arXiv · doi:10.48550/arxiv.1901.11173 · Cited by 146 · see citing works
cpSGD: communication-efficient and differentially-private distributed SGD
Agarwal, N., Suresh, A. T., Yu, F., Kumar, S., & McMahan, H. B. (2018) · neural information processing systems · Cited by 141 · see citing works
An Exact Quantized Decentralized Gradient Descent Algorithm
Reisizadeh, A., Mokhtari, A., Hassani, H., & Pedarsani, R. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2932876 · Cited by 138 · see citing works
Self-citationOne-Shot Federated LearningPreprint
Guha, N., Talwalkar, A., & Smith, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.11175 · Cited by 137 · see citing works
Balancing Communication and Computation in Distributed Optimization
Berahas, A. S., Bollapragada, R., Keskar, N. S., & Wei, E. (2018) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2018.2880407 · Cited by 108 · see citing works
Federated Heavy Hitters Discovery with Differential PrivacyPreprint
Zhu, W., Kairouz, P., McMahan, H. B., Sun, H., & Li, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08534 · Cited by 63 · see citing works
LoAdaBoost:Loss-Based AdaBoost Federated Machine Learning on medical DataPreprint
Li, H., Yin, Y., Fu, Z., Zhang, S., Deng, H., & Liu, D. (2018) · arXiv (Cornell University) · Cited by 57 · see citing works
CodedReduce: A Fast and Robust Framework for Gradient Aggregation in Distributed Learning
Reisizadeh, A., Prakash, S., Pedarsani, R., & Avestimehr, A. S. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3109097 · Cited by 30 · see citing works
Self-citationCompressed Distributed Gradient Descent: Communication-Efficient Consensus over Networks
Zhang, X., Liu, J., Zhu, Z., & Bentley, E. S. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737489 · Cited by 26 · see citing works
Accelerating the Convergence Rates of Distributed Subgradient Methods with Adaptive QuantizationPreprint
Doan, T. T., Maguluri, S. T., & Romberg, J. (2018) · arXiv (Cornell University) · Cited by 8 · see citing works
Flagged first, then by citations
3 of the 40 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · see citing works
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498 · see citing works
“FedPAQ [204] simultaneously incorporates these features and provides near-optimal theoretical guarantees for both strongly convex and non-convex loss functions, while empirically demonstrating the communication-computation tradeoff.”
BackgroundFederated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Vincent Poor, H. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3075439 · Cited by 1,503 · see citing works
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
Wang, J., Liu, Q., Liang, H., Joshi, G., & Poor, H. V. (2020) · arXiv · doi:10.48550/arxiv.2007.07481 · Cited by 927 · see citing works
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges
Khan, L. U., Saad, W., Han, Z., Hossain, E., & Hong, C. S. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3090430 · Cited by 892 · see citing works
Federated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Poor, H. (2021) · Deakin Research Online (Deakin University) · Cited by 889 · see citing works
Personalized Federated Learning with Moreau EnvelopesPreprint
Dinh, C. T., Tran, N. H. K., & Nguyen, T. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08848 · Cited by 644 · see citing works
Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach
Fallah, A., Mokhtari, A., & Ozdaglar, A. (2020) · neural information processing systems · Cited by 596 · see citing works
Self-citationDistributed Learning in Wireless Networks: Recent Progress and Future Challenges
Chen, M., Gündüz, D., Huang, K., Saad, W., Bennis, M., Feljan, A. V., & Poor, H. V. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118346 · Cited by 558 · see citing works
Wahab, O. A., Mourad, A., Otrok, H., & Taleb, T. (2021) · IEEE Communications Surveys and Tutorials · doi:10.1109/comst.2021.3058573 · Cited by 444 · see citing works
FedALA: Adaptive Local Aggregation for Personalized Federated Learning
Zhang, J., Yang, H., Wang, H., Song, T., Xue, Z., Ma, R., & Guan, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26330 · Cited by 466 · see citing works
Personalized Federated Learning: A Meta-Learning ApproachPreprint
Fallah, A., Mokhtari, A., & Ozdaglar, A. (2020) · arXiv.org · doi:10.48550/arxiv.2002.07948 · Cited by 751 · see citing works
Self-citationCommunication-efficient federated learning
Chen, M., Shlezinger, N., Poor, H. V., Eldar, Y. C., & Cui, S. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2024789118 · Cited by 397 · see citing works
Zhu, G., Du, Y., Gunduz, D., & Huang, K. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3039309 · Cited by 376 · see citing works
A survey of federated learning for edge computing: Research problems and solutions
Xia, Q., Ye, W., Tao, Z., Wu, J., & Li, Q. (2021) · High-Confidence Computing · doi:10.1016/j.hcc.2021.100008 · Cited by 290 · see citing works
UVeQFed: Universal Vector Quantization for Federated Learning
Shlezinger, N., Chen, M., Eldar, Y. C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.3046971 · Cited by 275 · see citing works
Privacy for Free: Wireless Federated Learning via Uncoded Transmission With Adaptive Power Control
Liu, D., & Simeone, O. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036948 · Cited by 234 · see citing works
Liu, S., Yu, J., Deng, X., & Wan, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3099368 · Cited by 234 · see citing works
Li, D., Han, D., Weng, T., Zheng, Z., Li, H., Liu, H., Castiglione, A., & Li, K. (2021) · Soft Computing · doi:10.1007/s00500-021-06496-5 · Cited by 208 · see citing works
Design and Analysis of Uplink and Downlink Communications for Federated Learning
Zheng, S., Shen, C., & Chen, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041388 · Cited by 191 · see citing works
Personalized Federated Learning using HypernetworksPreprint
Shamsian, A., Navon, A., Fetaya, E., & Chechik, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.04628 · Cited by 187 · see citing works
Hierarchical Federated Learning With Quantization: Convergence Analysis and System Design
Liu, L., Zhang, J., Song, S., & Letaief, K. B. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3190512 · Cited by 173 · see citing works
Communication and computation efficiency in Federated Learning: A survey
Almanifi, O. R. A., Chow, C., Tham, M.-L., Chuah, J. H., & Kanesan, J. (2023) · Internet of Things · doi:10.1016/j.iot.2023.100742 · Cited by 169 · see citing works
Duan, Q., Huang, J., Hu, S., Deng, R., Lu, Z., & Yu, S. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3316615 · Cited by 175 · see citing works
Aggregation Service for Federated Learning: An Efficient, Secure, and More Resilient Realization
Zheng, Y., Lai, S., Liu, Y., Yuan, X., Yi, X., & Wang, C. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3146448 · Cited by 163 · see citing works
Tighter Theory for Local SGD on Identical and Heterogeneous Data
Khaled, A., Mishchenko, K., & Richtárik, P. (2019) · arXiv · doi:10.48550/arxiv.1909.04746 · Cited by 162 · see citing works
Federated Learning Over Wireless IoT Networks With Optimized Communication and Resources
Chen, H., Huang, S., Zhang, D., Xiao, M., Skoglund, M., & Poor, H. V. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3151193 · Cited by 156 · see citing works
Federated learning in smart cities: Privacy and security survey
Al-Huthaifi, R., Li, T., Huang, W. X., Gu, J., & Li, C. (2023) · Information Sciences · doi:10.1016/j.ins.2023.03.033 · Cited by 160 · see citing works
FedBE: Making Bayesian Model Ensemble Applicable to Federated LearningPreprint
Chen, H.-Y., & Chao, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01974 · Cited by 148 · see citing works
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147 · see citing works
A Survey of Recent Advances in Optimization Methods for Wireless Communications
Liu, Y., Chang, T., Hong, M., Wu, Z., So, A. M., Jorswieck, E. A., & Yu, W. (2024) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2024.3443759 · Cited by 153 · see citing works
Dynamic Scheduling for Over-the-Air Federated Edge Learning With Energy Constraints
Sun, Y., Zhou, S., Niu, Z., & Gündüz, D. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126078 · Cited by 143 · see citing works
Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
Yang, C., Wang, Q., Xu, M., Chen, Z., Bian, K., Liu, Y., & Liu, X. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449851 · Cited by 141 · see citing works
A Novel Framework for the Analysis and Design of Heterogeneous Federated Learning
Wang, J., Liu, Q., Liang, H., Joshi, G., & Poor, H. V. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3106104 · Cited by 140 · see citing works
Federated Learning with Compression: Unified Analysis and Sharp GuaranteesPreprint
Haddadpour, F., Kamani, M. M., Mokhtari, A., & Mahdavi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01154 · Cited by 140 · see citing works
Self-citationReisizadeh, A., Tziotis, I., Hassani, H., Mokhtari, A., & Pedarsani, R. (2022) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2022.3205475 · Cited by 137 · see citing works
Self-citationCommunication-efficient distributed learning: An overview
Cao, X., Başar, T., Diggavi, S., Eldar, Y. C., Letaief, K. B., Poor, H. V., & Zhang, J. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3242710 · Cited by 139 · see citing works
Model Compression for Communication Efficient Federated Learning
Shah, S. M., & Lau, V. K. N. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3131614 · Cited by 129 · see citing works
Multi-Stage Hybrid Federated Learning Over Large-Scale D2D-Enabled Fog Networks
Hosseinalipour, S., Azam, S. S., Brinton, C. G., Michelusi, N., Aggarwal, V., Love, D. J., & Dai, H. (2022) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2022.3143495 · Cited by 138 · see citing works
Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning
Jhunjhunwala, D., Gadhikar, A., Joshi, G., & Eldar, Y. C. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413697 · Cited by 126 · see citing works
Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation
Wang, S., Lee, M., Hosseinalipour, S., Morabito, R., Chiang, M., & Brinton, C. G. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488906 · Cited by 123 · see citing works
Quantized Federated Learning Under Transmission Delay and Outage Constraints
Wang, Y., Xu, Y., Shi, Q.-J., & Chang, T.-H. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126081 · Cited by 138 · see citing works
AI augmented Edge and Fog computing: Trends and challenges
Tuli, S., Mirhakimi, F., Pallewatta, S., Zawad, S., Casale, G., Javadi, B., Yan, F., Buyya, R., & Jennings, N. R. (2023) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2023.103648 · Cited by 141 · see citing works
Adaptive Control of Local Updating and Model Compression for Efficient Federated Learning
Xu, Y., Liao, Y., Xu, H., Ma, Z., Wang, L., & Liu, J. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3186936 · Cited by 120 · see citing works
Communication-Efficient Federated Learning Based on Compressed Sensing
Li, C., Li, G., & Varshney, P. K. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3073112 · Cited by 116 · see citing works
Transmission Power Control for Over-the-Air Federated Averaging at Network Edge
Cao, X., Zhu, G., Xu, J., & Cui, S. R. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3143217 · Cited by 115 · see citing works
Green Edge AI: A Contemporary Survey
Mao, Y., Yu, X., Huang, K., Zhang, Y. A., & Zhang, J. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3437365 · Cited by 122 · see citing works
FedAS: Bridging Inconsistency in Personalized Federated Learning
Yang, X., Huang, W., & Ye, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01139 · Cited by 115 · see citing works
Federated Learning With Quantized Global Model UpdatesPreprint
Amiri, M. M., Gündüz, D., Kulkarni, S. R., & Poor, H. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10672 · Cited by 111 · see citing works
Duan, Q., Hu, S., Deng, R., & Lu, Z. (2022) · Sensors · doi:10.3390/s22165983 · Cited by 109 · see citing works
FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning
Xiong, Y., Wang, R., Cheng, M., Yu, F., & Hsieh, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01566 · Cited by 110 · see citing works
Context-Aware Online Client Selection for Hierarchical Federated Learning
Qu, Z., Duan, R., Chen, L., Xu, J., Lu, Z., & Liu, Y. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3186960 · Cited by 112 · see citing works
Federated Accelerated Stochastic Gradient DescentPreprint
Yuan, H., & Ma, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08950 · Cited by 106 · see citing works
Guo, H., Liu, A., & Lau, V. K. N. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3002925 · Cited by 148 · see citing works
Dynamic Aggregation for Heterogeneous Quantization in Federated Learning
Chen, S.-B., Shen, C., Zhang, L., & Tang, Y. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3076613 · Cited by 151 · see citing works
Computation and Communication Efficient Federated Learning With Adaptive Model Pruning
Jiang, Z., Xu, Y., Xu, H., Wang, Z., Liu, J., Qian, C., & Qiao, C. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3247798 · Cited by 105 · see citing works
Kang, J., Li, X., Nie, J., Liu, Y., Xu, M., Xiong, Z., Niyato, D. T., & Qiang, Y. (2022) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2022.3178970 · Cited by 102 · see citing works
Cluster-driven Graph Federated Learning over Multiple Domains
Caldarola, D., Mancini, M., Galasso, F., Ciccone, M., Rodolà, E., & Barbara Caputo, B. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00309 · Cited by 97 · see citing works
Accelerating Decentralized Federated Learning in Heterogeneous Edge Computing
Wang, L., Xu, Y., Xu, H., Chen, M., & Huang, L. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3178378 · Cited by 96 · see citing works
Trusted AI in Multiagent Systems: An Overview of Privacy and Security for Distributed Learning
Ma, C., Li, J., Wei, K., Liu, B., Ding, M., Yuan, L., Han, Z., & Vincent Poor, H. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3306773 · Cited by 96 · see citing works
Security and Privacy Threats to Federated Learning: Issues, Methods, and Challenges
Zhang, J., Zhu, H., Wang, F., Zhao, J., Xu, Q., & Li, H. (2022) · Security and Communication Networks · doi:10.1155/2022/2886795 · Cited by 94 · see citing works
Federated Learning via Unmanned Aerial Vehicle
Fu, M., Shi, Y.-M., & Zhou, Y. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3303492 · Cited by 91 · see citing works
Federated Learning Over Noisy Channels: Convergence Analysis and Design Examples
Wei, X., & Shen, C. (2022) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2022.3140788 · Cited by 90 · see citing works
Tariq, A., Serhani, M. A., Sallabi, F., Barka, E. S., Qayyum, T., Khater, H. M., & Shuaib, K. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3438264 · Cited by 110 · see citing works
Federated Learning With Spiking Neural Networks
Venkatesha, Y., Kim, Y., Tassiulas, L., & Panda, P. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3121632 · Cited by 87 · see citing works
A comprehensive review on blockchains for Internet of Vehicles: Challenges and directions
Hildebrand, B., Baza, M., Salman, T., Tabassum, S., Konatham, B., Amsaad, F., & Razaque, A. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2023.100547 · Cited by 88 · see citing works
A Comprehensive Empirical Study of Heterogeneity in Federated Learning
Abdelmoniem, A. M., Ho, C.-Y., Papageorgiou, P., & Canini, M. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3250275 · Cited by 85 · see citing works
Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning
Ji, S., Jiang, W., Walid, A., & Li, X. (2021) · IEEE Intelligent Systems · doi:10.1109/mis.2021.3114610 · Cited by 82 · see citing works
Mohammadi, S., Balador, A., Sinaei, S., & Flammini, F. (2024) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2024.104918 · Cited by 88 · see citing works
Chen, R., Li, L., Xue, K.-P., Zhang, C., Pan, M., & Fang, Y. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3213766 · Cited by 87 · see citing works
Towards efficient communications in federated learning: A contemporary survey
Zhao, Z., Mao, Y., Liu, Y., Song, L., Ouyang, Y., Chen, X., & Ding, W. (2023) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2022.12.053 · Cited by 80 · see citing works
Qiao, C., Li, M., Liu, Y., & Tian, Z. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3399612 · Cited by 83 · see citing works
Distilled One-Shot Federated LearningPreprint
Zhou, Y., Pu, G., Ma, X., Li, X., & Wu, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07999 · Cited by 79 · see citing works
A Review of Federated Learning in Agriculture
Žalik, K. R., & Žalik, M. (2023) · Sensors · doi:10.3390/s23239566 · Cited by 86 · see citing works
Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning
Lee, S., Zhang, T., & Avestimehr, A. S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26023 · Cited by 84 · see citing works
A survey: Distributed Machine Learning for 5G and beyond
Nassef, O., Sun, W., Purmehdi, H., Tatipamula, M., & Mahmoodi, T. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.108820 · Cited by 76 · see citing works
Secure Smart Communication Efficiency in Federated Learning: Achievements and Challenges
Pouriyeh, S., Shahid, O., Parizi, R. M., Sheng, Q. Z., Srivastava, G., Zhao, L., & Nasajpour, M. (2022) · Applied Sciences · doi:10.3390/app12188980 · Cited by 165 · see citing works
Neural network quantization in federated learning at the edge
Tonellotto, N., Gotta, A., Nardini, F. M., Gadler, D., & Silvestri, F. (2021) · Information Sciences · doi:10.1016/j.ins.2021.06.039 · Cited by 69 · see citing works
Liu, H., He, F., & Cao, G. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10228970 · Cited by 69 · see citing works
Evaluating the Communication Efficiency in Federated Learning Algorithms
Asad, M., Moustafa, A., Itō, T., & Aslam, M. (2021) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd49262.2021.9437738 · Cited by 68 · see citing works
1-Bit Compressive Sensing for Efficient Federated Learning Over the Air
Fan, X., Wang, Y., Huo, Y., & Tian, Z. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3209190 · Cited by 67 · see citing works
Joint Privacy Enhancement and Quantization in Federated Learning
Lang, N., Sofer, E., Shaked, T., & Shlezinger, N. (2023) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2023.3244092 · Cited by 69 · see citing works
Chai, Z., Chen, Y., Zhao, L., Cheng, Y., & Rangwala, H. (2020) · arXiv (Cornell University) · Cited by 67 · see citing works
Meta-heuristic as manager in federated learning approaches for image processing purposes
Połap, D., & Woźniak, M. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107872 · Cited by 65 · see citing works
A Systematic Survey of Blockchained Federated LearningPreprint
Wang, Z., Hu, Q., Xu, M., Zhuang, Y., Wang, Y., & Cheng, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02182 · Cited by 86 · see citing works
Communication-efficient distributed AI strategies for the IoT edge
Mwase, C., Jin, Y., Westerlund, T., Tenhunen, H., & Zou, Z. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.01.013 · Cited by 64 · see citing works
Communication Efficiency in Federated Learning: Achievements and ChallengesPreprint
Shahid, O., Pouriyeh, S., Parizi, R. M., Sheng, Q. Z., Srivastava, G., & Zhao, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10996 · Cited by 62 · see citing works
On the Convergence of Communication-Efficient Local SGD for Federated Learning
Gao, H., Xu, A. P., & Huang, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16920 · Cited by 59 · see citing works
FedMes: Speeding Up Federated Learning With Multiple Edge Servers
Han, D.-J., Choi, M., Park, J., & Moon, J. J. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118422 · Cited by 58 · see citing works
Lu, Z., Zhong, C., & Gursoy, M. C. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3131534 · Cited by 58 · see citing works
Communication-Efficient Federated Learning with Accelerated Client Gradient
Kim, G., Kim, J., & Han, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01177 · Cited by 57 · see citing works
Feraudo, A., Yadav, P., Safronov, V., Popescu, D. A., Mortier, R., Wang, S., Bellavista, P., & Crowcroft, J. (2020) · doi:10.1145/3378679.3394528 · Cited by 57 · see citing works
Enabling All In-Edge Deep Learning: A Literature Review
Joshi, P., Hasanuzzaman, M., Thapa, C., Afli, H., & Scully, T. (2023) · IEEE Access · doi:10.1109/access.2023.3234761 · Cited by 60 · see citing works
Scalable and Low-Latency Federated Learning With Cooperative Mobile Edge Networking
Zhang, Z., Gao, Z., Guo, Y., & Gong, Y. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3216837 · Cited by 55 · see citing works
Partial Synchronization to Accelerate Federated Learning Over Relay-Assisted Edge Networks
Qu, Z., Guo, S., Wang, H., Ye, B., Wang, Y., Zomaya, A. Y., & Tang, B. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3083154 · Cited by 54 · see citing works
A Primal-Dual SGD Algorithm for Distributed Nonconvex Optimization
Yi, X., Zhang, S., Yang, T., Chai, T., & Johansson, K. H. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105554 · Cited by 53 · see citing works
Differential Privacy Meets Federated Learning Under Communication Constraints
Mohammadi, N., Bai, J., Fan, Q., Song, Y., Yi, Y., & Liu, L. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3101991 · Cited by 55 · see citing works
Han, D.-J., Hosseinalipour, S., Love, D. J., Chiang, M., & Brinton, C. G. (2024) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2024.3365901 · Cited by 53 · see citing works
Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning
Elgabli, A., Park, J., Bedi, A. S., Issaid, C. B., Bennis, M., & Aggarwal, V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3026398 · Cited by 55 · see citing works
- Authors
- Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- Pages
- 2021-2031
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1909.13014
- OpenAlex
- W2975043678
- Cited by
- 587
- References
- 40
- Citations by year
- 2025: 41 · 2024: 146 · 2023: 156 · 2022: 96 · 2021: 100 · 2020: 46 · 2019: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Artificial Intelligence and Statistics
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Sensor Networks and Detection Algorithms
- Keywords
- federated learning, communication efficiency, scalability, computation-communication trade-off, strongly convex loss, non-convex loss
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review