Adaptive Federated Optimization
Open accessAbstract
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own training data. Standard federated optimization methods such as Federated Averaging (FedAvg) are often difficult to tune and exhibit unfavorable convergence behavior. In non-federated settings, adaptive optimization methods have had notable success in combating such issues. In this work, we propose federated versions of adaptive optimizers, including Adagrad, Adam, and Yogi, and analyze their convergence in the presence of heterogeneous data for general non-convex settings. Our results highlight the interplay between client heterogeneity and communication efficiency. We also perform extensive experiments on these methods and show that the use of adaptive optimizers can significantly improve the performance of federated learning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MPC approaches to private information retrieval break down into computational PIR, in which a single party can execute the entire server side, and information theoretic PIR, in which multiple non-colluding parties are required to execute the server side.
“MPC approaches to PIR break down into two main categories: computational PIR (cPIR), in which a single party can execute the entire server side of the protocol [286], and information theoretic PIR (itPIR), in which multiple non-colluding parties are required to execute the server side of the protocol [121].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [121]
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Self-citation(2010) · Cited by 8,261 · see citing works
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Self-citationFederated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · see citing works
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023 · see citing works
Self-citationWu, Y., & He, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_1 · Cited by 3,601 · see citing works
Towards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334 · see citing works
Self-citationAccurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · see citing works
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232 · see citing works
Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
Johnson, R., & Zhang, T. (2013) · Cited by 1,815 · see citing works
EMNIST: Extending MNIST to handwritten letters
Cohen, G., Afshar, S., Tapson, J. C., & André, V. S. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966217 · Cited by 1,782 · see citing works
On the Convergence of Adam and Beyond
Reddi, S. J., Kale, S., & Kumar, S. (2019) · arXiv · doi:10.48550/arxiv.1904.09237 · Cited by 1,771 · see citing works
Self-citationMeasuring the Effects of Non-Identical Data Distribution for Federated Visual ClassificationPreprint
Hsu, T., Qi, & Brown, M. (2019) · arXiv.org · doi:10.48550/arxiv.1909.06335 · Cited by 1,757 · see citing works
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., & Suresh, A. T. (2019) · arXiv · doi:10.48550/arxiv.1910.06378 · Cited by 1,734 · see citing works
Self-citationOn the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575 · see citing works
One weird trick for parallelizing convolutional neural networksPreprint
Krizhevsky, A. (2014) · arXiv.org · doi:10.48550/arxiv.1404.5997 · Cited by 1,435 · see citing works
Parallelized Stochastic Gradient Descent
Zinkevich, M., Weimer, M., Li, L., & Smola, A. J. (2010) · Cited by 1,124 · see citing works
LEAF: A Benchmark for Federated Settings
Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Konečný, J., McMahan, H. B., Smith, V., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.01097 · Cited by 894 · see citing works
Self-citationYu, H., Yang, S., & Zhu, S.-H. (2018) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015693 · Cited by 696 · see citing works
Local SGD Converges Fast and Communicates LittlePreprint
Stich, S. U. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09767 · Cited by 679 · see citing works
Li, W., & McCallum, A. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143917 · Cited by 612 · see citing works
Lookahead Optimizer: k steps forward, 1 step backPreprint
Zhang, M. R., Lucas, J., Hinton, G., & Ba, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.08610 · Cited by 470 · see citing works
Stochastic Variance Reduction for Nonconvex OptimizationPreprint
Reddi, S. J., Hefny, A., Sra, S., Póczos, B., & Smola, A. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06160 · Cited by 415 · see citing works
Self-citationSAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite ObjectivesPreprint
Defazio, A., Bach, F., & Lacoste-Julien, S. (2014) · arXiv (Cornell University) · Cited by 372 · see citing works
Adaptive Gradient Methods with Dynamic Bound of Learning RatePreprint
Luo, L., Xiong, Y., Liu, Y., & Sun, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09843 · Cited by 353 · see citing works
The Non-IID Data Quagmire of Decentralized Machine LearningPreprint
Hsieh, K., Phanishayee, A., Mutlu, O., & Gibbons, P. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00189 · Cited by 337 · see citing works
Wang, J., & Joshi, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07576 · Cited by 302 · see citing works
How to Escape Saddle Points EfficientlyPreprint
Jin, C., Ge, R., Netrapalli, P., Kakade, S. M., & Jordan, M. I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00887 · Cited by 291 · see citing works
Adaptive Methods for Nonconvex Optimization
Zaheer, M., Reddi, S. J., Sachan, D. S., Kale, S., & Kumar, S. (2018) · neural information processing systems · Cited by 267 · see citing works
Self-citationSCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning.
Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., & Suresh, A. T. (2019) · Cited by 227 · see citing works
Self-citationAdaptive bound optimization for online convex optimization
McMahan, H. B., & Streeter, M. (2010) · arXiv · doi:10.48550/arxiv.1002.4908 · Cited by 227 · see citing works
Self-citationFirst Analysis of Local GD on Heterogeneous Data
Khaled, A., Mishchenko, K., & Richtárik, P. (2019) · arXiv · doi:10.48550/arxiv.1909.04715 · Cited by 103 · see citing works
Stich, S. U., & Karimireddy, S. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05350 · Cited by 91 · see citing works
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes, from any initializationPreprint
Ward, R., Wu, X., & Bottou, L. (2018) · arXiv (Cornell University) · Cited by 59 · see citing works
Why ADAM Beats SGD for Attention Models
Zhang, J., Karimireddy, S. P., Veit, A., Kim, S., Reddi, S. J., Kumar, S., & Sra, S. (2019) · Cited by 54 · see citing works
Self-citationA Generic Approach for Escaping Saddle pointsPreprint
Reddi, S. J., Zaheer, M., Sra, S., Póczos, B., Bach, F., Salakhutdinov, R., & Smola, A. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01434 · Cited by 54 · see citing works
Self-citationEscaping Saddle Points with Adaptive Gradient MethodsPreprint
Staib, M., Reddi, S. J., Kale, S., Kumar, S., & Sra, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09149 · Cited by 49 · see citing works
Self-citationGlobal Convergence of Adaptive Gradient Methods for An Over-parameterized Neural NetworkPreprint
Wu, X., Du, S. S., & Ward, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07111 · Cited by 37 · see citing works
On the Convergence of Stochastic Gradient Descent with Adaptive StepsizesPreprint
Li, X., & Orabona, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08114 · Cited by 32 · see citing works
Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning RatesPreprint
Xie, C., Koyejo, S., Gupta, I., & Lin, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09030 · Cited by 21 · see citing works
Flagged first, then by citations
3 of the 45 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Debraj Basu, Deepesh Data, Can Karakus, and Suhas Diggavi. Qsparse-local-SGD: Distributed SGD with quantization, sparsification and local computations. In Advances in Neural Information Processing Systems , pp. 14668-14679, 2019.
- Aaron Defazio and Léon Bottou. On the ineffectiveness of variance reduced optimization for deep learning. arXiv preprint arXiv:1812.04529 , 2018. · arXiv
- H. Brendan McMahan and Matthew J. Streeter. Adaptive bound optimization for online convex optimization. In COLT The 23rd Conference on Learning Theory , 2010b.
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Self-citationA Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407 · see citing works
Federated Learning on Non-IID Data Silos: An Experimental Study
Li, Q., Diao, Y., Chen, Q., & He, B. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00077 · Cited by 1,281 · see citing works
Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
Wang, J., Liu, Q., Liang, H., Joshi, G., & Poor, H. V. (2020) · arXiv · doi:10.48550/arxiv.2007.07481 · Cited by 927 · see citing works
Robust Aggregation for Federated Learning
Pillutla, K., Kakade, S. M., & Harchaoui, Z. (2022) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2022.3153135 · Cited by 947 · see citing works
Inverting Gradients -- How easy is it to break privacy in federated learning?Preprint
Geiping, J., Bauermeister, H., Dröge, H., & Moeller, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.14053 · Cited by 749 · see citing works
Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications
Aledhari, M., Razzak, R., Parizi, R. M., & Saeed, F. (2020) · IEEE Access · doi:10.1109/access.2020.3013541 · Cited by 733 · see citing works
Ensemble Distillation for Robust Model Fusion in Federated LearningPreprint
Lin, T., Kong, L., Stich, S. U., & Jaggi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07242 · Cited by 733 · see citing works
Personalized Federated Learning with Moreau EnvelopesPreprint
Dinh, C. T., Tran, N. H. K., & Nguyen, T. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08848 · Cited by 644 · see citing works
FedBN: Federated Learning on Non-IID Features via Local Batch NormalizationPreprint
Li, X., Jiang, M., Zhang, X., Kamp, M., & Dou, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07623 · Cited by 700 · see citing works
End-to-end privacy preserving deep learning on multi-institutional medical imaging
Kaissis, G., Ziller, A., Passerat-Palmbach, J., Ryffel, T., Usynin, D., Trask, A., Lima, I., Mancuso, J., Jungmann, F., Steinborn, M.-M., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00337-8 · Cited by 513 · see citing works
Exploiting Shared Representations for Personalized Federated LearningPreprint
Collins, L., Hassani, H., Mokhtari, A., & Shakkottai, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07078 · Cited by 453 · see citing works
A state-of-the-art survey on solving non-IID data in Federated Learning
Ma, X., Zhu, J., Lin, Z., Chen, S., & Qin, Y. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.003 · Cited by 404 · see citing works
FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction
Gao, L., Fu, H., Li, L., Chen, Y., Xu, M., & Xu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00987 · Cited by 395 · see citing works
Decentralized Federated Averaging
Sun, T., Li, D., & Wang, B. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3196503 · Cited by 408 · see citing works
Federated Learning Based on Dynamic RegularizationPreprint
Acar, D. A. E., Zhao, Y., Navarro, R. M., Mattina, M., Whatmough, P. N., & Saligrama, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04263 · Cited by 391 · see citing works
FedML: A Research Library and Benchmark for Federated Machine LearningPreprint
He, C., Li, S., So, J., Zeng, X., Zhang, M., Wang, H., Wang, X., Vepakomma, P., Singh, A., Qiu, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13518 · Cited by 504 · see citing works
Client Selection in Federated Learning: Principles, Challenges, and Opportunities
Fu, L., Zhang, H., Gao, G., Zhang, M., & Liu, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3299573 · Cited by 310 · see citing works
Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computing
Mills, J., Hu, J. W., & Min, G. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3098467 · Cited by 303 · see citing works
Cho, Y. J., Wang, J., & Joshi, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01243 · Cited by 302 · see citing works
Fast-Convergent Federated Learning
Nguyen, H. T., Sehwag, V., Hosseinalipour, S., Brinton, C. G., Chiang, M., & Poor, H. V. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036952 · Cited by 281 · see citing works
Group Knowledge Transfer: Federated Learning of Large CNNs at the EdgePreprint
He, C., Annavaram, M., & Avestimehr, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14513 · Cited by 309 · see citing works
Shejwalkar, V., Houmansadr, A., Kairouz, P., & Ramage, D. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833647 · Cited by 292 · see citing works
End-Edge-Cloud Collaborative Computing for Deep Learning: A Comprehensive Survey
Wang, Y., Yang, C., Lan, S.-L., Zhu, L.-H., & Zhang, Y. (2024) · IEEE Communications Surveys and Tutorials · doi:10.1109/comst.2024.3393230 · Cited by 255 · see citing works
Chellapandi, V. P., Yuan, L., Brinton, C. G., Żak, S. H., & Wang, Z. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3332675 · Cited by 249 · see citing works
Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark
Huang, W., Ye, M., Shi, Z., Wan, G., He, L., Du, B., & Yang, Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3418862 · Cited by 251 · see citing works
AUCTION: Automated and Quality-Aware Client Selection Framework for Efficient Federated Learning
Deng, Y., Lyu, F., Ren, J., Wu, H., Zhou, Y., Zhang, Y., & Shen, X. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3134647 · Cited by 220 · see citing works
FairFed: Enabling Group Fairness in Federated Learning
Ezzeldin, Y. H., Yan, S., He, C., Ferrara, E., & Avestimehr, A. S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25911 · Cited by 209 · see citing works
Rethinking Federated Learning with Domain Shift: A Prototype View
Huang, W., Ye, M., Shi, Z., He, L., & Du, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01565 · Cited by 206 · see citing works
Design and Analysis of Uplink and Downlink Communications for Federated Learning
Zheng, S., Shen, C., & Chen, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041388 · Cited by 191 · see citing works
Federated Learning with Buffered Asynchronous AggregationPreprint
Nguyen, J., Malik, K., Zhan, H., Yousefpour, A., Rabbat, M., Esmaeili, M. M., & Huba, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06639 · Cited by 182 · see citing works
Differentially Private Learning with Adaptive ClippingPreprint
Andrew, G., Thakkar, O. D., McMahan, H. B., & Ramaswamy, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03871 · Cited by 197 · see citing works
Self-citationFederated Learning Meets Multi-Objective Optimization
Hu, Z., Shaloudegi, K., Zhang, G., & Yu, Y. (2022) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2022.3169117 · Cited by 173 · see citing works
HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images
Jiang, M., Wang, Z.-R., & Dou, Q. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19993 · Cited by 174 · see citing works
A multi-centre polyp detection and segmentation dataset for generalisability assessment
Ali, S., Jha, D., Ghatwary, N., Realdon, S., Cannizzaro, R., Salem, O. E., Lamarque, D., Daul, C., Riegler, M., Ånonsen, K., et al. (2021) · Scientific Data · doi:10.1038/s41597-023-01981-y · Cited by 192 · see citing works
Federated Learning for Internet of Things
Zhang, T., He, C., Ma, T., Gao, L., Ma, M., & Avestimehr, S. (2021) · Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/3485730.3493444 · Cited by 157 · see citing works
Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT
Li, Z., He, Y., Yu, H., Kang, J., Li, X., Xu, Z., & Niyato, D. T. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3161943 · Cited by 157 · see citing works
Applications of Federated Learning; Taxonomy, Challenges, and Research Trends
Shaheen, M., Farooq, M. S., Umer, T., & Kim, B.-S. (2022) · Electronics · doi:10.3390/electronics11040670 · Cited by 157 · see citing works
Bhavsar, M., Bekele, Y. B., Roy, K., Kelly, J. C., & Limbrick, D. B. (2024) · IEEE Access · doi:10.1109/access.2024.3386631 · Cited by 160 · see citing works
Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated LearningPreprint
Yang, H., Fang, M., & Liu, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.11203 · Cited by 150 · see citing works
FedBE: Making Bayesian Model Ensemble Applicable to Federated LearningPreprint
Chen, H.-Y., & Chao, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01974 · Cited by 148 · see citing works
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147 · see citing works
Sáez-de-Cámara, X., Flores, J. L., Arellano, C., Urbieta, A., & Zurutuza, U. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103299 · Cited by 147 · see citing works
Adaptive Federated Learning on Non-IID Data With Resource Constraint
Zhang, J., Guo, S., Qu, Z., Zeng, D., Zhan, Y., Liu, Q., & Akerkar, R. A. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3099723 · Cited by 144 · see citing works
A Novel Framework for the Analysis and Design of Heterogeneous Federated Learning
Wang, J., Liu, Q., Liang, H., Joshi, G., & Poor, H. V. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3106104 · Cited by 140 · see citing works
Federated Learning with Compression: Unified Analysis and Sharp GuaranteesPreprint
Haddadpour, F., Kamani, M. M., Mokhtari, A., & Mahdavi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01154 · Cited by 140 · see citing works
Reisizadeh, A., Tziotis, I., Hassani, H., Mokhtari, A., & Pedarsani, R. (2022) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2022.3205475 · Cited by 137 · see citing works
Hu, R., Guo, Y., & Gong, Y. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3343288 · Cited by 138 · see citing works
FedCorr: Multi-Stage Federated Learning for Label Noise Correction
Xu, J., Chen, Z., Quek, T. Q. S., & Chong, K. F. E. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00994 · Cited by 133 · see citing works
FedTP: Federated Learning by Transformer Personalization
Li, H., Cai, Z., Wang, J., Tang, J., Ding, W., Lin, C., & Shi, Y. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3269062 · Cited by 129 · see citing works
Do Gradient Inversion Attacks Make Federated Learning Unsafe?
Hatamizadeh, A., Yin, H., Molchanov, P. A., Myronenko, A., Li, W., Dogra, P., Feng, A., Flores, M. G., Kautz, J., Xu, D., et al. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3239391 · Cited by 133 · see citing works
Differentially Private Federated Learning with Local Regularization and Sparsification
Cheng, A., Wang, P., Zhang, X. S., & Cheng, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00988 · Cited by 125 · see citing works
FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning
Sattler, F., Korjakow, T., Rischke, R., & Samek, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3129371 · Cited by 122 · see citing works
FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks
Lin, B. Y., He, C., Ze, Z., Wang, H., Hua, Y., Dupuy, C., Gupta, R., Soltanolkotabi, M., Ren, X., & Avestimehr, S. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.13 · Cited by 122 · see citing works
Al-Quraan, M. M., Mohjazi, L., Bariah, L., Centeno, A., Zoha, A., Arshad, K., Assaleh, K. T., Muhaidat, S., Debbah, M. A., & Imran, M. A. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3251404 · Cited by 145 · see citing works
Fairness and privacy preserving in federated learning: A survey
Rafi, T. H., Noor, F. A., Hussain, T., & Chae, D. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102198 · Cited by 131 · see citing works
FedGraphNN: A Federated Learning System and Benchmark for Graph Neural NetworksPreprint
He, C., Balasubramanian, K., Ceyani, E., Yang, C., Xie, H., Sun, L., He, L., Yang, L., Yu, P. S., Rong, Y., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07145 · Cited by 171 · see citing works
Reviewing Federated Machine Learning and Its Use in Diseases Prediction
Moshawrab, M., Adda, M., Bouzouane, A., Ibrahim, H., & Raad, A. (2023) · Sensors · doi:10.3390/s23042112 · Cited by 114 · see citing works
Federated Learning and Meta Learning: Approaches, Applications, and Directions
Liu, X., Deng, Y., Nallanathan, A., & Bennis, M. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3330910 · Cited by 115 · see citing works
Cost-Effective Federated Learning in Mobile Edge Networks
Luo, B., Li, X., Wang, S., Huang, J.-W., & Tassiulas, L. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118436 · Cited by 98 · see citing works
FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning
Xiong, Y., Wang, R., Cheng, M., Yu, F., & Hsieh, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01566 · Cited by 110 · see citing works
FedGKD: Toward Heterogeneous Federated Learning via Global Knowledge Distillation
Yao, D., Pan, W., Dai, Y., Wan, Y., Ding, X., Chen, Y., Jin, H., Xu, Z., & Sun, L. (2023) · IEEE Transactions on Computers · doi:10.1109/tc.2023.3315066 · Cited by 107 · see citing works
Make Landscape Flatter in Differentially Private Federated Learning
Shi, Y., Liu, Y., Wei, K., Shen, L., Wang, X., & Tao, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02352 · Cited by 106 · see citing works
Islam, A., Amin, A. A., & Shin, S. Y. (2022) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2022.3151873 · Cited by 103 · see citing works
ScaleFL: Resource-Adaptive Federated Learning with Heterogeneous Clients
Ilhan, F., Su, G., & Liu, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02350 · Cited by 104 · see citing works
Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better
Bibikar, S., Vikalo, H., Wang, Z., & Chen, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i6.20555 · Cited by 99 · see citing works
Faster Adaptive Federated Learning
Wu, X., Huang, F., Hu, Z., & Huang, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26235 · Cited by 96 · see citing works
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
Karimireddy, S. P., Jaggi, M., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., & Suresh, A. T. (2020) · arXiv · doi:10.48550/arxiv.2008.03606 · Cited by 96 · see citing works
Self-citationImproving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models
Zhang, Z. J., Yang, Y. L., Yao, Z., Yan, Y., Gonzalez, J. E., Ramchandran, K., & Mahoney, M. W. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671693 · Cited by 104 · see citing works
Improving Generalization in Federated Learning by Seeking Flat Minima
Caldarola, D., Barbara Caputo, B., & Ciccone, M. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20050-2_38 · Cited by 92 · see citing works
No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices
Liu, R., Wu, F., Wu, C., Wang, Y., Lyu, L., Chen, H., & Xie, X. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539086 · Cited by 91 · see citing works
A Golden Decade of Deep Learning: Computing Systems & Applications
Dean, J. (2022) · Daedalus · doi:10.1162/daed_a_01900 · Cited by 89 · see citing works
Personalized Federated Learning With a Graph
Chen, F., Long, G., Wu, Z., Zhou, T., & Jiang, J. B. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/357 · Cited by 88 · see citing works
Communication-Efficient Federated Learning via Optimal Client Sampling
Ribero, M., & Vikalo, H. (2020) · LatinX in AI at International Conference on Machine Learning (LXAI) · doi:10.52591/lxai2020071310 · Cited by 87 · see citing works
Federated Learning With Spiking Neural Networks
Venkatesha, Y., Kim, Y., Tassiulas, L., & Panda, P. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3121632 · Cited by 87 · see citing works
FedScale: Benchmarking Model and System Performance of Federated Learning at ScalePreprint
Lai, F., Dai, Y., Singapuram, S. S. V., Liu, J., Zhu, X., Madhyastha, H. V., & Chowdhury, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.11367 · Cited by 87 · see citing works
Advancements in Physics-Informed Neural Networks for Laminated Composites: A Comprehensive Review
Khalid, S., Yazdani, M. H., Azad, M. M., Elahi, M. U., Raouf, I., & Kim, H. S. (2024) · Mathematics · doi:10.3390/math13010017 · Cited by 86 · see citing works
Federated Learning for Privacy-Preserving Open Innovation Future on Digital Health
Long, G., Shen, T., Tan, Y., Gerrard, L., Clarke, A., & Jiang, J. (2022) · Humanity Driven AI · doi:10.1007/978-3-030-72188-6_6 · Cited by 85 · see citing works
Zhou, T., Zhang, J., & Tsang, D. H. K. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3325366 · Cited by 84 · see citing works
Federated learning for 5G base station traffic forecasting
Perifanis, V., Pavlidis, N., Koutsiamanis, R.-A., & Efraimidis, P. S. (2023) · Computer Networks · doi:10.1016/j.comnet.2023.109950 · Cited by 87 · see citing works
Qiao, C., Li, M., Liu, Y., & Tian, Z. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3399612 · Cited by 83 · see citing works
A review on federated learning towards image processing
Khokhar, F. A., Shah, J. H., Khan, M. A., Sharif, M. I., Tariq, U., & Kadry, S. (2022) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2022.107818 · Cited by 80 · see citing works
Distilled One-Shot Federated LearningPreprint
Zhou, Y., Pu, G., Ma, X., Li, X., & Wu, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07999 · Cited by 79 · see citing works
Fair Federated Medical Image Segmentation via Client Contribution Estimation
Jiang, M., Roth, H. R., Li, W., Yang, D. L., Zhao, C. H., Nath, V., Xu, D., Dou, Q., & Xu, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01564 · Cited by 79 · see citing works
Federated Reconstruction: Partially Local Federated LearningPreprint
Singhal, K., Sidahmed, H., Garrett, Z., Wu, S., Rush, K., & Prakash, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.03448 · Cited by 78 · see citing works
Self-citationLayer-Wise Adaptive Model Aggregation for Scalable Federated Learning
Lee, S., Zhang, T., & Avestimehr, A. S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26023 · Cited by 84 · see citing works
Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation
Xu, A. P., Li, W., Guo, P., Yang, D., Roth, H. R., Hatamizadeh, A., Zhao, C., Xu, D., Huang, H., & Xu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02020 · Cited by 77 · see citing works
Cho, Y. J., Wang, J., Chirvolu, T., & Joshi, G. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3231527 · Cited by 100 · see citing works
Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework
Guliani, D., Beaufays, F., & Motta, G. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413397 · Cited by 76 · see citing works
Mora, A., Bujari, A., & Bellavista, P. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.03.027 · Cited by 74 · see citing works
Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD
Wang, J., Wang, S., Chen, R., & Ji, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20832 · Cited by 73 · see citing works
FedSeg: Class-Heterogeneous Federated Learning for Semantic Segmentation
Miao, J., Yang, Z., Fan, L., & Yang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00777 · Cited by 72 · see citing works
Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics
Romandini, N., Mora, A., Mazzocca, C., Montanari, R., & Bellavista, P. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3478334 · Cited by 76 · see citing works
Integration of large language models and federated learning
Chen, C., Feng, X., Li, Y., Lyu, L., Zhou, J., Zheng, X., & Yin, J. (2024) · Patterns · doi:10.1016/j.patter.2024.101098 · Cited by 103 · see citing works
Papaya: Practical, Private, and Scalable Federated LearningPreprint
Huba, D., Nguyen, J., Malik, K., Zhu, R., Rabbat, M., Yousefpour, A., Wu, C.-J., Zhan, H., Ustinov, P., Srinivas, H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.04877 · Cited by 75 · see citing works
A Review of Federated Learning Applications in Intrusion Detection Systems
Belenguer, A., Pascual, J. A., & Navaridas, J. (2025) · Computer Networks · doi:10.1016/j.comnet.2024.111023 · Cited by 118 · see citing works
FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving
Fantauzzo, L., Fanì, E., Caldarola, D., Tavera, A., Cermelli, F., Ciccone, M., & Barbara Caputo, B. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981098 · Cited by 68 · see citing works
The Internet of Federated Things (IoFT)
Kontar, R. A., Shi, N., Yue, X., Chung, S., Byon, E., Chowdhury, M., Jin, J. J., Kontar, W., Masoud, N., Nouiehed, M., et al. (2021) · IEEE Access · doi:10.1109/access.2021.3127448 · Cited by 67 · see citing works
Convergence Analysis and System Design for Federated Learning Over Wireless Networks
Wan, S., Lu, J., Fan, P., Shao, Y., Peng, C., & Letaief, K. B. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118351 · Cited by 65 · see citing works
An Approach for Peer-to-Peer Federated Learning
Wink, T., & Nochta, Z. (2021) · International Conference on Dependable Systems and Networks workshops · doi:10.1109/dsn-w52860.2021.00034 · Cited by 64 · see citing works
- Authors
- Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečný, J., Kumar, S., & McMahan, H. B.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2003.00295
- OpenAlex
- W3008187686
- Cited by
- 840
- References
- 45
- Citations by year
- 2026: 10 · 2025: 127 · 2024: 233 · 2023: 222 · 2022: 120 · 2021: 92 · 2020: 35 · 2019: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- federated learning, adaptive optimization, FedAvg, AdaGrad, Adam, yogis, convergence analysis, non-convex optimization, client heterogeneity, communication efficiency, distributed learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
