The DOI matches its DataCite record; 42 of the 45 references it cites were found in a registry or an index.
Adaptive Federated Optimization
Open accessAbstract
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own training data. Standard federated optimization methods such as Federated Averaging (FedAvg) are often difficult to tune and exhibit unfavorable convergence behavior. In non-federated settings, adaptive optimization methods have had notable success in combating such issues. In this work, we propose federated versions of adaptive optimizers, including Adagrad, Adam, and Yogi, and analyze their convergence in the presence of heterogeneous data for general non-convex settings. Our results highlight the interplay between client heterogeneity and communication efficiency. We also perform extensive experiments on these methods and show that the use of adaptive optimizers can significantly improve the performance of federated learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“MPC approaches to PIR break down into two main categories: computational PIR (cPIR), in which a single party can execute the entire server side of the protocol [286], and information theoretic PIR (itPIR), in which multiple non-colluding parties are required to execute the server side of the protocol [121].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [121]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Self-citation(2010) · Cited by 8,261
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Self-citationFederated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023
Self-citationWu, Y., & He, K. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01198-w · Cited by 3,601
EMNIST: Extending MNIST to handwritten letters
Cohen, G., Afshar, S., Tapson, J. C., & André, V. S. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966217 · Cited by 2,410
Towards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334
Self-citation
Flagged first, then by citations
3 of the 45 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Debraj Basu, Deepesh Data, Can Karakus, and Suhas Diggavi. Qsparse-local-SGD: Distributed SGD with quantization, sparsification and local computations. In Advances in Neural Information Processing Systems , pp. 14668-14679, 2019.
- Aaron Defazio and Léon Bottou. On the ineffectiveness of variance reduced optimization for deep learning. arXiv preprint arXiv:1812.04529 , 2018. · arXiv
- H. Brendan McMahan and Matthew J. Streeter. Adaptive bound optimization for online convex optimization. In COLT The 23rd Conference on Learning Theory , 2010b.
Cited by
Sorted by citations
Items 801–844 of 844
TOFU: Toward Obfuscated Federated Updates by Encoding Weight Updates Into Gradients From Proxy Data
Nagaraj, M., Garg, I., & Roy, K. (2024) · IEEE Access · doi:10.1109/access.2024.3390716
Exploiting data diversity in multi-domain federated learning
Madni, H. A., Umer, R. M., & Foresti, G. L. (2024) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ad4768 · Cited by 2
Controlled Privacy Leakage Propagation Throughout Overlapping Grouped Learning
Kiani, S., Boenisch, F., & Draper, S. C. (2024) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2024.3416089
Toward Federated Learning Through Intent Detection Research
Deksne, D., Kapočiūtė-Dzikienė, J., & Skadiņš, R. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-63543-4_6
TurboFed: A Federated Learning Approach to The PHM of Distributed Wind Turbines
Chen, B., Zhu, Y., Yu, G., & Xu, S. Z. (2024) · IEEE International Conference on Big Data Security on Cloud, IEEE International Conference on High Performance and Smart Computing, and IEEE International Conference on Intelligent Data and Security (BigDataSecurity/HPSC/IDS) · doi:10.1109/hpsc62738.2024.00026
Dp-Perfedavg: Safeguarding Client Data Privacy in Personalized Federated AveragingPreprint
Wu, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4907858
Towards Addressing Heterogeneity Of Data In Federated Learning
Sun, M., Chen, Z., Yan, T., Yang, Y., Liu, S., & Rao, Z. (2024) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs61882.2024.10603332
Gao, Z., Zhang, J., & Yu, X. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5675-9_29
A Bijection-backdoor-based Adversarial Examples Defense Method in Federated LearningPreprint
Li, Y., Guo, Y., Chen, F., Wang, Y., Chen, Q., & Hu, Y. (2024) · Research Square · doi:10.21203/rs.3.rs-4718612/v1
Accelerated Federated Learning Using Self-Adapting Bat AlgorithmPreprint
Wang, J., Sun, C., & Peng, Y. (2024) · Research Square · doi:10.21203/rs.3.rs-4755684/v1
Federated Learning Optimization Algorithm Based on Dynamic Client Scale
Wang, L., Feng, W., & Luo, R. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-65123-6_8
Siomos, V., Marimont, S. N., Passerat‐Palmbach, J., & Tarroni, G. (2024) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi56570.2024.10635565
Valli, M., Costan, A., Tedeschi, C., & Cudennec, L. (2024) · doi:10.1145/3659995.3660042
Storage-Assisted Uploading Method for HFL over Optical AIGC Networks
Zhang, J., Lin, X., Lian, Z., Li, J., Chen, Z., Sun, W., & Lou, Z. (2024) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/iccc62479.2024.10682033
Nakai-Kasai, A., & Wadayama, T. (2024) · IEICE Transactions on Communications · doi:10.23919/transcom.2024ebp3068 · Cited by 2
Federated Learning for Sound Data: Accurate Fan Noise Classification with a Multi-Microphone Setup
Neuhäusler, K. N., Wittek, N. H., Klaiber, M., Gerschner, F., Fernandes, M., & Rössle, M. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.674
Alwadaie, S., Jamal, A. T., Alkhuraij, S., & Elrefaei, L. A. (2024) · Journal of King Abdulaziz University-Computing and Information Technology Sciences · doi:10.4197/comp.13-2.7
One-Shot Clustering for Federated Learning
Zuziak, M. K., Pellungrini, R., & Rinzivillo, S. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825763
Enhancing Urban Traffic Forecasting with Scalable, Privacy-Preserving Federated Learning Approach
Jiang, Z. (2024) · International Conference on Electronic Information Engineering and Computer Technology (EIECT) · doi:10.1109/eiect64462.2024.10866348
Local methods with adaptivity via scaling
Chezhegov, S., Skorik, S., Khachaturov, N., Shalagin, D., Avetisyan, A. A., Takáč, M., Kholodov, Y. A., & Beznosikov, A. N. (2024) · Russian Mathematical Surveys · doi:10.4213/rm10209e
Rajesh, S., Gakhar, I., Shorey, R., & Verma, R. (2025) · International Conference on Communication Systems & Networks (COMSNETS) · doi:10.1109/comsnets63942.2025.10885640 · Cited by 1
Towards WebAssembly-Based Federated Learning
Gottschalk, F., Schulte, S., Hemadasa, N., Ebrahimi, E., Edinger, J., & Kaaser, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-84617-5_4
FedLegal: A Real-World Federated Learning Benchmark for Legal Natural Language Processing
Zhang, Z., Hu, X., Zhang, J., Zhang, Y., Wang, H., Qu, L., & Xu, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82240-7_10
EdgeFedNet: Edge Server Based Communication and Computation Efficient Federated Learning
Gowtham, L., Annappa, B., & Sachin, D. N. (2025) · SN Computer Science · doi:10.1007/s42979-025-03789-6
FedSSL: Federated Learning with Shape-Sensitive Loss for Catheter and Guidewire Segmentation
Kongtongvattana, C., Huang, B., Nguyen, H. A., Olajide, O., & Nguyen, A. (2024) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio64047.2024.10907455
(2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323641-9.00026-1
Improving performance fairness in federated learning for heterogeneous medical images
Jiang, M., & Dou, Q. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323641-9.00016-9
Cheong, Y., Shin, K.-S., & Shin, S.-Y. (2025) · Journal of information and communication convergence engineering · doi:10.56977/jicce.2025.23.1.1
Islam, M. A., Karmaker, D., Bhowmik, A., Billah, M. M., Mobin, M. I., & Noor, N. M. (2025) · International Journal of Modern Education and Computer Science · doi:10.5815/ijmecs.2025.02.04
Zhao, L., Zhao, Y., Gong, P., Li, W., Shi, Y., Jin, Z., & Qin, S. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116432
Gradients Stand-in for Defending Deep Leakage in Federated Learning
Hu, Y., Ren, H., Chen, H., Li, Y., Deng, J., & Xie, X. (2024) · International Conference on Computing in Natural Sciences, Biomedicine and Engineering (COMCONF) · doi:10.1109/comconf63340.2024.00017
Overview of the New Perspectives on Optimization Algorithms Research
Lambert‐Torres, G., de Moraes, C. H. V., & de Andrade, G. C. C. (2025) · IntechOpen eBooks · doi:10.5772/intechopen.1010958
CrossFL: A Cross-Round Federated Learning Framework with Asynchronous Client Selection
Wen, Z., Yang, N., Zhang, Y., Li, Y., Yao, Z., Chen, H., & Yuan, D. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8173-0_12
FedQR: Communication-Efficient Federated Learning via QR Factorization
Ba, A. F., Mao, Y., Djigal, H., Muhammad, A. U., Arain, T. A., & Konate, S. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-6951-6_30
A Comprehensive Review of Big Data through Adaptive Sharding and Federated Learning
Hussen, S. H., & Kareem, S. W. (2024) · International Engineering Conference on Sustainable Technology and Development (IEC) · doi:10.1109/iec61018.2024.11064087
RDP-FedAB: A Federated Learning Framework Balancing Privacy Protection and Model Performance
Wang, X., Xie, F., Liu, S., & Lai, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9872-1_11
Afbs: Buffer Gradient Selection in Semi-Asynchronous Federated LearningPreprint
Lu, C., Sun, Y., Chen, J., Yang, Z. M., Pan, J., & Zhu, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5366908 · Cited by 1
Entropy-Based Federated Learning to Predict Students’ Dropout
Bernardi, M. L., Cimitile, M., & Usman, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94002-6_16
Analysis of federated learning on non-independent and identically distributed sleep data
Anido-Alonso, A., & Álvarez-Estévez, D. (2026) · Physiological Measurement · doi:10.1088/1361-6579/ae4a82
Sumadi, F. D. S., & Alsubhi, K. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03108-2
Secure Medical Image Cryptanalysis with Quantum Neural Networks for IoT-Enabled Cloud StoragePreprint
Priyatharsini, G. S., Sambandam, J., Kavitha, R., Saritha, R. M., Maheswari, A., & Kouki, S. (2026) · Research Square · doi:10.21203/rs.3.rs-9313076/v1
Zhou, W., Li, R., & Huan, D. (2026) · Cluster Computing · doi:10.1007/s10586-026-06539-2
Kang, H., & Ding, Z. (2026) · Integrated Computer-Aided Engineering · doi:10.1177/10692509261490475
Metadata
- Authors
- Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečný, J., Kumar, S., & McMahan, H. B.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2003.00295
- OpenAlex
- W3008187686
- Cited by
- 840
- References
- 45
- Citations by year
- 2026: 10 · 2025: 127 · 2024: 233 · 2023: 222 · 2022: 120 · 2021: 92 · 2020: 35
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- federated learning, adaptive optimization, FedAvg, AdaGrad, Adam, yogis, convergence analysis, non-convex optimization, client heterogeneity, communication efficiency, distributed learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
