The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
Robust Aggregation for Federated Learning
Open accessAbstract
We present a novel approach to federated learning that endows its aggregation process with greater robustness to potential poisoning of local data or model parameters of participating devices. The proposed approach, Robust Federated Aggregation (RFA), relies on the aggregation of updates using the geometric median, which can be computed efficiently using a Weiszfeld-type algorithm. RFA is agnostic to the level of corruption and aggregates model updates without revealing each device’s individual contribution. We establish the convergence of the robust federated learning algorithm for the stochastic learning of additive models with least squares. We also offer two variants of RFA: a faster one with one-step robust aggregation, and another one with on-device personalization. We present experimental results with additive models and deep networks for three tasks in computer vision and natural language processing. The experiments show that RFA is competitive with the classical aggregation when the level of corruption is low, while demonstrating greater robustness under high corruption.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 11,726
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Self-citationCalibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
Robust estimation of a location parameter
Huber, P. J. (1964) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177703732 · Cited by 7,228
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210
The Byzantine generals problem
Lamport, L., Shostak, R., & Pease, M. (1982) · ACM Transactions on Programming Languages and Systems · doi:10.1145/357172.357176 · Cited by 6,023
Wiley Series in Probability and Statistics
(2008) · Wiley series in probability and statistics · doi:10.1002/series1345 · Cited by 4,217
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023 · cited 2×
“Extensions were proposed in [35, 51, 56, 59] — see [33] for a survey.”
BackgroundPractical Secure Aggregation for Privacy-Preserving Machine Learning
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3133982 · Cited by 3,956
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · cited 1×
“[66] C.”
Understanding deep learning (still) requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021) · Communications of the ACM · doi:10.1145/3446776 · Cited by 2,400
Towards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334
A Survey on Wearable Sensor-Based Systems for Health Monitoring and Prognosis
Pantelopoulos, A., & Bourbakis, N. G. (2010) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2009.2032660 · Cited by 2,296
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., & Suresh, A. T. (2019) · arXiv · doi:10.48550/arxiv.1910.06378 · Cited by 1,734
Robust Statistics: Theory and Methods
Maronna, R. A., Martin, R. D., & Yohai, V. J. (2006) · Cited by 1,715
(2017) · neural information processing systems · Cited by 1,413
Problem complexity and method efficiency in optimization
(1984) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(84)90101-0 · Cited by 1,329
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
Yin, D., Chen, Y., Ramchandran, K., & Bartlett, P. (2018) · arXiv · doi:10.48550/arxiv.1803.01498 · Cited by 1,130
Lectures on Convex Optimization
Nesterov, Y. (2018) · Springer Optimization and Its Applications · doi:10.1007/978-3-319-91578-4 · Cited by 1,010
LEAF: A Benchmark for Federated Settings
Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Konečný, J., McMahan, H. B., Smith, V., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.01097 · Cited by 894
Adaptive Federated Optimization
Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečný, J., Kumar, S., & McMahan, H. B. (2020) · arXiv · doi:10.48550/arxiv.2003.00295 · Cited by 840
Applied Federated Learning: Improving Google Keyboard Query Suggestions
Yang, T., Andrew, G., Eichner, H., Sun, H., Li, W., Kong, N., Ramage, D., & Beaufays, F. (2018) · arXiv · doi:10.48550/arxiv.1812.02903 · Cited by 684
Computing arbitrary functions of encrypted data
Gentry, C. (2010) · Communications of the ACM · doi:10.1145/1666420.1666444 · Cited by 650
Another proof of the defect relation for moving targets
Shirosaki, M. (1991) · Tohoku Mathematical Journal · doi:10.2748/tmj/1178227459 · Cited by 614
Huber, P. J. (2011) · International Encyclopedia of Statistical Science · doi:10.1007/978-3-642-04898-2_594 · Cited by 595
Breakdown Points of Affine Equivariant Estimators of Multivariate Location and Covariance Matrices
Lopuhaa, H. P., & Rousseeuw, P. J. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347978 · Cited by 446
Secure Single-Server Aggregation with (Poly)Logarithmic Overhead
Bell, J. H., Bonawitz, K. A., Gascón, A., Lepoint, T., & Raykova, M. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417885 · Cited by 438
Huang, L., Shea, A. L., Qian, H., Masurkar, A., Deng, H., & Liu, D. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103291 · Cited by 437
EMNIST: an extension of MNIST to handwritten letters
Cohen, G., Afshar, S., Tapson, J. C., & André, V. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05373 · Cited by 419
Adaptation, Learning, and Optimization over Networks
Sayed, A. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988515 · Cited by 415
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
Chen, Y., Su, L., & Xu, J. (2017) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3154503 · Cited by 390 · cited 1×
“Byzantine robustness, resilience to arbitrary behavior of some devices [51], was studied in distributed optimization with gradient aggregation [5, 16, 20, 21, 23, 88].”
BackgroundMohri, M., Sivek, G., & Suresh, A. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00146 · Cited by 373
Li, L., Xu, W., Chen, T., Giannakis, G. B., & Ling, Q. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011544 · Cited by 373
Can You Really Backdoor Federated Learning?
Sun, Z., Kairouz, P., Suresh, A. T., & McMahan, H. B. (2019) · arXiv · doi:10.48550/arxiv.1911.07963 · Cited by 348 · cited 1×
“We now compare RFA with: (a) coordinate-wise median [88] and `2 norm clipping [82] which are agnostic to the actual corruption level ρ like RFA, and, (b) trimmed mean [88] and multi-Krum [16], that require exact knowledge of the level of corruption ρ in the problem.”
MethodKuhn, H. W. (1973) · Mathematical Programming · doi:10.1007/bf01584648 · Cited by 329
A Pragmatic Introduction to Secure Multi-Party Computation
Evans, D., Kolesnikov, V., & Rosulek, M. (2018) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000019 · Cited by 321
Smoothing and First Order Methods: A Unified Framework
Beck, A., & Teboulle, M. (2012) · SIAM Journal on Optimization · doi:10.1137/100818327 · Cited by 267
Geometric median and robust estimation in Banach spaces
Minsker, S. (2015) · Bernoulli · doi:10.3150/14-bej645 · Cited by 261
On the Convergence of Federated Optimization in Heterogeneous Networks.Preprint
Sahu, A. K., Li, T., Sanjabi, M., Zaheer, M., Talwalkar, A., & Smith, V. (2018) · arXiv (Cornell University) · Cited by 237
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning
Mairal, J. (2015) · SIAM Journal on Optimization · doi:10.1137/140957639 · Cited by 232
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
Chen, Y., Su, L., & Xu, J. (2018) · Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3219617.3219655 · Cited by 228
Federated Variance-Reduced Stochastic Gradient Descent With Robustness to Byzantine Attacks
Wu, Z., Ling, Q., Chen, T., & Giannakis, G. B. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.3012952 · Cited by 226
Beck, A. (2015) · SIAM Journal on Optimization · doi:10.1137/13094829x · Cited by 221
A Pragmatic Introduction to Secure Multi-Party Computation
Evans, D., Kolesnikov, V., & Rosulek, M. (2018) · now publishers, Inc. eBooks · doi:10.1561/9781680835090 · Cited by 204
Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
Ammad-ud-din, M., Ivannikova, E., Khan, S. A., Oyomno, W., Fu, Q., Tan, K. E., & Flanagan, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09888 · Cited by 203
Non-strongly-convex smooth stochastic approximation with convergence rate O(1/n)
Bach, F., & Moulines, É. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.2119 · Cited by 202
Robust Estimators in High Dimensions without the Computational Intractability
Diakonikolas, I., Kamath, G., Kane, D. M., Li, J., Moitra, A., & Stewart, A. (2016) · 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS) · doi:10.1109/focs.2016.85 · Cited by 200
CoCoA: A General Framework for Communication-Efficient Distributed Optimization
Smith, V., Forte, S., Ma, C., Takáč, M., Jordan, M. I., & Jaggi, M. (2016) · arXiv (Cornell University) · doi:10.3929/ethz-b-000282738 · Cited by 193
Distributed optimization with arbitrary local solvers
Ma, C., Konečný, J., Jaggi, M., Smith, V., Jordan, M. I., Richtárik, P., & Takáč, M. (2017) · Optimization Methods and Software · doi:10.1080/10556788.2016.1278445 · Cited by 191
Ren, J., Wang, H., Hou, T., Zheng, S., & Tang, C. (2019) · IEEE Access · doi:10.1109/access.2019.2919736 · Cited by 185
An Invitation to Statistics in Wasserstein Space
Panaretos, V. M., & Zemel, Y. (2020) · SpringerBriefs in Probability and Mathematical Statistics · doi:10.1007/978-3-030-38438-8 · Cited by 166
Byzantine Stochastic Gradient Descent
Alistarh, D., Allen-Zhu, Z., & Li, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08917 · Cited by 163
Risk minimization by median-of-means tournaments
Lugosi, G., & Mendelson, S. (2019) · Journal of the European Mathematical Society · doi:10.4171/jems/937 · Cited by 148
Robust covariance and scatter matrix estimation under Huber’s contamination model
Chen, M., Gao, C., & Ren, Z. (2018) · The Annals of Statistics · doi:10.1214/17-aos1607 · Cited by 146
High-Dimensional Robust Mean Estimation in Nearly-Linear Time
Cheng, Y., Diakonikolas, I., & Ge, R. (2019) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611975482.171 · Cited by 143
Collaborative Unsupervised Visual Representation Learning from Decentralized Data
Zhuang, W., Gan, X., Wen, Y., Zhang, S., & Yi, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00487 · Cited by 139
Optimization with First-Order Surrogate FunctionsPreprint
Mairal, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.3120 · Cited by 138
Geometric median in nearly linear time
Cohen, M. B., Lee, Y. T., Miller, G. L., Pachocki, J., & Sidford, A. (2016) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/2897518.2897647 · Cited by 132
Loss minimization and parameter estimation with heavy tailsPreprint
Hsu, D., & Sivan Sabato, A. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1307.1827 · Cited by 124
Weiszfeld’s Method: Old and New Results
Beck, A., & Sabach, S. (2014) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-014-0586-7 · Cited by 120
A Formal Foundation for Secure Remote Execution of Enclaves
Subramanyan, P., Sinha, R., Lebedev, I., Devadas, S., & Seshia, S. A. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134098 · Cited by 119
The distributed discrete gaussian mechanism for federated learning with secure aggregation
Kairouz, P., Liu, Z., & Steinke, T. (2021) · arXiv · doi:10.48550/arxiv.2102.06387 · Cited by 117
Jain, P., Kakade, S. M., Kidambi, R., Netrapalli, P., & Sidford, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03774 · Cited by 107
Self-citationDistributed Gradient Descent Algorithm Robust to an Arbitrary Number of Byzantine Attackers
Cao, X., & Lai, L. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2946020 · Cited by 100
A Collaborative Learning Framework via Federated Meta-Learning
Lin, S., Yang, G., & Zhang, J. (2020) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00032 · Cited by 95
Improved asynchronous parallel optimization analysis for stochastic incremental methodsPreprint
Leblond, R., Pedregosa, F., & Lacoste-Julien, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.03749 · Cited by 77
Robust Distributed Estimation by Networked Agents
Al-Sayed, S., Zoubir, A. M., & Sayed, A. H. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2703664 · Cited by 52
Optimal Convergence Rates for Convex Distributed Optimization in Networks
Scaman, K., Bach, F., Bubeck, S., Lee, Y. T., & Massoulié, L. (2019) · HAL · Cited by 45
Resilient Distributed Parameter Estimation With Heterogeneous Data
Chen, Y., Kar, S., & Moura, J. M. F. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2931171 · Cited by 40
Uniform bounds for robust mean estimators
Minsker, S. (2025) · Stochastic Processes and their Applications · doi:10.1016/j.spa.2025.104724 · Cited by 27
A Superquantile Approach to Federated Learning with Heterogeneous Devices
Laguel, Y., Pillutla, K., Malick, J., & Harchaoui, Z. (2021) · Conference on Information Sciences and Systems · doi:10.1109/ciss50987.2021.9400318 · Cited by 18
Self-citation
Cited by
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Self-citationPrivacy and Robustness in Federated Learning: Attacks and Defenses
Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., Yang, Q., & Yu, P. S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3216981 · Cited by 548
“Poisoning Defense Technique IID Data Non-IID Data Breaking Point Data Poisoning Model Poisoning RONI [28], [133] Error rate X × NA X × Auror [127] Clustering X × NA X × Krum [13] Euclidean distance X × (n− 2)/2n X × Coordinate-wise Median [14] Coordinate-wise median X × 1/2 X × Bulyan [128] Krum + trimmed median X × (…”
BackgroundInfluentialAttack of the tails: Yes, you really can backdoor federated learning
Wang, H., Sreenivasan, K., Rajput, S., Vishwakarma, H., Agarwal, S., Sohn, J.-Y., Lee, K., & Papailiopoulos, D. (2020) · arXiv · doi:10.48550/arxiv.2007.05084 · Cited by 406
From distributed machine learning to federated learning: a survey
Liu, J., Huang, J., Zhou, Y., Li, X., Ji, S., Xiong, H., & Dou, D. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01664-x · Cited by 356
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3162397 · Cited by 362
“Another class of approaches employs the coordinate-wise median gradient [192], geometric median of means [193], and approximate geometric median [194] since median-based computations are more resistant to outliers than mean-based aggregation.”
Background
Metadata
- Authors
- Pillutla, K., Kakade, S. M., & Harchaoui, Z.
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2022
- Volume
- 70
- Pages
- 1142-1154
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W2996769125
- Cited by
- 801
- References
- 78
- FWCI
- 44.91
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.