Manipulating the Byzantine: Optimizing Model Poisoning Attacks and Defenses for Federated Learning
Open accessAbstract
Federated learning (FL) enables many data owners (e.g., mobile devices) to train a joint ML model (e.g., a nextword prediction classifier) without the need of sharing their private training data.However, FL is known to be susceptible to poisoning attacks by malicious participants (e.g., adversaryowned mobile devices) who aim at hampering the accuracy of the jointly trained model through sending malicious inputs during the federated training process.In this paper, we present a generic framework for model poisoning attacks on FL.We show that our framework leads to poisoning attacks that substantially outperform state-of-the-art model poisoning attacks by large margins.For instance, our attacks result in 1.5× to 60× higher reductions in the accuracy of FL models compared to previously discovered poisoning attacks.Our work demonstrates that existing Byzantine-robust FL algorithms are significantly more susceptible to model poisoning than previously thought.Motivated by this, we design a defense against FL poisoning, called divide-and-conquer (DnC).We demonstrate that DnC outperforms all existing Byzantine-robust FL algorithms in defeating model poisoning attacks, specifically, it is 2.5× to 12× more resilient in our experiments with different datasets and models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Model poisoning backdoor attacks during the aggregation phase can modify the aggregation process by scaling the parameter of the uploaded malicious local model.
“Model poisoning backdoor attacks during the aggregation phase can modify the aggregation process by scaling the parameter of the uploaded malicious local model [181], [182].”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [181]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,189 · see citing works
EMNIST: Extending MNIST to handwritten letters
Cohen, G., Afshar, S., Tapson, J. C., & André, V. S. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966217 · Cited by 1,782 · see citing works
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · see citing works
(2017) · neural information processing systems · Cited by 1,413 · see citing works
Poisoning Attacks against Support Vector Machines
Biggio, B., Nelson, B., & Laskov, P. (2012) · arXiv · doi:10.48550/arxiv.1206.6389 · Cited by 1,331 · see citing works
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
Yin, D., Chen, Y., Ramchandran, K., & Bartlett, P. (2018) · arXiv · doi:10.48550/arxiv.1803.01498 · Cited by 1,130 · see citing works
Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
Fang, M., Cao, X., Jia, J., & Gong, N. Z. (2019) · arXiv · doi:10.48550/arxiv.1911.11815 · Cited by 828 · see citing works
Analyzing Federated Learning through an Adversarial Lens
Bhagoji, A. N., Chakraborty, S., Mittal, P., & Calo, S. (2018) · arXiv · doi:10.48550/arxiv.1811.12470 · Cited by 703 · see citing works
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization
Muñoz-González, L., Biggio, B., Demontis, A., Paudice, A., Wongrassamee, V., Lupu, E. C., & Roli, F. (2017) · Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3128572.3140451 · Cited by 549 · see citing works
Spectral Signatures in Backdoor AttacksPreprint
Tran, B., Li, J. Z., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00636 · Cited by 506 · see citing works
Can You Really Backdoor Federated Learning?
Sun, Z., Kairouz, P., Suresh, A. T., & McMahan, H. B. (2019) · arXiv · doi:10.48550/arxiv.1911.07963 · Cited by 348 · see citing works
(2019) · neural information processing systems · Cited by 318 · see citing works
Charikar, M., Steinhardt, J., & Valiant, G. (2017) · ACM SIGACT Symposium on Theory of Computing (STOC) · doi:10.1145/3055399.3055491 · Cited by 210 · see citing works
Agnostic Estimation of Mean and Covariance
Lai, K. A., Rao, A. B., & Vempala, S. (2016) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2016.76 · Cited by 208 · see citing works
Robust Federated Learning in a Heterogeneous EnvironmentPreprint
Ghosh, A., Hong, J., Yin, D., & Ramchandran, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.06629 · Cited by 204 · see citing works
DRACO: Byzantine-resilient Distributed Training via Redundant GradientsPreprint
Chen, L., Wang, H., Charles, Z., & Papailiopoulos, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09877 · Cited by 191 · see citing works
A Little Is Enough: Circumventing Defenses For Distributed LearningPreprint
Baruch, M., Baruch, G., & Goldberg, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06156 · Cited by 184 · see citing works
Generalized Byzantine-tolerant SGDPreprint
Xie, C., Koyejo, O. O., & Gupta, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10116 · Cited by 172 · see citing works
Byzantine Stochastic Gradient DescentPreprint
Alistarh, D., Allen-Zhu, Z., & Li, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08917 · Cited by 172 · see citing works
Sever: A Robust Meta-Algorithm for Stochastic OptimizationPreprint
Diakonikolas, I., Kamath, G., Kane, D. M., Li, J. Z., Steinhardt, J., & Stewart, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.02815 · Cited by 162 · see citing works
Byzantine-Robust Federated Machine Learning through Adaptive Model AveragingPreprint
Muñoz-González, L., Co, K. T., & Lupu, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05125 · Cited by 153 · see citing works
Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge TransferPreprint
Chang, H., Shejwalkar, V., Shokri, R., & Houmansadr, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11279 · Cited by 131 · see citing works
Self-citationBeing Robust (in High Dimensions) Can Be PracticalPreprint
Diakonikolas, I., Kamath, G., Kane, D. M., Li, J. Z., Moitra, A., & Stewart, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00893 · Cited by 91 · see citing works
Principled approaches to robust machine learning and beyond
Li, J. Z. (2018) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 28 · see citing works
Byzantine-Robust Learning on Heterogeneous Datasets via ResamplingPreprint
He, L., Karimireddy, S. P., & Jaggi, M. (2021) · arXiv (Cornell University) · Cited by 25 · see citing works
Universal Multi-Party Poisoning AttacksPreprint
Mahloujifar, S., Mahmoody, M., & Mohammed, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03474 · Cited by 9 · see citing works
The Hidden Vulnerability of Distributed Learning in Byzantium
Mhamdi, E. M. E., Guerraoui, R., & Rouault, S. (2018) · arXiv · doi:10.48550/arxiv.1802.07927
Flagged first, then by citations
Research works printed but not linked: 7
- Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. Analyzing federated learning through an adversarial lens. In International Conference on Machine Learning, pages 634-643, 2019.
- Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečnỳ, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018. · arXiv
- Matthew Jagielski, Aline Oprea, Battista Biggio, Chang Liu, Cristina Nita-Rotaru, and Bo Li. Manipulating machine learning: Poisoning attacks and countermeasures against regression learning. 39th IEEE Symposium on Security and Privacy, 2018.
- Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977, 2019. · arXiv
- Jakub Konečnỳ, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2016. · arXiv
- Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2017. · arXiv
- Cong Xie, Sanmi Koyejo, and Indranil Gupta. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation. arXiv preprint arXiv:1903.03936, 2019. · arXiv
Cited by
Sorted by citations
Shejwalkar, V., Houmansadr, A., Kairouz, P., & Ramage, D. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833647 · Cited by 292 · see citing works
Self-citationPoisoning attacks in federated learning: A survey
Xia, G., Chen, J., Yu, C., & Ma, J. (2023) · IEEE Access · doi:10.1109/access.2023.3238823 · Cited by 212 · see citing works
CONTRA: Defending Against Poisoning Attacks in Federated Learning
Awan, S., Luo, B., & Li, F. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-88418-5_22 · Cited by 182 · see citing works
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
Naseri, M., Hayes, J., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23054 · Cited by 188 · see citing works
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
“Model poisoning backdoor attacks during the aggregation phase can modify the aggregation process by scaling the parameter of the uploaded malicious local model [181], [182].”
BackgroundA Systematic Review of Adversarial Machine Learning Attacks, Defensive Controls and Technologies
Malik, J., Muthalagu, R., & Pawar, P. M. (2024) · IEEE Access · doi:10.1109/access.2024.3423323 · Cited by 76 · see citing works
A robust analysis of adversarial attacks on federated learning environments
Nair, A. K., Raj, E. D., & Sahoo, J. (2023) · Computer Standards & Interfaces · doi:10.1016/j.csi.2023.103723 · Cited by 70 · see citing works
Defense against backdoor attack in federated learning
Lu, S., Li, R., Liu, W., & Chen, X. (2022) · Computers & Security · doi:10.1016/j.cose.2022.102819 · Cited by 48 · see citing works
Adam, M., & Baroudi, U. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3506214 · Cited by 46 · see citing works
Threats and Defenses in the Federated Learning Life Cycle: A Comprehensive Survey and Challenges
Li, Y., Guo, Z., Yang, N., Chen, H., Yuan, D., & Ding, W. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3563537 · Cited by 38 · see citing works
Papadopoulos, C., Kollias, K.-F., & Fragulis, G. F. (2024) · Future Internet · doi:10.3390/fi16110415 · Cited by 35 · see citing works
Federated learning vulnerabilities, threats and defenses: A systematic review and future directions
Almutairi, S., & Barnawi, A. (2023) · Internet of Things · doi:10.1016/j.iot.2023.100947 · Cited by 35 · see citing works
FedRDF: A Robust and Dynamic Aggregation Function Against Poisoning Attacks in Federated Learning
Campos, E. M., Gonzalez-Vidal, A., Hernández-Ramos, J. L., & Skarmeta, A. (2025) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2024.3474484 · Cited by 22 · see citing works
Robust Federated Learning for Ubiquitous Computing through Mitigation of Edge-Case Backdoor Attacks
Elhattab, F., Bouchenak, S., Talbi, R., & Nitu, V. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3569492 · Cited by 11 · see citing works
- Authors
- Shejwalkar, V., & Houmansadr, A.
- Affiliations
- University of Massachusetts Amherst (US)
- Corresponding author
- Shejwalkar, Virat
- Venue
- Proceedings 2021 Network and Distributed System Security Symposium
- Publisher
- Internet Society
- Published
- 2021
- OpenAlex
- W3138153888
- Cited by
- 615
- References
- 32
- Citations by year
- 2026: 123 · 2025: 183 · 2024: 158 · 2023: 96 · 2022: 43 · 2021: 12
- FWCI
- 35.62
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Network and Distributed System Security Symposium (publishedVersion)
- Funders
- National Science Foundation (1739462)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- federated learning, model poisoning, Byzantine attacks, attack optimization, malicious participants, next-word prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
