MPAF: Model Poisoning Attacks to Federated Learning based on Fake Clients
Open accessAbstract
Existing model poisoning attacks to federated learning assume that an attacker has access to a large fraction of compromised genuine clients. However, such assumption is not realistic in production federated learning systems that involve millions of clients. In this work, we propose the first Model Poisoning Attack based on Fake clients called MPAF. Specifically, we assume the attacker injects fake clients to a federated learning system and sends carefully crafted fake local model updates to the cloud server during training, such that the learnt global model has low accuracy for many indiscriminate test inputs. Towards this goal, our attack drags the global model towards an attacker-chosen base model that has low accuracy. Specifically, in each round of federated learning, the fake clients craft fake local model updates that point to the base model and scale them up to amplify their impact before sending them to the cloud server. Our experiments show that MPAF can significantly decrease the test accuracy of the global model, even if classical defenses and norm clipping are adopted, highlighting the need for more advanced defenses.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An attacker can add random noise to the local model to affect the convergence of the global model.
“For instance, the attacker can add random noise to the local model to affect the convergence of the global model [34], [35].”
From Poisoning attacks in federated learning: A survey (2023) · cites this paper as [35]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · see citing works
Data Poisoning Attacks Against Federated Learning SystemsPreprint
Tolpegin, V., Truex, S., Gursoy, M. E., & Liu, L. (2020) · European Symposium on Research in Computer Security · doi:10.48550/arxiv.2007.08432 · Cited by 1,021 · see citing works
FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping
Cao, X., Fang, M., Liu, J., & Gong, N. Z. (2021) · Proceedings 2021 Network and Distributed System Security Symposium · doi:10.14722/ndss.2021.24434 · Cited by 999 · see citing works
Self-citationData Poisoning Attacks Against Federated Learning Systems
Tolpegin, V., Truex, S., Gursoy, M. E., & Liu, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58951-6_24 · Cited by 927 · see citing works
Federated Learning: Collaborative Machine Learning without Centralized Training Data
Abhishek, V. A., Binny, S., Johan, T. R., Raj, N., & Thomas, V. (2022) · International Journal of Engineering Technology and Management Sciences · doi:10.46647/ijetms.2022.v06i05.052 · Cited by 489 · see citing works
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
Chen, Y., Su, L., & Xu, J. (2017) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3154503 · Cited by 390 · see citing works
Can You Really Backdoor Federated Learning?
Sun, Z., Kairouz, P., Suresh, A. T., & McMahan, H. B. (2019) · arXiv · doi:10.48550/arxiv.1911.07963 · Cited by 348 · see citing works
Shejwalkar, V., Houmansadr, A., Kairouz, P., & Ramage, D. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833647 · Cited by 292 · see citing works
A <scp>uror</scp>: defending against poisoning attacks in collaborative deep learning systems
Shen, S., Tople, S., & Saxena, P. (2016) · Proceedings of the 32nd Annual Conference on Computer Security Applications · doi:10.1145/2991079.2991125 · Cited by 228 · see citing works
Provably Secure Federated Learning against Malicious Clients
Cao, X., Jia, J., & Gong, N. Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16849 · Cited by 141 · see citing works
Self-citationFree-riders in Federated Learning: Attacks and Defenses
Lin, J., Du, M., & Liu, J. (2019) · arXiv · doi:10.48550/arxiv.1911.12560 · Cited by 76 · see citing works
The Hidden Vulnerability of Distributed Learning in Byzantium
Mhamdi, E. M. E., Guerraoui, R., & Rouault, S. (2018) · arXiv · doi:10.48550/arxiv.1802.07927
Flagged first, then by citations
Research works printed but not linked: 11
- [9][9] Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. Analyzing federated learning through an adversarial lens. In ICML , 2019. 1, 3 -
- [10][10] Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In NeurIPS , 2017. 1, 3 -
- [14][14] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Local model poisoning attacks to byzantinerobust federated learning. In USENIX Security Symposium , 2020. 1, 2, 3, 5, 8 -
- [15][15] Yann Fraboni, Richard Vidal, and Marco Lorenzi. Free-rider attacks on model aggregation in federated learning. In AISTATS , 2021. 3 -
- [18][18] H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Ag¨ uera y Arcas. Communicationefficient learning of deep networks from decentralized data. In AISTATS , 2017. 1, 2 -
- [20][20] Shashank Rajput, Hongyi Wang, Zachary Charles, and Dimitris Papailiopoulos. Detox: A redundancy-based framework for faster and more robust gradient aggregation. In NIPS , 2019. 3 -
- [26][26] Chulin Xie, Minghao Chen, Pin-Yu Chen, and Bo Li. Crfl: Certifiably robust federated learning against backdoor attacks. In ICML , 2021. 3 -
- [27][27] Chulin Xie, Keli Huang, Pin-Yu Chen, and Bo Li. Dba: Distributed backdoor attacks against federated learning. In ICLR , 2020. 3 -
- [28][28] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation. In UAI , 2020. 1, 2, 3 -
- [29][29] Cong Xie, Sanmi Koyejo, and Indranil Gupta. Zeno: Distributed stochastic gradient descent with suspicion-based fault-tolerance. In ICML , 2019. 3 -
- [30][30] Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In ICML , 2018. 1, 2, 3
Cited by
Sorted by citations
Poisoning attacks in federated learning: A survey
Xia, G., Chen, J., Yu, C., & Ma, J. (2023) · IEEE Access · doi:10.1109/access.2023.3238823 · Cited by 212 · see citing works
Exploring privacy measurement in federated learning
Jagarlamudi, G. K., Yazdinejad, A., Parizi, R. M., & Pouriyeh, S. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-023-05846-4 · Cited by 52 · see citing works
Data-Agnostic Model Poisoning Against Federated Learning: A Graph Autoencoder Approach
Li, K., Zheng, J., Yuan, X., Ni, W., Akan, O. B., & Poor, H. V. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3362147 · Cited by 41 · see citing works
Poisoning Federated Recommender Systems with Fake Users
Yin, M., Xu, Y., Fang, M., & Gong, N. Z. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645492 · Cited by 27 · see citing works
Self-citation
- Authors
- Cao, X., & Gong, N. Z.
- Affiliations
- Duke University (US)
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 3395-3403
- OpenAlex
- W4221155325
- Cited by
- 205
- References
- 12
- Citations by year
- 2026: 32 · 2025: 81 · 2024: 60 · 2023: 24 · 2022: 8
- FWCI
- 23.90
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- Arm
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- model poisoning, federated learning, adversarial attacks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
