Provably Secure Federated Learning against Malicious Clients
Open accessAbstract
Federated learning enables clients to collaboratively learn a shared global model without sharing their local training data with a cloud server. However, malicious clients can corrupt the global model to predict incorrect labels for testing examples. Existing defenses against malicious clients leverage Byzantine-robust federated learning methods. However, these methods cannot provably guarantee that the predicted label for a testing example is not affected by malicious clients. We bridge this gap via ensemble federated learning. In particular, given any base federated learning algorithm, we use the algorithm to learn multiple global models, each of which is learnt using a randomly selected subset of clients. When predicting the label of a testing example, we take majority vote among the global models. We show that our ensemble federated learning with any base federated learning algorithm is provably secure against malicious clients. Specifically, the label predicted by our ensemble global model for a testing example is provably not affected by a bounded number of malicious clients. Moreover, we show that our derived bound is tight. We evaluate our method on MNIST and Human Activity Recognition datasets. For instance, our method can achieve a certified accuracy of 88% on MNIST when 20 out of 1,000 clients are malicious.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The non-IID nature of WFL training data makes it easier for malicious clients to intentionally send anomaly updates to bias the global model in their favor.
“Furthermore, it makes it easier for malicious clients to intentionally send anomaly updates to bias the global model in their favor [28].”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
THE USE OF CONFIDENCE OR FIDUCIAL LIMITS ILLUSTRATED IN THE CASE OF THE BINOMIAL
Clopper, C. J., & Pearson, E. S. (1934) · Biometrika · doi:10.1093/biomet/26.4.404 · Cited by 5,106 · see citing works
Practical Secure Aggregation for Privacy-Preserving Machine Learning
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3133982 · Cited by 3,956 · see citing works
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · see citing works
(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959 · see citing works
Zhu, L., & Han, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-63076-8_2 · Cited by 1,906 · see citing works
Exploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734 · see citing works
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575 · see citing works
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · see citing works
(2017) · neural information processing systems · Cited by 1,413 · see citing works
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Hitaj, B., Ateniese, G., & Perez-Cruz, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134012 · Cited by 1,377 · see citing works
QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
Alistarh, D., Grubic, D., Li, J. Z., Tomioka, R., & Vojnović, M. (2016) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · doi:10.48550/arxiv.1610.02132 · Cited by 1,189 · see citing works
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
Yin, D., Chen, Y., Ramchandran, K., & Bartlett, P. (2018) · arXiv · doi:10.48550/arxiv.1803.01498 · Cited by 1,130 · see citing works
Differentially Private Federated Learning: A Client Level Perspective
Geyer, R. C., Klein, T., & Nabi, M. (2017) · arXiv · doi:10.48550/arxiv.1712.07557 · Cited by 1,052 · see citing works
Speeding Up Distributed Machine Learning Using Codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2736066 · Cited by 829 · see citing works
Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
Fang, M., Cao, X., Jia, J., & Gong, N. Z. (2019) · arXiv · doi:10.48550/arxiv.1911.11815 · Cited by 828 · see citing works
Self-citationFederated Learning with Matched AveragingPreprint
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., & Khazaeni, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06440 · Cited by 725 · see citing works
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep LearningPreprint
Wen, W., Xu, C., Yan, F., Wu, C., Wang, Y., Chen, Y., & Li, H. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07878 · Cited by 718 · see citing works
Analyzing Federated Learning through an Adversarial Lens
Bhagoji, A. N., Chakraborty, S., Mittal, P., & Calo, S. (2018) · arXiv · doi:10.48550/arxiv.1811.12470 · Cited by 703 · see citing works
Agnostic Federated LearningPreprint
Mohri, M., Sivek, G., & Suresh, A. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00146 · Cited by 619 · see citing works
PowerSGD: Practical Low-Rank Gradient Compression for Distributed OptimizationPreprint
Vogels, T., Karimireddy, S. P., & Jaggi, M. (2019) · Neural Information Processing Systems · doi:10.48550/arxiv.1905.13727 · Cited by 497 · see citing works
Bayesian Nonparametric Federated Learning of Neural NetworksPreprint
Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K. H., Hoang, T. N., & Khazaeni, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12022 · Cited by 482 · see citing works
Fair Resource Allocation in Federated Learning
Li, T., Sanjabi, M., Beirami, A., & Smith, V. (2019) · arXiv · doi:10.48550/arxiv.1905.10497 · Cited by 475 · see citing works
DBA: Distributed Backdoor Attacks against Federated Learning
Xie, C., Huang, K., Chen, P., & Li, B. (2020) · International Conference on Learning Representations · Cited by 468 · see citing works
The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial
Clopper, C. J., & Pearson, E. S. (1934) · Biometrika · doi:10.2307/2331986 · Cited by 438 · see citing works
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2020) · International Conference on Learning Representations · Cited by 275 · see citing works
On the Convergence of Federated Optimization in Heterogeneous Networks.Preprint
Sahu, A. K., Li, T., Sanjabi, M., Zaheer, M., Talwalkar, A., & Smith, V. (2018) · arXiv (Cornell University) · Cited by 237 · see citing works
FetchSGD: Communication-Efficient Federated Learning with SketchingPreprint
Rothchild, D., Panda, A., Ullah, E., Ivkin, N. P., Stoica, I. G., Braverman, V., Gonzalez, J. E., & Arora, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07682 · Cited by 234 · see citing works
Deep Models Under the GAN: Information Leakage from Collaborative Deep LearningPreprint
Hitaj, B., Ateniese, G., & Pérez‐Cruz, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.07464 · Cited by 232 · see citing works
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
Chen, Y., Su, L., & Xu, J. (2018) · Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3219617.3219655 · Cited by 228 · see citing works
Federated Adversarial Domain AdaptationPreprint
Peng, X., Huang, Z., Zhu, Y., & Saenko, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02054 · Cited by 223 · see citing works
Practical Federated Gradient Boosting Decision Trees
Li, Q., Wen, Z., & He, B. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5895 · Cited by 206 · see citing works
DRACO: Byzantine-resilient Distributed Training via Redundant GradientsPreprint
Chen, L., Wang, H., Charles, Z., & Papailiopoulos, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09877 · Cited by 191 · see citing works
Analyzing Federated Learning through an Adversarial Lens
Bhagoji, A. N., Chakraborty, S., Mittal, P., & Calo, S. B. (2019) · International Conference on Machine Learning · Cited by 188 · see citing works
Byzantine Stochastic Gradient DescentPreprint
Alistarh, D., Allen-Zhu, Z., & Li, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08917 · Cited by 172 · see citing works
Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product ManipulationPreprint
Xie, C., Koyejo, S., & Gupta, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03936 · Cited by 153 · see citing works
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient DescentPreprint
Chen, Y., Su, L., & Xu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05491 · Cited by 136 · see citing works
Federated Latent Dirichlet Allocation: A Local Differential Privacy Based Framework
Wang, Y., Tong, Y., & Shi, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6096 · Cited by 122 · see citing works
Fair Resource Allocation in Federated Learning
Li, T., Sanjabi, M., Beirami, A., & Smith, V. (2020) · International Conference on Learning Representations · Cited by 120 · see citing works
Federated Learning for Vision-and-Language Grounding Problems
Liu, F., Wu, X., Ge, S., Fan, W., & Zou, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6824 · Cited by 115 · see citing works
Intrinsic Certified Robustness of Bagging against Data Poisoning Attacks
Jia, J., Cao, X., & Gong, N. Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16971 · Cited by 89 · see citing works
Self-citationAcceleration for Compressed Gradient Descent in Distributed and Federated OptimizationPreprint
Li, Z., Kovalev, D. A., Qian, X., & Richtárik, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11364 · Cited by 72 · see citing works
Defending Against Saddle Point Attack in Byzantine-Robust Distributed LearningPreprint
Yin, D., Chen, Y., Ramchandran, K., & Bartlett, P. L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05358 · Cited by 66 · see citing works
From Local SGD to Local Fixed Point Methods for Federated Learning
Malinovsky, G., Kovalev, D. A., Gasanov, E., Condat, L., & Richtárik, P. (2020) · International Conference on Machine Learning · Cited by 59 · see citing works
Practical Federated Gradient Boosting Decision TreesPreprint
Li, Q., Wen, Z., & He, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.04206 · Cited by 27 · see citing works
Intrinsic Certified Robustness of Bagging against Data Poisoning AttacksPreprint
Jia, J., Cao, X., & Gong, N. Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.04495 · Cited by 15 · see citing works
Self-citationFrom Local SGD to Local Fixed-Point Methods for Federated LearningPreprint
Malinovsky, G., Kovalev, D. A., Gasanov, E., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.01442 · Cited by 10 · see citing works
FedBoost: Communication-efficient algorithms for federated learning
Hamer, J., Mohri, M., & Suresh, A. T. (2020) · International Conference on Machine Learning · Cited by 3 · see citing works
The Hidden Vulnerability of Distributed Learning in Byzantium
Mhamdi, E. M. E., Guerraoui, R., & Rouault, S. (2018) · arXiv · doi:10.48550/arxiv.1802.07927
Flagged first, then by citations
Research works printed but not linked: 4
- Bernstein, J.; Wang, Y.-X.; Azizzadenesheli, K.; and Anandkumar, A. 2018. signSGD: Compressed Optimisation for Non-Convex Problems. In ICML .
- Cao, X.; Jia, J.; and Gong, N. Z. 2021. Provably Secure Federated Learning against Malicious Clients. arXiv preprint arXiv:2102.01854 . · arXiv
- Kairouz, P.; McMahan, H. B.; Avent, B.; Bellet, A.; Bennis, M.; Bhagoji, A. N.; Bonawitz, K.; Charles, Z.; Cormode, G.; Cummings, R.; et al. 2019. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 . · arXiv
- Smith, V.; Chiang, C.-K.; Sanjabi, M.; and Talwalkar, A. S. 2017. Federated multi-task learning. In NeurIPS .
Cited by
Sorted by citations
ShieldFL: Mitigating Model Poisoning Attacks in Privacy-Preserving Federated Learning
Ma, Z., Ma, J., Miao, Y., Li, Y., & Deng, R. H. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3169918 · Cited by 371 · see citing works
Shejwalkar, V., Houmansadr, A., Kairouz, P., & Ramage, D. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833647 · Cited by 292 · see citing works
MPAF: Model Poisoning Attacks to Federated Learning based on Fake Clients
Cao, X., & Gong, N. Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00383 · Cited by 205 · see citing works
Self-citationDeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
Rieger, P., Nguyen, T. D., Miettinen, M., & Sadeghi, A.-R. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23156 · Cited by 202 · see citing works
Deep Model Poisoning Attack on Federated Learning
Zhou, X., Xu, M., Wu, Y., & Zheng, N. (2021) · Future Internet · doi:10.3390/fi13030073 · Cited by 170 · see citing works
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
A Survey of Security Strategies in Federated Learning: Defending Models, Data, and Privacy
Manzoor, H. U., Shabbir, A., Chen, A., Flynn, D., & Zoha, A. (2024) · Future Internet · doi:10.3390/fi16100374 · Cited by 58 · see citing works
APFed: Anti-Poisoning Attacks in Privacy-Preserving Heterogeneous Federated Learning
Chen, X., Yu, H., Jia, X., & Yu, X. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3315125 · Cited by 50 · see citing works
BayBFed: Bayesian Backdoor Defense for Federated Learning
Kumari, K., Rieger, P., Fereidooni, H., Jadliwala, M., & Sadeghi, A.-R. (2023) · 2023 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46215.2023.10179362 · Cited by 37 · see citing works
RobustFed: A Truth Inference Approach for Robust Federated Learning
Tahmasebian, F., Lou, J., & Xiong, L. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · doi:10.1145/3511808.3557439 · Cited by 26 · see citing works
An adaptive robust defending algorithm against backdoor attacks in federated learning
Wang, Y., Zhai, D.-H., He, Y., & Xia, Y. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.01.026 · Cited by 25 · see citing works
- Authors
- Cao, X., Jia, J., & Gong, N. Z.
- Affiliations
- Duke University (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2021-05-18
- Volume
- 35
- Issue
- 8
- Pages
- 6885-6893
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W3127520698
- Cited by
- 141
- References
- 49
- Citations by year
- 2026: 8 · 2025: 29 · 2024: 39 · 2023: 31 · 2022: 26 · 2021: 8
- FWCI
- 12.01
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion, public-domain)
- Funders
- National Science Foundation (1937786)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
- Keywords
- federated learning, malicious clients, Byzantine attacks, ensemble learning, majority voting, provable security, robustness bounds
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
