BayBFed: Bayesian Backdoor Defense for Federated Learning
Open accessAbstract
Federated learning (FL) is an emerging technology that allows participants to jointly train a machine learning model without sharing their private data with others. However, FL is vulnerable to poisoning attacks such as backdoor attacks. Consequently, a variety of defenses have recently been proposed, which have primarily utilized intermediary states of the global model (i.e., logits) or distance of the local models (i.e., L2−norm) with respect to the global model to detect malicious backdoors in FL. However, as these approaches directly operate on client updates (or weights), their effectiveness depends on factors such as clients’ data distribution or the adversary’s attack strategies. In this paper, we introduce a novel and more generic backdoor defense framework, called BayBFed, which proposes to utilize probability distributions over client updates to detect malicious updates in FL: BayBFed computes a probabilistic measure over the clients’ updates to keep track of any adjustments made in the updates, and uses a novel detection algorithm that can leverage this probabilistic measure to efficiently detect and filter out malicious updates. Thus, it overcomes the shortcomings of previous approaches that arise due to the direct usage of client updates; nevertheless, our probabilistic measure will include all aspects of the local client training strategies. BayBFed utilizes two Bayesian NonParametric (BNP) extensions: (i) a Hierarchical Beta-Bernoulli process to draw a probabilistic measure given the clients’ updates, and (ii) an adaptation of the Chinese Restaurant Process (CRP), referred by us as CRP-Jensen, which leverages this probabilistic measure to detect and filter out malicious updates. We extensively evaluate our defense approach on five benchmark datasets: CIFAR10, Reddit, IoT intrusion detection, MNIST, and FMNIST, and show that it can effectively detect and eliminate malicious updates in FL without deteriorating the benign performance of the global model.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The decentralized nature of federated learning makes security measures difficult, with opposing and input manipulation attacks particularly concerning due to limited visibility of local model updates.
“In addition, the decentralized nature of FL makes security measures difficult, with opposing and input manipulation attacks being particularly concerning due to the limited visibility of local model updates [21].”
From Federated Learning in Smart Healthcare: A Comprehensive Review on Privacy, Security, and Predictive Analytics with IoT Integration (2024) · cites this paper as [21]
Cites
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · see citing works
Federated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2021) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-020-00082-4 · Cited by 1,542 · cited 1× · see citing works
“For instance, multiple hospitals can participate in training a global model for cancer classification without revealing individual patients’ cancer records [21], [36], [46].”
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · see citing works
(2017) · neural information processing systems · Cited by 1,413 · see citing works
FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping
Cao, X., Fang, M., Liu, J., & Gong, N. Z. (2021) · Proceedings 2021 Network and Distributed System Security Symposium · doi:10.14722/ndss.2021.24434 · Cited by 999 · see citing works
Robust Aggregation for Federated Learning
Pillutla, K., Kakade, S. M., & Harchaoui, Z. (2022) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2022.3153135 · Cited by 947 · see citing works
Learning Differentially Private Recurrent Language Models
McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L. (2017) · arXiv · doi:10.48550/arxiv.1710.06963 · Cited by 849 · see citing works
Classifying IoT Devices in Smart Environments Using Network Traffic Characteristics
Sivanathan, A., Gharakheili, H. H., Loi, F., Radford, A., Wijenayake, C. A., Vishwanath, A., & Sivaraman, V. (2018) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2018.2866249 · Cited by 812 · see citing works
How to Explain Individual Classification DecisionsPreprint
Baehrens, D., Schroeter, T. S., Stefan Harmeling, B., Kawanabe, M., Hansen, K., & Mueller, K.-R. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0912.1128 · Cited by 743 · see citing works
DÏoT: A Federated Self-learning Anomaly Detection System for IoT
Nguyen, T. D., Marchal, S., Miettinen, M., Fereidooni, H., Asokan, N., & Sadeghi, A.-R. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00080 · Cited by 663 · see citing works
Self-citationSheller, M. J., Reina, G. A., Edwards, B., Martin, J., & Bakas, S. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11723-8_9 · Cited by 609 · see citing works
Lectures on the Poisson Process
Last, G., & Penrose, M. D. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316104477 · Cited by 518 · see citing works
Federated Learning: Collaborative Machine Learning without Centralized Training Data
Abhishek, V. A., Binny, S., Johan, T. R., Raj, N., & Thomas, V. (2022) · International Journal of Engineering Technology and Management Sciences · doi:10.46647/ijetms.2022.v06i05.052 · Cited by 489 · see citing works
DBA: Distributed Backdoor Attacks against Federated Learning
Xie, C., Huang, K., Chen, P., & Li, B. (2020) · International Conference on Learning Representations · Cited by 468 · see citing works
Applied Federated Learning: Improving Google Keyboard Query SuggestionsPreprint
Yang, T. T., Andrew, G., Eichner, H., Sun, H., Li, W., Kong, N., Ramage, D., & Beaufays, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02903 · Cited by 450 · see citing works
Attack of the tails: Yes, you really can backdoor federated learning
Wang, H., Sreenivasan, K., Rajput, S., Vishwakarma, H., Agarwal, S., Sohn, J.-Y., Lee, K., & Papailiopoulos, D. (2020) · arXiv · doi:10.48550/arxiv.2007.05084 · Cited by 406 · see citing works
Can You Really Backdoor Federated Learning?
Sun, Z., Kairouz, P., Suresh, A. T., & McMahan, H. B. (2019) · arXiv · doi:10.48550/arxiv.1911.07963 · Cited by 348 · see citing works
SAFELearn: Secure Aggregation for private FEderated Learning
Fereidooni, H., Marchal, S., Miettinen, M., Mirhoseini, A., Möllering, H., Nguyen, T. D., Rieger, P., Sadeghi, A., Schneider, T. H., Yalame, H., et al. (2021) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw53761.2021.00017 · Cited by 281 · see citing works
Self-citationThe Limitations of Federated Learning in Sybil Settings
Fung, C., Yoon, C. J. M., & Beschastnikh, I. (2020) · Recent Advances in Intrusion Detection · Cited by 265 · see citing works
A <scp>uror</scp>: defending against poisoning attacks in collaborative deep learning systems
Shen, S., Tople, S., & Saxena, P. (2016) · Proceedings of the 32nd Annual Conference on Computer Security Applications · doi:10.1145/2991079.2991125 · Cited by 228 · see citing works
BaFFLe: Backdoor Detection via Feedback-based Federated Learning
Andreina, S., Marson, G. A., Mollering, H., & Karame, G. (2021) · 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs51616.2021.00086 · Cited by 205 · see citing works
Learning to Detect Malicious Clients for Robust Federated LearningPreprint
Li, S., Cheng, Y. J., Wang, W., Liu, Y., & Chen, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00211 · Cited by 202 · see citing works
DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection
Rieger, P., Nguyen, T. D., Miettinen, M., & Sadeghi, A.-R. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23156 · Cited by 202 · see citing works
Self-citationPoisoning Attacks on Federated Learning-based IoT Intrusion Detection System
Nguyen, T. D., Rieger, P., Miettinen, M., & Sadeghi, A.-R. (2020) · Proceedings 2020 Workshop on Decentralized IoT Systems and Security · doi:10.14722/diss.2020.23003 · Cited by 200 · see citing works
Self-citationFLAME: Taming Backdoors in Federated Learning
Nguyen, T. D., Rieger, P., Chen, H., Yalame, H., Möllering, H., Fereidooni, H., Marchal, S., Miettinen, M., Mirhoseini, A., Zeitouni, S., et al. (2022) · Aaltodoc (Aalto University) · Cited by 198 · see citing works
Self-citationLocal and Central Differential Privacy for Robustness and Privacy in Federated Learning
Naseri, M., Hayes, J., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23054 · Cited by 188 · see citing works
Byzantine-Robust Federated Machine Learning through Adaptive Model AveragingPreprint
Muñoz-González, L., Co, K. T., & Lupu, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05125 · Cited by 153 · see citing works
Robust Traceability from Trace Amounts
Dwork, C., Smith, A., Steinke, T., Ullman, J., & Vadhan, S. (2015) · Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2015.46 · Cited by 142 · see citing works
Provably Secure Federated Learning against Malicious Clients
Cao, X., Jia, J., & Gong, N. Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16849 · Cited by 141 · see citing works
Abnormal client behavior detection in federated learning
Li, S., Cheng, Y., Liu, Y., Wang, W., & Chen, T. (2019) · arXiv · doi:10.48550/arxiv.1910.09933 · Cited by 122 · see citing works
A tutorial on Dirichlet process mixture modeling
Li, Y., Schofield, E., & Gönen, M. (2019) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2019.04.004 · Cited by 90 · see citing works
Learning Differentially Private Language Models Without Losing Accuracy
McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L. (2017) · Cited by 81 · see citing works
Nonparametric Bayesian estimators for counting processes
Kim, Y. (1999) · The Annals of Statistics · doi:10.1214/aos/1018031207 · Cited by 77 · see citing works
Mitigating Backdoor Attacks in Federated LearningPreprint
Wu, C., Xian, Y., Zhu, S., & Mitra, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.01767 · Cited by 69 · see citing works
A Stick-Breaking Construction of the Beta Process
Paisley, J. W., Zaas, A. K., Woods, C. W., et al. (2010) · International Conference on Machine Learning · Cited by 56 · see citing works
MLGuard: Mitigating Poisoning Attacks in Privacy Preserving Distributed Collaborative Learning
Khazbak, Y., Tan, T., & Cao, G. (2020) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn49398.2020.9209670 · Cited by 50 · see citing works
Posteriors, conjugacy, and exponential families for completely random measures
Broderick, T., Wilson, A., & Jordan, M. I. (2018) · Bernoulli · doi:10.3150/16-bej855 · Cited by 31 · see citing works
FedCRI: Federated Mobile Cyber-Risk Intelligence
Fereidooni, H., Dmitrienko, A., Rieger, P., Miettinen, M., Sadeghi, A., & Madlener, F. (2022) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23153 · Cited by 26 · see citing works
Self-citationThe Hidden Vulnerability of Distributed Learning in Byzantium
Mhamdi, E. M. E., Guerraoui, R., & Rouault, S. (2018) · arXiv · doi:10.48550/arxiv.1802.07927
Flagged first, then by citations
Cited by
Sorted by citations
Abbas, S. R., Abbas, Z., Zahir, A., & Lee, S. W. (2024) · Healthcare · doi:10.3390/healthcare12242587 · Cited by 228 · see citing works
- Authors
- Kumari, K., Rieger, P., Fereidooni, H., Jadliwala, M., & Sadeghi, A.-R.
- Affiliations
- Technische Universität Darmstadt (DE)
- The University of Texas at San Antonio (US)
- Venue
- 2023 IEEE Symposium on Security and Privacy (SP)
- Publisher
- IEEE
- Published
- 2023-05
- Pages
- 737-754
- ISSN
- 1081-6011 (ISSN-L)
- OpenAlex
- W4385679828
- Cited by
- 37
- References
- 39
- Citations by year
- 2026: 9 · 2025: 16 · 2024: 8 · 2023: 4
- FWCI
- 6.56
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Symposium on Security and Privacy (S&P) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- TUbilio (Technical University of Darmstadt) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1943351)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- federated learning, backdoor attacks, backdoor detection, poisoning attacks, Chinese restaurant process
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
