CoChat
Sign Up
PreprintCited by 22Adversarial Robustness in Machine LearningPrivacy-Preserving Technologies in DataDigital Media Forensic Detection

Backdoor Attacks on Federated Meta-Learning

Open access
Chen, Chien-Lun, Golubchik, Leana, Paolieri, Marco·2020·arXiv
DOI 10.48550/arxiv.2006.07026arXiv 2006.07026
Open this paper in CoChat — free accountOpen PDF

Abstract

Federated learning allows multiple users to collaboratively train a shared classification model while preserving data privacy. This approach, where model updates are aggregated by a central server, was shown to be vulnerable to poisoning backdoor attacks: a malicious user can alter the shared model to arbitrarily classify specific inputs from a given class. In this paper, we analyze the effects of backdoor attacks on federated meta-learning, where users train a model that can be adapted to different sets of output classes using only a few examples. While the ability to adapt could, in principle, make federated learning frameworks more robust to backdoor attacks (when new training examples are benign), we find that even 1-shot~attacks can be very successful and persist after additional training. To address these vulnerabilities, we propose a defense mechanism inspired by matching networks, where the class of an input is predicted from the similarity of its features with a support set of labeled examples. By removing the decision logic from the model shared with the federation, success and persistence of backdoor attacks are greatly reduced.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2006.07026

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Hyperparameter optimization (HPO) has a long history under AutoML, but it mainly concerns improving model accuracy rather than communication and computing efficacy for mobile devices.

    “Hyperparameter optimization (HPO) has a long history under the framework of AutoML [395, 273, 277], but it mainly concerns how to improve the model accuracy [64, 426, 374, 180] rather than communication and computing efficacy for mobile devices.”

    From Advances and Open Problems in Federated Learning (2021) · cites this paper as [426]

    The cited paper discusses accuracy in the context of backdoor attacks and defenses (Section 4, p. 6; Appendix A.1, p. 10). Searches for hyperparameter optimization and AutoML found no passage addressing whether HPO primarily targets model accuracy rather than communication or computing efficiency on mobile devices. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 22

Research works printed but not linked: 32

  1. [1][1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, 'Communication-efficient learning of deep networks from decentralized data,' in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA , pp. 1273-1282, 2017. -
  2. [2][2] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, 'Practical secure aggregation for privacy preserving machine learning,' IACR Cryptology ePrint Archive , vol. 2017, p. 281, 2017. -
  3. [3][3] L. T. Phong, Y. Aono, T. Hayashi, L. Wang, and S. Moriai, 'Privacy-preserving deep learning via additively homomorphic encryption,' IEEE Trans. Information Forensics and Security , vol. 13, no. 5, pp. 1333-1345, 2018. -
  4. [4][4] X. Chen, C. Liu, B. Li, K. Lu, and D. Song, 'Targeted backdoor attacks on deep learning systems using data poisoning,' CoRR , vol. abs/1712.05526, 2017. - · arXiv
  5. [5][5] T. Gu, B. Dolan-Gavitt, and S. Garg, 'Badnets: Identifying vulnerabilities in the machine learning model supply chain,' CoRR , vol. abs/1708.06733, 2017. - · arXiv
  6. [6][6] A. N. Bhagoji, S. Chakraborty, P. Mittal, and S. B. Calo, 'Analyzing federated learning through an adversarial lens,' in Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , pp. 634-643, 2019. -
  7. [7][7] E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, 'How to backdoor federated learning,' CoRR , vol. abs/1807.00459, 2018. - · arXiv
  8. [8][8] G. Baruch, M. Baruch, and Y. Goldberg, 'A little is enough: Circumventing defenses for distributed learning,' in Advances in Neural Information Processing Systems 32, NIPS 2019, December 8-14, 2019, Vancouver, Canada , pp. 8632-8642, Curran Associates, Inc., 2019. -
  9. [9][9] O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra, 'Matching networks for one shot learning,' in Advances in Neural Information Processing Systems 29, NIPS 2016, December 5-10, 2016, Barcelona, Spain , pp. 3630-3638, 2016. -
  10. [10][10] C. Finn, P. Abbeel, and S. Levine, 'Model-agnostic meta-learning for fast adaptation of deep networks,' in Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 , pp. 1126-1135, 2017. -
  11. [11][11] A. Nichol, J. Achiam, and J. Schulman, 'On first-order meta-learning algorithms,' CoRR , vol. abs/1803.02999, 2018. - · arXiv
  12. [12][12] F. Chen, Z. Dong, Z. Li, and X. He, 'Federated meta-learning for recommendation,' CoRR , vol. abs/1802.07876, 2018. - · arXiv
  13. [13][13] J. Li, M. Khodak, S. Caldas, and A. Talwalkar, 'Differentially private meta-learning,' CoRR , vol. abs/1909.05830, 2019. - · arXiv
  14. [14][14] S. Shen, S. Tople, and P. Saxena, 'Auror: defending against poisoning attacks in collaborative deep learning systems,' in Proceedings of the 32nd Annual Conference on Computer Security Applications, ACSAC 2016, Los Angeles, CA, USA, December 5-9, 2016 , pp. 508-519, 2016. -
  15. [15][15] P. Blanchard, E. M. E. Mhamdi, R. Guerraoui, and J. Stainer, 'Machine learning with adversaries: Byzantine tolerant gradient descent,' in Advances in Neural Information Processing Systems 30, NIPS 2017, 4-9 December 2017, Long Beach, CA, USA , pp. 119-129, 2017. -
  16. [16][16] J. Steinhardt, P. W. Koh, and P. Liang, 'Certified defenses for data poisoning attacks,' in Advances in Neural Information Processing Systems 30, NIPS 2017, 4-9 December 2017, Long Beach, CA, USA , pp. 3517-3529, 2017. -
  17. [17][17] M. Qiao and G. Valiant, 'Learning discrete distributions from untrusted batches,' in 9th Innovations in Theoretical Computer Science Conference, ITCS 2018, January 11-14, 2018, Cambridge, MA, USA , pp. 47:1-47:20, 2018. -
  18. [18][18] D. Yin, Y. Chen, K. Ramchandran, and P. L. Bartlett, 'Byzantine-robust distributed learning: Towards optimal statistical rates,' in Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018 , pp. 5636-5645, 2018. -
  19. [19][19] C. Xie, O. Koyejo, and I. Gupta, 'Generalized byzantine-tolerant SGD,' CoRR , vol. abs/1802.10116, 2018. - · arXiv
  20. [20][20] B. Tran, J. Li, and A. Madry, 'Spectral signatures in backdoor attacks,' in Advances in Neural Information Processing Systems 31, NIPS 2018, 3-8 December 2018, Montréal, Canada , pp. 8011-8021, 2018. -
  21. [21][21] Y. Chen, L. Su, and J. Xu, 'Distributed statistical machine learning in adversarial settings: Byzantine gradient descent,' in Proceedings of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2018, Irvine, CA, USA, June 18-22, 2018 , p. 96, 2018. -
  22. [22][22] E. M. E. Mhamdi, R. Guerraoui, and S. Rouault, 'The hidden vulnerability of distributed learning in byzantium,' in Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018 , pp. 3518-3527, 2018. -
  23. [23][23] C. Fung, C. J. M. Yoon, and I. Beschastnikh, 'Mitigating sybils in federated learning poisoning,' CoRR , vol. abs/1808.04866, 2018. - · arXiv
  24. [24][24] B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, 'Human-level concept learning through probabilistic program induction,' Science , vol. 350, no. 6266, pp. 1332-1338, 2015. -
  25. [25][25] Q. Sun, Y. Liu, T. Chua, and B. Schiele, 'Meta-transfer learning for few-shot learning,' in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019. -
  26. [26][26] A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. P. Lillicrap, 'Meta-learning with memory-augmented neural networks,' in Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016 , pp. 1842-1850, 2016. -
  27. [27][27] D. Bahdanau, K. Cho, and Y. Bengio, 'Neural machine translation by jointly learning to align and translate,' in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015. -
  28. [28][28] T. Luong, H. Pham, and C. D. Manning, 'Effective approaches to attention-based neural machine translation,' in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015 , pp. 1412-1421, 2015. -
  29. [29][29] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, 'Attention is all you need,' in Advances in Neural Information Processing Systems 30, NIPS 2017, 4-9 December 2017, Long Beach, CA, USA , pp. 5998-6008, 2017. -
  30. [30][30] M. Collier and J. Beel, 'Implementing neural turing machines,' in 27th International Conference on Artificial Neural Networks, ICANN 2018, Rhodes, Greece, October 4-7, 2018, Proceedings, Part III , pp. 94-104, 2018. -
  31. [31][31] J. Snell, K. Swersky, and R. S. Zemel, 'Prototypical networks for few-shot learning,' in Advances in Neural Information Processing Systems 30, NIPS 2017, 4-9 December 2017, Long Beach, CA, USA , pp. 4077-4087, 2017. -
  32. [32][32] W. Chen, Y. Liu, Z. Kira, Y. F. Wang, and J. Huang, 'A closer look at few-shot classification,' CoRR , vol. abs/1904.04232, 2019. · arXiv

Authors
Chen, C.-L., Golubchik, L., & Paolieri, M.
Affiliations
University of Southern California (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2006.07026
arXiv
2006.07026
OpenAlex
W3215891850
Cited by
22
References
0
Citations by year
2025: 1 · 2024: 5 · 2023: 1 · 2022: 9 · 2021: 1 · 2020: 5
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • neural information processing systems
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
backdoor attacks, federated learning, federated meta-learning, poisoning attacks, data privacy, few-shot learning, matching networks, defense mechanisms

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review