Hidden Trigger Backdoor Attacks
Open accessAbstract
With the success of deep learning algorithms in various domains, studying adversarial attacks to secure deep models in real world applications has become an important research topic. Backdoor attacks are a form of adversarial attacks on deep networks where the attacker provides poisoned data to the victim to train the model with, and then activates the attack by showing a specific small trigger pattern at the test time. Most state-of-the-art backdoor attacks either provide mislabeled poisoning data that is possible to identify by visual inspection, reveal the trigger in the poisoned data, or use noise to hide the trigger. We propose a novel form of backdoor attack where poisoned data look natural with correct labels and also more importantly, the attacker hides the trigger in the poisoned data and keeps the trigger secret until the test time. We perform an extensive study on various image classification settings and show that our attack can fool the model by pasting the trigger at random locations on unseen images although the model performs well on clean data. We also show that our proposed attack cannot be easily defended using a state-of-the-art defense algorithm for backdoor attacks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In trigger-based backdoor attacks, triggers are designed to be injected into the training data under the targeted label.
“In trigger-based backdoor attacks, triggers are designed to be injected into the training data under the targeted label [128], [166].”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [128]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139 · see citing works
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602 · see citing works
Practical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495 · see citing works
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680 · see citing works
Evasion attacks against machine learning at test time
Biggio, B., Corona, I., Maiorca, D., Nelson, B. A., Srndic, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Cited by 1,658 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H.-T., & Zhao, B. Y. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00031 · Cited by 1,486 · see citing works
Poisoning Attacks against Support Vector Machines
Biggio, B., Nelson, B., & Laskov, P. (2012) · arXiv · doi:10.48550/arxiv.1206.6389 · Cited by 1,331 · see citing works
Trojaning Attack on Neural Networks
Liu, Y., Ma, S., Aafer, Y., Lee, W.-C., Zhai, J., Wang, W., & Zhang, X. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23291 · Cited by 1,270 · see citing works
BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
Gu, T., Dolan-Gavitt, B., & Garg, S. (2017) · arXiv · doi:10.48550/arxiv.1708.06733 · Cited by 1,061 · see citing works
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
Liu, K., Dolan-Gavitt, B., & Garg, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00470-5_13 · Cited by 1,055 · see citing works
Gao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D. C., & Nepal, . (2019) · Annual Computer Security Applications Conference · doi:10.1145/3359789.3359790 · Cited by 719 · see citing works
Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural NetworksPreprint
Shafahi, A., Huang, W., Najibi, M., Suciu, O., Studer, C., Dumitraş, T., & Goldstein, T. (2018) · arXiv (Cornell University) · doi:10.3929/ethz-b-000460348 · Cited by 626 · see citing works
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization
Muñoz-González, L., Biggio, B., Demontis, A., Paudice, A., Wongrassamee, V., Lupu, E. C., & Roli, F. (2017) · Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3128572.3140451 · Cited by 549 · see citing works
Spectral Signatures in Backdoor AttacksPreprint
Tran, B., Li, J. Z., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00636 · Cited by 506 · see citing works
Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
Chen, B., Carvalho, W., Baracaldo, N., Ludwig, H., Edwards, B., Lee, T., Molloy, I., & Srivastava, B. (2018) · arXiv · doi:10.48550/arxiv.1811.03728 · Cited by 485 · see citing works
Using Machine Teaching to Identify Optimal Training-Set Attacks on Machine Learners
Mei, S., & Zhu, X. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9569 · Cited by 346 · see citing works
Support vector machines under adversarial label contamination
Huang, X. X., Biggio, B., Nelson, B. A., Xiao, H., Eckert, C. M., & Roli, F. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.08.081 · Cited by 220 · see citing works
Stronger Data Poisoning Attacks Break Data Sanitization Defenses
Koh, P. W., Steinhardt, J., & Liang, P. (2018) · arXiv · doi:10.48550/arxiv.1811.00741 · Cited by 189 · see citing works
Adversarial Label Flips Attack on Support Vector Machines
Xiao, H., Huang, X. X., & Claudia, E. (2012) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-61499-098-7-870 · Cited by 180 · see citing works
Adversarial Manipulation of Deep RepresentationsPreprint
Sabour, S., Cao, Y., Faghri, F., & Fleet, D. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05122 · Cited by 164 · see citing works
When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning AttacksPreprint
Suciu, O., Mărginean, R., Kaya, Y., Daumé, H., & Dumitraş, T. (2018) · arXiv (Cornell University) · Cited by 150 · see citing works
Transferable Clean-Label Poisoning Attacks on Deep Neural NetsPreprint
Zhu, C., Huang, W., Shafahi, A., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05897 · Cited by 133 · see citing works
Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
Liao, C., Zhong, H., Squicciarini, A., Zhu, S., & Miller, D. (2018) · arXiv · doi:10.48550/arxiv.1808.10307 · Cited by 110 · see citing works
Turner, A., Tsipras, D., & Mądry, A. (2018) · Cited by 104 · see citing works
Liu, Y., Xie, Y., & Srivastava, A. (2017) · IEEE International Conference on Computer Design: VLSI in Computers and Processors (ICCD) · Cited by 81 · see citing works
Gotta Catch'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networks
Shan, S., Wenger, E., Wang, B., Li, B., Zheng, H.-T., & Zhao, B. Y. (2020) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3372297.3417231 · Cited by 75 · see citing works
On the Practicality of Integrity Attacks on Document-Level Sentiment Analysis
Newell, A., Potharaju, R., Xiang, L., & Nita-Rotaru, C. (2014) · doi:10.1145/2666652.2666661 · Cited by 46 · see citing works
Poisoning Attacks with Generative Adversarial NetsPreprint
Muñoz-González, L., Pfitzner, B., Russo, M., Carnerero-Cano, J., & Lupu, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07773 · Cited by 46 · see citing works
Analysis of Causative Attacks against SVMs Learning from Data Streams
Burkard, C., & Lagesse, B. (2017) · doi:10.1145/3041008.3041012 · Cited by 42 · see citing works
Gotta Catch 'Em All: Using Concealed Trapdoors to Detect Adversarial Attacks on Neural Networks.Preprint
Shan, S., Willson, E., Wang, B., et al. (2019) · arXiv (Cornell University) · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Gao, Y.; Xu, C.; Wang, D.; Chen, S.; Ranasinghe, D. C.; and Nepal, S. 2019. Strip: A defence against trojan attacks on deep neural networks. arXiv:1902.06531 . · arXiv
- Koh, P. W., and Liang, P. 2017. Understanding blackbox predictions via influence functions. In Proceedings of the 34th International Conference on Machine LearningVolume 70 , 1885-1894. JMLR. org.
- Rezaei, S., and Liu, X. 2019. A target-agnostic attack on deep models: Exploiting security vulnerabilities of transfer learning. arXiv:1904.04334 . · arXiv
- Steinhardt, J.; Koh, P. W. W.; and Liang, P. S. 2017. Certified defenses for data poisoning attacks. In Advances in neural information processing systems , 3517-3529.
Cited by
Sorted by citations
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682 · see citing works
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3162397 · Cited by 362 · see citing works
Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
Doan, B. G., Abbasnejad, E., & Ranasinghe, D. C. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427264 · Cited by 263 · see citing works
LIRA: Learnable, Imperceptible and Robust Backdoor Attacks
Doan, K., Lao, Y., Zhao, W., & Li, P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01175 · Cited by 211 · see citing works
Rethinking the Backdoor Attacks’ Triggers: A Frequency Perspective
Zeng, Y., Park, W., Mao, Z. M., & Jia, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01616 · Cited by 193 · see citing works
Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
Cinà, A. E., Grosse, K., Demontis, A., Vascon, S., Zellinger, W., Moser, B. A., Oprea, A., Biggio, B., Pelillo, M., & Roli, F. (2023) · ACM Computing Surveys · doi:10.1145/3585385 · Cited by 152 · see citing works
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
Poisoning Web-Scale Training Datasets is Practical
Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F. (2024) · 2024 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp54263.2024.00179 · Cited by 138 · see citing works
Poisoning Web-Scale Training Datasets is Practical
Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F. (2023) · arXiv · doi:10.48550/arxiv.2302.10149 · Cited by 138 · see citing works
Concealed Data Poisoning Attacks on NLP Models
Wallace, E., Zhao, T., Feng, S., & Singh, S. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.13 · Cited by 115 · see citing works
An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences
Guo, W., Tondi, B., & Barni, M. (2022) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2022.3190213 · Cited by 91 · see citing works
Black-box Detection of Backdoor Attacks with Limited Information and Data
Dong, Y., Yang, X., Deng, Z., Pang, T., Xiao, Z., Su, H., & Zhu, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01617 · Cited by 92 · see citing works
Lin, Y.-S., Lee, W.-C., & Celik, Z. B. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467213 · Cited by 64 · see citing works
Hard to Forget: Poisoning Attacks on Certified Machine Unlearning
Marchant, N. G., Rubinstein, B. I. P., & Alfeld, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20736 · Cited by 64 · see citing works
“…2017; Koh and Liang, 2017; Shafahi et al., 2018; C. Zhu et al., 2019; W. R. Huang et al., 2020), and backdoor attacks , which aim to cause misclassification of test-time samples that contain a trigger (e.g. X. Chen et al., 2017; Dai, C. Chen, and Li, 2019; Saha, Subramanya, and Pirsiavash, 2020).”
Athena: Probabilistic Verification of Machine Unlearning
Sommer, D. M., Song, L., Wagh, S., & Mittal, P. (2022) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2022-0072 · Cited by 51 · see citing works
Dispersed Pixel Perturbation-Based Imperceptible Backdoor Trigger for Image Classifier Models
Wang, Y., Zhao, M., Li, S., Yuan, X., & Ni, W. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3202687 · Cited by 26 · see citing works
Test-Time Detection of Backdoor Triggers for Poisoned Deep Neural Networks
Li, X., Xiang, Z., Miller, D. J., & Kesidis, G. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746573 · Cited by 11 · see citing works
Neural networks memorise personal information from one sample
Hartley, J., Sanchez, P. P., Haider, F., & Tsaftaris, S. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-48034-3 · Cited by 10 · see citing works
Chen, C., Wei, L., Zhang, L., Peng, Y., & Ning, J. (2022) · Security and Communication Networks · doi:10.1155/2022/2985308 · Cited by 8 · see citing works
A lightweight backdoor defense framework based on image inpainting
Wei, Y., Gao, H., Wang, Y., Gao, Y., & Liu, H. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.03.052 · Cited by 6 · see citing works
Ye, J., Mao, Y., Song, J., Wang, X., Jin, C., & Song, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20219 · Cited by 1 · see citing works
- Authors
- Saha, A., Subramanya, A., & Pirsiavash, H.
- Affiliations
- University of Maryland, Baltimore County (US)
- ORCID
- Pirsiavash, Hamed
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2020-04-03
- Volume
- 34
- Issue
- 07
- Pages
- 11957-11965
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2978514398
- Cited by
- 13
- References
- 35
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 1 · 2022: 2 · 2021: 2 · 2020: 6
- FWCI
- 1.05
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Maryland Shared Open Access Repository (USMAI Consortium) (submittedVersion)
- arXiv (Cornell University)
- Digital Repository at the University of Maryland (University of Maryland College Park)
- Funders
- National Science Foundation (1845216)
- U.S. Department of Commerce
- National Institute of Standards and Technology (60NANB18D279)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- backdoor attacks, adversarial attacks, data poisoning, image classification, deep neural network security, backdoor defense evasion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
