An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
With the widespread use of deep neural networks (DNNs) in high-stake applications, the security problem of the DNN models has received extensive attention. In this paper, we investigate a specific security problem called trojan attack, which aims to attack deployed DNN systems relying on the hidden trigger patterns inserted by malicious hackers. We propose a training-free attack approach which is different from previous work, in which trojaned behaviors are injected by retraining model on a poisoned dataset. Specifically, we do not change parameters in the original model but insert a tiny trojan module (TrojanNet) into the target model. The infected model with a malicious trojan can misclassify inputs into a target label when the inputs are stamped with the special trigger. The proposed TrojanNet has several nice properties including (1) it activates by tiny trigger patterns and keeps silent for other signals, (2) it is model-agnostic and could be injected into most DNNs, dramatically expanding its attack scenarios, and (3) the training-free mechanism saves massive training efforts comparing to conventional trojan attack methods. The experimental results show that TrojanNet can inject the trojan into all labels simultaneously (all-label trojan attack) and achieves 100% attack success rate without affecting model accuracy on original tasks. Experimental analysis further demonstrates that state-of-the-art trojan detection algorithms fail to detect TrojanNet attack. The code is available at https://github.com/trx14/TrojanNet.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“iggers. 1 INTRODUCTION DNNs have achieved state-of-the-art performance in a variety of applications, such as healthcare [26], autonomous driving [7], security supervisor [28] , and speech recognition [13]. There are already many emerging markets [1, 2] to trade pre-trained DNN models. Recently, considerable attention…”
BackgroundAdversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“1) Inspired by the curriculum learning[5], which gradually increases the complexity of inputs to benefit model training.”
BackgroundLeys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.03.013 · Cited by 4,147
Deep learning for healthcare: review, opportunities and challenges
Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbx044 · Cited by 3,076 · cited 1×
“DNNs have achieved state-of-the-art performance in a variety of applications, such as healthcare [26], autonomous driving [7], security supervisor [28] , and speech recognition [13].”
BackgroundTrojaning Attack on Neural Networks
Liu, Y., Ma, S., Aafer, Y., Lee, W.-C., Zhai, J., Wang, W., & Zhang, X. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23291 · Cited by 1,270 · cited 5×
“mples. Otherwise, it can cause a false-positive attack, and thus exposes our hidden trojans. Secondly, trojan related neurons should not influence original function of the target model. Previous work [25] points out that muting trojan related neurons can dramatically harm model performance on the original task, which…”
BackgroundMethodInfluentialTechniques for Interpretable Machine Learning
Du, M., Liu, N., & Hu, X. (2018) · arXiv · doi:10.48550/arxiv.1808.00033 · Cited by 1,261
Self-citationFine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
Liu, K., Dolan-Gavitt, B., & Garg, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00470-5_13 · Cited by 1,055
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1712.05526 · Cited by 1,039 · cited 3×
“One direction is to make the trojan triggersmore imperceptible to humans [9, 22, 24].”
BackgroundMethodDetecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
Chen, B., Carvalho, W., Baracaldo, N., Ludwig, H., Edwards, B., Lee, T., Molloy, I., & Srivastava, B. (2018) · arXiv · doi:10.48550/arxiv.1811.03728 · Cited by 485
DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks
Chen, H., Fu, C., Zhao, J., & Koushanfar, F. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/647 · Cited by 314 · cited 6×
“For detector resources, we follow the assumptions used in [8, 16, 33]: (1) Detectors can white-box access to the DNN model.”
BackgroundMethodInfluentialBackdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
Liao, C., Zhong, H., Squicciarini, A., Zhu, S., & Miller, D. (2018) · arXiv · doi:10.48550/arxiv.1808.10307 · Cited by 110
Cited by
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3162397 · Cited by 362
“[107] propose an approach that adds a small module to the existing model instead of modifying existing model weights.”
BackgroundDynamic Backdoor Attacks Against Machine Learning Models
Salem, A., Wen, R., Backes, M., Ma, S., & Zhang, Y. (2022) · 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp53844.2022.00049 · Cited by 224
“[50] present a different approach, where the adversary appends a small Trojan module (TrojanNet) to the target model instead of fully retraining it.”
BackgroundLin, Y.-S., Lee, W.-C., & Celik, Z. B. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467213 · Cited by 64
“The last approach integrates a trojan module into the target model, which combines the output of two networks for triggers that causes misclassification to different target labels [38].”
Background
Metadata
- Authors
- Tang, R., Du, M., Liu, N., Yang, F., & Hu, X.
- Venue
- Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2020-08-20
- Pages
- 218-228
- OpenAlex
- W3081178496
- Cited by
- 165
- References
- 13
- FWCI
- 7.76
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.