The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks
Open accessAbstract
Deep Neural Networks (DNNs) are vulnerable to Neural Trojan (NT) attacks where the adversary injects malicious behaviors during DNN training. This type of ‘backdoor’ attack is activated when the input is stamped with the trigger pattern specified by the attacker, resulting in an incorrect prediction of the model. Due to the wide application of DNNs in various critical fields, it is indispensable to inspect whether the pre-trained DNN has been trojaned before employing a model. Our goal in this paper is to address the security concern on unknown DNN to NT attacks and ensure safe model deployment. We propose DeepInspect, the first black-box Trojan detection solution with minimal prior knowledge of the model. DeepInspect learns the probability distribution of potential triggers from the queried model using a conditional generative model, thus retrieves the footprint of backdoor insertion. In addition to NT detection, we show that DeepInspect’s trigger generator enables effective Trojan mitigation by model patching. We corroborate the effectiveness, efficiency, and scalability of DeepInspect against the state-of-the-art NT attacks across various benchmarks. Extensive experiments show that DeepInspect offers superior detection performance and lower runtime overhead than the prior work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, DP may hurt the accuracy of the learned model [170], which may not be appealing to the industry.”
From Privacy and Robustness in Federated Learning: Attacks and Defenses (2024) · cites this paper as [170]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
On Assessing ML Model Robustness: A Methodological Framework (Academic Track)
Awadid, A., & Robert, B. (2025) · OASIcs : OpenAccess Series in Informatics · doi:10.4230/oasics.saia.2024.1 · Cited by 4,723
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H.-T., & Zhao, B. Y. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00031 · Cited by 1,577
Poisoning Attacks against Support Vector Machines
Biggio, B., Nelson, B., & Laskov, P. (2012) · arXiv · doi:10.48550/arxiv.1206.6389 · Cited by 1,331
Cited by
Sorted by citations
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682
“[103] — 3,425 videos of 1,595 people [26], [21], [59], [66], [20] [84] PubFig [104] — 58,797 images of 200 people [59], [8], [20], [81] VGGFace [105] — 2.6 million images of 2,622 people [30], [56], [60], [59], [57] [17], [79], [63] VGGFace2 [106] — 3.3 million images of 9,131 people [18], [55] LFW [107] — 13,233 imag…”
InfluentialPrivacy and Robustness in Federated Learning: Attacks and Defenses
Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., Yang, Q., & Yu, P. S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3216981 · Cited by 548
“In this section, we outline research directions which we believe are promising.”
BackgroundSelf-citationSecurity and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445
“…in LMs, like, TROJANLM [98], one possible defense technique is to adopt existing techniques from other domains such as images, e.g., detecting trigger-embedded inputs at inference time [155], [168], and finding suspicious LMs and retrieving triggers during the model evaluation phase [169], [170].”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3162397 · Cited by 362
“structing a single trigger, the authors of [140], [150], and 1572 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.”
BackgroundMethodDetecting AI Trojans Using Meta Neural Analysis
Xu, X., Wang, Q., Li, H., Borisov, N., Gunter, C. A., & Li, B. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00034 · Cited by 306
Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
Doan, B. G., Abbasnejad, E., & Ranasinghe, D. C. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427264 · Cited by 263
“The first and obvious difference is that this method only detects potential Trojans, while our method cleans the inputs.”
MethodResultInfluentialNeural Attention Distillation: Erasing Backdoor Triggers from Deep Neural NetworksPreprint
Li, Y., Lyu, X., Koren, N., Lyu, L., Li, B., & Ma, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05930 · Cited by 264
“On the other hand, even though detection-based approaches have been performing fairly well on identifying backdoored models (Chen et al., 2019a; Tran et al., 2018; Chen et al., 2019b; Kolouri et al., 2020), the identified backdoored models still need to be purified by backdoor erasing techniques.”
BackgroundMethodInfluentialLIRA: Learnable, Imperceptible and Robust Backdoor Attacks
Doan, K., Lao, Y., Zhao, W., & Li, P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01175 · Cited by 211
“Several categories of defensive mechanisms have been developed in recent literature to counter backdoor attacks, including detection [8, 48, 18, 45], input mitigation [32, 27], model mitigation [28, 11, 50, 9, 38] approaches.”
BackgroundDeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
Qiu, H., Zeng, Y., Guo, S., Zhang, T., Qiu, M., & Thuraisingham, B. M. (2021) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3433210.3453108 · Cited by 205
“different techniques to recover the trigger, such as Generative Adversarial Networks [5], new regularization terms [15], Generative Distribution Modeling [41], and Artificial Brain Stimulation [33].”
MethodRethinking the Backdoor Attacks’ Triggers: A Frequency Perspective
Zeng, Y., Park, W., Mao, Z. M., & Jia, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01616 · Cited by 193
“Other popular techniques include [29], [4], and [12], are based on reconstructing the trigger from model parameters and performs detection based on the reconstructed triggers.”
BackgroundA survey of Deep Neural Network watermarking techniques
Li, Y., Wang, H., & Barni, M. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.051 · Cited by 177
An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks
Tang, R., Du, M., Liu, N., Yang, F., & Hu, X. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403064 · Cited by 187
“For detector resources, we follow the assumptions used in [8, 16, 33]: (1) Detectors can white-box access to the DNN model.”
BackgroundMethodInfluentialBadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning
Jia, J., Liu, Y., & Gong, N. Z. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833644 · Cited by 192
Adversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 230
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
Naseri, M., Hayes, J., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23054 · Cited by 188
“In fact, even defenses that do not require training data, e.g., DeepInspect [14], require inverting the model in order to extract the training data, thus violating the privacy-preserving goal of FL. Similarly, if a model is trained over encrypted data, e.g., using CryptoNet [34] or SecureML [75], the server cannot det…”
MethodWild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
Cinà, A. E., Grosse, K., Demontis, A., Vascon, S., Zellinger, W., Moser, B. A., Oprea, A., Biggio, B., Pelillo, M., & Roli, F. (2023) · ACM Computing Surveys · doi:10.1145/3585385 · Cited by 152
“Ba ck do or T T [74, 148, 166, 170, 181, 184] [21, 51, 60, 78, 85, 101, 159, 174, 196] [5, 28, 30, 70, 77, 93, 150, 154, 162, 180, 183, 192, 203] [30, 99, 106, 137, 178, 196, 198, 199, 207, 212] [35, 50, 70, 72, 77, 108, 137, 168, 171, 180, 184] [37, 47, 59, 103, 136, 143, 169– 171, 199]”
Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive ReviewPreprint
Gao, Y., Doan, B. G., Zhang, Z., Ma, S., Zhang, J., Fu, A., Nepal, ., & Kim, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10760 · Cited by 276
“Rare defense requires no access to training data [118].”
BackgroundMethodInfluentialDeep Feature Space Trojan Attack of Neural Networks by Controlled Detoxification
Cheng, S., Liu, Y., Ma, S., & Zhang, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16201 · Cited by 150
FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information
Cao, X., Jia, J., Zhang, Z., & Gong, N. Z. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179336 · Cited by 126
“[15], [32] have been proposed to detect and remove backdoor in neural networks.”
BackgroundAdversarial Neuron Pruning Purifies Backdoored Deep ModelsPreprint
Wu, D., & Wang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14430 · Cited by 125
“To repair the risk model, one direct way is to first reconstruct an approximation of the backdoor trigger via adversarial perturbation [43] or generative adversarial network (GAN) [5, 33, 50].”
BackgroundMethodPractical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases
Wang, R., Zhang, G., Liu, S., Chen, P., Xiong, J., & Wang, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58592-1_14 · Cited by 117
RAB: Provable Robustness Against Backdoor Attacks
Weber, M., Xu, X., Karlaš, B., Zhang, C., & Li, B. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179451 · Cited by 151
“[7] improves upon the approach by using model inversion to obtain training data, and then apply GANs to generate the “short path” and apply anomaly detection algorithm as in Neural Cleanse.”
MethodWang, Z., Zhai, J., & Ma, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01465 · Cited by 124
“Neural Cleanse [66], DeepInspect [7], K-arm [57] and ABS [40] use reconstructed triggers to perform detection.”
BackgroundMethodAn Invisible Black-Box Backdoor Attack Through Frequency Domain
Wang, T., Yao, Y., Xu, F., An, S., Tong, H., & Wang, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_23 · Cited by 139
“DeepInspect [10] is similar to Neural Cleanse except that it does not require the access to training data and the model parameters.”
BackgroundMethodRethinking the Trigger of Backdoor AttackPreprint
Li, Y., Zhai, T., Wu, B., Jiang, Y., Li, Z., & Xia, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04692 · Cited by 187
“, 2017b) (which conducts data preprocessing before prediction), triggerreconstruction based defense (Wang et al., 2019; Chen et al., 2019; Qiao et al., 2019) (which reconstructs the triggers and then removes the backdoor to ’patch’ the infected model), and model-reconstruction based defense (Liu et al.”
BackgroundInfluentialTrojaning Language Models for Fun and Profit
Zhang, X., Zhang, Z., Ji, S., & Wang, T. (2021) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00022 · Cited by 117
Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning
Li, L., Song, D., Li, X., Zeng, J., Ma, R., & Qiu, X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.241 · Cited by 145
“To defend against these injected backdoors, Chen et al. (2019); Li et al. (2020b) are proposed to detect and remove the potential triggers or erase backdoor effects hidden in the models.”
BackgroundDesign and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks
Gao, Y., Kim, Y., Doan, B. G., Zhang, Z., Zhang, G., Nepal, ., Ranasinghe, D. C., & Kim, H. (2021) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3055844 · Cited by 102
Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models
Shan, S., Ding, W., Passananti, J., Wu, S. L., Zheng, H.-T., & Zhao, B. Y. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00207 · Cited by 166
“Some [46, 47, 48, 49, 50, 51] focus on detecting poison data by leveraging their unique behavior while others [52, 53, 54] advocate for robust training to mitigate the influence of poison data during training time.”
BackgroundYang, W., Li, L., Zhang, Z., Ren, X., Sun, X., & He, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.165 · Cited by 137
Backdoor Defense with Machine Unlearning
Liu, Y., Fan, M., Chen, C., Liu, X., Ma, Z., Wang, L., & Ma, J. (2022) · IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · doi:10.1109/infocom48880.2022.9796974 · Cited by 95
An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences
Guo, W., Tondi, B., & Barni, M. (2022) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2022.3190213 · Cited by 91
Black-box Detection of Backdoor Attacks with Limited Information and Data
Dong, Y., Yang, X., Deng, Z., Pang, T., Xiao, Z., Su, H., & Zhu, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01617 · Cited by 92
“A recent work [8] also claimed to perform “black-box” backdoor detection.”
BackgroundMethodInfluentialA Tale of Evil Twins: Adversarial Inputs versus Poisoned Models
Pang, R., Shen, H., Zhang, X., Ji, S., Vorobeychik, Y., Luo, X., Liu, A., & Wang, T. (2020) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3372297.3417253 · Cited by 87
“The existing defense methods against poisoned models mostly focus on the backdoor attacks, which, according to their strategies, can be categorized as: (i) cleansing potential contaminated data at the training stage [50], (ii) identifying suspicious models during model inspection [9, 31, 51], and (iii) detecting trigg…”
BackgroundZhao, Z., Chen, X., Xuan, Y., Dong, Y., Wang, D., & Liang, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01478 · Cited by 88
Rethinking Stealthiness of Backdoor Attack against NLP Models
Yang, W., Lin, Y., Li, P., Zhou, J., & Sun, X. (2021) · doi:10.18653/v1/2021.acl-long.431 · Cited by 89
“Several effective defense methods are introduced in CV (Huang et al., 2019; Wang et al., 2019; Chen et al., 2019; Gao et al., 2019), but there are only limited researches focusing on defending backdoor attacks against NLP models (Chen and Dai, 2020; Qi et al., 2020; Azizi et al., 2021).”
BackgroundTestability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives
Su, F., Liu, C., & Stratigopoulos, H. (2023) · IEEE Design and Test · doi:10.1109/mdat.2023.3241116 · Cited by 84
“Other security threats that will not be covered in this survey include DNN model IP theft [192], [193], [194], backdoor attacks on DNNs performed when training is outsourced [195], [196], and fault injection attacks [197], [198].”
BackgroundTrojDRL: Evaluation of Backdoor Attacks on Deep Reinforcement Learning
Kiourti, P., Wardega, K., Jha, S., & Li, W. (2020) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218663 · Cited by 79
DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection
Li, Y., Hua, J., Wang, H., Chen, C., & Liu, Y. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00035 · Cited by 75
“DeepInspect [36] addresses black-box backdoor detection by recovering a substitution training dataset through using model inversion and reconstructing triggers using a conditional Generative Adversarial Network (cGAN).”
BackgroundMethodA Systematic Review of Adversarial Machine Learning Attacks, Defensive Controls and Technologies
Malik, J., Muthalagu, R., & Pawar, P. M. (2024) · IEEE Access · doi:10.1109/access.2024.3423323 · Cited by 76
“DeepInspect [92] learns the probability distribution of trigger patterns using a conditional generative model, then applies model patching to eliminate the trigger.”
MethodLiu, Y., Mondal, A., Chakraborty, A., Zuzak, M., Jacobsen, N., Xing, D., & Srivastava, A. (2020) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed48828.2020.9137011 · Cited by 72
“A wide array of techniques to do this have been proposed [3, 7, 10, 16, 35].”
BackgroundInfluentialProFlip: Targeted Trojan Attack with Progressive Bit Flips
Chen, H., Fu, C., Zhao, J., & Koushanfar, F. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00762 · Cited by 76
“This type of trigger is effective in practice and has been widely used in previous Trojan attacks [4, 15, 9, 39].”
MethodSelf-citationTowards Inspecting and Eliminating Trojan Backdoors in Deep Neural Networks
Guo, W., Wang, L., Xu, Y. H., Xing, X., Du, M., & Song, D. X. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00025 · Cited by 68
“[4] further borrow the idea of Generative Adversarial Network [11] to generate the trigger.”
BackgroundMethodInfluentialAdversarial Unlearning of Backdoors via Implicit Hypergradient
Zeng, Y., Chen, S., Park, W., Mao, Z. M., Jin, M., & Jia, R. (2021) · VTechWorks (Virginia Tech) · doi:10.48550/arxiv.2110.03735 · Cited by 116
“We compare I-BAU with six state-of-art defenses: Neural Cleanse (NC) (Wang et al., 2019), Deepinspect (DI) (Chen et al., 2019), TABOR (Guo et al., 2019), Fine-pruning (FP) (Liu et al., 2018a), Neural Attention Distillation (NAD) (Li et al., 2020b), and Differential Privacy training (DP) (Du et al.,…”
BackgroundMethodInfluentialRAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models
Yang, W., Lin, Y., Li, P., Zhou, J., & Sun, X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.659 · Cited by 80
“…detect poisoned samples or pre-process inputs to avoid the activation of the backdoor in the inference time; (2) Offline defenses (Liu et al., 2018a; Chen et al., 2019; Wang et al., 2019; Li et al., 2021) which choose to remove or mitigate the backdoor effect in the model before models are…”
BackgroundInfluentialNeuronInspect: Detecting Backdoors in Neural Networks via Output ExplanationsPreprint
Huang, X., Alzantot, M., & Srivastava, M. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07399 · Cited by 100
On Certifying Robustness against Backdoor Attacks via Randomized SmoothingPreprint
Wang, B., Cao, X., jia, J., & Gong, N. Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11750 · Cited by 101
“Indeed, various empirical defenses [4, 7, 9, 5, 23, 26, 20] have been proposed to defend against backdoor attacks in the past few years.”
BackgroundADFL: Defending backdoor attacks in federated learning via adversarial distillation
Zhu, C., Zhang, J., Sun, X., Chen, B., & Meng, W. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103366 · Cited by 61
Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks
Qi, X., Xie, T., Pan, R., Zhu, J., Yang, Y., & Bu, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01299 · Cited by 64
“To this end, data poisoning based backdoor attacks [12,21,53,72] on deep neural networks (DNNs) in the production stage (or training stage) and corresponding defenses [11, 13, 74] are extensively explored in recent years.”
BackgroundPiccolo: Exposing Complex Backdoors in NLP Transformer Models
Liu, Y., Shen, G., Tao, G., An, S., Ma, S., & Zhang, X. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833579 · Cited by 65
“Other state-of-the-art scanners such as NC [10], ABS [11], DeepInspect [12], K-Arm backdoor scanning [48], ULP [15] are for computer vision and hence not applicable.”
BackgroundMethodNaseri, M., Hayes, J., & Cristofaro, E. D. (2020) · arXiv (Cornell University) · Cited by 54
Athena: Probabilistic Verification of Machine Unlearning
Sommer, D. M., Song, L., Wagh, S., & Mittal, P. (2022) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2022-0072 · Cited by 51
“One category is to directly update model parameters by fine-pruning inner neurons with a subset of benign samples [36], fine-tuning with knowledge distillation [20], or fine-tuning with reconstructed backdoor patterns [19, 37].”
BackgroundWang, H., Xiang, Z. J., Miller, D. J., & Kesidis, G. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00015 · Cited by 62
“Also, most of these reverse-engineering-based methods, except [79], [80], require some clean samples, which are not always available as we have discussed in Sec.”
MethodT-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text ClassificationPreprint
Azizi, A., Tahmid, I. A., Waheed, A., Mangaokar, N., Pu, J., Javed, M., Reddy, C. K. A., & Viswanath, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.04264 · Cited by 58
Imperceptible Backdoor Attack: From Input Space to Feature Representation
Zhong, N., Qian, Z., & Zhang, X. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/242 · Cited by 49
Defense-Resistant Backdoor Attacks Against Deep Neural Networks in Outsourced Cloud Environment
Gong, X., Chen, Y., Wang, Q., Huang, H., Meng, L., Shen, C. Q., & Zhang, Q. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087237 · Cited by 50
“DeepInspect [36] requires no access to the training dataset since its first step is to reverse training data.”
BackgroundYoshida, K., & Fujino, T. (2020) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3411508.3421375 · Cited by 49
“Some have used approaches such as detecting backdoor models [2, 4, 10], removing or disabling backdoors from backdoor models [2, 8, 12], and removing poison data from poison training datasets [1].”
BackgroundMethodBackdoor Defense via Deconfounded Representation Learning
Zhang, Z., Liu, Q., Wang, Z., Lu, Z., & Hu, Q. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01177 · Cited by 49
“Based on the target and the working stage, existing defenses can be divided into the following categories: (1) Detection based defenses aim to detect anomalies in input data [7, 12, 15, 21, 58, 62, 67] or whether a given model is backdoored [8, 27, 56, 65].”
BackgroundTransferable Graph Backdoor Attack
Yang, S., Doan, B. G., Montague, P., Vel, O. D., Abraham, T., Camtepe, S. A., Ranasinghe, D. C., & Kanhere, S. S. (2022) · International Symposium on Research in Attacks, Intrusions and Defenses (RAID) · doi:10.1145/3545948.3545976 · Cited by 48
“The existing defenses either perform inspections on the suspicious models [4, 24, 41] or detect the possibly poisoned inputs at inference time Table 9: Training with subsampling.”
BackgroundTrojanPuzzle: Covertly Poisoning Code-Suggestion Models
Aghakhani, H., Dai, W., Manoel, A., Fernandes, X., Kharkar, A., Kruegel, C., Vigna, G., Evans, D., Zorn, B., & Sim, R. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00140 · Cited by 72
“Related work also proposed defenses [32], [58], [5], [57], [11] that operate at the post-training state and aim to detect whether a model is poisoned (backdoored) or not.”
BackgroundTriggerless Backdoor Attack for NLP Tasks with Clean Labels
Gan, L., Li, J., Zhang, T., Li, X., Meng, Y., Wu, F. Q., Yang, Y., Guo, S., & Fan, C. C. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.214 · Cited by 51
Backdoor Attacks and Defenses Targeting Multi-Domain AI Models: A Comprehensive Review
Zhang, S., Pan, Y., Liu, Q., Yan, Z., Choo, K. R., & Wang, G. (2024) · ACM Computing Surveys · doi:10.1145/3704725 · Cited by 51
“If the defender’s access to knowledge is further restricted, then they can use model inversion techniques to generate a substitute training dataset [17] and employ a Conditional GAN to reverse-engineer the triggers.”
Backdoors in Neural Models of Source Code
Ramakrishnan, G., & Albarghouthi, A. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956690 · Cited by 61
“com/goutham7r/backdoors-for-code images [3, 5, 16, 17, 18, 19], with proposed defenses including neuron-pruning [20, 21], trigger detection [22] and trigger reconstruction [20, 23].”
BackgroundFew-shot Backdoor Defense Using Shapley Estimation
Guan, J., Tu, Z., He, R., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01300 · Cited by 42
“To improve Neural Cleanse, GangSweep [48], Tabor [17], and DeepInspect [3] were proposed to use GANs [14] and interpretable AI to generate better-reversed triggers.”
MethodInfluentialSecurity for Machine Learning-based Software Systems: A Survey of Threats, Practices, and Challenges
Chen, H., & Babar, M. A. (2023) · ACM Computing Surveys · doi:10.1145/3638531 · Cited by 66
“In following sections, we discuss the security threats of MLBSS from the system aspects of confidentiality, integrity and availability.”
BackgroundSelf-citationText Backdoor Detection Using an Interpretable RNN Abstract Model
Fan, M., Si, Z., Xie, X., Liu, Y., & Liu, T. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3103064 · Cited by 38
“Chen et al. [48] proposed DeepInspect, the first black-box backdoor detection solution with minimal prior model knowledge.”
You Are Catching My Attention: Are Vision Transformers Bad Learners under Backdoor Attacks?
Yuan, Z., Zhou, P., Zou, K., & Cheng, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02357 · Cited by 41
Complex Backdoor Detection by Symmetric Feature Differencing
Liu, Y., Shen, G., Tao, G., Wang, Z., Ma, S., & Zhang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01458 · Cited by 42
“R AY with four other state-of-the-art (SOTA) defenses, namely, Meta-Neural Analysis [73], DeepInspect [11], NeuronInspect [24], and TABOR [21], on reflection and composite attacks.”
BackgroundInfluentialDefense against neural trojan attacks: A survey
Kaviani, S., & Sohn, I. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.133 · Cited by 39
Defending against Backdoor Attacks in Natural Language Generation
Sun, X., Li, X., Meng, Y., Ao, X., Lyu, L., Li, J., & Zhang, T. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25656 · Cited by 48
Li, M., Zhong, Q., Zhang, L. Y., Du, Y., Zhang, J., & Xiang, Y. (2020) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom50675.2020.00062 · Cited by 36
“Similarly, if the feature pattern embedded in the trigger is visible [7], [12], then the pattern can be easily detected [25], and even removed [23], [24].”
BackgroundAli, H., Chen, D., Harrington, M., Salazar, N., Ameedi, M. A., Khan, A. F., Butt, A. R., & Cho, J.-H. (2023) · IEEE Access · doi:10.1109/access.2023.3326410 · Cited by 41
“• DeepInspect (DI) [72]: DI mainly detects backdoor attacks based on three stages: model inversion, trigger generation, and anomaly detection.”
BackgroundBackdoor Attack and Defense on Deep Learning: A Survey
Bai, Y., Xing, G., Wu, H., Rao, Z., Ma, C., Wang, S., Liu, X., Zhou, Y., Tang, J., Huang, K., et al. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3482723 · Cited by 42
“Therefore, Chen et al. [190] proposed an approach that requires less prior knowledge of the model and does not rely on the original clean dataset.”
AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value AnalysisPreprint
Guo, J., Li, A., & Liu, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14880 · Cited by 51
“In this setting, the typical optimization objective of backdoor detection (Chen et al., 2019b; Liu et al., 2019) becomes impossible to solve due to the limited information.”
BackgroundMethodInfluentialMengara, O., Avila, A. R., & Falk, T. H. (2024) · IEEE Access · doi:10.1109/access.2024.3355816 · Cited by 35
“Representative examples include [167], [168], [169]: For trigger synthesis, defenses first produce a trigger for the backdoor and then eliminate its effects.”
BackgroundEnhancing Backdoor Attacks With Multi-Level MMD Regularization
Xia, P., Niu, H., Li, Z., & Li, B. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3161477 · Cited by 32
“Several defense approaches have been proposed to defend against these attacks, such as backdoor detection [13], [50], trigger synthesis [32], [51], model reconstruction [42], [52], and model diagnosis [53], [54].”
BackgroundQuantization Backdoors to Deep Learning Commercial Frameworks
Ma, H., Qiu, H., Gao, Y., Zhang, Z., Abuadbba, A., Xue, M., Fu, A., Zhang, J., Al-Sarawi, S. F., & Abbott, D. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3271956 · Cited by 40
“Great efforts have been made to mitigate backdoor attacks recently [11], [12], [13], [14], [34], [58].”
BackgroundMethodInfluentialJigsaw Puzzle: Selective Backdoor Attack to Subvert Malware Classifiers
Li, Y., Chen, Z., Cortellazzi, J., Pendlebury, F., Tu, K., Pierazzi, F., Cavallaro, L., & Wang, G. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179347 · Cited by 42
“For model inspection, existing methods are designed to synthesize or search for trigger patterns that allow any samples from all different classes to be universally classified to the target label [11, 15, 68].”
MethodFederatedReverse: A Detection and Defense Method Against Backdoor Attacks in Federated Learning
Zhao, C., Wen, Y., Li, S., Liu, F., & Meng, D. (2021) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3437880.3460403 · Cited by 29
“However, common anomaly detection algorithms require to access the local data of all parties [9, 13].”
BackgroundMethodStealthy Backdoors as Compression Artifacts
Tian, Y., Suya, F., Xu, F., & Evans, D. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3160359 · Cited by 31
Redeem Myself: Purifying Backdoors in Deep Learning Models using Self Attention Distillation
Gong, X., Chen, Y., Wang, Y., Wang, Q., Gu, Y., Huang, H., & Shen, C. Q. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179375 · Cited by 31
“STRIP [14] data ■ ■ # # # # # Sentinet [9] data ■ ■ # # # # # SP [57] data ■ ■ ? # # # # AC [5] data ■ ■ # ? ? ? ? SCAn [56] data ■ ■ ■ ? ? ? ? NC [59] data ■ ■ # # # # # ABS [39] model ■ ■ # ■ ? ? ? MNTD [65] model ■ ■ ■ ? ? ? # NeuronInspect [23] model ■ ? # ? ? # ? DeepInspect [6] model ■ ■ # ? ? ? ?”
MethodInfluentialBackdoor Cleansing with Unlabeled Data
Pang, L., Sun, T., Ling, H., & Chen, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01176 · Cited by 33
“Most methods in this category first synthesize possible triggers and then utilize synthesized triggers to mitigate backdoor effects [3, 4, 10, 35, 46].”
MethodXiang, Z. J., Miller, D. J., & Kesidis, G. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102280 · Cited by 27
Zhou, Y., Ni, T., Lee, W., & Zhao, Q. (2025) · Transactions on Artificial Intelligence · doi:10.53941/tai.2025.100003 · Cited by 66
“While NC [113] relies on a clean training dataset which limits its application scenarios, DeepInspect (DI ) [185], another black-box Trojan detection framework through model inspection, requires minimal prior knowledge about the backdoored model, it first employs model inversion to obtain a…”
Detection of Lying Electrical Vehicles in Charging Coordination Using Deep Learning
Shafee, A. A., Fouda, M. M., Mahmoud, M. M. E. A., Aljohani, A. J., Alasmary, W., & Amsaad, F. (2020) · IEEE Access · doi:10.1109/access.2020.3028097 · Cited by 25
“For comparison with the GRU detector, another DNN detector based on a multilayer perceptron (MLP) model [42] was also trained and tested to evaluate how the GRU model benefits from the ability to exploit the time-series nature of the data.”
MethodFragile Neural Network Watermarking with Trigger Image Set
Zhu, R., Ping, W., Li, S., Yin, Z., Zhang, X., & Qian, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-82136-4_23 · Cited by 25
A Case Study of Integrating Fairness Visualization Tools in Machine Learning Education
Mashhadi, A., Zolyomi, A., & Quedado, J. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3503568 · Cited by 25
“DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks..”
MethodStealthy and Flexible Trojan in Deep Learning Framework
Wang, Y., Chen, K., Tan, Y., Huang, S., Ma, W., & Li, Y. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3164073 · Cited by 27
“DeepInspect [41] uses model inversion and conditional Generative Adversarial Network (cGAN) to reconstruct the substitution training dataset and realizes model backdoor detection under black box conditions.”
MethodA defense method against backdoor attacks on neural networks
Kaviani, S., Shamshiri, S., & Sohn, I. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118990 · Cited by 25
Robust federated learning with attack-adaptive aggregation
Wan, C. P. (2022) · doi:10.14711/thesis-991012994506603412 · Cited by 24
TrojanZoo: Towards Unified, Holistic, and Practical Evaluation of Neural Backdoors
Pang, R., Zhang, Z., Gao, X., Xi, Z., Ji, S., Cheng, P., Luo, X., & Wang, T. (2022) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp53844.2022.00048 · Cited by 36
“– DeepInspect (DI) [11] follows a similar pipeline but uses a generative network to generate trigger candidates.”
BackgroundInfluentialA Comprehensive Survey on Backdoor Attacks and Their Defenses in Face Recognition Systems
Roux, Q. L., Bourbao, E., Teglia, Y., & Kallas, K. (2024) · IEEE Access · doi:10.1109/access.2024.3382584 · Cited by 30
“…specifications methods (how the defense acts against backdoor attacks), alongside the respective number of identified papers associated with each subcategory (note: some papers cover more than one case). datapoints directly from the scrutinized DNN via model inversion [130] or distillation [131].”
BackgroundMethodInfluentialAI-Lancet: Locating Error-inducing Neurons to Optimize Neural Networks
Zhao, Y., Zhu, H. G., Chen, K., & Zhang, S. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484818 · Cited by 25
“Recently, both backdoor detection [8, 30, 36, 58] and backdoor removal [35, 58] approaches have been proposed.”
BackgroundVerification of integrity of deployed deep learning models using Bayesian Optimization
Kuttichira, D. P., Gupta, S., Nguyen, D., Rana, S., & Venkatesh, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108238 · Cited by 23
NTD: Non-Transferability Enabled Deep Learning Backdoor Detection
Li, Y., Ma, H., Zhang, Z., Gao, Y., Abuadbba, A., Xue, M., Fu, A., Zheng, Y., Al-Sarawi, S. F., & Abbott, D. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3312973 · Cited by 27
“Considering the severe consequences of backdoor attacks, great efforts have been made in detecting or eliminating backdoors both from either offline inspection [6], [7], [12], [18], [19] or online inspection [8], [11], [20].”
BackgroundMethodInfluentialResisting Distributed Backdoor Attacks in Federated Learning: A Dynamic Norm Clipping Approach
Guo, Y., Wang, Q., Ji, T., Wang, X., & Li, P. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671910 · Cited by 22
Backdoor Attacks on Time Series: A Generative Approach
Jiang, Y., Ma, X., Erfani, S., & Bailey, J. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00034 · Cited by 24
“Following [31], improved detection techniques were introduced in [32], [33].”
MethodEvaluating the Robustness of Trigger Set-Based Watermarks Embedded in Deep Neural Networks
Lee, S., Song, W., Jana, S., Cha, M., & Son, S. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3196790 · Cited by 21
“In this regard, we suggest future research conduct at least a complete set of state-of-the-art attacks at the time of suggesting a new approach.”
MethodTutorial: Toward Robust Deep Learning against Poisoning Attacks
Chen, H., & Koushanfar, F. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3574159 · Cited by 24
“This paper [9] proposes the first black-box backdoor detection and mitigation technique.”
BackgroundMethodSelf-citation
Metadata
- Authors
- Chen, H., Fu, C., Zhao, J., & Koushanfar, F.
- Affiliations
- University of California San Diego (US)
- Venue
- Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
- Published
- 2019-08
- Pages
- 4658-4664
- OpenAlex
- W2966689772
- Cited by
- 314
- References
- 17
- Citations by year
- 2026: 15 · 2025: 43 · 2024: 51 · 2023: 55 · 2022: 49 · 2021: 69 · 2020: 30 · 2019: 1
- FWCI
- 14.25
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Physical Unclonable Functions (PUFs) and Hardware Security
- Keywords
- backdoor attacks, deep neural network security, conditional generative models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
