The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation
Open accessAbstract
This paper presents a technique to scan neural network based AI models to determine if they are trojaned. Pre-trained AI models may contain back-doors that are injected through training or by transforming inner neuron weights. These trojaned models operate normally when regular inputs are provided, and mis-classify to a specific output label when the input is stamped with some special pattern called trojan trigger. We develop a novel technique that analyzes inner neuron behaviors by determining how output activations change when we introduce different levels of stimulation to a neuron. The neurons that substantially elevate the activation of a particular output label regardless of the provided input is considered potentially compromised. Trojan trigger is then reverse-engineered through an optimization procedure using the stimulation analysis results, to confirm that a neuron is truly compromised. We evaluate our system ABS on 177 trojaned models that are trojaned with various attack methods that target both the input space and the feature space, and have various trojan trigger sizes and shapes, together with 144 benign models that are trained with different data and initial weight values. These models belong to 7 different model structures and 6 different datasets, including some complex ones such as ImageNet, VGG-Face and ResNet110. Our results show that ABS is highly effective, can achieve over 90% detection rate for most cases (and many 100%), when only one input sample is provided for each output label. It substantially out-performs the state-of-the-art technique Neural Cleanse that requires a lot of input samples and small trojan triggers to achieve good performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citationVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
Practical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv · doi:10.48550/arxiv.1802.00420 · Cited by 2,172
Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition
Stallkamp, J., Schlipsing, M., Salmen, J., & Igel, C. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.016 · Cited by 1,537
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H., & Zhao, B. Y. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00031 · Cited by 1,368
Trojaning Attack on Neural Networks
Liu, Y., Ma, S., Aafer, Y., Lee, W.-C., Zhai, J., Wang, W., & Zhang, X. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23291 · Cited by 1,270
Self-citationAge and gender classification using convolutional neural networks
Levi, G., & Hassncer, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301352 · Cited by 1,151
The German Traffic Sign Recognition Benchmark: A multi-class classification competition
Stallkamp, J., Schlipsing, M., Salmen, J., & Igel, C. (2011) · The 2011 International Joint Conference on Neural Networks · doi:10.1109/ijcnn.2011.6033395 · Cited by 1,144
GradientBased Learning Applied to Document Recognition
(2009) · Intelligent Signal Processing · doi:10.1109/9780470544976.ch9 · Cited by 1,135
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
Liu, K., Dolan-Gavitt, B., & Garg, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00470-5_13 · Cited by 1,055
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1712.05526 · Cited by 1,039
Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., & Li, B. (2018) · 2018 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2018.00057 · Cited by 861
Systematic Poisoning Attacks on and Defenses for Machine Learning in Healthcare
Mozaffari-Kermani, M., Sur-Kolay, S., Raghunathan, A., & Jha, N. K. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2014.2344095 · Cited by 383
Liu, Y., Xie, Y., & Srivastava, A. (2017) · 2017 IEEE International Conference on Computer Design (ICCD) · doi:10.1109/iccd.2017.16 · Cited by 241
Self-citationPoison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks
Shafahi, A., Huang, W., Najibi, M., Suciu, O., Studer, C., Dumitraş, T., & Goldstein, T. (2018) · arXiv (Cornell University) · doi:10.3929/ethz-b-000460348 · Cited by 236
NIC: Detecting Adversarial Samples with Neural Network Invariant Checking
Ma, S., Liu, Y., Tao, G., Lee, W.-C., & Zhang, X. (2019) · Proceedings 2019 Network and Distributed System Security Symposium · doi:10.14722/ndss.2019.23415 · Cited by 235
Self-citationPoisoning Attacks to Graph-Based Recommender Systems
Fang, M., Yang, G., Gong, N. Z., & Liu, J. (2018) · Proceedings of the 34th Annual Computer Security Applications Conference · doi:10.1145/3274694.3274706 · Cited by 204
Label Sanitization Against Label Flipping Poisoning Attacks
Paudice, A., Muñoz-González, L., & Lupu, E. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-13453-2_1 · Cited by 176
On the Suitability of Lp-Norms for Creating and Preventing Adversarial Examples
Sharif, M., Bauer, L., & Reiter, M. K. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00211 · Cited by 172
Model-Reuse Attacks on Deep Learning Systems
Ji, Y., Zhang, X., Ji, S., Luo, X., & Wang, T. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243757 · Cited by 167
Self-citationYang, W. M., Kong, D., Xie, T., & Gunter, C. A. (2017) · Annual Computer Security Applications Conference · doi:10.1145/3134600.3134642 · Cited by 132
STRIP: A Defence Against Trojan Attacks on Deep Neural NetworksPreprint
Gao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D. C., & Nepal, . (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06531 · Cited by 127
Turner, A., Tsipras, D., & Mądry, A. (2018) · Cited by 125
Attacks Meet Interpretability: Attribute-steered Detection of Adversarial SamplesPreprint
Tao, G., Ma, S., Liu, Y., & Zhang, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11580 · Cited by 117
Self-citationBackdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
Liao, C., Zhong, H., Squicciarini, A., Zhu, S., & Miller, D. (2018) · arXiv · doi:10.48550/arxiv.1808.10307 · Cited by 110
SentiNet: Detecting Physical Attacks Against Deep Learning SystemsPreprint
Chou, E., Tramèr, F., Pellegrino, G., & Boneh, D. (2018) · arXiv (Cornell University) · Cited by 83
Backdoor attacks against learning systems
Ji, Y., Zhang, X.-Y., & Wang, T. (2017) · IEEE Conference on Communications and Network Security · doi:10.1109/cns.2017.8228656 · Cited by 73
Self-citationPoTrojan: powerful neural-level trojan designs in deep learning modelsPreprint
Zou, M., Shi, Y., Wang, C., Li, F., Song, W., & Wang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03043 · Cited by 73
Hu-Fu: Hardware and Software Collaborative Attack Framework Against Neural Networks
Li, W., Yu, J., Ning, X., Wang, P., Wei, Q., Wang, Y., & Yang, H. (2018) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2018.00093 · Cited by 73
Efficient Repair of Polluted Machine Learning Systems via Causal Unlearning
Cao, Y., Yu, A. F., Aday, A., Stahl, E., Merwine, J., & Yang, J. (2018) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3196494.3196517 · Cited by 50
Traffic sign detection for U.S. roads: Remaining challenges and a case for tracking
Møgelmose, A., Liu, D., & Trivedi, M. M. (2014) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2014.6957882 · Cited by 41
Neural TrojansPreprint
Liu, Y., Xie, Y., & Srivastava, A. (2017) · arXiv (Cornell University) · Cited by 29
Self-citation
Cited by
Privacy-Enhanced Federated Learning Against Poisoning Adversaries
Liu, X., Li, H., Xu, G., Chen, Z., Huang, X., & Lu, R. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3108434 · Cited by 351
Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization
Li, S., Xue, M., Zhao, B., Zhu, H., & Zhang, X. (2020) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2020.3021407 · Cited by 310
Self-citationFebruus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
Doan, B. G., Abbasnejad, E., & Ranasinghe, D. C. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427264 · Cited by 263
Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
Ashmore, R., Calinescu, R., & Paterson, C. (2022) · ACM Computing Surveys · doi:10.1145/3453444 · Cited by 256
Backdoor Attacks to Graph Neural Networks
Zhang, Z., Jia, J., Wang, B., & Gong, N. Z. (2021) · Proceedings of the 26th ACM Symposium on Access Control Models and Technologies · doi:10.1145/3450569.3463560 · Cited by 220
Metadata
- Authors
- Liu, Y., Lee, W.-C., Tao, G., Ma, S., Aafer, Y., & Zhang, X.
- Venue
- Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2019-11-06
- Pages
- 1265-1282
- OpenAlex
- W2986013765
- Cited by
- 404
- References
- 43
- FWCI
- 14.12
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.