SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems
Open accessAbstract
SentiNet is a novel detection framework for localized universal attacks on neural networks. These attacks restrict adversarial noise to contiguous portions of an image and are reusable with different images-constraints that prove useful for generating physically-realizable attacks. Unlike most other works on adversarial detection, SentiNet does not require training a model or preknowledge of an attack prior to detection. Our approach is appealing due to the large number of possible mechanisms and attack-vectors that an attack-specific defense would have to consider. By leveraging the neural network's susceptibility to attacks and by using techniques from model interpretability and object detection as detection mechanisms, SentiNet turns a weakness of a model into a strength. We demonstrate the effectiveness of SentiNet on three different attacks-i.e., data poisoning attacks, trojaned networks, and adversarial patches (including physically realizable attacks)-and show that our defense is able to achieve very competitive performance metrics for all three threats. Finally, we show that SentiNet is robust against strong adaptive adversaries, who build adversarial patches that specifically target the components of SentiNet's architecture.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most existing robustness defenses are gradient aggregation methods mainly developed for defending against untargeted Byzantine attackers.
“Most existing robustness defenses are gradient aggregation methods mainly developed for defending against the untargeted Byzantine attackers, such as Krum/Multi-Krum [22], AGGREGATHOR [56], Byzantine Gradient Descent (BGD) [57], Median-based Gradient Descent [23], Trimmed-mean-based Gradient Descent [23] and SIGNSGD [39].”
From Privacy and Robustness in Federated Learning: Attacks and Defenses (2024) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
YOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
(2011) · Cited by 6,993 · see citing works
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602 · see citing works
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980 · see citing works
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479 · see citing works
Adversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375 · see citing works
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00097 · Cited by 3,639 · see citing works
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv · doi:10.48550/arxiv.1802.00420 · Cited by 2,172 · see citing works
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021 · see citing works
Understanding Neural Networks Through Deep VisualizationPreprint
Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,687 · see citing works
Adversarial Examples Are Not Easily Detected
Carlini, N., & Wagner, D. (2017) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3128572.3140444 · Cited by 1,626 · see citing works
Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition
Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978392 · Cited by 1,530 · see citing works
Trojaning Attack on Neural Networks
Liu, Y., Ma, S., Aafer, Y., Lee, W.-C., Zhai, J., Wang, W., & Zhang, X. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23291 · Cited by 1,270 · see citing works
BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
Gu, T., Dolan-Gavitt, B., & Garg, S. (2017) · arXiv · doi:10.48550/arxiv.1708.06733 · Cited by 1,061 · see citing works
A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data
Goldstein, M., & Uchida, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0152173 · Cited by 1,032 · see citing works
Meng, D., & Chen, H. (2017) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3133956.3134057 · Cited by 1,013 · see citing works
Synthesizing Robust Adversarial ExamplesPreprint
Athalye, A., Engstrom, L., Ilyas, A., & Kwok, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07397 · Cited by 973 · see citing works
Deep Learning of Representations for Unsupervised and Transfer Learning.
Bengio, Y. (2011) · Cited by 885 · see citing works
Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural NetworksPreprint
Shafahi, A., Huang, W., Najibi, M., Suciu, O., Studer, C., Dumitraş, T., & Goldstein, T. (2018) · arXiv (Cornell University) · doi:10.3929/ethz-b-000460348 · Cited by 626 · see citing works
Detecting Adversarial Samples from Artifacts
Feinman, R., Curtin, R. R., Shintre, S., & Gardner, A. B. (2017) · arXiv · doi:10.48550/arxiv.1703.00410 · Cited by 606 · see citing works
On Evaluating Adversarial RobustnessPreprint
Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Mądry, A., & Kurakin, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06705 · Cited by 562 · see citing works
On the (Statistical) Detection of Adversarial ExamplesPreprint
Grosse, K., Manoharan, P., Papernot, N., Michael Backes, A., & McDaniel, P. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06280 · Cited by 546 · see citing works
Brown, T. B., Mané, D., Roy, A., Abadi, M., & Gilmer, J. (2017) · arXiv · doi:10.48550/arxiv.1712.09665 · Cited by 509 · see citing works
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2016) · arXiv · doi:10.48550/arxiv.1612.08242 · Cited by 488 · see citing works
Robust Physical-World Attacks on Deep Learning ModelsPreprint
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08945 · Cited by 443 · see citing works
Anomaly Detection using One-Class Neural NetworksPreprint
Chalapathy, R., Menon, A. K., & Chawla, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06360 · Cited by 442 · see citing works
SafetyNet: Detecting and Rejecting Adversarial Examples Robustly
Lu, J., Issaranon, T., & Forsyth, D. A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.56 · Cited by 397 · see citing works
Physical Adversarial Examples for Object DetectorsPreprint
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Tramèr, F., Prakash, A., Kohno, T., & Song, D. X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.07769 · Cited by 260 · see citing works
Adversarial and Clean Data Are Not Twins
Gong, Z., & Wang, W. (2023) · International Workshop on Exploiting Artificial Intelligence Techniques for Data Management (aiDM) · doi:10.1145/3593078.3593935 · Cited by 255 · see citing works
MagNet: a Two-Pronged Defense against Adversarial ExamplesPreprint
Meng, D., & Chen, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09064 · Cited by 234 · see citing works
Adversarial Examples: Attacks and Defenses for Deep LearningPreprint
Yuan, X., He, P. L., Zhu, Q., & Li, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.07107 · Cited by 212 · see citing works
LaVAN: Localized and Visible Adversarial NoisePreprint
Karmon, D., Zoran, D., & Goldberg, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02608 · Cited by 203 · see citing works
Learning Deep Features for Discriminative LocalizationPreprint
Zhou, B., Khosla, A., Lapedriza, À., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.04150 · Cited by 202 · see citing works
Early Methods for Detecting Adversarial ImagesPreprint
Hendrycks, D., & Gimpel, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.00530 · Cited by 191 · see citing works
Robust Physical-World Attacks on Machine Learning Models.Preprint
Evtimov, I., Eykholt, K., Fernandes, E., Kohno, T., Li, B., Prakash, A., Rahmati, A., & Song, D. X. (2017) · arXiv (Cornell University) · Cited by 187 · see citing works
Model-Reuse Attacks on Deep Learning Systems
Ji, Y., Zhang, X., Ji, S., Luo, X., & Wang, T. (2018) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3243734.3243757 · Cited by 170 · see citing works
DeepFool: a simple and accurate method to fool deep neural networksPreprint
Moosavi-Dezfooli, S.-M., Fawzi, A., & Pascal Frossard, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04599 · Cited by 154 · see citing works
On the convergence of derivative-free methods for unconstrained optimization
Conn, A., Scheinberg, K., & Toint, P. L. (1997) · Repository of the University of Namur · Cited by 149 · see citing works
Adversarial and Clean Data Are Not TwinsPreprint
Gong, Z., Wang, W., & Ku, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04960 · Cited by 101 · see citing works
Universal adversarial perturbationsPreprint
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Pascal Frossard, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.08401 · Cited by 82 · see citing works
Hardware Trojan Attacks on Neural NetworksPreprint
Clements, J., & Lao, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05768 · Cited by 73 · see citing works
Traffic sign detection for U.S. roads: Remaining challenges and a case for tracking
Møgelmose, A., Liu, D., & Trivedi, M. M. (2014) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2014.6957882 · Cited by 41 · see citing works
SafetyNet: Detecting and Rejecting Adversarial Examples RobustlyPreprint
Lu, J., Issaranon, T., & Forsyth, D. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.00103 · Cited by 41 · see citing works
Review of Deep LearningPreprint
Zhang, R., Li, W., & Mo, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.01653 · Cited by 28 · see citing works
Ad-versarial: Defeating Perceptual Ad-BlockingPreprint
Tramèr, F., Dupré, P., Rusak, G., et al. (2018) · arXiv (Cornell University) · Cited by 13 · see citing works
Evaluation and Design of Robust Neural Network Defenses
Carlini, N. (2018) · eScholarship (California Digital Library) · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- J. Hendrik Metzen, T. Genewein, V. Fischer, and B. Bischoff, "On Detecting Adversarial Perturbations," ArXiv e-prints, Feb. 2017.
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682 · see citing works
Privacy and Robustness in Federated Learning: Attacks and Defenses
Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., Yang, Q., & Yu, P. S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3216981 · Cited by 548 · see citing works
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3162397 · Cited by 362 · see citing works
Rethinking the Backdoor Attacks’ Triggers: A Frequency Perspective
Zeng, Y., Park, W., Mao, Z. M., & Jia, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01616 · Cited by 193 · see citing works
On the Robustness of Backdoor-based Watermarking in Deep Neural Networks
Shafieinejad, M., Lukas, N., Wang, J., Li, X., & Kerschbaum, F. (2021) · Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3437880.3460401 · Cited by 98 · see citing works
An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences
Guo, W., Tondi, B., & Barni, M. (2022) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2022.3190213 · Cited by 91 · see citing works
Black-box Detection of Backdoor Attacks with Limited Information and Data
Dong, Y., Yang, X., Deng, Z., Pang, T., Xiao, Z., Su, H., & Zhu, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01617 · Cited by 92 · see citing works
Lin, Y.-S., Lee, W.-C., & Celik, Z. B. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467213 · Cited by 64 · see citing works
Athena: Probabilistic Verification of Machine Unlearning
Sommer, D. M., Song, L., Wagh, S., & Mittal, P. (2022) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2022-0072 · Cited by 51 · see citing works
Text Backdoor Detection Using an Interpretable RNN Abstract Model
Fan, M., Si, Z., Xie, X., Liu, Y., & Liu, T. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3103064 · Cited by 38 · see citing works
Chen, C., Wei, L., Zhang, L., Peng, Y., & Ning, J. (2022) · Security and Communication Networks · doi:10.1155/2022/2985308 · Cited by 8 · see citing works
A lightweight backdoor defense framework based on image inpainting
Wei, Y., Gao, H., Wang, Y., Gao, Y., & Liu, H. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.03.052 · Cited by 6 · see citing works
- Authors
- Chou, E., Tramer, F., & Pellegrino, G.
- Affiliations
- Carnegie Mellon University (US)
- Helmholtz Center for Information Security (DE)
- Venue
- 2020 IEEE Security and Privacy Workshops (SPW)
- Publisher
- IEEE
- Published
- 2020-05
- Pages
- 48-54
- OpenAlex
- W3011696425
- Cited by
- 281
- References
- 57
- Citations by year
- 2026: 20 · 2025: 62 · 2024: 62 · 2023: 55 · 2022: 26 · 2021: 40 · 2020: 10 · 2019: 6
- FWCI
- 12.33
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- IEEE Security and Privacy Workshops (SPW) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- universal adversarial attacks, adversarial attack detection, adversarial patches, data poisoning, model interpretability, object detection, adaptive adversaries, deep learning security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
