Towards Evaluating the Robustness of Neural Networks
Open accessAbstract
Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input x and any target classification t, it is possible to find a new input x' that is similar to x but classified as t. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from 95% to 0.5%. In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with 100% probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Carlini and Farid (2020) add imperceptible adversarial perturbations to fake/real faces and show that even state-of-the-art DeepFake detectors are vulnerable to both white-box and black-box attacks with significant accuracy reduction on public datasets.
“For example, Carlini and Farid (2020) add imperceptible adversarial perturbations to the fake/real faces and show that even the state-of-the-art DeepFake detectors are vulnerable to both white-box and black-box attacks (Carlini and Wagner 2017; Brown et al. 2017) with significant accuracy reduction on the public datasets (Wang et al. 2020e; Frank et al. 2020).”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Carlini and Wagner 2017
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredExisting detection methods face challenges with generalization across datasets and vulnerability to adversarial attacks.
“Existing detection methods face challenges with generalization across datasets and vulnerability to adversarial attacks.”
From Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial Attacks (2025) · cites this paper as [4]
The Introduction says neural networks are vulnerable to adversarial examples, and the Original Adversarial section shows that a proposed defense, defensive distillation, fails against stronger attacks (page numbers unavailable). The paper does not establish that detection methods struggle to generalize across datasets; testing attacks on three datasets is not a finding about cross-dataset generalization. (LLM, conf=0.83)
Experimental — accuracy not yet measuredAdversarial attacks have been widely studied against ML models under both white-box and black-box assumptions.
“Adversarial attacks have been widely studied against ML models [47,50] under both white-box [9, 21, 31, 55] and black-box assumptions [33, 43].”
From Adversarial Detection Avoidance Attacks: Evaluating the robustness of perceptual hashing-based client-side scanning (2021) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · see citing works
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
Adversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375 · see citing works
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Clevert, D.-A., Unterthiner, T., & Hochreiter, S. (2015) · arXiv · doi:10.48550/arxiv.1511.07289 · Cited by 4,513 · see citing works
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198 · see citing works
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
Papernot, N., McDaniel, P. D., Wu, X., Jha, S., & Swami, A. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.41 · Cited by 3,123 · see citing works
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Hinton, G. E., Deng, L., Yu, D., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. (2012) · Cited by 1,839 · see citing works
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326 · see citing works
A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots
Giusti, A., Guzzi, J., Ciresan, D. C., He, F.-L., Rodriguez, J. P., Fontana, F., Faessler, M., Forster, C., Schmidhuber, J., Caro, G. D., et al. (2016) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2015.2509024 · Cited by 665 · see citing works
Towards Deep Neural Network Architectures Robust to Adversarial ExamplesPreprint
Gu, S., & Rigazio, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.5068 · Cited by 658 · see citing works
Globally Normalized Transition-Based Neural Networks
Andor, D., Alberti, C., Weiss, D., Severyn, A., Presta, A., Ganchev, K., Petrov, S., & Collins, M. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1231 · Cited by 494 · see citing works
Carlini, N., Mishra, P., Vaidya, T., Zhang, Y., Sherr, M., Shields, C., Wagner, D., & Zhou, W. (2016) · USENIX Security Symposium · Cited by 455 · see citing works
Self-citationMalware classification with recurrent networks
Pascanu, R., Stokes, J. W., Sanossian, H. Y. Y., Marinescu, M., & Thomas, A. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178304 · Cited by 454 · see citing works
Large-scale malware classification using random projections and neural networks
Dahl, G. E., Stokes, J. W., Deng, L., & Yu, D. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638293 · Cited by 450 · see citing works
Learning with a Strong AdversaryPreprint
Huang, R., Xu, B., Schuurmans, D., & Szepesvari, C. (2015) · arXiv.org · doi:10.48550/arxiv.1511.03034 · Cited by 381 · see citing works
All you need is a good initPreprint
Mishkin, D., & Matas, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06422 · Cited by 374 · see citing works
Shaham, U., Yamada, Y., & Negahban, S. N. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.04.027 · Cited by 355 · see citing works
Graham, B. (2014) · arXiv · doi:10.48550/arxiv.1412.6071 · Cited by 337 · see citing works
Adversarial Perturbations Against Deep Neural Networks for Malware ClassificationPreprint
Grosse, K., Papernot, N., Manoharan, P., Michael Backes, A., & McDaniel, P. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04435 · Cited by 315 · see citing works
Measuring Neural Net Robustness with ConstraintsPreprint
Bastani, O., Ioannou, Y., Lampropoulos, L., Vytiniotis, D., Nori, A., & Criminisi, A. (2016) · arXiv (Cornell University) · doi:10.17863/cam.26484 · Cited by 257 · see citing works
Fast, Lean, and Accurate: Modeling Password Guessability Using Neural Networks
Melicher, W., Ur, B., Segreti, S. M., Komanduri, S., Bauer, L., Christin, N., & Cranor, L. F. (2016) · USENIX Security Symposium · Cited by 250 · see citing works
Neural Networks in Mobile Robot Motion
Janglová, D. (2004) · International Journal of Advanced Robotic Systems · doi:10.5772/5615 · Cited by 189 · see citing works
Yuan, Z., Lü, Y., Wang, Z., & Xue, Y. (2014) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2740070.2631434 · Cited by 167 · see citing works
DeepFool: a simple and accurate method to fool deep neural networksPreprint
Moosavi-Dezfooli, S.-M., Fawzi, A., & Pascal Frossard, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04599 · Cited by 154 · see citing works
The Limitations of Deep Learning in Adversarial SettingsPreprint
Papernot, N., McDaniel, P. D., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.07528 · Cited by 107 · see citing works
On the Effectiveness of Defensive DistillationPreprint
Papernot, N., & McDaniel, P. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.05113 · Cited by 57 · see citing works
Neural Networks in Mobile Robot Motion
Janglová, D. (2005) · Pro Literatur Verlag, Germany eBooks · doi:10.5772/10672 · Cited by 11 · see citing works
Flagged first, then by citations
2 of the 45 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- GOODFELLOW, I. J., SHLENS, J., AND SZEGEDY, C. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014). · arXiv
- HUANG, X., KWIATKOWSKA, M., WANG, S., AND WU, M. Safety verification of deep neural networks. arXiv preprint arXiv:1610.06940 (2016). · arXiv
- MAAS, A. L., HANNUN, A. Y., AND NG, A. Y. Rectifier nonlinearities improve neural network acoustic models. In Proc. ICML (2013), vol. 30.
- PAPERNOT, N., GOODFELLOW, I., SHEATSLEY, R., FEINMAN, R., AND MCDANIEL, P. cleverhans v1.0.0: an adversarial machine learning library. arXiv preprint arXiv:1610.00768 (2016). · arXiv
- SZEGEDY, C., VANHOUCKE, V., IOFFE, S., SHLENS, J., AND WOJNA, Z. Rethinking the Inception architecture for computer vision. arXiv preprint arXiv:1512.00567 (2015). · arXiv
- YUAN, Z., LU, Y., WANG, Z., AND XUE, Y. Droid-sec: Deep learning in android malware detection. In ACM SIGCOMM Computer Commu- nication Review (2014), vol. 44, ACM, pp. 371-372. APPENDIX
Cited by
Sorted by citations
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602 · see citing works
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Boosting Adversarial Attacks with Momentum
Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., & Li, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00957 · Cited by 3,168 · see citing works
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922 · see citing works
One Pixel Attack for Fooling Deep Neural Networks
Su, J., Vargas, D. V., & Sakurai, K. (2019) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2019.2890858 · Cited by 2,406 · see citing works
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021 · see citing works
Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
Xu, W., Evans, D., & Qi, Y. (2018) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23198 · Cited by 1,859 · see citing works
Adversarial Examples: Attacks and Defenses for Deep Learning
Yuan, X., He, P., Zhu, Q., & Li, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886017 · Cited by 1,733 · see citing works
Chen, P., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C. (2017) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3128572.3140448 · Cited by 1,782 · see citing works
Robust physical-world attacks on deep learning visual classification
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00175 · Cited by 2,173 · see citing works
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Frankle, J., & Carbin, M. (2018) · arXiv · doi:10.48550/arxiv.1803.03635 · Cited by 1,529 · see citing works
Theoretically principled trade-off between robustness and accuracy
Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., & Jordan, M. I. (2019) · arXiv · doi:10.48550/arxiv.1901.08573 · Cited by 1,560 · see citing works
Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H.-T., & Zhao, B. Y. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00031 · Cited by 1,486 · see citing works
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477 · see citing works
“measurement artifacts or adversarial perturbations [182]; 2) resilience to drifting data distributions [49]; 3) ability to accurately assess the confidence of its own predictions [135], [139]; 4) safety and security aspects [21], [26], [84], [193]; 5) legal requirements or adherence to social norms [54], [60]; 6) abil…”
BackgroundWild patterns: Ten years after the rise of adversarial machine learning
Biggio, B., & Roli, F. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.023 · Cited by 1,444 · see citing works
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Carlini, N., & Wagner, D. (2017) · AISec@CCS · doi:10.1145/3128572.3140444 · Cited by 2,079 · see citing works
Self-citationImproving Transferability of Adversarial Examples With Input Diversity
Xie, C., Zhang, Z., Zhou, Y., Bai, S., Wang, J., Ren, Z., & Yuille, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00284 · Cited by 1,353 · see citing works
Ahmad, Z., Khan, A. S., Shiang, C. W., Abdullah, J., & Ahmad, F. (2020) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.4150 · Cited by 1,313 · see citing works
Pei, K., Cao, Y., Yang, J., & Jana, S. (2017) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3132747.3132785 · Cited by 1,296 · see citing works
Trojaning Attack on Neural Networks
Liu, Y., Ma, S., Aafer, Y., Lee, W.-C., Zhai, J., Wang, W., & Zhang, X. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23291 · Cited by 1,270 · see citing works
Tian, Y., Pei, K., Jana, S., & Ray, B. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180220 · Cited by 1,400 · see citing works
Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
Carlini, N., & Wagner, D. (2018) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2018.00009 · Cited by 1,071 · see citing works
Self-citationRobustness May Be at Odds with AccuracyPreprint
Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12152 · Cited by 1,240 · see citing works
Deep Variational Information BottleneckPreprint
Alemi, A. A., Fischer, I., Dillon, J. V., & Murphy, K. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00410 · Cited by 1,291 · see citing works
A Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 1,080 · see citing works
Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
Dong, Y., Pang, T., Su, H., & Zhu, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00444 · Cited by 1,032 · see citing works
Countering Adversarial Images using Input TransformationsPreprint
Guo, C., Rana, M., Cissé, M., & van der Maaten, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00117 · Cited by 1,151 · see citing works
Meng, D., & Chen, H. (2017) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3133956.3134057 · Cited by 1,013 · see citing works
Synthesizing Robust Adversarial ExamplesPreprint
Athalye, A., Engstrom, L., Ilyas, A., & Kwok, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07397 · Cited by 973 · see citing works
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacksPreprint
Croce, F., & Hein, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01690 · Cited by 965 · see citing works
Adversarial Examples for Semantic Segmentation and Object Detection
Xie, C., Wang, J., Zhang, Z., Zhou, Y., Xie, L., & Yuille, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.153 · Cited by 955 · see citing works
HotFlip: White-Box Adversarial Examples for Text Classification
Ebrahimi, J., Rao, A., Lowd, D., & Dou, D. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2006 · Cited by 948 · see citing works
Generating Adversarial Examples with Adversarial Networks
Xiao, C., Li, B., Zhu, J.-Y., He, W., Liu, M., & Song, D. X. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/543 · Cited by 988 · see citing works
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897 · see citing works
Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative ModelsPreprint
Samangouei, P., Kabkab, M., & Chellappa, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06605 · Cited by 838 · see citing works
Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
Fang, M., Cao, X., Jia, J., & Gong, N. Z. (2019) · arXiv · doi:10.48550/arxiv.1911.11815 · Cited by 828 · see citing works
Mitigating Adversarial Effects Through RandomizationPreprint
Xie, C., Wang, J., Zhang, Z., Ren, Z., & Yuille, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01991 · Cited by 905 · see citing works
Generating Natural Language Adversarial Examples
Alzantot, M., Sharma, Y., Elgohary, A., Ho, B.-J., Srivastava, M. B., & Chang, K.-W. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1316 · Cited by 895 · see citing works
AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation
Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, P., Chaudhuri, S., & Vechev, M. T. (2018) · IEEE Symposium on Security and Privacy · doi:10.1109/sp.2018.00058 · Cited by 1,038 · see citing works
Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., & Li, B. (2018) · 2018 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2018.00057 · Cited by 861 · see citing works
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762 · see citing works
Certified Robustness to Adversarial Examples with Differential Privacy
Lecuyer, M., Atlidakis, V., Geambasu, R., Hsu, D., & Jana, S. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00044 · Cited by 750 · see citing works
Fooling Automated Surveillance Cameras: Adversarial Patches to Attack Person Detection
Thys, S., Ranst, W. V., & Goedemé, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00012 · Cited by 770 · see citing works
Raschka, S., Patterson, J. M., & Nolet, C. (2020) · Information · doi:10.3390/info11040193 · Cited by 777 · see citing works
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review
Xu, H., Ma, Y., Liu, H., Deb, D., Liu, H., Tang, J., & Jain, A. K. (2020) · Machine Intelligence Research · doi:10.1007/s11633-019-1211-x · Cited by 716 · see citing works
Black-Box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers
Gao, J., Lanchantin, J., Soffa, M. L., & Qi, Y. (2018) · 2018 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2018.00016 · Cited by 693 · see citing works
Fast is better than free: Revisiting adversarial trainingPreprint
Wong, E., Rice, L., & Kolter, J. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2001.03994 · Cited by 1,444 · see citing works
High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
Wang, H., Wu, X., Huang, Z., & Xing, E. P. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00871 · Cited by 679 · see citing works
HopSkipJumpAttack: A Query-Efficient Decision-Based Attack
Chen, J., Jordan, M. I., & Wainwright, M. J. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00045 · Cited by 718 · see citing works
TextBugger: Generating Adversarial Text Against Real-world Applications
Li, J., Ji, S., Du, T., Li, B., & Wang, T. (2019) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2019.23138 · Cited by 634 · see citing works
An abstract domain for certifying neural networks
Singh, G., Gehr, T., Püschel, M., & Vechev, M. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290354 · Cited by 625 · see citing works
DÏoT: A Federated Self-learning Anomaly Detection System for IoT
Nguyen, T. D., Marchal, S., Miettinen, M., Fereidooni, H., Asokan, N., & Sadeghi, A.-R. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00080 · Cited by 663 · see citing works
EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
Chen, P., Sharma, Y., Zhang, H., Yi, J., & Hsieh, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11302 · Cited by 594 · see citing works
Deep Fingerprinting: Undermining Website Fingerprinting Defenses with Deep Learning
Sirinam, P., Imani, M., Juarez, M., & Wright, M. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243768 · Cited by 592 · see citing works
Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving
Cao, Y., Xiao, C., Cyr, B., Zhou, Y., Park, W., Rampazzi, S., Chen, Q. A., Fu, K. E., & Mao, Z. M. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3339815 · Cited by 577 · see citing works
Adversarial Attacks and Defenses in Deep Learning
Ren, K., Zheng, T., Qin, Z., & Liu, X. (2020) · Engineering · doi:10.1016/j.eng.2019.12.012 · Cited by 678 · see citing works
Certified Defenses against Adversarial ExamplesPreprint
Raghunathan, A., Steinhardt, J., & Liang, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.09344 · Cited by 572 · see citing works
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612 · see citing works
“empirical robustness, on a test dataset has been widely used [65, 312].”
BackgroundOn Evaluating Adversarial RobustnessPreprint
Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Mądry, A., & Kurakin, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06705 · Cited by 562 · see citing works
Self-citationProtecting Intellectual Property of Deep Neural Networks with Watermarking
Zhang, J., Gu, Z., Jang, J., Wu, H., Stoecklin, M. P., Huang, H., & Molloy, I. (2018) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3196494.3196550 · Cited by 556 · see citing works
Understanding adversarial attacks on deep learning based medical image analysis systems
Ma, X., Niu, Y., Gu, L., Wang, Y., Zhao, Y., Bailey, J., & Lu, F. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2020.107332 · Cited by 553 · see citing works
On the (Statistical) Detection of Adversarial ExamplesPreprint
Grosse, K., Manoharan, P., Papernot, N., Michael Backes, A., & McDaniel, P. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06280 · Cited by 546 · see citing works
Huang, X., Kroening, D., Ruan, W., Sharp, J. J., Sun, Y., Thamo, E., Wu, M., & Yi, X. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100270 · Cited by 567 · see citing works
Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz, M., & Backes, M. (2019) · Proceedings 2019 Network and Distributed System Security Symposium · doi:10.14722/ndss.2019.23119 · Cited by 852 · see citing works
Enhancing the Transferability of Adversarial Attacks through Variance Tuning
Wang, X., & He, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00196 · Cited by 514 · see citing works
Characterizing Adversarial Subspaces Using Local Intrinsic DimensionalityPreprint
Ma, X., Li, B., Wang, Y., Erfani, S., Wijewickrema, S. N. R., Schoenebeck, G. R., Song, D. X., Houle, M. E., & Bailey, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02613 · Cited by 508 · see citing works
SoK: Security and Privacy in Machine Learning
Papernot, N., McDaniel, P., Sinha, A., & Wellman, M. P. (2018) · 2018 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2018.00035 · Cited by 502 · see citing works
Stealing Hyperparameters in Machine Learning
Wang, B., & Gong, N. Z. (2018) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2018.00038 · Cited by 491 · see citing works
On Adaptive Attacks to Adversarial Example Defenses
Tramer, F., Carlini, N., Brendel, W., & Madry, A. (2020) · arXiv · doi:10.48550/arxiv.2002.08347 · Cited by 486 · see citing works
Self-citationAdversarial Machine Learning for Network Intrusion Detection Systems: A Comprehensive Survey
He, K., Kim, D. S., & Asghar, M. R. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3233793 · Cited by 493 · see citing works
Overfitting in adversarially robust deep learningPreprint
Rice, L., Wong, E., & Kolter, J. Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11569 · Cited by 472 · see citing works
Explainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., van Gerven, M. A. J., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 486 · see citing works
Guiding Deep Learning System Testing Using Surprise Adequacy
Kim, J.-H., FELDT, R. H., & Yoo, S. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00108 · Cited by 451 · see citing works
Robust Physical-World Attacks on Deep Learning ModelsPreprint
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08945 · Cited by 443 · see citing works
Hein, M., Andriushchenko, M., & Bitterwolf, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00013 · Cited by 444 · see citing works
Efficient Decision-Based Black-Box Adversarial Attacks on Face Recognition
Dong, Y., Su, H., Wu, B., Li, Z., Liu, W., Zhang, T., & Zhu, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00790 · Cited by 433 · see citing works
SoK: Towards the Science of security and privacy in machine learning
Papernot, N., McDaniel, P. D., Sinha, A., & Wellman, M. P. (2018) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 427 · see citing works
Certifying Some Distributional Robustness with Principled Adversarial TrainingPreprint
Sinha, A., Namkoong, H., Volpi, R., & Duchi, J. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10571 · Cited by 515 · see citing works
Improving Adversarial Robustness Requires Revisiting Misclassified Examples
Wang, Y., Zou, D., Yi, J., Bailey, J., Ma, X., & Gu, Q. (2020) · International Conference on Learning Representations · Cited by 430 · see citing works
Towards Fast Computation of Certified Robustness for ReLU NetworksPreprint
Weng, T.-W., Zhang, H., Chen, H., Song, Z., Hsieh, C., Boning, D. S., Dhillon, I. S., & Daniel, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09699 · Cited by 460 · see citing works
A survey on adversarial attacks and defences
Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A., & Mukhopadhyay, D. (2021) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12028 · Cited by 1,042 · see citing works
Adversarial Risk and the Dangers of Evaluating Against Weak AttacksPreprint
Uesato, J., O'Donoghue, B., van den Oord, A., & Kohli, P. (2018) · International Conference on Machine Learning · doi:10.48550/arxiv.1802.05666 · Cited by 713 · see citing works
Automated Website Fingerprinting through Deep Learning
Rimmer, V., Preuveneers, D., Juarez, M., Goethem, T. V., & Joosen, W. (2018) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23105 · Cited by 450 · see citing works
Unlabeled Data Improves Adversarial RobustnessPreprint
Carmon, Y., Raghunathan, A., Schmidt, L., Liang, P., & Duchi, J. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13736 · Cited by 440 · see citing works
Review of Artificial Intelligence Adversarial Attack and Defense Technologies
Qiu, S., Liu, Q., Zhou, S., & Wu, C. (2019) · Applied Sciences · doi:10.3390/app9050909 · Cited by 412 · see citing works
Towards Robust Neural Networks via Random Self-ensemble
Liu, X., Cheng, M., Zhang, H., & Hsieh, C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_23 · Cited by 408 · see citing works
Attack of the tails: Yes, you really can backdoor federated learning
Wang, H., Sreenivasan, K., Rajput, S., Vishwakarma, H., Agarwal, S., Sohn, J.-Y., Lee, K., & Papailiopoulos, D. (2020) · arXiv · doi:10.48550/arxiv.2007.05084 · Cited by 406 · see citing works
A comprehensive survey of loss functions and metrics in deep learning
Terven, J. R., Esparza, D. M. C., Romero-González, J.-A., Ramírez-Pedraza, A., & Chávez‐Urbiola, E. A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11198-7 · Cited by 517 · see citing works
SafetyNet: Detecting and Rejecting Adversarial Examples Robustly
Lu, J., Issaranon, T., & Forsyth, D. A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.56 · Cited by 397 · see citing works
Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective
Xu, K., Chen, H., Liu, S., Chen, P., Weng, T.-W., Hong, M., & Lin, X. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/550 · Cited by 418 · see citing works
Deep Text Classification Can be Fooled
Liang, B., Li, H., Su, M., Bian, P., Li, X., & Shi, W. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/585 · Cited by 392 · see citing works
Provable defenses against adversarial examples via the convex outer adversarial polytopePreprint
Kolter, J. Z., & Wong, E. (2017) · arXiv (Cornell University) · Cited by 387 · see citing works
DeepMutation: Mutation Testing of Deep Learning Systems
Ma, L., Zhang, F., Sun, J., Xue, M., Li, B., Juefei-Xu, F., Xie, C., Li, L., Liu, Y., Zhao, J., et al. (2018) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2018.00021 · Cited by 385 · see citing works
Understanding the role of individual units in a deep neural network
Bau, D., Zhu, J.-Y., Strobelt, H., Lapedriza, A., Zhou, B., & Torralba, A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907375117 · Cited by 375 · see citing works
AutoZOOM: Autoencoder-Based Zeroth Order Optimization Method for Attacking Black-Box Neural Networks
Tu, C., Ting, P., Chen, P., Liu, S., Zhang, H., Yi, J., Hsieh, C., & Cheng, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301742 · Cited by 387 · see citing works
ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector
Chen, S.-T., Cornelius, C., Martin, J., & Chau, D. H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10925-7_4 · Cited by 376 · see citing works
Nesterov Accelerated Gradient and Scale Invariance for Adversarial AttacksPreprint
Lin, J., Song, C., He, K., Wang, L., & Hopcroft, J. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06281 · Cited by 455 · see citing works
Efficient Neural Network Robustness Certification with General Activation FunctionsPreprint
Zhang, H., Weng, T.-W., Chen, P., Hsieh, C., & Daniel, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00866 · Cited by 372 · see citing works
Making machine learning robust against adversarial inputs
Goodfellow, I., McDaniel, P., & Papernot, N. (2018) · Communications of the ACM · doi:10.1145/3134599 · Cited by 358 · see citing works
Generative Adversarial Perturbations
Poursaeed, O., Katsman, I., Gao, B., & Belongie, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00465 · Cited by 370 · see citing works
- Authors
- Carlini, N., & Wagner, D.
- Affiliations
- University of California, Berkeley (US)
- Venue
- 2017 IEEE Symposium on Security and Privacy (SP)
- Publisher
- IEEE
- Published
- 2017-05
- Pages
- 39-57
- ISSN
- 1081-6011 (ISSN-L)
- OpenAlex
- W2516574342
- Cited by
- 8,859
- References
- 45
- Citations by year
- 2026: 566 · 2025: 983 · 2024: 1222 · 2023: 1200 · 2022: 1111 · 2021: 1308 · 2020: 1181 · 2019: 801 · 2018: 393 · 2017: 86 · 2016: 5 · 2014: 1 · 2012: 1
- FWCI
- 306.99
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Symposium on Security and Privacy (S&P) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Cisco Systems
- International Science and Technology Center
- Multidisciplinary University Research Initiative (FA9550-12–1-0040)
- Air Force Office of Scientific Research (FA9550-, FA9550-12, FA9550‐12‐1, FA9550-12-1-0040)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- adversarial examples, defensive distillation, adversarial attacks, neural network robustness, attack transferability, distance measures, robustness analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
