Adversarial Examples: Attacks and Defenses for Deep Learning
Abstract
With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks (DNNs) have been recently found vulnerable to well-designed input samples called adversarial examples. Adversarial perturbations are imperceptible to human but can easily fool DNNs in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying DNNs in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for DNNs, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The ultimate level of transparency is retained by simulable models, next by decomposable structures, and subsequently by computationally transparent configurations.
“The ultimate level of transparency is retained by simulable models, next by decomposable structures, and subsequently by computationally transparent configurations [46], [50].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [50]
Nine criteria have been proposed in the National Association of City Transportation Officials' automated vehicle manifesto as a framework for upcoming autonomous driving governance.
“Nine criteria have been proposed in the National Association of City Transportation Officials' automated vehicle manifesto as a framework for upcoming autonomous driving governance (Fig. 13) [50], [79].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [50]
Post-hoc explainability methodologies establish lesser intricate surrogate models to emulate complicated black-box ML techniques.
“Post-hoc explainability methodologies establish lesser intricate surrogate models to emulate complicated black-box ML techniques [1], [50].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [50]
A novel hybrid algorithm is used in [50] to determine a subset of parameters that are most significant to the categorization job.
“To determine a subset of parameters that are most significant to the categorization job, a novel hybrid algorithm is used in [50].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [50]
Together, the medial and lateral optimizations yield greater local search effectiveness and growing global model accuracy.
“Together, the medial and lateral optimizations yield greater local search effectiveness as well as growing global model accuracy.”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [50]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
Rethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517 · see citing works
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Adversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375 · see citing works
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317 · see citing works
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · see citing works
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258 · see citing works
Boosting Adversarial Attacks with Momentum
Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., & Li, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00957 · Cited by 3,168 · see citing works
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
Papernot, N., McDaniel, P. D., Wu, X., Jha, S., & Swami, A. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.41 · Cited by 3,123 · see citing works
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · see citing works
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680 · see citing works
Adversarial machine learning at scale
Kurakin, A., Goodfellow, I., & Bengio, S. (2016) · arXiv · doi:10.48550/arxiv.1611.01236 · Cited by 2,325 · see citing works
Ensemble Adversarial Training: Attacks and Defenses
Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2017) · arXiv · doi:10.48550/arxiv.1705.07204 · Cited by 2,176 · see citing works
Chen, P., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C. (2017) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3128572.3140448 · Cited by 1,782 · see citing works
Can we open the black box of AI?
Castelvecchi, D. (2016) · Nature · doi:10.1038/538020a · Cited by 1,773 · see citing works
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks
Katz, G., Barrett, C., Dill, D. L., Julian, K., & Kochenderfer, M. J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-63387-9_5 · Cited by 1,706 · see citing works
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695 · see citing works
Opening the Black Box of Deep Neural Networks via InformationPreprint
Shwartz-Ziv, R., & Tishby, N. (2017) · arXiv.org · doi:10.48550/arxiv.1703.00810 · Cited by 1,674 · see citing works
Evasion attacks against machine learning at test time
Biggio, B., Corona, I., Maiorca, D., Nelson, B. A., Srndic, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Cited by 1,658 · see citing works
Adversarial Examples Are Not Easily Detected
Carlini, N., & Wagner, D. (2017) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3128572.3140444 · Cited by 1,626 · see citing works
Do Deep Nets Really Need to be Deep?Preprint
Ba, J. L., & Caruana, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6184 · Cited by 1,592 · see citing works
Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition
Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978392 · Cited by 1,530 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
Adversarial Examples for Evaluating Reading Comprehension Systems
Jia, R., & Liang, P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1215 · Cited by 1,410 · see citing works
Poisoning Attacks against Support Vector Machines
Biggio, B., Nelson, B., & Laskov, P. (2012) · arXiv · doi:10.48550/arxiv.1206.6389 · Cited by 1,331 · see citing works
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326 · see citing works
Autoencoding beyond pixels using a learned similarity metric
Larsen, A., Sønderby, S. K., Larochelle, H., & Winther, O. (2016) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 1,317 · see citing works
Delving into Transferable Adversarial Examples and Black-box Attacks
Liu, Y., Chen, X., Liu, C., & Song, D. (2016) · arXiv · doi:10.48550/arxiv.1611.02770 · Cited by 1,310 · see citing works
Pei, K., Cao, Y., Yang, J., & Jana, S. (2017) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3132747.3132785 · Cited by 1,296 · see citing works
Dalvi, N. N., Domingos, P. M., Mausam, Sanghai, S. K., & Verma, D. (2004) · Knowledge Discovery and Data Mining · doi:10.1145/1014052.1014066 · Cited by 1,085 · see citing works
Meng, D., & Chen, H. (2017) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3133956.3134057 · Cited by 1,013 · see citing works
Robots that can adapt like animals
Cully, A., Clune, J., Tarapore, D., & Mouret, J.-B. (2015) · Nature · doi:10.1038/nature14422 · Cited by 979 · see citing works
Adversarial Examples for Semantic Segmentation and Object Detection
Xie, C., Wang, J., Zhang, Z., Zhou, Y., Xie, L., & Yuille, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.153 · Cited by 955 · see citing works
(2007) · Cited by 937 · see citing works
Full Resolution Image Compression with Recurrent Neural Networks
Toderici, G. D., Vincent, D., Johnston, N., Hwang, S. J., Minnen, D. C., Shor, J., & Covell, M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.577 · Cited by 920 · see citing works
Adversarially Robust Generalization Requires More DataPreprint
Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., & Ma̧dry, A. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1804.11285 · Cited by 891 · see citing works
Safety Verification of Deep Neural Networks
Huang, X., Kwiatkowska, M., Wang, S., & Wu, M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-63387-9_1 · Cited by 859 · see citing works
Can machine learning be secure?
Barreno, M., Nelson, B. A., Sears, R., Joseph, A. D., & Tygar, J. D. (2006) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/1128817.1128824 · Cited by 852 · see citing works
The security of machine learning
Barreno, M., Nelson, B. A., Joseph, A. D., & Tygar, J. D. (2010) · Machine Learning · doi:10.1007/s10994-010-5188-5 · Cited by 839 · see citing works
Deep neural network based malware detection using two dimensional binary program features
Saxe, J., & Berlin, K. (2015) · International Conference on Malicious and Unwanted Software · doi:10.1109/malware.2015.7413680 · Cited by 781 · see citing works
Lowd, D., & Meek, C. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081950 · Cited by 755 · see citing works
Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization
Evangelidis, G., & Psarakis, E. Ζ. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.113 · Cited by 749 · see citing works
On Detecting Adversarial PerturbationsPreprint
Metzen, J. H., Genewein, T., Fischer, V., & Bischoff, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.04267 · Cited by 747 · see citing works
Autoencoding beyond pixels using a learned similarity metricPreprint
Larsen, A., Sønderby, S. K., Larochelle, H., & Winther, O. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.09300 · Cited by 712 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · arXiv · doi:10.48550/arxiv.1606.05250 · Cited by 678 · see citing works
Towards Deep Neural Network Architectures Robust to Adversarial ExamplesPreprint
Gu, S., & Rigazio, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.5068 · Cited by 658 · see citing works
The Space of Transferable Adversarial ExamplesPreprint
Tramèr, F., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2017) · arXiv.org · doi:10.48550/arxiv.1704.03453 · Cited by 649 · see citing works
Detecting Adversarial Samples from Artifacts
Feinman, R., Curtin, R. R., Shintre, S., & Gardner, A. B. (2017) · arXiv · doi:10.48550/arxiv.1703.00410 · Cited by 606 · see citing works
Salimans, T., Karpathy, A., Chen, X., & Kingma, D. P. (2017) · arXiv · doi:10.48550/arxiv.1701.05517 · Cited by 584 · see citing works
Adversarial Examples for Malware Detection
Grosse, K., Papernot, N., Manoharan, P., Michael Backes, A., & McDaniel, P. D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66399-9_4 · Cited by 566 · see citing works
PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
Song, Y., Kim, T., Nowozin, S., Ermon, S., & Kushman, N. (2017) · arXiv · doi:10.48550/arxiv.1710.10766 · Cited by 562 · see citing works
On the (Statistical) Detection of Adversarial ExamplesPreprint
Grosse, K., Manoharan, P., Papernot, N., Michael Backes, A., & McDaniel, P. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06280 · Cited by 546 · see citing works
Generating Natural Adversarial Examples
Zhao, Z., Dua, D., & Singh, S. (2017) · arXiv · doi:10.48550/arxiv.1710.11342 · Cited by 508 · see citing works
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2016) · arXiv · doi:10.48550/arxiv.1612.08242 · Cited by 488 · see citing works
Carlini, N., Mishra, P., Vaidya, T., Zhang, Y., Sherr, M., Shields, C., Wagner, D., & Zhou, W. (2016) · USENIX Security Symposium · Cited by 455 · see citing works
Large-scale malware classification using random projections and neural networks
Dahl, G. E., Stokes, J. W., Deng, L., & Yu, D. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638293 · Cited by 450 · see citing works
Robust Physical-World Attacks on Deep Learning ModelsPreprint
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08945 · Cited by 443 · see citing works
SoK: Towards the Science of security and privacy in machine learning
Papernot, N., McDaniel, P. D., Sinha, A., & Wellman, M. P. (2018) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 427 · see citing works
SafetyNet: Detecting and Rejecting Adversarial Examples Robustly
Lu, J., Issaranon, T., & Forsyth, D. A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.56 · Cited by 397 · see citing works
Systematic Poisoning Attacks on and Defenses for Machine Learning in Healthcare
Kermani, M. M., Sur‐Kolay, S., Raghunathan, A., & Jha, N. K. (2014) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2014.2344095 · Cited by 383 · see citing works
Learning with a Strong AdversaryPreprint
Huang, R., Xu, B., Schuurmans, D., & Szepesvari, C. (2015) · arXiv.org · doi:10.48550/arxiv.1511.03034 · Cited by 381 · see citing works
Technical Report on the CleverHans v2.1.0 Adversarial Examples LibraryPreprint
Papernot, N., Faghri, F., Carlini, N., Goodfellow, I., Feinman, R., Kurakin, A., Xie, C., Sharma, Y., Brown, T., Roy, A., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.00768 · Cited by 380 · see citing works
Yuan, Z., Lü, Y., Wang, Z., & Xue, Y. (2014) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2619239.2631434 · Cited by 373 · see citing works
Generating Adversarial Malware Examples for Black-Box Attacks Based on GANPreprint
Hu, W., & Tan, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05983 · Cited by 361 · see citing works
Single Shot Text Detector with Regional Attention
He, P. L., Huang, W., He, T., Zhu, Q., Qiao, Y., & Li, X. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.331 · Cited by 347 · see citing works
Self-citationAdversarial Attacks and Defences Competition
Kurakin, A., Goodfellow, I., Samy Bengio, A., Dong, Y., Liao, F., Liang, M., Pang, T., Zhu, J., Hu, X., Xie, C., et al. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-94042-7_11 · Cited by 321 · see citing works
Is feature selection secure against training data poisoning?Preprint
Huang, X. X., Biggio, B., Gavin Brown, B., Fumera, G., Eckert, C. M., & Roli, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.07933 · Cited by 314 · see citing works
Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning modelsPreprint
Rauber, J., Brendel, W., & Bethge, M. (2017) · arXiv (Cornell University) · Cited by 305 · see citing works
Analysis of classifiers’ robustness to adversarial perturbations
Fawzi, A., Fawzi, O., & Pascal Frossard, B. (2017) · Machine Learning · doi:10.1007/s10994-017-5663-3 · Cited by 289 · see citing works
Universal Adversarial Perturbations Against Semantic Image Segmentation
Metzen, J. H., Kumar, M. C., Brox, T., & Fischer, V. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.300 · Cited by 287 · see citing works
Enhancing robustness of machine learning systems via data transformations
Bhagoji, A. N., Cullina, D., Sitawarin, C., & Mittal, P. (2018) · Conference on Information Sciences and Systems · doi:10.1109/ciss.2018.8362326 · Cited by 275 · see citing works
Adversarial Diversity and Hard Positive Generation
Rozsa, A., Rudd, E. M., & Boult, T. E. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.58 · Cited by 268 · see citing works
Adversarial and Clean Data Are Not Twins
Gong, Z., & Wang, W. (2023) · International Workshop on Exploiting Artificial Intelligence Techniques for Data Management (aiDM) · doi:10.1145/3593078.3593935 · Cited by 255 · see citing works
Adversarial Examples for Generative Models
Kos, J., Fischer, I., & Song, D. X. (2018) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2018.00014 · Cited by 249 · see citing works
Adversarial Training for Relation Extraction
Wu, Y., Bamman, D., & Russell, S. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1187 · Cited by 229 · see citing works
Adversarial Example Defense: Ensembles of Weak Defenses are not Strong
He, W., Wei, J., Chen, X., Carlini, N., & Song, D. X. (2017) · Cited by 221 · see citing works
Towards the Science of Security and Privacy in Machine LearningPreprint
Papernot, N., McDaniel, P. D., Sinha, A., & Wellman, M. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03814 · Cited by 217 · see citing works
Data Poisoning Attacks against Autoregressive Models
Alfeld, S., Zhu, X., & Barford, P. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10237 · Cited by 205 · see citing works
Anderson, H. S., Woodbridge, J., & Filar, B. (2016) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/2996758.2996767 · Cited by 202 · see citing works
Early Methods for Detecting Adversarial ImagesPreprint
Hendrycks, D., & Gimpel, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.00530 · Cited by 191 · see citing works
Multiple classifier systems for robust classifier design in adversarial environments
Biggio, B., Fumera, G., & Roli, F. (2010) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-010-0007-7 · Cited by 187 · see citing works
A Fast Uyghur Text Detector for Complex Background Images
Yan, C. C., Xie, H., Chen, J., Zha, Z.-J., Hao, X., Zhang, Y., & Dai, Q. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2838320 · Cited by 183 · see citing works
A Boundary Tilting Persepective on the Phenomenon of Adversarial ExamplesPreprint
Tanay, T., & Griffin, L. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07690 · Cited by 180 · see citing works
Exploring the space of adversarial images
Tabacof, P., & Junior, E. A. D. V. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727230 · Cited by 176 · see citing works
Yuan, Z., Lü, Y., Wang, Z., & Xue, Y. (2014) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2740070.2631434 · Cited by 167 · see citing works
MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial ExamplesPreprint
Carlini, N., & Wagner, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08478 · Cited by 154 · see citing works
Towards Robust Detection of Adversarial ExamplesPreprint
Pang, T., Du, C., Dong, Y., & Zhu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.00633 · Cited by 154 · see citing works
Adversarial vulnerability for any classifierPreprint
Fawzi, A., Fawzi, H., & Fawzi, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08686 · Cited by 153 · see citing works
Delving into adversarial attacks on deep policies
Kos, J., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1705.06452 · Cited by 135 · see citing works
DolphinAtack: Inaudible Voice CommandsPreprint
Zhang, G., Yan, C., Ji, X., Zhang, T., Zhang, T., & Xu, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.09537 · Cited by 127 · see citing works
Adversarial SpheresPreprint
Gilmer, J., Metz, L., Faghri, F., Schoenholz, S. S., Raghu, M., WATTENBERG, M. P., & Goodfellow, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02774 · Cited by 124 · see citing works
Towards Proving the Adversarial Robustness of Deep Neural Networks
Katz, G., Barrett, C., Dill, D. L., Julian, K., & Kochenderfer, M. J. (2017) · Electronic Proceedings in Theoretical Computer Science · doi:10.4204/eptcs.257.3 · Cited by 115 · see citing works
Bradshaw, J., Matthews, A. G. D. G., & Ghahramani, Z. (2017) · arXiv · doi:10.48550/arxiv.1707.02476 · Cited by 109 · see citing works
Adversarial and Clean Data Are Not TwinsPreprint
Gong, Z., Wang, W., & Ku, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04960 · Cited by 101 · see citing works
Provably Minimally-Distorted Adversarial ExamplesPreprint
Carlini, N., Katz, G., Barrett, C., & Dill, D. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.10207 · Cited by 97 · see citing works
DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural NetworksPreprint
Gopinath, D., Katz, G., Păsăreanu, C., & Barrett, C. W. (2017) · arXiv.org · doi:10.48550/arxiv.1710.00486 · Cited by 94 · see citing works
The IBM 2015 English conversational telephone speech recognition system
Saon, G., Kuo, H.-K. J., Rennie, S. J., & Picheny, M. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-632 · Cited by 92 · see citing works
Adversarial Example Defenses: Ensembles of Weak Defenses are not StrongPreprint
He, W., Wei, J., Chen, X., Carlini, N., & Song, D. X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04701 · Cited by 82 · see citing works
Towards Interpretable Deep Neural Networks by Leveraging Adversarial ExamplesPreprint
Dong, Y., Bao, F., Su, H., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09035 · Cited by 77 · see citing works
Adversarial Images for Variational AutoencodersPreprint
Tabacof, P., Tavares, J., & Junior, E. A. D. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00155 · Cited by 61 · see citing works
A Survey of Modern Questions and Challenges in Feature Extraction.
Storcheus, D., Rostamizadeh, A., & Kumar, S. (2015) · neural information processing systems · Cited by 52 · see citing works
Boosting Adversarial Attacks with MomentumPreprint
Dong, Y., Liao, F., Pang, T., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06081 · Cited by 50 · see citing works
Standard detectors aren't (currently) fooled by physical adversarial stop signsPreprint
Lu, J., Sibai, H., Fabry, E., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03337 · Cited by 47 · see citing works
Image Quality Assessment Using the SSIM and the Just Noticeable Difference Paradigm
Flynn, J. R., Ward, S., Abich, J., et al. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39360-0_3 · Cited by 45 · see citing works
Detecting Adversarial Attacks on Neural Network Policies with Visual ForesightPreprint
Lin, Y.-C., Liu, M.-Y., Sun, M., & Huang, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.00814 · Cited by 40 · see citing works
Fundamental limits on adversarial robustness
Fawzi, A., Fawzi, O., & Frossard, P. (2015) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 39 · see citing works
Towards Interpretable Deep Neural Networks by Leveraging Adversarial ExamplesPreprint
Dong, Y., Su, H., Zhu, J. J., & Bao, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05493 · Cited by 39 · see citing works
Adversarial examples for generative modelsPreprint
Kos, J., Fischer, I., & Song, D. X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06832 · Cited by 31 · see citing works
Pattern Recognition Systems under Attack
Roli, F., Biggio, B., & Fumera, G. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41822-8_1 · Cited by 24 · see citing works
Learning Fast and Slow: PROPEDEUTICA for Real-time Malware DetectionPreprint
Sun, R., Yuan, X., He, P. L., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01145 · Cited by 12 · see citing works
Self-citationBlind attacks on machine learners
Beatson, A., Wang, Z., & Liu, H. (2016) · neural information processing systems · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,062 · see citing works
Digital twin: values, challenges and enablers from a modeling perspective
Rasheed, A., San, O., & Kvamsdal, T. (2020) · IEEE Access · doi:10.1109/access.2020.2970143 · Cited by 1,952 · see citing works
Rethinking drug design in the artificial intelligence era
Schneider, P., Walters, W. P., Plowright, A. T., Sieroka, N., Listgarten, J., Goodnow, R. A., Fisher, J., Jansen, J. M., Duca, J. S., Rush, T. S., et al. (2020) · Nature Reviews Drug Discovery · doi:10.1038/s41573-019-0050-3 · Cited by 894 · see citing works
Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study
Hosny, A., Parmar, C., Coroller, T. P., Grossmann, P., Zeleznik, R., Kumar, A., Bussink, J., Gillies, R. J., Mak, R. H., & Aerts, H. J. W. L. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002711 · Cited by 579 · see citing works
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612 · see citing works
“Comprehensive surveys can be found in the literature such as [12, 19, 45, 69, 213, 304, 373].”
BackgroundInfluentialRecent Advances in Adversarial Training for Adversarial Robustness
Bai, T., Luo, J., Zhao, J., Wen, B., & Wang, Q. (2021) · arXiv · doi:10.48550/arxiv.2102.01356 · Cited by 490 · see citing works
“Toour best knowledge, surveys focusing on adversarial training do not exist so far, and recently proposed surveys [Yuan et al. , 2019; Silva and Najafirad, 2020] mainly summarize existing adversarial attacks and defense methods.”
BackgroundMethodRecent Advances in Adversarial Training for Adversarial Robustness
Bai, T., Luo, J., Zhao, J., Wen, B., & Wang, Q. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/591 · Cited by 490 · see citing works
A Review of Trustworthy and Explainable Artificial Intelligence (XAI)
Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023) · IEEE Access · doi:10.1109/access.2023.3294569 · Cited by 352 · see citing works
“To determine a subset of parameters that are most significant to the categorization job, a novel hybrid algorithm is used in [50].”
BackgroundMethodInfluentialDataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3162397 · Cited by 362 · see citing works
Data Poisoning Attacks on Federated Machine Learning
Sun, G., Cong, Y., Dong, J., Wang, Q., Lyu, L., & Liu, J. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3128646 · Cited by 310 · see citing works
PoisonGAN: Generative Poisoning Attacks Against Federated Learning in Edge Computing Systems
Zhang, J., Chen, B., Cheng, X., Binh, H. T. T., & Yu, S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3023126 · Cited by 291 · see citing works
A Survey of Data-driven and Knowledge-aware eXplainable AI
Li, X.-H., Cao, C. C., Shi, Y., Bai, W., Gao, H., Qiu, L., Wang, C., Gao, Y., Zhang, S., Xue, X., et al. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2983930 · Cited by 224 · see citing works
Self-citationBringing ai to edge: from deep learning’s perspective
Liu, D., Kong, H., Luo, X., Liu, W., & Subramaniam, R. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.04.141 · Cited by 171 · see citing works
“Moreover, DNN models are found to be vulnerable to adversary attack [202], so security is emerging as another new design concern for ML systems.”
BackgroundEling, M., Nuessle, D., & Staubli, J. (2022) · The Geneva Papers on Risk and Insurance Issues and Practice · doi:10.1057/s41288-020-00201-7 · Cited by 156 · see citing works
González-Gonzalo, C., Thee, E. F., Klaver, C. C. W., Lee, A. Y., Schlingemann, R. O., Tufail, A., Verbraak, F., & Sánchez, C. I. (2022) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2021.101034 · Cited by 103 · see citing works
Artificial intelligence in safety-critical systems: a systematic review
Wang, Y., & Chung, S. H. (2022) · Industrial Management & Data Systems · doi:10.1108/imds-07-2021-0419 · Cited by 96 · see citing works
Adversarial attack and defense technologies in natural language processing: A survey
Qiu, S., Liu, Q., Zhou, S., & Huang, W. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.020 · Cited by 94 · see citing works
Guo, R., Liu, H., & Liu, D. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2023.3297663 · Cited by 79 · see citing works
A Survey of Deep Learning Applications to Autonomous Vehicle Control
Kuutti, S., Bowden, R., Jin, Y., Barber, P., & Fallah, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2962338 · Cited by 96 · see citing works
Secure and robust machine learning for healthcare: A survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3013489 · Cited by 59 · see citing works
Minimalistic Attacks: How Little It Takes to Fool Deep Reinforcement Learning Policies
Qu, X., Sun, Z., Ong, Y.-S., Gupta, A., & Wei, P. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.2974509 · Cited by 40 · see citing works
Salimparsa, M., Sedig, K., Lizotte, D. J., Abdullah, S. S., Chalabianloo, N., & Muanda, F. T. (2025) · Informatics · doi:10.3390/informatics12040119 · Cited by 39 · see citing works
“Adversarial examples are a related concept: these are small, often imperceptible perturbations to inputs designed to intentionally cause misclassification, thereby revealing model vulnerabilities or sensitivities [37].”
Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning
Zhang, Y., Song, Y., Liang, J., Bai, K., & Yang, Q. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403349 · Cited by 29 · see citing works
A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
Su, H., Luo, J., Liu, C., Yang, X., Zhang, Y., Dong, Y., & Zhu, J. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3688650 · Cited by 11 · see citing works
Truthful AI: Developing and governing AI that does not lie
Evans, O., Cotton-Barratt, O., Finnveden, L., Bales, A., Balwit, A., Wills, P., Righetti, L., & Saunders, W. (2021) · arXiv · doi:10.48550/arxiv.2110.06674 · Cited by 39 · see citing works
- Authors
- Yuan, X., He, P., Zhu, Q., & Li, X.
- Affiliations
- University of Florida (US)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2019-09
- Volume
- 30
- Issue
- 9
- Pages
- 2805-2824
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2963178695
- Semantic Scholar
- 03a507a0876c7e1a26608358b1a9dd39f1eb08e0
- Cited by
- 1,733
- References
- 137
- Citations by year
- 2026: 106 · 2025: 170 · 2024: 199 · 2023: 258 · 2022: 258 · 2021: 327 · 2020: 280 · 2019: 119 · 2018: 14 · 2017: 1 · 2012: 1
- FWCI
- 112.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Science Foundation (CNS-1842407, CNS-1747783, CNS-1624782, OAC-1229576)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- adversarial examples, adversarial attacks, adversarial defense, deep neural network, adversarial perturbations, adversarial example generation, safety-critical systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
