Boosting Fast Adversarial Training With Learnable Adversarial Initialization
Open accessAbstract
Adversarial training (AT) has been demonstrated to be effective in improving model robustness by leveraging adversarial examples for training. However, most AT methods are in face of expensive time and computational cost for calculating gradients at multiple steps in generating adversarial examples. To boost training efficiency, fast gradient sign method (FGSM) is adopted in fast AT methods by calculating gradient only once. Unfortunately, the robustness is far from satisfactory. One reason may arise from the initialization fashion. Existing fast AT generally uses a random sample-agnostic initialization, which facilitates the efficiency yet hinders a further robustness improvement. Up to now, the initialization in fast AT is still not extensively explored. In this paper, focusing on image classification, we boost fast AT with a sample-dependent adversarial initialization, i.e., an output from a generative network conditioned on a benign image and its gradient information from the target network. As the generative network and the target network are optimized jointly in the training phase, the former can adaptively generate an effective initialization with respect to the latter, which motivates gradually improved robustness. Experimental evaluations on four benchmark databases demonstrate the superiority of our proposed method over state-of-the-art fast AT methods, as well as comparable robustness to advanced multi-step AT methods. The code is released at https://github.com//jiaxiaojunQAQ//FGSM-SDI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
With additional knowledge of local models and global model, model-dependent triggers can be potentially further enhanced, e.g., by employing adversarial generative networks and training.
“With the additional knowledge of the local models along with the global model, the model-dependent triggers can be potentially further enhanced, e.g., by employing adversarial generative networks and training [174].”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [174]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Part of this paper’s reference list could not be split into single entries.
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741 · see citing works
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · see citing works
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909 · see citing works
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602 · see citing works
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317 · see citing works
On Assessing ML Model Robustness: A Methodological Framework (Academic Track)
Awadid, A., & Robert, B. (2025) · OASIcs : OpenAccess Series in Informatics · doi:10.4230/oasics.saia.2024.1 · Cited by 4,723 · see citing works
Deep Learning using Rectified Linear Units (ReLU)Preprint
Agarap, A. F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08375 · Cited by 3,792 · see citing works
Boosting Adversarial Attacks with Momentum
Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., & Li, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00957 · Cited by 3,168 · see citing works
Cyclical Learning Rates for Training Neural Networks
Smith, L. N. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.58 · Cited by 2,931 · see citing works
On the momentum term in gradient descent learning algorithms
Ning Qian, A. (1999) · Neural Networks · doi:10.1016/s0893-6080(98)00116-6 · Cited by 2,318 · see citing works
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks
Katz, G., Barrett, C., Dill, D. L., Julian, K., & Kochenderfer, M. J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-63387-9_5 · Cited by 1,706 · see citing works
Improving Transferability of Adversarial Examples With Input Diversity
Xie, C., Zhang, Z., Zhou, Y., Bai, S., Wang, J., Ren, Z., & Yuille, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00284 · Cited by 1,356 · see citing works
Self-citationDeng, L., Hinton, G. E., & Kingsbury, B. E. D. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,222 · see citing works
Deep convolutional neural networks for LVCSR
Sainath, T. N., Mohamed, A., Kingsbury, B. E. D., & Ramabhadran, B. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639347 · Cited by 1,066 · see citing works
Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
Dong, Y., Pang, T., Su, H., & Zhu, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00444 · Cited by 1,034 · see citing works
Defense Against Adversarial Attacks Using High-Level Representation Guided Denoiser
Liao, F., Liang, M., Dong, Y., Pang, T., Hu, X., & Zhu, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00191 · Cited by 967 · see citing works
Square Attack: A Query-Efficient Black-Box Adversarial Attack via Random Search
Andriushchenko, M., Croce, F., Flammarion, N., & Hein, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58592-1_29 · Cited by 880 · see citing works
Enhancing the Transferability of Adversarial Attacks through Variance Tuning
Wang, X., & He, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00196 · Cited by 514 · see citing works
Shaham, U., Yamada, Y., & Negahban, S. N. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.04.027 · Cited by 355 · see citing works
ComDefend: An Efficient Image Compression Model to Defend Adversarial Examples
Jia, X., Wei, X., Cao, X., & Foroosh, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00624 · Cited by 341 · see citing works
Self-citationTransferable Adversarial Attacks for Image and Video Object Detection
Wei, X., Liang, S., Chen, N., & Cao, X. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/134 · Cited by 219 · see citing works
Self-citationAdversarial Vertex Mixup: Toward Better Adversarially Robust Generalization
Lee, S., Lee, H., & Yoon, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00035 · Cited by 119 · see citing works
Training Robust Deep Neural Networks via Adversarial Noise Propagation
Liu, A., Liu, X., Yu, H., Zhang, C., Liu, Q., & Tao, D. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3082317 · Cited by 117 · see citing works
Understanding Catastrophic Overfitting in Single-step Adversarial Training
Kim, H., Lee, W., & Lee, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16989 · Cited by 90 · see citing works
Defense Against Adversarial Attacks by Reconstructing Images
Zhang, S., Gao, H., & Rao, Q. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3092582 · Cited by 69 · see citing works
A Reinforced Generation of Adversarial Examples for Neural Machine Translation
zou, W., Huang, S., Xie, J., Dai, X., & Chen, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.319 · Cited by 67 · see citing works
Deep image prior based defense against adversarial examples
Dai, T., Feng, Y., Chen, B., Lu, J. J., & Xia, S. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108249 · Cited by 54 · see citing works
Improving Query Efficiency of Black-Box Adversarial Attack
Bai, Y., Zeng, Y., Jiang, Y., Wang, Y., Xia, S., & Guo, W. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58595-2_7 · Cited by 48 · see citing works
Adversarial Attack Against Deep Saliency Models Powered by Non-Redundant Priors
Che, Z., Borji, A., Zhai, G., et al. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3050303 · Cited by 28 · see citing works
Removing Adversarial Noise via Low-Rank Completion of High-Sensitivity Points
Zhao, Z., Wang, H., Sun, H., et al. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3086596 · Cited by 15 · see citing works
Adversarial Training for Solving Inverse Problems in Image Processing
Zou, Z., Shi, T., Shi, Z., et al. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3053398 · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 21
- [4][4] D. Amodei, S. Ananthanarayanan, R. Anubhai, J. Bai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, Q. Cheng, G. Chen et al. , 'Deep speech 2: End-to-end speech recognition in english and mandarin,' in ICML , 2016, pp. 173-182. -
- [5][5] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus, 'Intriguing properties of neural networks,' in ICLR , 2014. -
- [6][6] I. J. Goodfellow, J. Shlens, and C. Szegedy, 'Explaining and harnessing adversarial examples,' in ICLR , 2015. -
- [11][11] Y. Song, T. Kim, S. Nowozin, S. Ermon, and N. Kushman, 'Pixeldefend: Leveraging generative models to understand and defend against adversarial examples,' in ICLR , 2018. -
- [17][17] Y. Bai, Y. Zeng, Y. Jiang, S.-T. Xia, X. Ma, and Y. Wang, 'Improving adversarial robustness via channel-wise activation suppressing,' in ICLR , 2021. -
- [18][18] D. Wu, S. Xia, and Y. Wang, 'Adversarial weight perturbation helps robust generalization,' in NeurIPS , 2020. -
- [23][23] Y. Wang, X. Ma, J. Bailey, J. Yi, B. Zhou, and Q. Gu, 'On the convergence and robustness of adversarial training,' in ICML , 2019, pp. 6586-6595. -
- [25][25] T. Weng, H. Zhang, H. Chen, Z. Song, C. Hsieh, L. Daniel, D. S. Boning, and I. S. Dhillon, 'Towards fast computation of certified robustness for relu networks,' in ICML , 2018, pp. 5273-5282. -
- [26][26] H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan, 'Theoretically principled trade-off between robustness and accuracy,' in ICML , 2019, pp. 7472-7482. -
- [27][27] L. Rice, E. Wong, and J. Z. Kolter, 'Overfitting in adversarially robust deep learning,' in ICML , 2020, pp. 8093-8104. -
- [29][29] F. Tramèr, A. Kurakin, N. Papernot, I. J. Goodfellow, D. Boneh, and P. D. McDaniel, 'Ensemble adversarial training: Attacks and defenses,' in ICLR , 2018. -
- [30][30] A. Shafahi, M. Najibi, A. Ghiasi, Z. Xu, J. P. Dickerson, C. Studer, L. S. Davis, G. Taylor, and T. Goldstein, 'Adversarial training for free!' in NeurIPS , 2019, pp. 3353-3364. -
- [31][31] E. Wong, L. Rice, and J. Z. Kolter, 'Fast is better than free: Revisiting adversarial training,' in ICLR , 2020. -
- [32][32] M. Andriushchenko and N. Flammarion, 'Understanding and improving fast adversarial training,' in NeurIPS , 2020. -
- [37][37] J. Lin, C. Song, K. He, L. Wang, and J. E. Hopcroft, 'Nesterov accelerated gradient and scale invariance for adversarial attacks,' in ICLR , 2020. -
- [41][41] F. Croce and M. Hein, 'Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,' in ICML , 2020, pp. 2206-2216. -
- [42][42] --, 'Minimally distorted adversarial examples with a fast adaptive boundary attack,' in ICML , 2020, pp. 2196-2205. -
- [44][44] S. Ioffe and C. Szegedy, 'Batch normalization: Accelerating deep network training by reducing internal covariate shift,' in ICML , 2015, pp. 448-456. -
- [45][45] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, 'Spectral normalization for generative adversarial networks,' in ICLR , 2018. -
- [52][52] T. Pang, X. Yang, Y. Dong, H. Su, and J. Zhu, 'Bag of tricks for adversarial training,' in CVPR 2021, , 2021. -
- [55][55] H. Zhang, H. Chen, C. Xiao, B. Li, M. Liu, D. S. Boning, and C. Hsieh, 'Robust deep reinforcement learning against adversarial perturbations on state observations,' in NeurIPS , 2020. -
Cited by
Sorted by citations
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
“With the additional knowledge of the local models along with the global model, the model-dependent triggers can be potentially further enhanced, e.g., by employing adversarial generative networks and training [174].”
BackgroundSurvey: federated learning data security and privacy-preserving in edge-Internet of Things
Li, H., Ge, L., & Tian, L. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10774-7 · Cited by 112 · see citing works
Guo, R., Liu, H., & Liu, D. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2023.3297663 · Cited by 79 · see citing works
- Authors
- Jia, X., Zhang, Y., Wu, B., Wang, J., & Cao, X.
- Affiliations
- Chinese Academy of Sciences (CN)
- Tencent (China) (CN)
- Shenzhen Research Institute of Big Data (CN)
- Sun Yat-sen University (CN)
- Venue
- IEEE Transactions on Image Processing
- Published
- 2022
- Volume
- 31
- Pages
- 4417-4430
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3206581298
- Cited by
- 69
- References
- 36
- Citations by year
- 2026: 11 · 2025: 17 · 2024: 22 · 2023: 14 · 2022: 5
- FWCI
- 4.77
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key Research and Development Program of China (2019YFB1406500)
- National Natural Science Foundation of China (62025604, 62132006, U1803264, 62076213)
- Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (VRLAB2021C06)
- Tencent AI Laboratory Rhino-Bird Focused Research Program (JR202123)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- FGSM, adversarial robustness, image classification, generative adversarial networks, training efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
