Adversarial Noise Layer: Regularize Neural Network by Adding Noise
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
In this paper, we introduce a novel regularization method called Adversarial Noise Layer (ANL), which are able to significantly improve CNN's generalization ability by adding carefully crafted noise into the intermediate layer activations. ANL can be easily implemented and integrated with most of the mainstream CNN-based models. We compared the effects of the different types of noise and visually demonstrate that our proposed adversarial noise instruct CNN models to learn to extract cleaner feature maps, which further reduce the risk of over-fitting. We also conclude that models trained with ANL are more robust to the adversarial examples generated by FGSM than the traditional adversarial training approaches.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Tabular data can be easily shuffled, but the perturbation of images can have unpredictable effects, with research demonstrating this type of manipulation (for example, adversarial noise) both as an attack against algorithms 65 and a regularization mechanism leading to increased robustness 66 and resilience against inversion attacks.”
From Secure, privacy-preserving and federated machine learning in medical imaging (2020) · cites this paper as [66]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“We also explore the effects of different noises on the VGG-16 (Si-monyan and Zisserman 2015) and ResNet (He et al. 2016a) models using CIFAR-10 and CIFAR-100 (Krizhevsky and Hinton 2009) datasets.”
MethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 11×
“[21] introduced the methods to avoid over-fitting by random horizontal flipping train data, which directly enlarge training dataset.”
BackgroundMethodInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“Many regularization methods [3, 4, 5] have been proposed to prevent over-fitting by adding noise into the training data.”
BackgroundMethodInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 5×
“We trained a LeNet-5 (LeCun et al. 1998) network on MNIST for 3 epochs.”
MethodInfluentialThe Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
(2014) · Cited by 33,419
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 2×
“The modern view of deep CNN architectures is that deep neural network extracts the vision features layer by layer (Zeiler and Fergus 2014).”
BackgroundIdentity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 17×
“We also test the AT algorithm introduced by (Goodfellow, Shlens, and Szegedy 2015), which enhances the robustness of the network by using the adversarial examples for training.”
BackgroundMethodInfluentialZagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108 · cited 4×
“We demonstrate the experiment with the LeNet-5 1 [11] network on Fashion-MNIST [25] dataset.”
MethodInfluentialRandom Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2017) · arXiv · doi:10.48550/arxiv.1708.04896 · Cited by 3,617 · cited 4×
“Many regularization methods [3, 4, 5] have been proposed to prevent over-fitting by adding noise into the training data.”
BackgroundMethodInfluentialRandom Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7000 · Cited by 3,617
Practical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495 · cited 2×
“Plenty of literatures (Goodfellow, Shlens, and Szegedy 2015; Papernot et al. 2017; Kurakin, Goodfellow, and Bengio 2017; Moosavi-Dezfooli et al. 2017) show that small perturbations cause well-designed deep networks to misclassify the image easily.”
BackgroundUniversal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680
Adversarial machine learning at scale
Kurakin, A., Goodfellow, I., & Bengio, S. (2016) · arXiv · doi:10.48550/arxiv.1611.01236 · Cited by 2,325 · cited 2×
“Plenty of literatures (Goodfellow, Shlens, and Szegedy 2015; Papernot et al. 2017; Kurakin, Goodfellow, and Bengio 2017; Moosavi-Dezfooli et al. 2017) show that small perturbations cause well-designed deep networks to misclassify the image easily.”
BackgroundHoward, A., Zhmoginov, A., Chen, L.-C., Sandler, M., & Zhu, M. (2018) · arXiv (Cornell University) · Cited by 896
Attentive Generative Adversarial Network for Raindrop Removal from A Single Image
Qian, R., Tan, R. T., Yang, W., Su, J., & Liu, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00263 · Cited by 821
Deep Pyramidal Residual Networks
Han, D., Kim, J., & Kim, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.668 · Cited by 715 · cited 2×
“Six architectures are adopted: MobileNet [17], MobileNet v2 [18], VGG [4], ResNet [13], PyramidNet [19] and Wide Residual Networks(WRN) [15].”
MethodDisturbLabel: Regularizing CNN on the Loss Layer
Xie, L., Wang, J., Wei, Z., Wang, M., & Tian, Q. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.514 · Cited by 200 · cited 3×
“Besides, methods like DisturbLabel [6] randomly changed the label of a small subset of samples to incorrect value each iteration, thereby regularizing the CNNs on loss layer.”
BackgroundMethodInfluentialRegularizing Deep Networks Using Efficient Layerwise Adversarial Training
Sankaranarayanan, S., Jain, A., Chellappa, R., & Lim, S. N. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11688 · Cited by 82
Research works printed but not linked: 1
- rosenblatt, 1957, The Perceptron A perceiving and recognizing automaton Project PARA
Cited by
Secure, privacy-preserving and federated machine learning in medical imaging
Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0186-1 · Cited by 1,676
“…but the perturbation of images can have unpredictable effects, with research demonstrating this type of manipulation (for example, adversarial noise) both as an attack against algorithms 65 and a regularization mechanism leading to increased robustness 66 and resilience against inversion attacks.”
BackgroundDigital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs)
Bangari, V., Marquez, B. A., Miller, H., Tait, A. N., Nahmias, M. A., de Lima, T. F., Peng, H.-T., Prucnal, P. R., & Shastri, B. J. (2020) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2945540 · Cited by 241
SecureBERT: A Domain-Specific Language Model for Cybersecurity
Aghaei, E., Niu, X., Shadid, W., & Al-Shaer, E. (2023) · Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-25538-0_3 · Cited by 210
“Referring to previous works on maintaining robust neural networks [18,31,33], incorporation of noise to an unstable neural network model with a limited training set can act as a regularizer and help reduce overfitting during the training.”
Rubik’s Cube+: A self-supervised feature learning framework for 3D medical image analysis
Zhu, J., Li, Y., Hu, Y., Ma, K., Zhou, S. K., & Zheng, Y. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101746 · Cited by 161
Deep learning in electron microscopy
Ede, J. M. (2021) · Machine Learning Science and Technology · doi:10.1088/2632-2153/abd614 · Cited by 144
Metadata
- Authors
- You, Z., Ye, J., Li, K., Xu, Z., & Wang, P.
- Affiliations
- Peking University (CN)
- University of Electronic Science and Technology of China (CN)
- Australian National University (AU)
- Venue
- 2019 IEEE International Conference on Image Processing (ICIP)
- Publisher
- IEEE
- Published
- 2019-09
- Pages
- 909-913
- ISSN
- 1522-4880 (ISSN-L)
- OpenAlex
- W2971018118
- Semantic Scholar
- a39149da8f9ec8fc398b9f1506c61e17650e3118
- Cited by
- 80
- References
- 23
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 15 · 2023: 12 · 2022: 11 · 2021: 11 · 2020: 15 · 2019: 7
- FWCI
- 4.21
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Proceedings - International Conference on Image Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- neural network regularization, convolutional neural networks, generalization, overfitting, feature maps, adversarial examples, FGSM, adversarial robustness, noise injection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.