The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Understanding adversarial training: Increasing local stability of supervised models through robust optimization
Open accessAbstract
We show that adversarial training of supervised learning models is in fact a robust optimization procedure. To do this, we establish a general framework for increasing local stability of supervised learning models using robust optimization. The framework is general and broadly applicable to differentiable non-parametric models, e.g., Artificial Neural Networks (ANNs). Using an alternating minimization-maximization procedure, the loss of the model is minimized with respect to perturbed examples that are generated at each parameter update, rather than with respect to the original training data. Our proposed framework generalizes adversarial training, as well as previous approaches for increasing local stability of ANNs. Experimental results reveal that our approach increases the robustness of the network to existing adversarial examples, while making it harder to generate new ones. Furthermore, our algorithm improves the accuracy of the networks also on the original test data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
(2014) · Cited by 33,419
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
(1999) · Springer Series in Operations Research and Financial Engineering · doi:10.1007/b98874 · Cited by 9,482
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Adversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
Papernot, N., McDaniel, P., Wu, X., Jha, S., & Swami, A. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.41 · Cited by 2,982
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,846
Introductory Lectures on Convex Optimization
Nesterov, Y. (2004) · Applied Optimization · doi:10.1007/978-1-4419-8853-9 · Cited by 2,698
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680
Theory and Applications of Robust Optimization
Brown, D. B., Caramanis, C., & Bertsimas, D. (2010) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,524
Smooth minimization of non-smooth functions
Nesterov, Y. (2005) · Mathematical Programming · doi:10.1007/s10107-004-0552-5 · Cited by 2,455
Neural Networks: A Systematic Introduction
Rojas, R. (1996) · Medical Entomology and Zoology · Cited by 2,138
xgboost: Extreme Gradient Boosting
Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., Chen, K., Mitchell, R., Cano, I., Zhou, T., et al. (2014) · CRAN: Contributed Packages · doi:10.32614/cran.package.xgboost · Cited by 2,025
Adversarial Examples: Attacks and Defenses for Deep Learning
Yuan, X., He, P., Zhu, Q., & Li, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886017 · Cited by 1,733
Chen, P.-Y., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C.-J. (2017) · Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3128572.3140448 · Cited by 1,656
Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Rifai, S., Vincent, P., Muller, X., Glorot, X., & Bengio, Y. (2011) · International Conference on Machine Learning · Cited by 1,600
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326
Numerical Optimization (Springer Series in Operations Research and Financial Engineering)
Nocedal, J., & Wright, S. J. (2000) · Cited by 1,112
Robust Solutions to Least-Squares Problems with Uncertain Data
El Ghaoui, L., & Lebret, H. (1997) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/s0895479896298130 · Cited by 1,076
Optimization for Machine Learning
(2011) · The MIT Press eBooks · doi:10.7551/mitpress/8996.001.0001 · Cited by 875
Learning Representations by Back-Propagating Errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (2002) · Cognitive Modeling · doi:10.7551/mitpress/1888.003.0013 · Cited by 759
Robust solution to least squares problems with uncertain data
Ghaoui, L. E., & Lebret, H. (1997) · Cited by 638
Towards Deep Neural Network Architectures Robust to Adversarial Examples
Gu, S., & Rigazio, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.5068 · Cited by 622
Distributional Smoothing with Virtual Adversarial Training
Miyato, T., Maeda, S., Koyama, M., Nakae, K., & Ishii, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00677 · Cited by 317
Robustness and Regularization of Support Vector Machines
Xu, H., Caramanis, C., & Mannor, S. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0803.3490 · Cited by 301
Analysis of classifiers’ robustness to adversarial perturbations
Fawzi, A., Fawzi, O., & Frossard, P. (2018) · Machine Learning · doi:10.1007/s10994-017-5663-3 · Cited by 287
Adversarial Diversity and Hard Positive Generation
Rozsa, A., Rudd, E. M., & Boult, T. E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.58 · Cited by 258
Dropout Training as Adaptive Regularization
Wager, S., Wang, S., & Liang, P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1307.1493 · Cited by 245
Xu, H., Caramanis, C., & Mannor, S. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2048503 · Cited by 230
Deep roto-translation scattering for object classification
Oyallon, E., & Mallat, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298904 · Cited by 220
Chen, P.-Y., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C.-J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.03999 · Cited by 181
Cutting-set methods for robust convex optimization with pessimizing oracles
Mutapcic, A., & Boyd, S. (2009) · Optimization Methods and Software · doi:10.1080/10556780802712889 · Cited by 170
Deep Learning and Music Adversaries
Kereliuk, C., Sturm, B. L. T., & Larsen, J. O. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2478068 · Cited by 164
The Manifold Tangent Classifier
Rifai, S., Dauphin, Y., Vincent, P., Bengio, Y., & Muller, X. (2011) · Cited by 162
Parseval Networks: Improving Robustness to Adversarial Examples
Cissé, M., Bojanowski, P., Grave, É., Dauphin, Y., & Usunier, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.08847 · Cited by 158
Convex Learning with Invariances
Teo, C. H., Roweis, S. T., Globerson, A., & Smola, A. J. (2007) · Cited by 90
Oracle-Based Robust Optimization via Online Learning
Ben‐Tal, A., Hazan, E., Koren, T., & Mannor, S. (2015) · Operations Research · doi:10.1287/opre.2015.1374 · Cited by 62
Chapter Fourteen. Robust Adjustable Multistage Optimization
(2009) · Princeton University Press eBooks · doi:10.1515/9781400831050.355 · Cited by 59
Deep Roto-Translation Scattering for Object ClassificationPreprint
Oyallon, E., & Mallat, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.8659 · Cited by 7
Cited by
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021
Theoretically principled trade-off between robustness and accuracy
Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., & Jordan, M. I. (2019) · arXiv · doi:10.48550/arxiv.1901.08573 · Cited by 1,560
DeepXplore: Automated Whitebox Testing of Deep Learning Systems
Pei, K., Cao, Y., Yang, J., & Jana, S. (2017) · Proceedings of the 26th Symposium on Operating Systems Principles · doi:10.1145/3132747.3132785 · Cited by 1,217
Metadata
- Authors
- Shaham, U., Yamada, Y., & Negahban, S.
- Venue
- Neurocomputing
- Publisher
- Elsevier BV
- Published
- 2018-09
- Volume
- 307
- Pages
- 195-204
- ISSN
- 0925-2312 (ISSN-L)
- OpenAlex
- W2269778407
- Cited by
- 355
- References
- 51
- FWCI
- 29.56
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.