CoChat
Sign Up
PreprintCited by 5

The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.

Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

Croce, Francesco, Hein, Matthias·2020·arXiv
DOI 10.48550/arxiv.2003.01690arXiv 2003.01690
Open this paper in CoChat — free account

Abstract

The field of defense strategies against adversarial attacks has significantly grown over the last years, but progress is hampered as the evaluation of adversarial defenses is often insufficient and thus gives a wrong impression of robustness. Many promising defenses could be broken later on, making it difficult to identify the state-of-the-art. Frequent pitfalls in the evaluation are improper tuning of hyperparameters of the attacks, gradient obfuscation or masking. In this paper we first propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness. We apply our ensemble to over 50 models from papers published at recent top machine learning and computer vision venues. In all except one of the cases we achieve lower robust test accuracy than reported in these papers, often by more than $10\%$, identifying several broken defenses.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2003.01690

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 49

  1. Alayrac, J.-B., Uesato, J., Huang, P.-S., Fawzi, A., Stan- forth, R., and Kohli, P. Are labels required for improving adversarial robustness? In NeurIPS, 2019.
  2. Andriushchenko, M., Croce, F., Flammarion, N., and Hein, M. Square attack: a query-efficient black-box adversarial attack via random search. In ECCV, 2020.
  3. Athalye, A., Carlini, N., and Wagner, D. A. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
  4. Atzmon, M., Haim, N., Yariv, L., Israelov, O., Maron, H., and Lipman, Y. Controlling neural level sets. In NeurIPS, 2019.
  5. Augustin, M., Meinke, A., and Hein, M. Adversarial robust- ness on in-and out-distribution improves explainability. In ECCV, 2020.
  6. Carlini, N. and Wagner, D. Adversarial examples are not easily detected: Bypassing ten detection methods. In ACM Workshop on Artificial Intelligence and Security, 2017b.
  7. Carmon, Y., Raghunathan, A., Schmidt, L., Duchi, J. C., and Liang, P. S. Unlabeled data improves adversarial robustness. In NeurIPS, pp. 11190-11201, 2019.
  8. Chan, A., Tay, Y., Ong, Y. S., and Fu, J. Jacobian adver- sarially regularized networks for robustness. In ICLR, 2020.
  9. Cohen, J. M., Rosenfeld, E., and Kolter, J. Z. Certified adversarial robustness via randomized smoothing. In NeurIPS, 2019.
  10. Croce, F. and Hein, M. Minimally distorted adversarial examples with a fast adaptive boundary attack. In ICML, 2020.
  11. Croce, F., Andriushchenko, M., and Hein, M. Provable robustness of relu networks via maximization of linear regions. In AISTATS, 2019a.
  12. Croce, F., Rauber, J., and Hein, M. Scaling up the random- ized gradient-free adversarial attack reveals overestima- tion of robustness using established attacks. International J. of Computer Vision (IJCV), 2019b.
  13. Ding, G. W., Wang, L., and Jin, X. AdverTorch v0.1: An adversarial robustness toolbox based on pytorch. arXiv preprint arXiv:1902.07623, 2019. · arXiv
  14. Ding, G. W., Sharma, Y., Lui, K. Y. C., and Huang, R. Mma training: Direct input space margin maximization through adversarial training. In ICLR, 2020.
  15. Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. In ICLR, 2015.
  16. Gowal, S., Dvijotham, K., Stanforth, R., Bunel, R., Qin, C., Uesato, J., Arandjelovic, R., Mann, T. A., and Kohli, P. On the effectiveness of interval bound propagation for training verifiably robust models. In ICCV, 2019.
  17. Grathwohl, W., Wang, K.-C., Jacobsen, J.-H., Duvenaud, D., Norouzi, M., and Swersky, K. Your classifier is secretly an energy based model and you should treat it like one. In ICLR, 2020.
  18. Hendrycks, D., Lee, K., and Mazeika, M. Using pre-training can improve model robustness and uncertainty. In ICML, pp. 2712-2721, 2019.
  19. Jang, Y., Zhao, T., Hong, S., and Lee, H. Adversarial defense via learning to generate diverse attacks. In ICCV, 2019.
  20. Kim, J. and Wang, X. Sensible adversarial learning, 2020. URL https://openreview.net/forum? id=rJlf_RVKwr.
  21. Kumari, N., Singh, M., Sinha, A., Machiraju, H., Krish- namurthy, B., and Balasubramanian, V. N. Harnessing the vulnerability of latent layers in adversarially trained models. In IJCAI, pp. 2779-2785, 7 2019.
  22. Lecuyer, M., Atlidakis, V., Geambasu, R., Hsu, D., and Jana, S. Certified robustness to adversarial examples with differential privacy. In IEEE Symposium on Security and Privacy (SP), 2019.
  23. Li, B., Chen, C., Wang, W., and Carin, L. Certified adver- sarial robustness with additive noise. In NeurIPS, 2019.
  24. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Valdu, A. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
  25. Mao, C., Zhong, Z., Yang, J., Vondrick, C., and Ray, B. Metric learning for adversarial robustness. In NeurIPS, pp. 478-489, 2019.
  26. Mirman, M., Gehr, T., and Vechev, M. Differentiable ab- stract interpretation for provably robust neural networks. In ICML, 2018.
  27. Moosavi-Dezfooli, S.-M., Fawzi, A., Uesato, J., and Frossard, P. Robustness via curvature regularization, and vice versa. In CVPR, 2019.
  28. Mosbach, M., Andriushchenko, M., Trost, T., Hein, M., and Klakow, D. Logit pairing methods can fool gradient- based attacks. In NeurIPS 2018 Workshop on Security in Machine Learning, 2018.
  29. Mustafa, A., Khan, S., Hayat, M., Goecke, R., Shen, J., and Shao, L. Adversarial defense by restricting the hidden space of deep neural networks. In ICCV, 2019.
  30. Pang, T., Xu, K., Du, C., Chen, N., and Zhu, J. Improving adversarial robustness via promoting ensemble diversity. In ICML, 2019.
  31. Pang, T., Xu, K., Dong, Y., Du, C., Chen, N., and Zhu, J. Rethinking softmax cross-entropy loss for adversarial robustness. In ICLR, 2020.
  32. Qin, C., Martens, J., Gowal, S., Krishnan, D., Dvijotham, K., Fawzi, A., De, S., Stanforth, R., and Kohli, P. Adversarial robustness through local linearization. In NeurIPS, 2019.
  33. Rice, L., Wong, E., and Kolter, J. Z. Overfitting in adversar- ially robust deep learning. In ICML, 2020.
  34. Rony, J., Hafemann, L. G., Oliveira, L. S., Ayed, I. B., Sabourin, R., and Granger, E. Decoupling direction and norm for efficient gradient-based l2 adversarial attacks and defenses. In CVPR, 2019.
  35. Shafahi, A., Najibi, M., Ghiasi, M. A., Xu, Z., Dickerson, J., Studer, C., Davis, L. S., Taylor, G., and Goldstein, T. Adversarial training for free! In NeurIPS, pp. 3353-3364, 2019.
  36. Song, C., He, K., Wang, L., and Hopcroft, J. E. Improving the generalization of adversarial training with domain adaptation. In ICLR, 2019.
  37. Taghanaki, S. A., Abhishek, K., Azizi, S., and Hamarneh, G. A kernelized manifold mapping to diminish the effect of adversarial perturbations. In CVPR, 2019.
  38. Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., and Madry, A. Robustness may be at odds with accuracy. In ICLR, 2019.
  39. Wang, B., Shi, Z., and Osher, S. Resnets ensemble via the feynman-kac formalism to improve natural and robust accuracies. In NeurIPS, 2019.
  40. Wang, J. and Zhang, H. Bilateral adversarial training: To- wards fast training of more robust models against adver- sarial attacks. In ICCV, 2019.
  41. Wong, E., Schmidt, F., Metzen, J. H., and Kolter, J. Z. Scaling provable adversarial defenses. In NeurIPS, 2018.
  42. Wong, E., Rice, L., and Kolter, J. Z. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.
  43. Xiao, C., Zhong, P., and Zheng, C. Enhancing adversarial defense by k-winners-take-all. In ICLR, 2020.
  44. Yang, Y., Zhang, G., Katabi, D., and Xu, Z. ME-net: To- wards effective adversarial robustness with matrix estima- tion. In ICML, 2019.
  45. Zhang, D., Zhang, T., Lu, Y., Zhu, Z., and Dong, B. You only propagate once: Accelerating adversarial training via maximal principle. In NeurIPS, pp. 227-238, 2019a.
  46. Zhang, H. and Wang, J. Defense against adversarial attacks using feature scattering-based adversarial training. In NeurIPS, pp. 1829-1839, 2019.
  47. Zhang, H. and Xu, W. Adversarial interpolation train- ing: A simple approach for improving model robustness, 2020. URL https://openreview.net/forum? id=Syejj0NYvr.
  48. Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., and Jordan, M. I. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019b.
  49. Zhang, H., Chen, H., Xiao, C., Gowal, S., Stanforth, R., Li, B., Boning, D., and Hsieh, C.-J. Towards stable and efficient training of verifiably robust neural networks. In ICLR, 2020.

Cited by

Sorted by citations

Metadata

Authors
Croce, F., & Hein, M.
Venue
arXiv
Publisher
arXiv
Published
2020
DOI
10.48550/arxiv.2003.01690
arXiv
2003.01690
References
2
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review