CoChat
Sign Up
PreprintCited by 373Adversarial Robustness in Machine LearningSecurity and Verification in ComputingMachine Learning and Data Classification

The DOI matches its DataCite record; 128 of the 135 references it cites were found in a registry or an index.

RobustBench: a standardized adversarial robustness benchmark

Open access
Croce, Francesco, Andriushchenko, Maksym, Sehwag, Vikash, Debenedetti, Edoardo, Flammarion, Nicolas, Chiang, Mung, Mittal, Prateek, Hein, Matthias·2020·arXiv
DOI 10.48550/arxiv.2010.09670arXiv 2010.09670
Open this paper in CoChat — free accountOpen PDF

Abstract

As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in training robust models. A key challenge in benchmarking robustness is that its evaluation is often error-prone leading to robustness overestimation. Our goal is to establish a standardized benchmark of adversarial robustness, which as accurately as possible reflects the robustness of the considered models within a reasonable computational budget. To this end, we start by considering the image classification task and introduce restrictions (possibly loosened in the future) on the allowed models. We evaluate adversarial robustness with AutoAttack, an ensemble of white- and black-box attacks, which was recently shown in a large-scale study to improve almost all robustness evaluations compared to the original publications. To prevent overadaptation of new defenses to AutoAttack, we welcome external evaluations based on adaptive attacks, especially where AutoAttack flags a potential overestimation of robustness. Our leaderboard, hosted at https://robustbench.github.io/, contains evaluations of 120+ models and aims at reflecting the current state of the art in image classification on a set of well-defined tasks in $\ell_\infty$- and $\ell_2$-threat models and on common corruptions, with possible extensions in the future. Additionally, we open-source the library https://github.com/RobustBench/robustbench that provides unified access to 80+ robust models to facilitate their downstream applications. Finally, based on the collected models, we analyze the impact of robustness on the performance on distribution shifts, calibration, out-of-distribution detection, fairness, privacy leakage, smoothness, and transferability.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2010.09670

Cites

128 of 128 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 135 references

Research works printed but not linked: 47

  1. [3][3] M. Andriushchenko and N. Flammarion. Understanding and improving fast adversarial training. NeurIPS , 2020. -
  2. [4][4] M. Andriushchenko, F. Croce, N. Flammarion, and M. Hein. Square attack: a query-efficient black-box adversarial attack via random search. In ECCV , 2020. -
  3. [6][6] M. Atzmon, N. Haim, L. Yariv, O. Israelov, H. Maron, and Y. Lipman. Controlling neural level sets. NeurIPS , 2019. -
  4. [14][14] D. A. Calian, F. Stimberg, O. Wiles, S.-A. Rebuffi, A. Gyorgy, T. Mann, and S. Gowal. Defending against image corruptions through adversarial augmentations. arXiv , 2021. -
  5. [16][16] N. Carlini. A complete list of all (arxiv) adversarial example papers. https://nicholas.carlini. com/writing/2019/all-adversarial-example-papers.html , 2021. Accessed: 2021-06-08. -
  6. [21][21] J. Chen and Q. Gu. Rays: A ray searching method for hard-label adversarial attack. In KDD , 2020. -
  7. [23][23] T. Chen, S. Liu, S. Chang, Y. Cheng, L. Amini, and Z. Wang. Adversarial robustness: From self-supervised pre-training to fine-tuning. In CVPR , 2020. -
  8. [29][29] F. Croce, M. Andriushchenko, N. D. Singh, N. Flammarion, and M. Hein. Sparse-rs: a versatile framework for query-efficient sparse black-box adversarial attacks. In ECCV Workshop on Adversarial Robustness in the Real World , 2020. -
  9. [35][35] Y. Dong, Q.-A. Fu, X. Yang, T. Pang, H. Su, Z. Xiao, and J. Zhu. Benchmarking adversarial robustness on image classification. In CVPR , 2020. -
  10. [38][38] L. Engstrom, A. Ilyas, S. Santurkar, D. Tsipras, B. Tran, and A. Madry. Adversarial robustness as a prior for learned representations. arXiv preprint arXiv:1906.00945 , 2019. - · arXiv
  11. [39][39] L. Engstrom, B. Tran, D. Tsipras, L. Schmidt, and A. Madry. Exploring the landscape of spatial robustness. In ICML , 2019. -
  12. [42][42] A. Fawzi and P. Frossard. Manitest: Are classifiers really invariant? In BMVC , 2015. -
  13. [46][46] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In ICLR , 2015. -
  14. [48][48] S. Gowal, K. D. Dvijotham, R. Stanforth, R. Bunel, C. Qin, J. Uesato, R. Arandjelovic, T. Mann, and P. Kohli. Scalable verified training for provably robust image classification. In ICCV , 2019. -
  15. [51][51] K. Grosse, N. Papernot, P. Manoharan, M. Backes, and P. McDaniel. Adversarial examples for malware detection. In European symposium on research in computer security , 2017. -
  16. [53][53] C. Guo, M. Rana, M. Cisse, and L. van der Maaten. Countering adversarial images using input transformations. In ICLR , 2018. -
  17. [54][54] C. Guo, J. R. Gardner, Y. You, A. G. Wilson, and K. Q. Weinberger. Simple black-box adversarial attacks. In ICML , 2019. -
  18. [56][56] M. Hein and M. Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In NeurIPS , 2017. -
  19. [57][57] M. Hein, M. Andriushchenko, and J. Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In CVPR , 2019. -
  20. [63][63] J. E. Hu, A. Swaminathan, H. Salman, and G. Yang. Improved image wasserstein attacks and defenses. ICLR Workshop: Towards Trustworthy ML: Rethinking Security and Privacy for ML , 2020. -
  21. [66][66] C. Jin and M. Rinard. Manifold regularization for adversarial robustness. arXiv , 2020. -
  22. [67][67] A. Ju. Generative models as a robust alternative for image classification: Progress and challenges. PhD thesis, UC Berkeley , 2021. -
  23. [76][76] B. Kulynych, R. Overdorf, C. Troncoso, and S. Gürses. Pots: protective optimization technologies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , pages 177-188, 2020. -
  24. [82][82] M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu, and S. Jana. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE S&P , 2019. -
  25. [83][83] L. Li, M. Weber, X. Xu, L. Rimanic, B. Kailkhura, T. Xie, C. Zhang, and B. Li. Tss: Transformationspecific smoothing for robustness certification. In ACM CCS , 2021. -
  26. [84][84] Y. Li, J. Bradshaw, and Y. Sharma. Are generative classifiers more robust to adversarial attacks? In ICML , 2019. -
  27. [88][88] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR , 2018. -
  28. [98][98] M.-I. Nicolae, M. Sinn, M. N. Tran, B. Buesser, A. Rawat, M. Wistuba, V. Zantedeschi, N. Baracaldo, B. Chen, H. Ludwig, I. Molloy, and B. Edwards. Adversarial robustness toolbox v1.2.0. arXiv preprint arXiv:1807.01069 , 2018. - · arXiv
  29. [109][109] M. S. Rahman, M. Imani, N. Mathews, and M. Wright. Mockingbird: Defending against deep-learningbased website fingerprinting attacks with adversarial traces. IEEE Transactions on Information Forensics and Security , 16:1594-1609, 2020. -
  30. [112][112] B. Recht, R. Roelofs, L. Schmidt, and V. Shankar. Do imagenet classifiers generalize to imagenet? In ICML , 2019. -
  31. [113][113] L. Rice, E. Wong, and J. Z. Kolter. Overfitting in adversarially robust deep learning. In ICML , 2020. -
  32. [117][117] H. Salman, A. Ilyas, L. Engstrom, A. Kapoor, and A. Madry. Do adversarially robust imagenet models transfer better? NeurIPS , 2020. -
  33. [121][121] V. Sehwag, S. Wang, P. Mittal, and S. Jana. Hydra: Pruning adversarially robust neural networks. NeurIPS , 2020. -
  34. [122][122] V. Sehwag, S. Wang, P. Mittal, and S. Jana. On pruning adversarially robust neural networks. NeurIPS , 2020. -
  35. [124][124] A. Shafahi, M. Najibi, A. Ghiasi, Z. Xu, J. Dickerson, C. Studer, L. S. Davis, G. Taylor, and T. Goldstein. Adversarial training for free! NeurIPS , 2019. -
  36. [126][126] C. Shi, C. Holtz, and G. Mishne. Online adversarial purification based on self-supervised learning. In International Conference on Learning Representations , 2021. URL https://openreview.net/ forum?id=\_i3ASPp12WS . -
  37. [133][133] K. Sridhar, O. Sokolsky, I. Lee, and J. Weimer. Robust learning via persistency of excitation. arXiv , 2021. -
  38. [134][134] D. Stutz, M. Hein, and B. Schiele. Relating adversarially robust generalization to flat minima. In ICCV , 2021. -
  39. [139][139] G. Tolias, F. Radenovic, and O. Chum. Targeted mismatch adversarial attack: Query with a flower to retrieve the tower. In ICCV , 2019. -
  40. [141][141] F. Tramèr, P. Dupré, G. Rusak, G. Pellegrino, and D. Boneh. Adversarial: Perceptual ad blocking meets adversarial machine learning. In ACM SIGSAC CCS , 2019. -
  41. [144][144] J. Uesato, J.-B. Alayrac, P.-S. Huang, R. Stanforth, A. Fawzi, and P. Kohli. Are labels required for improving adversarial robustness? NeurIPS , 2019. -
  42. [148][148] Y. Wang, D. Zou, J. Yi, J. Bailey, X. Ma, and Q. Gu. Improving adversarial robustness requires revisiting misclassified examples. ICLR , 2020. -
  43. [149][149] T.-W. Weng, H. Zhang, H. Chen, Z. Song, C.-J. Hsieh, D. Boning, I. S. Dhillon, and L. Daniel. Towards fast computation of certified robustness for relu networks. In ICML , 2018. -
  44. [154][154] B. Wu, J. Chen, D. Cai, X. He, and Q. Gu. Do wider neural networks really help adversarial robustness? arXiv , 2020. -
  45. [162][162] Y.-Y. Yang, C. Rashtchian, H. Zhang, R. Salakhutdinov, and K. Chaudhuri. A closer look at accuracy vs. robustness. Advances in Neural Information Processing Systems , 33, 2020. -
  46. [165][165] D. Zhang, T. Zhang, Y. Lu, Z. Zhu, and B. Dong. You only propagate once: Accelerating adversarial training via maximal principle. NeurIPS , 2019. -
  47. [169][169] H. Zhang, H. Chen, C. Xiao, S. Gowal, R. Stanforth, B. Li, D. Boning, and C.-J. Hsieh. Towards stable and efficient training of verifiably robust neural networks. ICLR , 2020. -

Cited by

Sorted by citations

See all 373

Related

Metadata

Authors
Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., & Hein, M.
Affiliations
  • University of Tübingen (DE)
  • École Polytechnique Fédérale de Lausanne (CH)
  • Princeton University (US)
  • Purdue University West Lafayette (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2010.09670
arXiv
2010.09670
OpenAlex
W3093235747
Cited by
284
References
135
Citations by year
2025: 41 · 2024: 86 · 2023: 62 · 2022: 48 · 2021: 46 · 2020: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (1553437, 1704105)
  • Deutsche Forschungsgemeinschaft (206/4-1, EXC2064, TRR 248, EXC 2064_1, 390727645, 389792660)
  • Bundesministerium für Bildung und Forschung (01IS18039A)
  • Office of Naval Research
  • Army Research Office
  • Army Research Laboratory
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords
adversarial robustness, benchmarking, image classification, AutoAttack, adaptive attacks, corruption, distribution shift, calibration, out-of-distribution detection, fairness, privacy leakage, transferability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review