The DOI matches its DataCite record; 51 of the 57 references it cites were found in a registry or an index.
On Adaptive Attacks to Adversarial Example Defenses
Open accessAbstract
Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS---and chosen for illustrative and pedagogical purposes---can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result---showing that a defense was ineffective---this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“even for settings where powerful optimization attacks exist, it is extremely easy to build non-robust defenses that thwart any specific attack, and require an adaptive attack evaluation [58].”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [58]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859
Self-citationBoosting Adversarial Attacks with Momentum
Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., & Li, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00957 · Cited by 3,334
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
Papernot, N., McDaniel, P. D., Wu, X., Jha, S., & Swami, A. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.41 · Cited by 3,251
Adversarial machine learning at scale
Kurakin, A., Goodfellow, I., & Bengio, S. (2016) · arXiv · doi:10.48550/arxiv.1611.01236 · Cited by 2,325
Ensemble Adversarial Training: Attacks and Defenses
Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2017) · arXiv · doi:10.48550/arxiv.1705.07204 · Cited by 2,176
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv · doi:10.48550/arxiv.1802.00420 · Cited by 2,172
Self-citationChen, P., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C. (2017) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3128572.3140448 · Cited by 1,864
Evasion attacks against machine learning at test time
Biggio, B., Corona, I., Maiorca, D., Nelson, B. A., Srndic, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Cited by 1,658
Robustness May Be at Odds with AccuracyPreprint
Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12152 · Cited by 1,513
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326
Countering Adversarial Images using Input TransformationsPreprint
Guo, C., Rana, M., Cissé, M., & van der Maaten, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00117 · Cited by 1,151
On Evaluating Adversarial RobustnessPreprint
Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Mądry, A., & Kurakin, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06705 · Cited by 1,066
Self-citationReliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacksPreprint
Croce, F., & Hein, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01690 · Cited by 1,045
Decision-based adversarial attacks: Reliable attacks against black-box machine learning models
Brendel, W., Rauber, J., & Bethge, M. (2017) · arXiv · doi:10.48550/arxiv.1712.04248 · Cited by 1,018
Self-citationExact matrix completion via convex optimization
Candès, E. J., & Recht, B. (2012) · Communications of the ACM · doi:10.1145/2184319.2184343 · Cited by 1,011
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Carlini, N., & Wagner, D. (2017) · Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3128572.3140444 · Cited by 966
Self-citationBlack-box Adversarial Attacks with Limited Queries and Information
Ilyas, A., Engstrom, L., Athalye, A., & Lin, J. (2018) · arXiv · doi:10.48550/arxiv.1804.08598 · Cited by 806
Technical Report on the CleverHans v2.1.0 Adversarial Examples LibraryPreprint
Papernot, N., Faghri, F., Carlini, N., Goodfellow, I., Feinman, R., Kurakin, A., Xie, C., Sharma, Y., Brown, T., Roy, A., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.00768 · Cited by 766
Self-citationEAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
Chen, P., Sharma, Y., Zhang, H., Yi, J., & Hsieh, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11302 · Cited by 618
Adversarial Risk and the Dangers of Evaluating Against Weak AttacksPreprint
Uesato, J., O’Donoghue, B., van den Oord, A., & Kohli, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05666 · Cited by 467
Matrix estimation by Universal Singular Value Thresholding
Chatterjee, S. (2014) · The Annals of Statistics · doi:10.1214/14-aos1272 · Cited by 464
Synthesizing Robust Adversarial Examples
Athalye, A., Engstrom, L., Ilyas, A., & Kwok, K. (2018) · International Conference on Machine Learning · Cited by 432
Deflecting Adversarial Attacks with Pixel Deflection
Prakash, A., Moran, N., Garber, S., DiLillo, A., & Storer, J. A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00894 · Cited by 342
Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning modelsPreprint
Rauber, J., Brendel, W., & Bethge, M. (2017) · arXiv (Cornell University) · Cited by 325
Self-citationDefensive Distillation is Not Robust to Adversarial ExamplesPreprint
Carlini, N., & Wagner, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.04311 · Cited by 318
Self-citationTowards the first adversarially robust neural network model on MNISTPreprint
Schott, L., Rauber, J., Bethge, M., & Brendel, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09190 · Cited by 310
Self-citationImproving Adversarial Robustness via Promoting Ensemble DiversityPreprint
Pang, T., Xu, K., Du, C., Chen, N., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08846 · Cited by 300
Adversarial Robustness as a Prior for Learned RepresentationsPreprint
Engstrom, L., Ilyas, A., Santurkar, S., Tsipras, D., Tran, B., & Mądry, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00945 · Cited by 236
Adversarial Example Defense: Ensembles of Weak Defenses are not Strong
He, W., Wei, J., Chen, X., Carlini, N., & Song, D. X. (2017) · Cited by 221
Self-citationMitigating Evasion Attacks to Deep Neural Networks via Region-based Classification
Cao, X., & Gong, N. Z. (2017) · Annual Computer Security Applications Conference · doi:10.1145/3134600.3134606 · Cited by 190
On the Robustness of the CVPR 2018 White-Box Adversarial Example DefensesPreprint
Athalye, A., & Carlini, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.03286 · Cited by 151
Self-citationRethinking Softmax Cross-Entropy Loss for Adversarial RobustnessPreprint
Pang, T., Xu, K., Dong, Y., Du, C., Chen, N., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10626 · Cited by 136
The Odds are Odd: A Statistical Test for Detecting Adversarial ExamplesPreprint
Roth, K., Kilcher, Y., & Hofmann, T. (2019) · arXiv (Cornell University) · doi:10.3929/ethz-b-000391937 · Cited by 132
Characterizing Audio Adversarial Examples Using Temporal DependencyPreprint
Yang, Z., Li, B., Chen, P., & Song, D. X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10875 · Cited by 122
Decision Boundary Analysis of Adversarial Examples
He, W., Li, B., & Song, D. X. (2018) · International Conference on Learning Representations · Cited by 99
Mixup Inference: Better Exploiting Mixup to Defend Adversarial AttacksPreprint
Pang, T., Xu, K., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11515 · Cited by 94
Accurate, reliable and fast robustness evaluationPreprint
Brendel, W., Rauber, J., Kümmerer, M., Ustyuzhaninov, I., & Bethge, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01003 · Cited by 87
Self-citationAn Alternative Surrogate Loss for PGD-based Adversarial TestingPreprint
Gowal, S., Uesato, J., Qin, C., Huang, P.-S., Mann, T., & Kohli, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09338 · Cited by 83
A New Defense Against Adversarial Images: Turning a Weakness into a StrengthPreprint
Yu, T., Hu, S., Guo, C., Chao, W., & Weinberger, K. Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07629 · Cited by 66
Sen, S., Ravindran, B., & Raghunathan, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.10162 · Cited by 53
Verma, G., & Swami, A. (2019) · neural information processing systems · Cited by 50
Max-Mahalanobis Linear Discriminant Analysis NetworksPreprint
Pang, T., Du, C., & Zhu, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09308 · Cited by 42
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial RobustnessPreprint
Jacobsen, J.-H., Behrmann, J., Carlini, N., Tramèr, F., & Papernot, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10484 · Cited by 27
Self-citationAre Generative Classifiers More Robust to Adversarial AttacksPreprint
Li, Y., Bradshaw, J., & Sharma, Y. (2018) · arXiv (Cornell University) · Cited by 23
Computer Vision with a Single (Robust) Classifier.Preprint
Santurkar, S., Tsipras, D., Tran, B., Ilyas, A., Engstrom, L., & Mądry, A. (2019) · arXiv (Cornell University) · Cited by 19
Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?Preprint
Carlini, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02322 · Cited by 17
Self-citationRobust Perception through Analysis by Synthesis.Preprint
Schott, L., Rauber, J., Brendel, W., & Bethge, M. (2018) · arXiv (Cornell University) · Cited by 17
Self-citationResisting Adversarial Attacks by k-Winners-Take-All.Preprint
Xiao, C., Zhong, P., & Zheng, C. (2019) · arXiv (Cornell University) · Cited by 13
Are Odds Really Odd? Bypassing Statistical Detection of Adversarial ExamplesPreprint
Hosseini, H., Kannan, S., & Poovendran, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.12138 · Cited by 13
Adversarial Example Detection and Classification With Asymmetrical Adversarial TrainingPreprint
Yin, X., Kolouri, S., & Rohde, G. K. (2019) · arXiv (Cornell University) · Cited by 9
Flagged first, then by citations
6 of the 57 references couldn’t be matched to a known paper.
Research works printed but not linked: 8
- Bafna, M., Murtagh, J., and Vyas, N. Thwarting adversarial examples: An l0-robust sparse fourier transform. In Advances in Neural Information Processing Systems , pp. 10096-10106, 2018.
- Chen, P.-Y., Zhang, H., Sharma, Y., Yi, J., and Hsieh, C.-J. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security , pp. 15-26, 2017.
- Dhillon, G. S., Azizzadenesheli, K., Lipton, Z. C., Bernstein, J. D., Kossaifi, J., Khanna, A., and Anandkumar, A. Stochastic activation pruning for robust adversarial defense. In International Conference on Learning Representations , 2018.
- Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations , 2017.
- Sabour, S., Cao, Y., Faghri, F., and Fleet, D. J. Adversarial manipulation of deep representations. International Conference on Learning Representations , 2016.
- Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR) , 2014.
- Tramèr, F. and Boneh, D. Adversarial training and robustness for multiple perturbations. In Advances in Neural Information Processing Systems , pp. 5858-5868, 2019.
- Yang, Y., Zhang, G., Katabi, D., and Xu, Z. Me-net: Towards effective adversarial robustness with matrix estimation. In International Conference on Machine Learning , pp. 7025-7034, 2019a.
Cited by
Sorted by citations
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682
Adversarial Attacks Against Network Intrusion Detection in IoT Systems
Qiu, H., Dong, T., Zhang, T., Lu, J., Memmi, G., & Meikang Qiu, A. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3048038 · Cited by 334
Advances in Adversarial Attacks and Defenses in Computer Vision: A Survey
Akhtar, N., Mian, A. S., Kardan, N., & Shah, M. (2021) · IEEE Access · doi:10.1109/access.2021.3127960 · Cited by 355
Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain
Rosenberg, I., Shabtai, A., Elovici, Y., & Rokach, L. (2022) · ACM Computing Surveys · doi:10.1145/3453158 · Cited by 295
RobustBench: a standardized adversarial robustness benchmark
Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., & Hein, M. (2020) · arXiv · doi:10.48550/arxiv.2010.09670 · Cited by 284
Metadata
- Authors
- Tramer, F., Carlini, N., Brendel, W., & Madry, A.
- Affiliations
- Google (United States) (US)
- University of Tübingen (DE)
- ORCID
- Carlini, Nicholas
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2002.08347
- OpenAlex
- W3007305010
- Cited by
- 486
- References
- 57
- Citations by year
- 2025: 31 · 2024: 68 · 2023: 75 · 2022: 76 · 2021: 164 · 2020: 69
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- adversarial examples, adaptive attacks, robustness, adversarial robustness, machine learning security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
