On Adaptive Attacks to Adversarial Example Defenses
Open accessAbstract
Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS---and chosen for illustrative and pedagogical purposes---can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result---showing that a defense was ineffective---this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It is extremely easy to build non-robust defenses that thwart any specific attack, and require an adaptive attack evaluation.
“even for settings where powerful optimization attacks exist, it is extremely easy to build non-robust defenses that thwart any specific attack, and require an adaptive attack evaluation [58].”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [58]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · see citing works
Self-citationBoosting Adversarial Attacks with Momentum
Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., & Li, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00957 · Cited by 3,168 · see citing works
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
Papernot, N., McDaniel, P. D., Wu, X., Jha, S., & Swami, A. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.41 · Cited by 3,123 · see citing works
Adversarial machine learning at scale
Kurakin, A., Goodfellow, I., & Bengio, S. (2016) · arXiv · doi:10.48550/arxiv.1611.01236 · Cited by 2,325 · see citing works
Ensemble Adversarial Training: Attacks and Defenses
Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2017) · arXiv · doi:10.48550/arxiv.1705.07204 · Cited by 2,176 · see citing works
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv · doi:10.48550/arxiv.1802.00420 · Cited by 2,172 · see citing works
Self-citationAdversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Carlini, N., & Wagner, D. (2017) · AISec@CCS · doi:10.1145/3128572.3140444 · Cited by 2,079 · see citing works
Self-citationChen, P., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C. (2017) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3128572.3140448 · Cited by 1,782 · see citing works
Evasion attacks against machine learning at test time
Biggio, B., Corona, I., Maiorca, D., Nelson, B. A., Srndic, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Cited by 1,622 · see citing works
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326 · see citing works
Robustness May Be at Odds with AccuracyPreprint
Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12152 · Cited by 1,240 · see citing works
Countering Adversarial Images using Input TransformationsPreprint
Guo, C., Rana, M., Cissé, M., & van der Maaten, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00117 · Cited by 1,151 · see citing works
Decision-based adversarial attacks: Reliable attacks against black-box machine learning models
Brendel, W., Rauber, J., & Bethge, M. (2017) · arXiv · doi:10.48550/arxiv.1712.04248 · Cited by 1,018 · see citing works
Self-citationExact matrix completion via convex optimization
Candès, E. J., & Recht, B. (2012) · Communications of the ACM · doi:10.1145/2184319.2184343 · Cited by 1,011 · see citing works
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacksPreprint
Croce, F., & Hein, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01690 · Cited by 965 · see citing works
Black-box Adversarial Attacks with Limited Queries and Information
Ilyas, A., Engstrom, L., Athalye, A., & Lin, J. (2018) · arXiv · doi:10.48550/arxiv.1804.08598 · Cited by 806 · see citing works
Adversarial Risk and the Dangers of Evaluating Against Weak AttacksPreprint
Uesato, J., O'Donoghue, B., van den Oord, A., & Kohli, P. (2018) · International Conference on Machine Learning · doi:10.48550/arxiv.1802.05666 · Cited by 713 · see citing works
EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
Chen, P., Sharma, Y., Zhang, H., Yi, J., & Hsieh, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11302 · Cited by 594 · see citing works
On Evaluating Adversarial RobustnessPreprint
Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Mądry, A., & Kurakin, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06705 · Cited by 562 · see citing works
Self-citationMatrix estimation by Universal Singular Value Thresholding
Chatterjee, S. (2014) · The Annals of Statistics · doi:10.1214/14-aos1272 · Cited by 464 · see citing works
Synthesizing Robust Adversarial Examples
Athalye, A., Engstrom, L., Ilyas, A., & Kwok, K. (2018) · International Conference on Machine Learning · Cited by 432 · see citing works
Towards the first adversarially robust neural network model on MNISTPreprint
Schott, L., Rauber, J., Bethge, M., & Brendel, W. (2018) · International Conference on Learning Representations · doi:10.48550/arxiv.1805.09190 · Cited by 397 · see citing works
Self-citationTechnical Report on the CleverHans v2.1.0 Adversarial Examples LibraryPreprint
Papernot, N., Faghri, F., Carlini, N., Goodfellow, I., Feinman, R., Kurakin, A., Xie, C., Sharma, Y., Brown, T., Roy, A., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.00768 · Cited by 380 · see citing works
Self-citationDeflecting Adversarial Attacks with Pixel Deflection
Prakash, A., Moran, N., Garber, S., DiLillo, A., & Storer, J. A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00894 · Cited by 328 · see citing works
Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning modelsPreprint
Rauber, J., Brendel, W., & Bethge, M. (2017) · arXiv (Cornell University) · Cited by 305 · see citing works
Self-citationImproving Adversarial Robustness via Promoting Ensemble DiversityPreprint
Pang, T., Xu, K., Du, C., Chen, N., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08846 · Cited by 293 · see citing works
Defensive Distillation is Not Robust to Adversarial ExamplesPreprint
Carlini, N., & Wagner, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.04311 · Cited by 222 · see citing works
Self-citationAdversarial Example Defense: Ensembles of Weak Defenses are not Strong
He, W., Wei, J., Chen, X., Carlini, N., & Song, D. X. (2017) · Cited by 221 · see citing works
Self-citationMitigating Evasion Attacks to Deep Neural Networks via Region-based Classification
Cao, X., & Gong, N. Z. (2017) · Annual Computer Security Applications Conference · doi:10.1145/3134600.3134606 · Cited by 176 · see citing works
On the Robustness of the CVPR 2018 White-Box Adversarial Example DefensesPreprint
Athalye, A., & Carlini, N. (2018) · arXiv.org · doi:10.48550/arxiv.1804.03286 · Cited by 176 · see citing works
Self-citationAdversarial Robustness as a Prior for Learned RepresentationsPreprint
Engstrom, L., Ilyas, A., Santurkar, S., Tsipras, D., Tran, B., & Mądry, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00945 · Cited by 147 · see citing works
Characterizing Audio Adversarial Examples Using Temporal DependencyPreprint
Yang, Z., Li, B., Chen, P., & Song, D. X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10875 · Cited by 122 · see citing works
The Odds are Odd: A Statistical Test for Detecting Adversarial ExamplesPreprint
Roth, K., Kilcher, Y., & Hofmann, T. (2019) · arXiv (Cornell University) · doi:10.3929/ethz-b-000391937 · Cited by 122 · see citing works
Rethinking Softmax Cross-Entropy Loss for Adversarial RobustnessPreprint
Pang, T., Xu, K., Dong, Y., Du, C., Chen, N., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10626 · Cited by 108 · see citing works
Decision Boundary Analysis of Adversarial Examples
He, W., Li, B., & Song, D. X. (2018) · International Conference on Learning Representations · Cited by 99 · see citing works
Accurate, reliable and fast robustness evaluationPreprint
Brendel, W., Rauber, J., Kümmerer, M., Ustyuzhaninov, I., & Bethge, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01003 · Cited by 81 · see citing works
Self-citationA New Defense Against Adversarial Images: Turning a Weakness into a StrengthPreprint
Yu, T., Hu, S., Guo, C., Chao, W., & Weinberger, K. Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07629 · Cited by 66 · see citing works
Mixup Inference: Better Exploiting Mixup to Defend Adversarial AttacksPreprint
Pang, T., Xu, K., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11515 · Cited by 63 · see citing works
Verma, G., & Swami, A. (2019) · neural information processing systems · Cited by 50 · see citing works
An Alternative Surrogate Loss for PGD-based Adversarial TestingPreprint
Gowal, S., Uesato, J., Qin, C., Huang, P.-S., Mann, T., & Kohli, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09338 · Cited by 47 · see citing works
Sen, S., Ravindran, B., & Raghunathan, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.10162 · Cited by 44 · see citing works
Max-Mahalanobis Linear Discriminant Analysis NetworksPreprint
Pang, T., Du, C., & Zhu, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09308 · Cited by 42 · see citing works
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial RobustnessPreprint
Jacobsen, J.-H., Behrmann, J., Carlini, N., Tramèr, F., & Papernot, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10484 · Cited by 27 · see citing works
Self-citationAre Generative Classifiers More Robust to Adversarial AttacksPreprint
Li, Y., Bradshaw, J., & Sharma, Y. (2018) · arXiv (Cornell University) · Cited by 23 · see citing works
Computer Vision with a Single (Robust) Classifier.Preprint
Santurkar, S., Tsipras, D., Tran, B., Ilyas, A., Engstrom, L., & Mądry, A. (2019) · arXiv (Cornell University) · Cited by 19 · see citing works
Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?Preprint
Carlini, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02322 · Cited by 17 · see citing works
Self-citationRobust Perception through Analysis by Synthesis.Preprint
Schott, L., Rauber, J., Brendel, W., & Bethge, M. (2018) · arXiv (Cornell University) · Cited by 14 · see citing works
Self-citationAre Odds Really Odd? Bypassing Statistical Detection of Adversarial ExamplesPreprint
Hosseini, H., Kannan, S., & Poovendran, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.12138 · Cited by 13 · see citing works
Resisting Adversarial Attacks by k-Winners-Take-All.Preprint
Xiao, C., Zhong, P., & Zheng, C. (2019) · arXiv (Cornell University) · Cited by 10 · see citing works
Adversarial Example Detection and Classification With Asymmetrical Adversarial TrainingPreprint
Yin, X., Kolouri, S., & Rohde, G. K. (2019) · arXiv (Cornell University) · Cited by 9 · see citing works
Flagged first, then by citations
6 of the 57 references couldn’t be matched to a known paper.
Research works printed but not linked: 8
- Bafna, M., Murtagh, J., and Vyas, N. Thwarting adversarial examples: An l0-robust sparse fourier transform. In Advances in Neural Information Processing Systems , pp. 10096-10106, 2018.
- Chen, P.-Y., Zhang, H., Sharma, Y., Yi, J., and Hsieh, C.-J. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security , pp. 15-26, 2017.
- Dhillon, G. S., Azizzadenesheli, K., Lipton, Z. C., Bernstein, J. D., Kossaifi, J., Khanna, A., and Anandkumar, A. Stochastic activation pruning for robust adversarial defense. In International Conference on Learning Representations , 2018.
- Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations , 2017.
- Sabour, S., Cao, Y., Faghri, F., and Fleet, D. J. Adversarial manipulation of deep representations. International Conference on Learning Representations , 2016.
- Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR) , 2014.
- Tramèr, F. and Boneh, D. Adversarial training and robustness for multiple perturbations. In Advances in Neural Information Processing Systems , pp. 5858-5868, 2019.
- Yang, Y., Zhang, G., Katabi, D., and Xu, Z. Me-net: Towards effective adversarial robustness with matrix estimation. In International Conference on Machine Learning , pp. 7025-7034, 2019a.
Cited by
Sorted by citations
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682 · see citing works
Adversarial Attacks Against Network Intrusion Detection in IoT Systems
Qiu, H., Dong, T., Zhang, T., Lu, J., Memmi, G., & Meikang Qiu, A. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3048038 · Cited by 334 · see citing works
Advances in Adversarial Attacks and Defenses in Computer Vision: A Survey
Akhtar, N., Mian, A. S., Kardan, N., & Shah, M. (2021) · IEEE Access · doi:10.1109/access.2021.3127960 · Cited by 319 · see citing works
Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain
Rosenberg, I., Shabtai, A., Elovici, Y., & Rokach, L. (2022) · ACM Computing Surveys · doi:10.1145/3453158 · Cited by 295 · see citing works
RobustBench: a standardized adversarial robustness benchmark
Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., & Hein, M. (2020) · arXiv · doi:10.48550/arxiv.2010.09670 · Cited by 284 · see citing works
Privacy and Security Issues in Deep Learning: A Survey
Liu, X., Xie, L., Wang, Y., Zou, J., Xiong, J., Ying, Z., & Vasilakos, A. V. (2020) · IEEE Access · doi:10.1109/access.2020.3045078 · Cited by 285 · see citing works
On the Robustness of Vision Transformers to Adversarial Examples
Mahmood, K., Mahmood, R., & van Dijk, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00774 · Cited by 217 · see citing works
Toward the third generation artificial intelligence
Zhang, B., Zhu, J., & Su, H. (2023) · Science China Information Sciences · doi:10.1007/s11432-021-3449-x · Cited by 217 · see citing works
Adversarial Machine Learning in Image Classification: A Survey Toward the Defender’s Perspective
Machado, G. R., Silva, E., & Goldschmidt, R. R. (2021) · ACM Computing Surveys · doi:10.1145/3485133 · Cited by 186 · see citing works
Evading Deepfake-Image Detectors with White- and Black-Box Attacks
Carlini, N., & Farid, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00337 · Cited by 170 · see citing works
Self-citationAdversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 193 · see citing works
Bag of Tricks for Adversarial TrainingPreprint
Pang, T., Yang, X. H., Dong, Y., Su, H., & Zhu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00467 · Cited by 170 · see citing works
Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability
Xiong, Y., Lin, J., Zhang, M., Hopcroft, J. E., & He, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01456 · Cited by 152 · see citing works
SoK: Certified Robustness for Deep Neural Networks
Li, L., Xie, T., & Li, B. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179303 · Cited by 143 · see citing works
Simulating a Primary Visual Cortex at the Front of CNNs Improves Robustness to Image PerturbationsPreprint
Dapello, J., Marques, T., Schrimpf, M., Geiger, F., Cox, D., & DiCarlo, J. J. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.16.154542 · Cited by 137 · see citing works
Poison Ink: Robust and Invisible Backdoor Attack
Zhang, J., Chen, D., Huang, Q., Liao, J., Zhang, W., Feng, H., Hua, G., & Yu, N. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3201472 · Cited by 121 · see citing works
Shen, J., & Robertson, N. M. (2020) · Information Sciences · doi:10.1016/j.ins.2020.11.026 · Cited by 113 · see citing works
Improving Robustness using Generated DataPreprint
Gowal, S., Rebuffi, S.-A., Wiles, O., Stimberg, F., Calian, D. A., & Mann, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09468 · Cited by 114 · see citing works
A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking
Liu, C., Dong, Y., Xiang, W., Yang, X. H., Su, H., Zhu, J., Chen, Y., He, Y., Xue, H., & Zheng, S. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02196-3 · Cited by 117 · see citing works
Unsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95 · see citing works
Self-citationImproving Adversarial Robustness via Channel-wise Activation SuppressingPreprint
Bai, Y., Zeng, Y., Jiang, Y., Xia, S., Ma, X., & Wang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.08307 · Cited by 89 · see citing works
Adversarial Training Against Location-Optimized Adversarial Patches
Rao, S., Stutz, D., & Schiele, B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-68238-5_32 · Cited by 85 · see citing works
Petersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 83 · see citing works
Adversarial Distributional Training for Robust Deep LearningPreprint
Dong, Y., Deng, Z., Pang, T., Su, H., & Zhu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.05999 · Cited by 82 · see citing works
Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges
Dong, J., Chen, J., Xie, X., Lai, J., & Chen, H. (2024) · ACM Computing Surveys · doi:10.1145/3702638 · Cited by 88 · see citing works
DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of EnsemblesPreprint
Yang, H., Zhang, J., Dong, H., Inkawhich, N., Gardner, A., Touchet, A., Wilkes, W., Berry, H., & Li, H. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14720 · Cited by 73 · see citing works
On Success and Simplicity: A Second Look at Transferable Targeted AttacksPreprint
Zhao, Z., Liu, Z., & Larson, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.11207 · Cited by 69 · see citing works
Defense Against Adversarial Attacks by Reconstructing Images
Zhang, S., Gao, H., & Rao, Q. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3092582 · Cited by 69 · see citing works
Block-Wise Image Transformation With Secret Key for Adversarially Robust Defense
MaungMaung, A., & Kiya, H. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3062977 · Cited by 66 · see citing works
Exploring misclassifications of robust neural networks to enhance adversarial attacks
Schwinn, L., Raab, R., Nguyen, A., Zanca, D., & Eskofier, B. M. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04532-5 · Cited by 70 · see citing works
Adversarial purification with Score-based generative modelsPreprint
Yoon, J., Hwang, S. J., & Lee, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06041 · Cited by 65 · see citing works
Towards Practical Certifiable Patch Defense with Vision Transformer
Chen, Z., Li, B., Xu, J., Wu, S., Ding, S., & Zhang, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01472 · Cited by 64 · see citing works
GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization
Lee, S., Kim, H., & Lee, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3169217 · Cited by 63 · see citing works
TnT Attacks! Universal Naturalistic Adversarial Patches Against Deep Neural Network Systems
Doan, B. G., Xue, M., Ma, S., Abbasnejad, E., & Ranasinghe, D. C. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3198857 · Cited by 62 · see citing works
Zhang, B., Tondi, B., & Barni, M. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2020.102988 · Cited by 61 · see citing works
Nanda, V., Dooley, S. W., Singla, S., Feizi, S., & Dickerson, J. P. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445910 · Cited by 58 · see citing works
Dynamic Time Warping Based Adversarial Framework for Time-Series Domain
Belkhouja, T., Yan, Y., & Doppa, J. R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3224754 · Cited by 54 · see citing works
Joshi, S., Villalba, J., Zelasko, P., Moro-Velazquez, L., & Dehak, N. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3116438 · Cited by 53 · see citing works
A Survey on Cyber Situation-awareness Systems: Framework, Techniques, and Insights
Alavizadeh, H., Jang‐Jaccard, J., Enoch, S. Y., Al-Sahaf, H., Welch, I. S., Camtepe, S. A., & Kim, D. S. (2022) · ACM Computing Surveys · doi:10.1145/3530809 · Cited by 62 · see citing works
FENCE: Feasible Evasion Attacks on Neural Networks in Constrained Environments
Chernikova, A., & Oprea, A. (2022) · ACM Transactions on Privacy and Security · doi:10.1145/3544746 · Cited by 52 · see citing works
Practical Evaluation of Adversarial Robustness via Adaptive Auto Attack
Ye, L., Cheng, Y., Gao, L., Liu, X., Zhang, Q., & Song, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01468 · Cited by 49 · see citing works
SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain
Harder, P., Pfreundt, F., Keuper, M., & Keuper, J. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533442 · Cited by 48 · see citing works
Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognition
Chen, G., Zhao, Z., Song, F., Chen, S., Fan, L., Wang, F., & Wang, J. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3220673 · Cited by 48 · see citing works
Defense Against Adversarial Attacks Using Topology Aligning Adversarial Training
Kuang, H., Liu, H., Lin, X., & Ji, R. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3359820 · Cited by 46 · see citing works
Machine Learning Security in Industry: A Quantitative Survey
Grosse, K., Bieringer, L., Besold, T. R., Biggio, B., & Krombholz, K. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3251842 · Cited by 50 · see citing works
Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?Preprint
Sehwag, V., Mahloujifar, S., Handina, T., Dai, S., Xiang, C., Chiang, M., & Mittal, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09425 · Cited by 60 · see citing works
Robust Single Image Reflection Removal Against Adversarial Attacks
Song, Z., Zhang, Z., Zhang, K., Luo, W., Fan, Z., Ren, W., & Lu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02365 · Cited by 44 · see citing works
Bugs in our pockets: the risks of client-side scanning
Abelson, H., Anderson, R., Bellovin, S. M., Benaloh, J., Blaze, M., Callas⋆, J., Diffie, W., Landau, S. M., Neumann, P. G., Rivest, R. L., et al. (2024) · Journal of Cybersecurity · doi:10.1093/cybsec/tyad020 · Cited by 43 · see citing works
Robust Deep Reinforcement Learning through Adversarial LossPreprint
Oikarinen, T., Zhang, W., Megretski, A., Daniel, L., & Weng, T.-W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01976 · Cited by 46 · see citing works
Feature Separation and Recalibration for Adversarial Robustness
Kim, W. J., Cho, Y., Jung, J., & Yoon, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00791 · Cited by 42 · see citing works
Certified Patch Robustness via Smoothed Vision Transformers
Salman, H., Jain, S., Wong, E., & Mądry, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01471 · Cited by 41 · see citing works
Fast Minimum-norm Adversarial Attacks through Adaptive Norm ConstraintsPreprint
Pintor, M., Roli, F., Brendel, W., & Biggio, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.12827 · Cited by 44 · see citing works
Self-citationAn Efficient Preprocessing-Based Approach to Mitigate Advanced Adversarial Attacks
Qiu, H., Zeng, Y., Zheng, Q., Guo, S., Zhang, T., & Li, H. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3076826 · Cited by 39 · see citing works
One Prompt Word is Enough to Boost Adversarial Robustness for Pre-Trained Vision-Language Models
Li, L., Guan, H., Qiu, J., & Spratling, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02304 · Cited by 39 · see citing works
Low Curvature Activations Reduce Overfitting in Adversarial Training
Singla, V., Singla, S., Feizi, S., & Jacobs, D. W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01611 · Cited by 37 · see citing works
WaveGuard: Understanding and Mitigating Audio Adversarial ExamplesPreprint
Hussain, S., Neekhara, P., Dubnov, S., McAuley, J., & Koushanfar, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03344 · Cited by 41 · see citing works
Adversarial Attacks are Reversible with Natural Supervision
Mao, C., Chiquier, M., Wang, H., Yang, J., & Vondrick, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00070 · Cited by 36 · see citing works
AI and the Future of Cyber Competition
Hoffman, W. (2021) · doi:10.51593/2020ca007 · Cited by 35 · see citing works
Li, X., Wang, Z., Zhang, B., Sun, F., & Hu, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3237935 · Cited by 35 · see citing works
PatchCleanser: Certifiably Robust Defense against Adversarial Patches for Any Image ClassifierPreprint
Xiang, C., Mahloujifar, S., & Mittal, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09135 · Cited by 34 · see citing works
Efficient Exact Verification of Binarized Neural Networks
Jia, K., & Rinard, M. (2020) · arXiv · doi:10.48550/arxiv.2005.03597 · Cited by 33 · see citing works
Adversarial Attacks in Machine Learning: Key Insights and Defense Approaches
Khaleel, Y. L., Habeeb, M. A., & Alnabulsi, H. (2024) · Applied Data Science and Analysis · doi:10.58496/adsa/2024/011 · Cited by 32 · see citing works
Do gradient-based explanations tell anything about adversarial robustness to android malware?
Melis, M., Scalas, M., Demontis, A., Maiorca, D., Biggio, B., Giacinto, G., & Roli, F. (2021) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-021-01393-7 · Cited by 30 · see citing works
Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based ModelsPreprint
Hill, M., Mitchell, J., & Zhu, S. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.13525 · Cited by 32 · see citing works
Three Challenges to Secure AI Systems in the Context of AI Regulations
Hamon, R., Junklewitz, H., Garrido, J. S., & Sánchez, I. (2024) · IEEE Access · doi:10.1109/access.2024.3391021 · Cited by 34 · see citing works
Learning defense transformations for counterattacking adversarial examples
Li, J., Zhang, S., Cao, J., & Tan, M. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.03.008 · Cited by 31 · see citing works
Exploiting Verified Neural Networks via Floating Point Numerical Error
Jia, K., & Rinard, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88806-0_9 · Cited by 30 · see citing works
Vulnerability of Person Re-Identification Models to Metric Adversarial Attacks
Bouniot, Q., Audigier, R., & Loesch, A. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00405 · Cited by 29 · see citing works
Random Noise Defense Against Query-Based Black-Box AttacksPreprint
Qin, Z., Fan, Y., Hongyuan Zha, B., & Wu, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11470 · Cited by 30 · see citing works
How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?Preprint
Dong, X., Tuan, L. A., Lin, M., Yan, S., & Zhang, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.11668 · Cited by 31 · see citing works
Randomization matters. How to defend against strong adversarial attacksPreprint
Pinot, R., Ettedgui, R., Rizk, G., Chevaleyre, Y., & Jamal Atif, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11565 · Cited by 29 · see citing works
PointCAT: Contrastive Adversarial Training for Robust Point Cloud Recognition
Huang, Q., Dong, X., Chen, D., Zhou, H., Zhang, W., Zhang, K., Hua, G., Cheng, Y., & Yu, N. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3372456 · Cited by 28 · see citing works
The Best Defense is a Good Offense: Adversarial Augmentation Against Adversarial Attacks
Frosio, I., & Kautz, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00396 · Cited by 28 · see citing works
Adversarially Robust One-class Novelty Detection
Lo, S.-Y., Oza, P., & Patel, V. M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3189638 · Cited by 28 · see citing works
CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature SelectionPreprint
Yan, H., Zhang, J., Niu, G., Feng, J., Tan, V. Y. F., & Sugiyama, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05311 · Cited by 27 · see citing works
Adversarial Robustness through the Lens of Causality.Preprint
Zhang, Y., Gong, M., Liu, T., Niu, G., Tian, X., Han, B., Schölkopf, B., & Zhang, K. (2021) · arXiv (Cornell University) · Cited by 26 · see citing works
MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-Based Malware Detection
Rashid, A., & Such, J. M. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3293959 · Cited by 26 · see citing works
Ma, L., & Liang, L. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105345 · Cited by 26 · see citing works
“That Is a Suspicious Reaction!”: Interpreting Logits Variation to Detect NLP Adversarial Attacks
Mosca, E., Agarwal, S., Rando, J., & Groh, G. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.538 · Cited by 26 · see citing works
Mitigating Accuracy-Robustness Trade-Off via Balanced Multi-Teacher Adversarial Distillation
Zhao, S., Wang, X., & Wei, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3416308 · Cited by 28 · see citing works
Ptolemy: Architecture Support for Robust Deep Learning
Gan, Y., Qiu, Y., Leng, J., Guo, M., & Zhu, Y. (2020) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro50266.2020.00031 · Cited by 25 · see citing works
Disguising Attacks with Explanation-Aware Backdoors
Noppel, M., Peter, L., & Wressnegger, C. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179308 · Cited by 29 · see citing works
Liu, Z., & Larson, M. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449891 · Cited by 24 · see citing works
Universal 3-Dimensional Perturbations for Black-Box Attacks on Video Recognition Systems
Xie, S., Wang, H., Kong, Y., & Hong, Y. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833776 · Cited by 23 · see citing works
A Hierarchical Feature Constraint to Camouflage Medical Adversarial Attacks
Yao, Q., He, Z., Lin, Y., Ma, K., Zheng, Y., & Zhou, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_4 · Cited by 25 · see citing works
Defending Against Multiple and Unforeseen Adversarial Videos
Lo, S.-Y., & Patel, V. M. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3137648 · Cited by 24 · see citing works
Towards Understanding and Improving Adversarial Robustness of Vision Transformers
Jain, S., & Dutta, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02336 · Cited by 22 · see citing works
On Lipschitz Regularization of Convolutional Layers using Toeplitz Matrix Theory
Araujo, A., Négrevergne, B., Chevaleyre, Y., & Jamal Atif, C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16824 · Cited by 1 · see citing works
Privacy Leakage of Adversarial Training Models in Federated Learning Systems
Zhang, J., Chen, Y., & Li, H. H. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00021 · Cited by 22 · see citing works
Adversarial training with distribution normalization and margin balance
Cheng, Z., Zhu, F., Zhang, X.-Y., & Liu, C. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.109182 · Cited by 22 · see citing works
Detecting Adversarial Examples Is (Nearly) As Hard As Classifying ThemPreprint
Tramèr, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.11630 · Cited by 22 · see citing works
FACESEC: A Fine-grained Robustness Evaluation Framework for Face Recognition Systems
Tong, L., Chen, Z., Ni, J., Cheng, W. D., Song, D., Chen, H., & Vorobeychik, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01305 · Cited by 21 · see citing works
The Art of Defense: Letting Networks Fool the Attacker
Zhang, J., Dong, Y., Kuang, M., Liu, B., Ouyang, B., Zhu, J., Wang, H., & Meng, Y. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3278458 · Cited by 21 · see citing works
Beware the Black-Box: On the Robustness of Recent Defenses to Adversarial Examples
Mahmood, K., Gurevin, D., van Dijk, M., & Nguyen, P. H. (2021) · Entropy · doi:10.3390/e23101359 · Cited by 21 · see citing works
Towards Understanding Fast Adversarial TrainingPreprint
Li, B., Wang, S., Jana, S., & Carin, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03089 · Cited by 27 · see citing works
RP2K: A Large-Scale Retail Product Dataset for Fine-Grained Image ClassificationPreprint
Peng, J., Xiao, C., Wei, X., & Li, Y. (2020) · arXiv.org · doi:10.48550/arxiv.2006.12634 · Cited by 66 · see citing works
MimicDiffusion: Purifying Adversarial Perturbation via Mimicking Clean Diffusion Model
Song, K., Lai, H., Pan, Y., & Yin, J. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02329 · Cited by 21 · see citing works
Combating Adversaries with Anti-adversaries
Alfarra, M., Perez, J. C., Thabet, A., Bibi, A., Torr, P. H. S., & Ghanem, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i6.20545 · Cited by 20 · see citing works
kherchouche, A., Fezza, S. A., & Hamidouche, W. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06330-x · Cited by 21 · see citing works
Voting for the Right Answer: Adversarial Defense for Speaker Verification
Wu, H., Zhang, Y., Wu, Z., Wang, D., & Lee, H.-Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1452 · Cited by 22 · see citing works
- Authors
- Tramer, F., Carlini, N., Brendel, W., & Madry, A.
- Affiliations
- Google (United States) (US)
- University of Tübingen (DE)
- ORCID
- Carlini, Nicholas
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2002.08347
- OpenAlex
- W3007305010
- Cited by
- 486
- References
- 57
- Citations by year
- 2025: 31 · 2024: 68 · 2023: 75 · 2022: 76 · 2021: 164 · 2020: 69 · 2019: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- adversarial examples, adaptive attacks, robustness, adversarial robustness, machine learning security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
