CoChat
Sign Up
Cited by 152Adversarial Robustness in Machine LearningSecurity and Verification in ComputingDigital Media Forensic Detection

DEEPSEC: A Uniform Platform for Security Analysis of Deep Learning Model

Open access
Ling, Xiang, Ji, Shouling, Zou, Jiaxu, Wang, Jiannan, Wu, Chunming, Li, Bo, Wang, Ting·2019·2019 IEEE Symposium on Security and Privacy (SP)
DOI 10.1109/sp.2019.00023
Open this paper in CoChat — free accountOpen PDF

Abstract

Deep learning (DL) models are inherently vulnerable to adversarial examples - maliciously crafted inputs to trigger target DL models to misbehave - which significantly hinders the application of DL in security-sensitive domains. Intensive research on adversarial learning has led to an arms race between adversaries and defenders. Such plethora of emerging attacks and defenses raise many questions: Which attacks are more evasive, preprocessing-proof, or transferable? Which defenses are more effective, utility-preserving, or general? Are ensembles of multiple defenses more robust than individuals? Yet, due to the lack of platforms for comprehensive evaluation on adversarial attacks and defenses, these critical questions remain largely unsolved. In this paper, we present the design, implementation, and evaluation of DEEPSEC, a uniform platform that aims to bridge this gap. In its current implementation, DEEPSEC incorporates 16 state-of-the-art attacks with 10 attack utility metrics, and 13 state-of-the-art defenses with 5 defensive utility metrics. To our best knowledge, DEEPSEC is the first platform that enables researchers and practitioners to (i) measure the vulnerability of DL models, (ii) evaluate the effectiveness of various attacks/defenses, and (iii) conduct comparative studies on attacks/defenses in a comprehensive and informative manner. Leveraging DEEPSEC, we systematically evaluate the existing adversarial attack and defense methods, and draw a set of key findings, which demonstrate DEEPSEC's rich functionality, such as (1) the trade-off between misclassification and imperceptibility is empirically confirmed; (2) most defenses that claim to be universally applicable can only defend against limited types of attacks under restricted settings; (3) it is not necessary that adversarial examples with higher perturbation magnitude are easier to be detected; (4) the ensemble of multiple defenses cannot improve the overall defense capability, but can improve the lower bound of the defense effectiveness of individuals. Extensive analysis on DEEPSEC demonstrates its capabilities and advantages as a benchmark platform which can benefit future adversarial learning research.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/sp.2019.00023

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Advanced detection methods mainly rely on DNN-based binary classification models, which are known to be vulnerable to adversarial examples.

    “the advanced detection methods mainly rely on DNN-based binary classiication models, which are known to be vulnerable to adversarial examples”

    From DEEPFAKER: A Unified Evaluation Platform for Facial Deepfake and Detection Models (2024) · cites this paper as [38]

    The Introduction states that existing deep-learning models are vulnerable to adversarial examples. The “Detection-only Defenses” section describes several detection methods, but does not say that advanced detection methods mainly rely on DNN-based binary classifiers. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 42

  1. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al., "Mastering the game of go with deep neural networks and tree search," nature, vol. 529, no. 7587, p. 484, 2016.
  2. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, "Intriguing properties of neural networks," in ICLR, 2014.
  3. M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang et al., "End to end learning for self-driving cars," arXiv:1604.07316, 2016. · arXiv
  4. O. M. Parkhi, A. Vedaldi, A. Zisserman et al., "Deep face recognition." in BMVC, 2015.
  5. P. Rajpurkar, J. Irvin, A. Bagul, D. Ding, T. Duan, H. Mehta, B. Yang, K. Zhu, D. Laird, R. L. Ball et al., "Mura dataset: Towards radiologist-level abnormality detection in musculoskeletal radiographs," arXiv:1712.06957, 2017. · arXiv
  6. N. Papernot, P. McDaniel, A. Sinha, and M. Wellman, "Sok: Towards the science of security and privacy in machine learning," arXiv:1611.03814, 2016. · arXiv
  7. X. Yuan, P. He, Q. Zhu, R. R. Bhat, and X. Li, "Adversarial examples: Attacks and defenses for deep learning," arXiv:1712.07107, 2017. · arXiv
  8. A. Athalye, N. Carlini, and D. Wagner, "Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples," in ICML, 2018.
  9. A. S. Ross and F. Doshi-Velez, "Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients," in AAAI, 2018.
  10. N. Papernot, F. Faghri, N. Carlini, I. Goodfellow, R. Feinman, A. Ku- rakin, C. Xie, Y. Sharma, T. Brown, A. Roy, A. Matyasko, V. Behzadan, K. Hambardzumyan, Z. Zhang, Y.-L. Juang, Z. Li, R. Sheatsley, A. Garg, J. Uesato, W. Gierke, Y. Dong, D. Berthelot, P. Hendricks, J. Rauber, and R. Long, "cleverhans v2.1.0: an adversarial machine learning library," arXiv:1610.00768, 2016. · arXiv
  11. F. Tramèr, A. Kurakin, N. Papernot, D. Boneh, and P. McDaniel, "Ensemble adversarial training: Attacks and defenses," in ICLR, 2018.
  12. A. Kurakin, I. Goodfellow, and S. Bengio, "Adversarial examples in the physical world," in ICLR, 2017.
  13. Y. Dong, F. Liao, T. Pang, H. Su, X. Hu, J. Li, and J. Zhu, "Boosting adversarial attacks with momentum," arXiv:1710.06081, 2017. · arXiv
  14. S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, "Deepfool: A simple and accurate method to fool deep neural networks," in CVPR, 2016.
  15. S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, "Univer- sal adversarial perturbations," in CVPR, 2017.
  16. W. He, B. Li, and D. Song, "Decision boundary analysis of adversarial examples," in ICLR, 2018.
  17. P. Chen, Y. Sharma, H. Zhang, J. Yi, and C. Hsieh, "EAD: elastic- net attacks to deep neural networks via adversarial examples," in AAAI, 2018.
  18. C. Guo, M. Rana, M. Cissé, and L. van der Maaten, "Countering adversarial images using input transformations," in ICLR, 2018.
  19. C. Xie, J. Wang, Z. Zhang, Z. Ren, and A. Yuille, "Mitigating adversarial effects through randomization," in ICLR, 2018.
  20. Y. Song, T. Kim, S. Nowozin, S. Ermon, and N. Kushman, "Pixelde- fend: Leveraging generative models to understand and defend against adversarial examples," in ICLR, 2018.
  21. J. Buckman, A. Roy, C. Raffel, and I. Goodfellow, "Thermometer encoding: One hot way to resist adversarial examples," in ICLR, 2018.
  22. X. Ma, B. Li, Y. Wang, S. M. Erfani, S. Wijewickrema, M. E. Houle, G. Schoenebeck, D. Song, and J. Bailey, "Characterizing adversarial subspaces using local intrinsic dimensionality," in ICLR, 2018.
  23. W. Xu, D. Evans, and Y. Qi, "Feature squeezing: Detecting adversarial examples in deep neural networks," in NDSS, 2018.
  24. I. Evtimov, K. Eykholt, E. Fernandes, T. Kohno, B. Li, A. Prakash, A. Rahmati, and D. Song, "Robust physical-world attacks on deep learning models," arXiv:1707.08945, 2017. · arXiv
  25. A. Liu, W. Lin, M. Paul, C. Deng, and F. Zhang, "Just noticeable difference for images with decomposition model for separating edge and textured regions," IEEE Transactions on Circuits and Systems for Video Technology, 2010.
  26. B. Luo, Y. Liu, L. Wei, and Q. Xu, "Towards imperceptible and robust adversarial example attacks against neural networks," in AAAI, 2018.
  27. A. v. d. Oord, N. Kalchbrenner, and K. Kavukcuoglu, "Pixel recurrent neural networks," arXiv:1601.06759, 2016. · arXiv
  28. N. Akhtar and A. Mian, "Threat of adversarial attacks on deep learning in computer vision: A survey," arXiv:1801.00553, 2018. · arXiv
  29. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, 1998.
  30. J. Alakuijala, R. Obryk, O. Stoliarchuk, Z. Szabadka, L. Vande- venne, and J. Wassenberg, "Guetzli: Perceptually guided jpeg encoder," arXiv:1703.04421, 2017. · arXiv
  31. N. Papernot, P. McDaniel, and I. Goodfellow, "Transferability in ma- chine learning: from phenomena to black-box attacks using adversarial samples," arXiv:1605.07277, 2016. · arXiv
  32. Y. Liu, X. Chen, C. Liu, and D. Song, "Delving into transferable adversarial examples and black-box attacks," arXiv:1611.02770, 2016. · arXiv
  33. L. Rokach, "Taxonomy for characterizing ensemble methods in clas- sification tasks: A review and annotated bibliography," Computational Statistics & Data Analysis, 2009.
  34. T. Strauss, M. Hanselmann, A. Junginger, and H. Ulmer, "Ensemble methods as a defense to adversarial perturbations against deep neural networks," arXiv:1709.03423, 2017. · arXiv
  35. A. Bagnall, R. Bunescu, and G. Stewart, "Training ensembles to detect adversarial examples," arXiv:1712.04006, 2017. · arXiv
  36. Y. Song, R. Shu, N. Kushman, and S. Ermon, "Generative adversarial examples," arXiv:1805.07894, 2018. · arXiv
  37. C. Xiao, B. Li, J.-Y. Zhu, W. He, M. Liu, and D. Song, "Generating adversarial examples with adversarial networks," arXiv:1801.02610, 2018. · arXiv
  38. J. Rauber, W. Brendel, and M. Bethge, "Foolbox v0. 8.0: A python toolbox to benchmark the robustness of machine learning models," arXiv:1707.04131, 2017. · arXiv
  39. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al., "Tensorflow: A system for large-scale machine learning." in OSDI, 2016.
  40. R. Al-Rfou, G. Alain, A. Almahairi, C. Angermueller, D. Bahdanau, N. Ballas, F. Bastien, J. Bayer, A. Belikov, A. Belopolsky et al., "Theano: A python framework for fast computation of mathematical expressions," arXiv:1605.02688, 2016. · arXiv
  41. T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, and Z. Zhang, "Mxnet: A flexible and efficient machine learn- ing library for heterogeneous distributed systems," arXiv:1512.01274, 2015. · arXiv
  42. G. Huang, Z. Liu, K. Q. Weinberger, and L. van der Maaten, "Densely connected convolutional networks," in CVPR, 2017.

Cited by

Sorted by citations

Authors
Ling, X., Ji, S., Zou, J., Wang, J., Wu, C., Li, B., & Wang, T.
Affiliations
  • Zhejiang University (CN)
  • Alibaba Group (China) (CN)
  • University of Illinois Urbana-Champaign (US)
  • Lehigh University (US)
ORCID
Venue
2019 IEEE Symposium on Security and Privacy (SP)
Publisher
IEEE
Published
2019-05
Pages
673-690
ISSN
1081-6011 (ISSN-L)
DOI
10.1109/sp.2019.00023
OpenAlex
W2902543210
Cited by
152
References
10
Citations by year
2026: 8 · 2025: 12 · 2024: 18 · 2023: 21 · 2022: 21 · 2021: 30 · 2020: 30 · 2019: 11 · 2018: 1
FWCI
8.80
Type
conference paper
Language
en
Open access
Licence
Locations
IEEE Symposium on Security and Privacy (S&P) (publishedVersion)
Funders
  • National Science Foundation (1566526, 1718787)
  • National Natural Science Foundation of China (2016YFB0800201, 61772466)
  • National Key Research and Development Program of China (2016YFB0800201, 2016YFB0800102)
  • Fundamental Research Funds for the Central Universities (2016XZZX001‐04)
  • Natural Science Foundation of Zhejiang Province (2016YFB0800201, R19F020013)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
adversarial attacks, adversarial defense, deep learning security, adversarial examples, benchmark platform, transferability, perturbation magnitude

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review