CoChat
Sign Up
Cited by 21Adversarial Robustness in Machine LearningDigital Media Forensic DetectionSecurity and Verification in Computing

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Minimum Uncertainty Based Detection of Adversaries in Deep Neural Networks

Open access
Sheikholeslami, Fatemeh, Jain, Swayambhoo, Giannakis, Georgios B.·2020·2020 Information Theory and Applications Workshop (ITA)
DOI 10.1109/ita50056.2020.9244964
Open this paper in CoChat — free accountOpen PDF

Abstract

Despite their unprecedented performance in various domains, utilization of Deep Neural Networks (DNNs) in safety-critical environments is severely limited in the presence of even small adversarial perturbations. The present work develops a randomized approach to detecting such perturbations based on minimum uncertainty metrics that rely on sampling at the hidden layers during the DNN inference stage. Inspired by Bayesian approaches to uncertainty estimation, the sampling probabilities are designed for effective detection of the adversarially corrupted inputs. Being modular, the novel detector of adversaries can be conveniently employed by any pre-trained DNN at no extra training overhead. Selecting which units to sample per hidden layer entails quantifying the amount of DNN output uncertainty, where the overall uncertainty is expressed in terms of its layer-wise components - what also promotes scalability. Sampling probabilities are then sought by minimizing uncertainty measures layer-by-layer, leading to a novel convex optimization problem that admits an exact solver with superlinear convergence rate. By simplifying the objective function, low-complexity approximate solvers are also developed. In addition to valuable insights, these approximations link the novel approach with state-of-the-art randomized adversarial detectors. The effectiveness of the novel detectors in the context of competing alternatives is highlighted through extensive tests for various types of adversarial attacks with variable levels of strength.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/ita50056.2020.9244964

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • MultiSWAG can entirely alleviate double descent behaviour.

    “They also showed that it can entirely alleviate double descent behaviour.”

    From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [216]

    The cited paper’s Section 3.2 discusses minimizing predictive variance for adversarial-input detection, and Section 6.2 compares detection methods on attack datasets; page numbers are unavailable in our copy. Searches for “MultiSWAG,” “double descent,” and alleviation of the generalization curve found no passage addressing the claim. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • Deep ensemble models can provide a better approximation to the Bayesian model average in DL compared to standard Bayesian methods such as BBB.

    “Wilson and Izmailov [216] showed how deep ensemble models can provide a better approximation to the Bayesian model average in DL compared to standard Bayesian methods (such as BBB).”

    From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [216]

    Section 2, “Bayesian Neural Network Preliminaries” (page unavailable), discusses approximating a Bayesian predictive distribution using variational inference. Searches for deep ensembles, Bayesian model averaging, BBB, and Wilson and Izmailov found no passage comparing their performance. Our copy therefore does not verify the attributed claim. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 49

  1. kendall, 2017, What uncertainties do we need in bayesian deep learning for computer vision?, Advances in neural information processing systems, 5574
  2. smith, 2018, Understanding measures of uncertainty for adversarial example detection, Conf on Uncertainty in Artificial Intelligence
  3. zhang, 2017, Understanding deep learning requires rethinking generalization, International Conference on Learning Representations
  4. gomez, 2018, Targetted dropout, Advances in Neural Information Processing Workshop Track
  5. guo, 2018, Sparse DNNs with improved adversarial robustness, Advances in neural information processing systems, 240
  6. zhang, 2019, Towards stable and efficient training of verifiably robust neural networks, arXiv preprint arXiv 1906 03008
  7. malinin, 2018, Predictive uncertainty estimation via prior networks, Advances in neural information processing systems, 7047
  8. hendrycks, 2017, A baseline for detecting misclassified and out-of-distribution examples in neural networks, Proc Int Conf Learn Represent
  9. gouk, 2018, Regularisation of neural networks by enforcing Lipschitz continuity, arXiv preprint arXiv 1804 04121
  10. sinha, 2018, Certifying some distributional robustness with principled adversarial training, International Conference on Learning Representations
  11. schmidt, 2018, Ad-versarially robust generalization requires more data, Advances in neural information processing systems
  12. wong, 2018, Provable defenses against adversarial examples via the convex outer adversarial polytope, Proc Int Conf Mach Learn (ICML)
  13. 10.1109/CVPR.2017.690 · DOI · Paper page
  14. simonyan, 2015, Very deep convolutional networks for large-scale image recognition, International Conference on Learning Representations
  15. metzen, 2017, On detecting adversarial perturbations, International Conference on Learning Representations
  16. feinman, 2017, Detecting adversarial samples from artifacts, arXiv preprint arXiv 1703 00410
  17. 10.1109/ICCV.2017.56 · DOI
  18. guo, 2018, Countering adversarial images using input transformations, International Conference on Learning Representations
  19. cohen, 2019, Certified adversarial robustness via randomized smoothing, Intl Conf on Machine Learning (ICML)
  20. miyato, 2018, Virtual adversarial training: A regularization method for supervised and semi- supervised learning, IEEE Trans on Pattern Analysis and Machine Intelligence
  21. gopalakrishnan, 2018, Combating adversarial attacks using sparse representations, arXiv preprint arXiv 1803 03880
  22. goodfellow, 2015, Explaining and harnessing adversarial examples, International Conference on Learning Representations
  23. kurakin, 2017, Adversarial examples in the physical world, International Conference on Learning Representations
  24. 10.1109/CVPR.2018.00957 · DOI
  25. 10.1145/3133956.3134052 · DOI
  26. goswami, 2018, Unravelling robustness of deep learning based face recognition against adversarial attacks, AAAI Conf on Artificial Intelligence, 10.1609/aaai.v32i1.12341 · DOI
  27. sheikholeslami, 2019, Efficient random-ized defense against adversarial attacks in deep convolutional networks, Proc of Intl Conf on Acoustics Speech and Signal Processing
  28. 10.1109/MMSP.2018.8547128 · DOI
  29. 10.1145/3128572.3140444 · DOI
  30. 10.1109/SP.2017.49 · DOI · Paper page
  31. papernot, 2016, Transferability in machine learning: From phenomena to black-box attacks using adversarial samples, Technical Report arXiv preprint arXiv 1605 07277
  32. zhang, 2019, The limitations of adversarial training and the blind-spot attack, International Conference on Learning Representations
  33. liu, 2019, Signsgd via zeroth-order oracle, International Conference on Learning Representations
  34. 10.1109/EuroSP.2016.36 · DOI · Paper page
  35. gu, 2015, Towards deep neural network architectures robust to adversarial examples, International Conference on Learning Representations
  36. sutskever, 2014, Sequence to sequence learning with neural networks, Advances in neural information processing systems, 3104
  37. 10.1145/2976749.2978392 · DOI · Paper page
  38. szegedy, 2014, Intriguing properties of neural networks, International Conference on Learning Representations
  39. engel, 2017, Neural audio synthesis of musical notes with wavenet autoencoders, Intl conf on Machine Learning, 1068
  40. tsipras, 2019, Robustness may be at odds with accuracy, International Conference on Learning Representations
  41. 10.1109/CVPR.2018.00175 · DOI · Paper page
  42. bishop, 2006, Pattern Recognition and Machine Learning
  43. srivastava, 2014, Dropout: A simple way to prevent neural networks from overfitting, Journal of Machine Learning Research, 15, 1929
  44. 10.1613/jair.5756 · DOI
  45. neal, 2012, Bayesian learning for neural networks
  46. gal, 2016, A theoretically grounded application of dropout in recurrent neural networks, Advances in neural information processing systems, 1019
  47. gal, 2016, Bayesian convolutional neural networks with Bernoulli approximate variational inference, International Conference on Learning Representations Workshops Track
  48. dhillon, 2018, Stochastic activation pruning for robust adversarial defense, International Conference on Learning Representations
  49. 10.24963/ijcai.2019/789 · DOI

Cited by

Sorted by citations

Metadata

Authors
Sheikholeslami, F., Jain, S., & Giannakis, G. B.
Affiliations
Technicolor (United States) (US)
ORCID
Venue
2020 Information Theory and Applications Workshop (ITA)
Publisher
IEEE
Published
2020-02-02
Pages
1-16
DOI
10.1109/ita50056.2020.9244964
OpenAlex
W2940465284
Cited by
21
References
3
Citations by year
2026: 1 · 2025: 1 · 2024: 2 · 2023: 1 · 2022: 6 · 2021: 4 · 2020: 6
FWCI
1.62
Type
conference paper
Language
en
Open access
Licence
Locations
  • Information Theory and Applications Workshop (ITA) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Science Foundation (1711471, 1505970, 1901134)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
Keywords
adversarial attack detection, adversarial perturbations, deep neural network, uncertainty estimation, Bayesian neural networks, convex optimization, safety-critical systems, adversarial robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review