CoChat
Sign Up
PreprintCited by 10,381Adversarial Robustness in Machine LearningSecurity and Verification in ComputingSoftware Testing and Debugging Techniques

Explaining and Harnessing Adversarial Examples

Open access
Goodfellow, Ian J., Shlens, Jonathon, Szegedy, Christian·2014·arXiv
DOI 10.48550/arxiv.1412.6572arXiv 1412.6572
Open this paper in CoChat — free accountOpen PDF

Abstract

Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Early attempts at explaining this phenomenon focused on nonlinearity and overfitting. We argue instead that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Using this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1412.6572

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Adversarial training typically only improves robustness to the specific type of adversarial examples incorporated during training, potentially leaving the trained model vulnerable to other forms of adversarial noise.

    “Moreover, adversarial training typically only improves robustness to the specific type of adversarial examples incorporated during training, potentially leaving the trained model vulnerable to other forms of adversarial noise [176, 448, 414].”

    From Advances and Open Problems in Federated Learning (2021) · cites this paper as [414]

    Section 6 (p. 5) reports that training with fast-gradient-sign adversarial examples reduced the error rate on that type of example from 89.4% to 17.9%. The paper does not establish that adversarial training typically improves robustness only to the type used in training, or test whether that trained model remains vulnerable to other forms of adversarial noise. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Adversarial attacks help adversaries evade FL system security detection and can generate poisoned samples to undermine the FL system accuracy.

    “Adversarial attacks help adversaries evade FL system security detection and can generate poisoned samples to undermine the FL system accuracy (Goodfellow et al. 2014;Nair et al. 2023).”

    From Survey: federated learning data security and privacy-preserving in edge-Internet of Things (2024) · cites this paper as Goodfellow et al. 2014

    The Introduction (p. 1) says adversarial examples cause machine-learning models to misclassify slightly altered inputs, and Section 4 (p. 3) describes generating such examples. The paper does not establish that these attacks evade federated-learning security detection or that poisoned samples undermine a federated-learning system’s accuracy; its discussion of adversarial training is not evidence of a poisoning attack. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Neural network-based and zero-shot detectors rely on a deep net for their detection and can be vulnerable to adversarial and poisoning attacks.

    “Additionally, neural network-based and zero-shot detectors rely on a deep net for their detection, and they can be vulnerable to adversarial and poisoning attacks (Goodfellow et al., 2014; Sadasivan et al., 2023; Kumar et al., 2022; Wang et al., 2022).”

    From Can AI-Generated Text be Reliably Detected? (2023) · cites this paper as Goodfellow et al., 2014

    The Introduction (p. 1) says neural networks can misclassify adversarial examples, and Section 4 (p. 2) demonstrates an adversarial perturbation of a deep network. The paper does not establish that neural network-based or zero-shot detectors rely on deep nets, or that those detectors are vulnerable to poisoning attacks. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Tabular data can be easily shuffled, but the perturbation of images can have unpredictable effects, with research demonstrating this type of manipulation both as an attack against algorithms and as a regularization mechanism leading to increased robustness and resilience against inversion attacks.

    “Tabular data can be easily shuffled, but the perturbation of images can have unpredictable effects, with research demonstrating this type of manipulation (for example, adversarial noise) both as an attack against algorithms 65 and a regularization mechanism leading to increased robustness 66 and resilience against inversion attacks.”

    From Secure, privacy-preserving and federated machine learning in medical imaging (2020) · cites this paper as [65]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Deep neural networks are vulnerable to adversarial examples, which are carefully chosen inputs that cause the network to change output without a visible change to a human.

    “On the other hand, it is worth noting that researchers have recently that noticed deep neural networks are vulnerable to adversarial examples, which are carefully chosen inputs that cause the network to change output without a visible change to a human (Fig. 15) [60–63].”

    From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [61]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 14 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. Bergstra, James, Breuleux, Olivier, Bastien, Fr´ ed´ eric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) , June 2010. Oral Presentation.
  2. Chalupka, K., Perona, P., and Eberhardt, F. Visual Causal Feature Learning. ArXiv e-prints , December 2014.
  3. Glorot, Xavier, Bordes, Antoine, and Bengio, Yoshua. Deep sparse rectifier neural networks. In JMLR W&CP: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2011) , April 2011.
  4. Goodfellow, Ian J., Warde-Farley, David, Mirza, Mehdi, Courville, Aaron, and Bengio, Yoshua. Maxout networks. In Dasgupta, Sanjoy and McAllester, David (eds.), International Conference on Machine Learning , pp. 1319-1327, 2013c.
  5. Nguyen, A., Yosinski, J., and Clune, J. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. ArXiv e-prints , December 2014.
  6. Rust, Nicole, Schwartz, Odelia, Movshon, J. Anthony, and Simoncelli, Eero. Spatiotemporal elements of macaque V1 receptive fields. Neuron , 46(6):945-956, 2005.

Cited by

Sorted by citations

Related

Authors
Goodfellow, I. J., Shlens, J., & Szegedy, C.
Affiliations
Google (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2014
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1412.6572
arXiv
1412.6572
OpenAlex
W1945616565
Cited by
10,381
References
14
Citations by year
2026: 66 · 2025: 843 · 2024: 1806 · 2023: 1966 · 2022: 1649 · 2021: 1599 · 2020: 1112 · 2019: 767 · 2018: 383 · 2017: 124 · 2016: 43 · 2015: 15 · 2014: 1 · 2012: 5
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • International Conference on Learning Representations
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
adversarial examples, adversarial perturbations, neural network vulnerabilities, linearization, adversarial training, MNIST, robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review