Recent Advances in Adversarial Training for Adversarial Robustness

“Su et al. (2018) evaluated the recent SOTA ImageNet-based DNN models on multiple robustness metrics.”

“However, it is found [Andriushchenko and Flammarion, 2020] that the above fast training methods [Shafahi et al.”

“The adversarial counterpart x ′ of x is expressed as There are some common attacks following this formulation like Fast Gradient Sign Method (FGSM) [Goodfellow et al. , 2015], iterative FGSM [Kurakin et al. , 2016], and Projected Gradient Descent (PGD) attack [Madry et al. , 2018].”

“Current adversarial training methods [Madry et al. , 2018] can produce a robust model with worst-case accuracy of around 90% on MNIST.”

“(2020) empirically investigated various techniques like `1 and `2 regularization, cutout [DeVries and Taylor, 2017], mixup [Zhang et al., 2018] and early stopping, where early stopping is found to be the most effective.”

“, 2015], iterative FGSM [Kurakin et al., 2016], and Projected Gradient Descent (PGD) attack [Madry et al.”

“(2020) empirically investigated various techniques like `1 and `2 regularization, cutout [DeVries and Taylor, 2017], mixup [Zhang et al.”

“Kurakin et al. (2017) extended this FGSM-based regularization term by controlling the ratio of adversarial examples in batches so that it can scale up to ImageNet.”

“Adversarial training methods [6], [7] are popular for de-fending against adversarial examples and noisy inputs.”

“Adversarial training, a technique where a dataset augmented with adversarial examples is utilized to train the model, has shown promising results in improving robustness [10].”

“( Bai et al. , 2021; Good-fellow et al. , 2016).”

“Defensive strategies against adversarial attacks in image processing are still under development, as discussed in a review paper [10] and Section 2 of this paper.”

“Ongoing research [3], [4] endeavors focus on developing robust models capable of withstanding such attacks and enhancing the model’s resilience against adversarial examples.”

“Another model can also be created for adding noise [1]. a) Error in the input y : This project will train an adversarial network to learn worst-case noise and another network to learn to defend jointly [17].”

“Adversarial training (AT) [19] emerged as a promising solution to improve the robustness of models against adversarial attacks.”

“Furthermore, in the literature of deep learning security, adversarial fine-tuning, where the model learned from its problematic outputs, is shown to be an effective solution (Bai et al., 2021).”

“Adversarial training remains one of the most effective methods for defending against adversarial attacks Bai et al. (2021).”

“Several defenses have been proposed against adversarial attacks, which can be classified into heuristic defenses and certified defenses [3, 12, 21].”

“It is reasonable to sacrifice some accuracy on the clean dataset for better robustness(like the idea of adversarial training [34]).”

“The second line of studies sought algorithms that can evaluate the robustness of DL’s decision-making and enhance it [13, 14, 15, 16].”

“The gradient of the network contains enough information about the local sensitivity [55] that it can be used to design these attacks, and train the network to withstand them [24, 5].”

“Adversarial training [5] [6] [7] is the most effective way to improve the robustness of the model itself.”

“Existing approaches for countering adversarial attacks generally follow a dual strategy: generating adversarial examples and retraining the classifier by incorporating these examples into the training set (as seen in [8], for example).”

“Given the implications AE can have on Deep learning based NIDS, several AE mitigation techniques have been developed over the years which includes Adversarial training [9], Defensive Distillation [10], Randomization [11], Gradient Masking [12], Ensemble methods [13] and certified defense [14].”

“Perhaps the most successful of these methods is adversarial training [3, 23], which adversarially perturbs each training minibatch by using Projected Gradient De-scent to modify the data in the minibatch such that the loss is maximized.”

“All rights reserved. and Kolter 2020; Bai et al. 2021), which usually constructs a virtual attacker that generates the worst adversarial examples maximizing the loss in the neighborhood of clean examples and seeks a robust classification model (classifier) by minimizing the loss on the generated…”

“The recent literature has seen a surging interest in this field, resulting in a large number of techniques to improve the robustness of deep learning models to adversarial attacks, such as adversarial training [3, 15, 35], and data augFigure 1.”

“A variety of approaches attempted to improve the reliability of fp, such as adversarial attacking [3], special network structure design [4], mega data/model [5], symbolic learning [6], domain-knowledge regularization [7], learning neural operator [8], etc.”

“Adversarial training [8][9] proves to be one of the most effective approaches to defending deep learning models against adversarial examples.”

“A classical review of defense methods can be obtained in [Bai et al., 2021].”

“Adversarial training is an state-of-the-art empirically robust defense of which the idea is to train a better classifier [14].”

“Compared with traditional model training, the cost of adversarial training are more than 10 times higher for achieving the same level of accuracy [15].”

“We refer readers to [1] for a recent survey on AT.”

“Evaluating policy stability under sensor perturbation is supported by solid robot learning research [26].”

“In the field of image processing, numerous mature defense strategies have been proposed [24], such as adversarial training [25] and model distillation-based defenses [26].”

“Researchers such as Bai et al. [30], Vitorino et al. [31], and Krishnan and Shrinath [32] have shown that incorporating carefully designed adversarial examples into the training process can substantially enhance model resilience.”

“Currently, it is the de-facto method for robust training [11].”

“One of the most effective methods is adversarial training [15], [17], [34], [35].”

“A common single task strategy to defend against adversarial attacks is to leverage adversarial training [19, 35, 36].”

“Active defense algorithms optimize the model by adding additional training samples (adversarial training [2]) or modifying the network structure (Defensive distillation [46], gradient masking [18], etc.”

“Create and inject adversarial samples into the learning datasets, those that would be used by an adversary, intended to trick the model so that it is able to learn and become resilient to manipulations [16].”