The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
Open accessAbstract
Deep learning algorithms have been shown to perform extremely well on many classical machine learning problems. However, recent studies have shown that deep learning, like other machine learning techniques, is vulnerable to adversarial samples: inputs crafted to force a deep neural network (DNN) to provide adversary-selected outputs. Such attacks can seriously undermine the security of the system supported by the DNN, sometimes with devastating consequences. For example, autonomous vehicles can be crashed, illicit or illegal content can bypass content filters, or biometric authentication systems can be manipulated to allow improper access. In this work, we introduce a defensive mechanism called defensive distillation to reduce the effectiveness of adversarial samples on DNNs. We analytically investigate the generalizability and robustness properties granted by the use of defensive distillation when training DNNs. We also empirically study the effectiveness of our defense mechanisms on two DNNs placed in adversarial settings. The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN. Such dramatic gains can be explained by the fact that distillation leads gradients used in adversarial sample creation to be reduced by a factor of 1030. We also find that distillation increases the average minimum number of features that need to be modified to create adversarial samples by about 800% on one of the DNNs we tested.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
(2012) · Cited by 8,009
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
Self-citationDeep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Research works printed but not linked: 6
- fawzi, 2015, Analysis of classifiers' robustness to adversarial perturbations, Deep Learning Workshop at ICML 2015 arXiv preprint arXiv 1502 02590
- bergstra, 2010, Theano: a cpu and gpu math expression compiler, Proceedings of the Python for Scientific Computing Conference (SciPy), 4, 3
- 2015, Nvidia tegra drive px: Self-driving car computer, Nvidia
- sermanet, 2014, Overfeat: Integrated recognition, localization and detection using convolutional networks, International Conference on Learning Representations (ICLR 2014) arXiv preprint arXiv 1312 6229
- miyato, 2015, Distributional smoothing by virtual adversarial examples, CoRR vol abs/1507 00677
- krizhevsky, 2012, Imagenet classification with deep convolutional neural networks, Advances in neural information processing systems, 1097
Cited by
Items 101–137 of 137
Artificial Intelligence Security: Threats and Countermeasures
Hu, Y., Kuang, W., Qin, Z., Li, K., Zhang, J., Gao, Y., Li, W., & Li, K. (2023) · ACM Computing Surveys · doi:10.1145/3487890 · Cited by 196
Human Uncertainty Makes Classification More Robust
Peterson, J., Battleday, R., Griffiths, T., & Russakovsky, O. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00971 · Cited by 196
Stronger Data Poisoning Attacks Break Data Sanitization Defenses
Koh, P. W., Steinhardt, J., & Liang, P. (2018) · arXiv · doi:10.48550/arxiv.1811.00741 · Cited by 189
Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning
Ilahi, I., Usama, M., Qadir, J., Janjua, M. U., Al-Fuqaha, A., Hoang, D. T., & Niyato, D. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3111139 · Cited by 170
Bringing ai to edge: from deep learning’s perspective
Liu, D., Kong, H., Luo, X., Liu, W., & Subramaniam, R. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.04.141 · Cited by 171
“Moreover, knowledge distillation can also be used to design robust networks [151].”
BackgroundDEEPSEC: A Uniform Platform for Security Analysis of Deep Learning Model
Ling, X., Ji, S., Zou, J., Wang, J., Wu, C., Li, B., & Wang, T. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00023 · Cited by 152
Wasserstein Adversarial Examples via Projected Sinkhorn Iterations
Wong, E., Schmidt, F. R., & Kolter, J. Z. (2019) · arXiv · doi:10.48550/arxiv.1902.07906 · Cited by 145
“To name a few, distillation was proposed as a defense but was defeated (Papernot et al., 2016; Carlini & Wagner, 2017), realistic transformations seen by vehicles were thought to be safe until more robust adversarial examples were created (Lu et al., 2017; Athalye et al., 2018b), and many defenses…”
BackgroundAlgorithmic Decision-Making in AVs: Understanding Ethical and Technical Concerns for Smart Cities
Lim, H. S. M., & Taeihagh, A. (2019) · Sustainability · doi:10.3390/su11205791 · Cited by 136
Machine Learning in Adversarial Settings
McDaniel, P., Papernot, N., & Celik, Z. B. (2016) · IEEE Security & Privacy · doi:10.1109/msp.2016.51 · Cited by 120
Self-citationStakeholders in Explainable AI
Preece, A., Harborne, D., Braines, D., Tomsett, R., & Chakraborty, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00184 · Cited by 117
Adversarial Threats to DeepFake Detection: A Practical Perspective
Neekhara, P., Dolhansky, B., Bitton, J., & Ferrer, C. C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00103 · Cited by 117
Bradshaw, J., Matthews, A. G. D. G., & Ghahramani, Z. (2017) · arXiv · doi:10.48550/arxiv.1707.02476 · Cited by 109
Toward Trustworthy Artificial Intelligence (TAI) in the Context of Explainability and Robustness
chander, B., John, C., Warrier, L., & Gopalakrishnan, K. (2025) · ACM Computing Surveys · doi:10.1145/3675392 · Cited by 108
Curls & Whey: Boosting Black-Box Adversarial Attacks
Shi, Y., Wang, S., & Han, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00668 · Cited by 103
An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences
Guo, W., Tondi, B., & Barni, M. (2022) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2022.3190213 · Cited by 91
(2018) · Neural Information Processing Systems · Cited by 89
Model Evasion Attack on Intrusion Detection Systems using Adversarial Machine Learning
Ayub, M. A., Johnson, W. A., Talbert, D. A., & Siraj, A. (2020) · 2020 54th Annual Conference on Information Sciences and Systems (CISS) · doi:10.1109/ciss48834.2020.1570617116 · Cited by 86
Detecting computer-generated disinformation
Stiff, H., & Johansson, F. (2022) · International Journal of Data Science and Analytics · doi:10.1007/s41060-021-00299-5 · Cited by 79
Survey: Leakage and Privacy at Inference Time
Jegorova, M., Kaul, C., Mayor, C., O'Neil, A. Q., Weir, A., Murray-Smith, R., & Tsaftaris, S. A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3229593 · Cited by 77
Domain Robustness in Neural Machine Translation
Müller, M., Rios, A., & Sennrich, R. (2019) · arXiv · doi:10.48550/arxiv.1911.03109 · Cited by 68
Must-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning
Koçak, B., Cuocolo, R., Santos, D. P. D., Stanzione, A., & Ugga, L. (2023) · Balkan Medical Journal · doi:10.4274/balkanmedj.galenos.2022.2022-11-51 · Cited by 64
Secure and robust machine learning for healthcare: A survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3013489 · Cited by 59
Security and Privacy Issues in Deep Learning
Bae, H., Jang, J., Jung, D., Jang, H., Ha, H., Lee, H., & Yoon, S. (2018) · arXiv · doi:10.48550/arxiv.1807.11655 · Cited by 56
Residual error based knowledge distillation
Gao, M., Wang, Y., & Wan, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.10.113 · Cited by 45
Exploring Adversarial Fake Images on Face Manifold
Li, D., Wang, W., Fan, H., & Dong, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00573 · Cited by 43
Rizvi, S. K. J., Azad, M. A., & Fraz, M. M. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-021-09543-4 · Cited by 37
Are Adversarial Robustness and Common Perturbation Robustness Independent Attributes ?
Laugros, A., Caplier, A., & Ospici, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00134 · Cited by 33
Anelli, V. W., Deldjoo, Y., Di Noia, T., Malitesta, D., & Merra, F. A. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462848 · Cited by 31
Zhou, C., Kadambi, P., Mattina, M., & Whatmough, P. N. (2020) · arXiv · doi:10.48550/arxiv.2001.04974 · Cited by 27
Robust Pervasive Detection for Adversarial Samples of Artificial Intelligence in IoT Environments
Wang, S., & Qiao, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2919695 · Cited by 23
Robust or Private? Adversarial Training Makes Models More Vulnerable to Privacy Attacks
Mejia, F. A., Gamble, P., Hampel-Arias, Z., Lomnitz, M., Lopatina, N., Tindall, L., & Barrios, M. A. (2019) · arXiv · doi:10.48550/arxiv.1906.06449 · Cited by 13
Kazmi, S. M. K. A., Aafaq, N., Khan, M. A., Khalil, M., & Saleem, A. (2023) · IEEE Access · doi:10.1109/access.2023.3299878 · Cited by 13
Thomaidou, A., & Limniotis, K. (2025) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp5010007 · Cited by 11
A Synergetic Attack against Neural Network Classifiers combining Backdoor and Adversarial Examples
Liu, G., Khalil, I., Khreishah, A., & Phan, N. (2021) · 2021 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata52589.2021.9671964 · Cited by 11
Mitigating Black-Box Adversarial Attacks via Output Noise Perturbation
Aithal, M. B., & Li, X. (2022) · IEEE Access · doi:10.1109/access.2022.3146198 · Cited by 7
(2018) · arXiv (Cornell University) · Cited by 6
Robust Federated Learning with Adaptable Learning Rate
Zhou, J., Zhong, Z., Wang, J., Zhang, X., Chen, F., & Yan, C. (2021) · 2021 7th International Conference on Big Data and Information Analytics (BigDIA) · doi:10.1109/bigdia53151.2021.9619731 · Cited by 2
Metadata
- Authors
- Papernot, N., McDaniel, P., Wu, X., Jha, S., & Swami, A.
- Venue
- 2016 IEEE Symposium on Security and Privacy (SP)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 582-597
- ISSN
- 1081-6011 (ISSN-L)
- OpenAlex
- W2174868984
- Cited by
- 2,982
- References
- 39
- FWCI
- 132.14
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.