Mitigating Black-Box Adversarial Attacks via Output Noise Perturbation
Open accessAbstract
In black-box adversarial attacks, attackers query the deep neural network (DNN) and use the query results to optimize the adversarial samples iteratively. In this paper, we study the method of adding white noise to the DNN output to mitigate such attacks. One of our unique contributions is a theoretical analysis of gradient signal-to-noise ratio (SNR), which shows the trade-off between the defense noise level and the attack query cost. The attacker’s query count (QC) is derived mathematically as a function of noise standard deviation. This will guide the defender to find the appropriate noise level for mitigating attacks to the desired security level specified by QC and DNN performance loss. Our analysis shows that the added noise is drastically magnified by the small variation of DNN outputs, which makes the reconstructed gradient have an extremely low SNR. Adding slight white noise with a very small standard deviation, e.g., less than 0.01, is enough to increase QC by many orders of magnitude yet without introducing any noticeable classification accuracy reduction. Our experiments demonstrate that this method can effectively mitigate both soft-label and hard-label black-box attacks under realistic QC constraints. We also prove that this method outperforms many other defense methods and is robust to the attacker’s countermeasures.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The signal-to-noise ratio of the gradient estimation can fall well below -40 dB.
“As per [25], the signal-to-noise-ratio (SNR) of the gradient estimation can fall well below -40 dB”
From Robustness of Practical Perceptual Hashing Algorithms to Hash-Evasion and Hash-Inversion Attacks (2024) · cites this paper as [25]
Cites
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602 · see citing works
Practical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495 · see citing works
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
Papernot, N., McDaniel, P. D., Wu, X., Jha, S., & Swami, A. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.41 · Cited by 3,123 · see citing works
Ensemble Adversarial Training: Attacks and Defenses
Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2017) · arXiv · doi:10.48550/arxiv.1705.07204 · Cited by 2,176 · see citing works
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv · doi:10.48550/arxiv.1802.00420 · Cited by 2,172 · see citing works
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Carlini, N., & Wagner, D. (2017) · AISec@CCS · doi:10.1145/3128572.3140444 · Cited by 2,079 · see citing works
Chen, P., Zhang, H., Sharma, Y., Yi, J., & Hsieh, C. (2017) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3128572.3140448 · Cited by 1,782 · see citing works
Delving into Transferable Adversarial Examples and Black-box Attacks
Liu, Y., Chen, X., Liu, C., & Song, D. (2016) · arXiv · doi:10.48550/arxiv.1611.02770 · Cited by 1,310 · see citing works
Countering Adversarial Images using Input TransformationsPreprint
Guo, C., Rana, M., Cissé, M., & van der Maaten, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00117 · Cited by 1,151 · see citing works
Decision-based adversarial attacks: Reliable attacks against black-box machine learning models
Brendel, W., Rauber, J., & Bethge, M. (2017) · arXiv · doi:10.48550/arxiv.1712.04248 · Cited by 1,018 · see citing works
Mitigating Adversarial Effects Through RandomizationPreprint
Xie, C., Wang, J., Zhang, Z., Ren, Z., & Yuille, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01991 · Cited by 905 · see citing works
Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative ModelsPreprint
Samangouei, P., Kabkab, M., & Chellappa, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06605 · Cited by 838 · see citing works
Black-box Adversarial Attacks with Limited Queries and Information
Ilyas, A., Engstrom, L., Athalye, A., & Lin, J. (2018) · arXiv · doi:10.48550/arxiv.1804.08598 · Cited by 806 · see citing works
HopSkipJumpAttack: A Query-Efficient Decision-Based Attack
Chen, J., Jordan, M. I., & Wainwright, M. J. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00045 · Cited by 718 · see citing works
On Adaptive Attacks to Adversarial Example Defenses
Tramer, F., Carlini, N., Brendel, W., & Madry, A. (2020) · arXiv · doi:10.48550/arxiv.2002.08347 · Cited by 486 · see citing works
Simple Black-box Adversarial Attacks
Guo, C., Gardner, J. R., You, Y., Wilson, A. G., & Weinberger, K. Q. (2019) · arXiv · doi:10.48550/arxiv.1905.07121 · Cited by 427 · see citing works
Towards Robust Neural Networks via Random Self-ensemble
Liu, X., Cheng, M., Zhang, H., & Hsieh, C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_23 · Cited by 408 · see citing works
SafetyNet: Detecting and Rejecting Adversarial Examples Robustly
Lu, J., Issaranon, T., & Forsyth, D. A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.56 · Cited by 397 · see citing works
AutoZOOM: Autoencoder-Based Zeroth Order Optimization Method for Attacking Black-Box Neural Networks
Tu, C., Ting, P., Chen, P., Liu, S., Zhang, H., Yi, J., Hsieh, C., & Cheng, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301742 · Cited by 387 · see citing works
He, Z., Rakin, A. S., & Fan, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00068 · Cited by 280 · see citing works
Benchmarking Adversarial Robustness on Image Classification
Dong, Y., Fu, Q.-A., Yang, X. F., Pang, T., Su, H., Xiao, Z., & Zhu, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00040 · Cited by 275 · see citing works
Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach
Cheng, M., Le, T., Chen, P.-Y., Yi, J., Zhang, H., & Hsieh, C.-J. (2018) · arXiv · doi:10.48550/arxiv.1807.04457 · Cited by 269 · see citing works
Certified Adversarial Robustness with Additive NoisePreprint
Li, B., Chen, C., Wang, W., & Carin, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03113 · Cited by 225 · see citing works
Alzantot, M., Sharma, Y., Chakraborty, S., Zhang, H., Hsieh, C., & Srivastava, M. B. (2019) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3321707.3321749 · Cited by 215 · see citing works
Improving Black-box Adversarial Attacks with a Transfer-based PriorPreprint
Cheng, S., Dong, Y., Pang, T., Su, H., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.06919 · Cited by 199 · see citing works
On the Robustness of the CVPR 2018 White-Box Adversarial Example DefensesPreprint
Athalye, A., & Carlini, N. (2018) · arXiv.org · doi:10.48550/arxiv.1804.03286 · Cited by 176 · see citing works
Li, Y., Li, L., Wang, L., Zhang, T., & Gong, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00441 · Cited by 175 · see citing works
Learning Universal Adversarial Perturbations with Generative Models
Hayes, J., & Danezis, G. (2018) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2018.00015 · Cited by 167 · see citing works
Simple Black-Box Adversarial Perturbations for Deep NetworksPreprint
Narodytska, N., & Kasiviswanathan, S. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.06299 · Cited by 166 · see citing works
Díaz‐Francés, E., & Rubio, F. J. (2012) · Statistical Papers · doi:10.1007/s00362-012-0429-2 · Cited by 160 · see citing works
Sign-OPT: A Query-Efficient Hard-label Adversarial AttackPreprint
Cheng, M., Singh, S., Chen, P. M., Chen, P., Liu, S., & Hsieh, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10773 · Cited by 116 · see citing works
Exact Distribution for the Product of Two Correlated Gaussian Random Variables
Cui, G., Yu, X., Iommelli, S., & Kong, L. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2614539 · Cited by 74 · see citing works
Wang, X., Wang, S., Chen, P., Wang, Y., Kulis, B., Lin, X., & Chin, S. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/833 · Cited by 50 · see citing works
Machine Learning as an Adversarial Service: Learning Black-Box Adversarial Examples.
Hayes, J., & Danezis, G. (2017) · Cited by 44 · see citing works
Defending Against Machine Learning Model Stealing Attacks Using Deceptive PerturbationsPreprint
Lee, T., Edwards, B., Molloy, I., & Su, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00054 · Cited by 30 · see citing works
Fan, W., Sun, G., Su, Y., Liu, Z., & Lu, X. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-7353-6 · Cited by 25 · see citing works
Flagged first, then by citations
4 of the 44 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A Comprehensive Review of Adversarial Attacks and Defense Strategies in Deep Neural Networks
Abomakhelb, A. I. A., Jalil, K. A., Buja, A. G., Alhammadi, A., & Alenezi, A. M. (2025) · Technologies · doi:10.3390/technologies13050202 · Cited by 31 · see citing works
Federated and Transfer Learning: A Survey on Adversaries and Defense Mechanisms
Hallaji, E., Razavi‐Far, R., & Saif, M. (2022) · Adaptation, learning, and optimization · doi:10.1007/978-3-031-11748-0_3 · Cited by 22 · see citing works
Boundary Defense Against Black-box Adversarial Attacks
Aithal, M. B., & Li, X. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956476 · Cited by 6 · see citing works
Self-citationWang, L., Wan, Q., Wang, J., Chen, M., Peng, L., & Fu, X. (2025) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2025.103424 · Cited by 2 · see citing works
Robustness of Practical Perceptual Hashing Algorithms to Hash-Evasion and Hash-Inversion Attacks
Madden, J., Bhavsar, M., Dorje, L., & Li, X. (2024) · arXiv · doi:10.48550/arxiv.2406.00918
“Following the analysis of [25], this noise is amplified by 1 /β , leading to the gradient estimate g being affected by a noise power σ 2 /β 2 , hence making it unreliable.”
Self-citationTowards Adversarial Attacks Mitigation for License Plate Recognition Systems
Safina, A., & Sadovykh, A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-94770-4_32
Mamba Driven Channel Estimation for High-Mobility Massive MIMO Systems
Wu, L., Wang, Y., Zheng, Y., Gong, C., & Wang, J. (2026) · International Conference on Communication, Remote Sensing and Information Technology (CRSIT) · doi:10.1109/crsit69981.2026.11605319
On Evaluating Stateful Defence Models Against Query-Based Black-Box Attacks
Ali, Z. T. M., Azad, R. M. A., Azad, M. A., Rice, I., Daraz, U., Imran, A. S., & Holyhead, J. (2026) · doi:10.1109/cvprf71909.2026.00082
- Authors
- Aithal, M. B., & Li, X.
- Affiliations
- Binghamton University (US)
- Venue
- IEEE Access
- Published
- 2022
- Volume
- 10
- Pages
- 12395-12411
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3203257711
- Cited by
- 7
- References
- 44
- Citations by year
- 2026: 2 · 2025: 3 · 2022: 2
- FWCI
- 0.41
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University) (cc-by)
- Funders
- U.S. Air Force Office of Scientific Research (FA9550-21-1-0229)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- black-box adversarial attacks, query cost, adversarial robustness, deep neural network security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
