Cited by 92Adversarial Robustness in Machine LearningDigital Media Forensic DetectionSecurity and Verification in Computing

Black-box Detection of Backdoor Attacks with Limited Information and Data

Open access
Dong, Yinpeng, Yang, Xiao, Deng, Zhijie, Pang, Tianyu, Xiao, Zihao, Su, Hang, Zhu, Jun·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)

Abstract

Although deep neural networks (DNNs) have made rapid progress in recent years, they are vulnerable in adversarial environments. A malicious backdoor could be embedded in a model by poisoning the training dataset, whose intention is to make the infected model give wrong predictions during inference when the specific trigger appears. To mitigate the potential threats of backdoor attacks, various backdoor detection and defense methods have been proposed. However, the existing techniques usually require the poisoned training data or access to the white-box model, which is commonly unavailable in practice. In this paper, we propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. We introduce a gradient-free optimization algorithm to reverse-engineer the potential trigger for each class, which helps to reveal the existence of backdoor attacks. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models. Extensive experiments on hundreds of DNN models trained on several datasets corroborate the effectiveness of our method under the black-box setting against various backdoor attacks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.01617

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

7 of the 57 references couldn’t be matched to a known paper.

Research works printed but not linked: 5

  1. Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural networks. In Pro- ceedings of the 35th Annual Computer Security Applications Conference (ACSAC), pages 113-125, 2019. 2, 8, 9
  2. Kaidi Jin, Tianwei Zhang, Chao Shen, Yufei Chen, Ming Fan, Chenhao Lin, and Ting Liu. A unified framework for analyzing and detecting malicious examples of dnn models. arXiv preprint arXiv:2006.14871, 2020. 8, 9 · arXiv
  3. Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. Re- flection backdoor: A natural backdoor attack on deep neu- ral networks. In European Conference on Computer Vision (ECCV), 2020. 2
  4. Yuntao Liu, Yang Xie, and Ankur Srivastava. Neural tro- jans. In IEEE International Conference on Computer Design (ICCD), pages 45-48. IEEE, 2017. 2
  5. Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. Natural evolu- tion strategies. Journal of Machine Learning Research, 15(27):949-980, 2014. 4, 11

Cited by

Sorted by citations

Authors
Dong, Y., Yang, X., Deng, Z., Pang, T., Xiao, Z., Su, H., & Zhu, J.
Affiliations
  • Tsinghua University (CN)
  • Pazhou Lab,Guangzhou,China,510330
  • Hohai University (CN)
Venue
2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2021-10
Pages
16462-16471
OpenAlex
W3138646989
Cited by
92
References
57
Citations by year
2026: 6 · 2025: 19 · 2024: 34 · 2023: 22 · 2022: 6 · 2021: 5
FWCI
7.74
Type
conference paper
Language
en
Open access
Locations
  • IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Key Research and Development Program of China
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
Keywords
backdoor attacks, backdoor detection, black-box attacks, data poisoning, derivative-free optimization, adversarial robustness, deep neural network

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review