Increasing depth, distribution distillation, and model soup: erasing backdoor triggers for deep neural networks
Abstract
Deep neural networks are vulnerable to backdoor attacks, in which the adversary injects a trigger embedded set into the training process. Inputs marked with the trigger provide incorrect predictions, whereas clean inputs remain unaffected. To erase the latent triggers in models, increasing depth, distribution distillation, and model soup (ID3MS), a defensive solution that operates without prior knowledge of triggers and relies on a small clean set is introduced. The depth of the backdoor model is increased by adding fully connected layer(s) at the penultimate layer. Without a classification layer, the original backdoor and increased depth models are considered as teacher and student, respectively. The student model applies distribution distillation to refit the distribution of the clean set and erase the backdoor triggers. The distilled student model is then recovered with the classification layer and model soup is used to ensemble a collection of models generated by various fine-tuning hyperparameters. The experimental results validate the superior performance of the ID3MS compared with existing defensive techniques against several attacks across datasets.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The knowledge distillation method stores historical parameters, subtracts the contribution of targeted malicious clients to derive a skewed unlearning model, and performs knowledge distillation with the original model as teacher and the skewed unlearning model as student.
“The knowledge distillation method is proposed in [216], [226] to restore the global model's performance. In this design, the central server stores the historical parameters of the participating clients. Once the global model is obtained. The contribution of the targeted malicious clients is subtracted, and a skewed unlearning model is derived. Then, the knowledge distillation is performed. The”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [216]
Because the dataset used in distillation does not contain backdoor patterns, the student model will not learn the backdoors injected in the teacher model.
“Due to the fact that the data set used in the distillation does not contain backdoor patterns, the student model will not learn the backdoors injected in the teacher model.”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [216]
The knowledge distillation method is not compatible with SecAgg, and participant privacy is at risk of leakage.
“Meanwhile, the proposed method is not compatible with SecAgg, and the privacy of the participants is at risk of leakage.”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [216]
The unlearning mechanism requires knowledge of the target clients, and attackers are normally stealthy in practical scenarios.
“The unlearning mechanism requires the knowledge of the target clients, and the attackers are normally stealthy in practical scenarios.”
From Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey (2024) · cites this paper as [216]
Cited by
Sorted by citations
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
“3) Model Similarity-Based Defense: Another commonly adopted strategy of defense during the aggregation phase is the model similarity-based defense method [215], [216], [217].”
MethodSeeing Through the Threat: Adaptive Attention-Guided Backdoor Defense in Non-IID Federated Learning
You, X.-C., & Karati, A. (2025) · IEEE Conference on Communications and Network Security · doi:10.1109/cns66487.2025.11194979 · Cited by 1 · see citing works
“Thus, it maintains the integrity of the global model [6].”
- Authors
- Zhang, Y., Zhang, T., Liu, Q., Sun, G., & Wu, H.
- Affiliations
- Shanghai University (CN)
- Corresponding author
- Sun, Guangling
- Venue
- Journal of Electronic Imaging
- Publisher
- SPIE-Intl Soc Optical Eng
- Published
- 2022-11-08
- Volume
- 31
- Issue
- 06
- Pages
- 1-10
- ISSN
- 1017-9909 (ISSN-L)
- 1560-229X
- OpenAlex
- W4308587686
- Cited by
- 2
- References
- 0
- Citations by year
- 2025: 1 · 2024: 1
- FWCI
- 0.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Electronic Imaging (publishedVersion)
- Funders
- National Natural Science Foundation of China (61902235)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- backdoor attacks, deep neural network defense, ensemble learning, teacher-student distillation, adversarial robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
