Cited by 38Adversarial Robustness in Machine LearningExplainable Artificial Intelligence (XAI)Software Testing and Debugging Techniques

Text Backdoor Detection Using an Interpretable RNN Abstract Model

Open access
Fan, Ming, Si, Ziliang, Xie, Xiaofei, Liu, Yang, Liu, Ting·2021·IEEE Transactions on Information Forensics and Security

Abstract

Deep neural networks (DNNs) are known to be inherently vulnerable to malicious attacks such as the adversarial attack and the backdoor attack. The former is crafted by adding small perturbations to benign inputs so as to fool a DNN. The latter generally embeds a hidden pattern in a DNN by poisoning the dataset during the training process, which causes the infected model to misbehave on predefined inputs with a specific trigger and normally perform for others. Much work has been conducted on defending against the adversarial samples, while the backdoor attack received much less attention, especially in recurrent neural networks (RNNs), which play an important role in the text processing field. Two main limitations make it hard to directly apply existing image backdoor detection approaches to RNN-based text classification systems. First, a layer in an RNN does not preserve the same feature latent space function for different inputs, making it impossible to map the inserted specific pattern with the neural activations. Second, the text data is inherently discrete, making it hard to optimize the text like image pixels. In this work, we propose a novel backdoor detection approach named InterRNN for RNN-based text classification systems from the interpretation perspective. Specifically, we first propose a novel RNN interpretation technique by constructing a nondeterministic finite automaton (NFA) based abstract model, which can effectively reduce the analysis complexity of an RNN while preserving its original logic rules. Then, based on the abstract model, we can obtain interpretation results that explain the fundamental reason behind the decision for each input. We then detect trigger words by leveraging the differences between the behaviors in the backdoor sentences and those in the normal sentences. The extensive experiment results on four benchmark datasets demonstrate that our approach can generate better interpretation results compared to state-of-the-art approaches and effectively detect backdoors in RNNs.

Integrity signals

Registry

Crossref · 10.1109/tifs.2021.3103064

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Fan, M., Si, Z., Xie, X., Liu, Y., & Liu, T.
Affiliations
  • Xi'an Jiaotong University (CN)
  • Nanyang Technological University (SG)
Venue
IEEE Transactions on Information Forensics and Security
Published
2021
Volume
16
Pages
4117-4132
ISSN
  • 1556-6013 (ISSN-L)
  • 1556-6021
OpenAlex
W3190075653
Cited by
38
References
46
Citations by year
2026: 4 · 2025: 8 · 2024: 9 · 2023: 8 · 2022: 8 · 2021: 1
FWCI
2.30
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Information Forensics and Security (publishedVersion)
  • Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, other-oa)
  • DR-NTU (Nanyang Technological University) (submittedVersion)
Funders
  • National Natural Science Foundation of China (61902306, 61632015, 61602369, U1766215, 61772408, 61702414, 61833015)
  • China Postdoctoral Science Foundation (2019TQ0251, 2020M673439)
  • Innovative Research Group of the National Natural Science Foundation of China (61721002)
  • Ministry of Education Innovation Research Team (IRT_17R86)
  • Youth Talent Support Plan of Xi’an Association for Science and Technology (095920201303)
  • National Research Foundation, Prime Minister’s Office, Singapore, under its National Cybersecurity Research and Development Program (NRF2018NCR-NCR005-0001)
  • NRF Investigatorship (NRFI06-2020-0022)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
backdoor attack detection, recurrent neural networks, text classification, nondeterministic finite automata, abstract models, adversarial attacks, data poisoning, explainable AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review