Adversarial Detection with Model Interpretation
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Machine learning (ML) systems have been increasingly applied in web security applications such as spammer detection, malware detection and fraud detection. These applications have an intrinsic adversarial nature where intelligent attackers can adaptively change their behaviors to avoid being detected by the deployed detectors. Existing efforts against adversaries are usually limited by the type of applied ML models or the specific applications such as image classification. Additionally, the working mechanisms of ML models usually cannot be well understood by users, which in turn impede them from understanding the vulnerabilities of models nor improving their robustness. To bridge the gap, in this paper, we propose to investigate whether model interpretation could potentially help adversarial detection. Specifically, we develop a novel adversary-resistant detection framework by utilizing the interpretation of ML models. The interpretation process explains the mechanism of how the target ML model makes prediction for a given instance, thus providing more insights for crafting adversarial samples. The robustness of detectors is then improved through adversarial training with the adversarial samples. A data-driven method is also developed to empirically estimate costs of adversaries in feature manipulation. Our approach is model-agnostic and can be applied to various types of classification models. Our experimental results on two real-world datasets demonstrate the effectiveness of interpretation-based attacks and how estimated feature manipulation cost would affect the behavior of adversaries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317 · cited 2×
“Third, along with the popularity of deep neural networks (DNNs) and its vulnerability to adversaries [48], various approaches have been developed for DNNs to defend against adversaries [22, 28, 35, 38] using fast gradient method or its variations.”
BackgroundMethodThe Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068
Jindal, N., & Liu, B. (2008) · Proceedings of the international conference on Web search and web data mining - WSDM '08 · doi:10.1145/1341531.1341560 · Cited by 1,542 · cited 1×
“Classification models are increasingly used in security-related applications like spam filtering, biometric authentication and malware detection [4, 15, 25, 36, 39].”
MethodTransferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326
Delving into Transferable Adversarial Examples and Black-box Attacks
Liu, Y., Chen, X., Liu, C., & Song, D. (2016) · arXiv · doi:10.48550/arxiv.1611.02770 · Cited by 1,310
Evasion Attacks against Machine Learning at Test Time
Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40994-3_25 · Cited by 890 · cited 8×
“• Gradient Descent Attacking method (GDA) [6]: An attacking method moving samples along the direction of gradient vectors.”
MethodInfluentialSpotting fake reviewer groups in consumer reviews
Mukherjee, A., Liu, B., & Glance, N. (2012) · Proceedings of the 21st international conference on World Wide Web · doi:10.1145/2187836.2187863 · Cited by 773 · cited 2×
“5 RELATEDWORK Classification models are increasingly used in security-related applications like spam filtering, biometric authentication and malware detection [4, 15, 25, 36, 39].”
MethodDie Free or Live Hard? Empirical Evaluation and New Design for Fighting Evolving Twitter Spammers
Yang, C., Harkreader, R. C., & Gu, G. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-23644-0_17 · Cited by 235
Stackelberg games for adversarial prediction problems
Brückner, M., & Scheffer, T. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020495 · Cited by 229 · cited 2×
“To estimate feature robustness, a straightforward way is to find the closed-form formula for each feature and explicitly compute the difficulty of changing its value [17, 53], or to unify the cost to a constant value [11].”
BackgroundMethodData Poisoning Attacks against Autoregressive Models
Alfeld, S., Zhu, X., & Barford, P. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10237 · Cited by 205
Contextual Outlier Interpretation
Liu, N., Shin, D., & Hu, X. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/341 · Cited by 74
Self-citationHow to Explain Individual Classification Decisions
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859912 · Cited by 54 · cited 5×
“To tackle this issue, some recent work tries to identify how individual decisions are made [3, 30, 46] or to provide intuitive descriptions to the concepts learned by the models [16, 26].”
BackgroundMethodInfluential
Cited by
Techniques for interpretable machine learning
Du, M., Liu, N., & Hu, X. (2019) · Communications of the ACM · doi:10.1145/3359786 · Cited by 1,268
Self-citationTechniques for Interpretable Machine Learning
Du, M., Liu, N., & Hu, X. (2018) · arXiv · doi:10.48550/arxiv.1808.00033 · Cited by 1,261
Self-citationdEFEND: Explainable Fake News Detection
Shu, K., Cui, L., Wang, S., Lee, D., & Liu, H. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330935 · Cited by 604
Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590
Zogan, H., Razzak, I., Wang, X., Jameel, S., & Xu, G. (2022) · World Wide Web · doi:10.1007/s11280-021-00992-2 · Cited by 197
“As a result, understanding the representations acquired by neurones in intermediate levels of DNNs is important to the explanation of deep neural networks (DNNs) [27, 28].”
Background
Metadata
- Authors
- Liu, N., Yang, H., & Hu, X.
- Venue
- Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2018-07-19
- Pages
- 1803-1811
- OpenAlex
- W2809376420
- Cited by
- 56
- References
- 17
- FWCI
- 3.23
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.