CoChat

Black Box Attacks on Explainable Artificial Intelligence(XAI) methods in Cyber Security

Kuppa, Aditya, Le-Khac, Nhien-An · 2020 · 2020 International Joint Conference on Neural Networks (IJCNN)

The DOI matches its Crossref record; 33 of the 35 references it cites were found in a registry or an index.

Abstract

Cybersecurity community is slowly leveraging Machine Learning (ML) to combat ever evolving threats. One of the biggest drivers for successful adoption of these models is how well domain experts and users are able to understand and trust their functionality. As these black-box models are being employed to make important predictions, the demand for transparency and explainability is increasing from the stakeholders.Explanations supporting the output of ML models are crucial in cyber security, where experts require far more information from the model than a simple binary output for their analysis. Recent approaches in the literature have focused on three different areas: (a) creating and improving explainability methods which help users better understand the internal workings of ML models and their outputs; (b) attacks on interpreters in white box setting; (c) defining the exact properties and metrics of the explanations generated by models. However, they have not covered, the security properties and threat models relevant to cybersecurity domain, and attacks on explainable models in black box settings.In this paper, we bridge this gap by proposing a taxonomy for Explainable Artificial Intelligence (XAI) methods, covering various security properties and threat models relevant to cyber security domain. We design a novel black box attack for analyzing the consistency, correctness and confidence security properties of gradient based XAI methods. We validate our proposed system on 3…

Overview

Integrity signals

Registry

Crossref · 10.1109/ijcnn48605.2020.9206780

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 33 references

Research works printed but not linked: 8

  1. Swartout, 1983, XPLAIN: A system for creating and explaining expert consulting programs Artificial intelligence
  2. 10.1145/279232.279236 · DOI · Paper page
  3. Jones, 2014, {SciPy}: Open source scientific tools for {Python}
  4. 10.1111/rssb.12508 · DOI
  5. 10.1080/00401706.1987.10488218 · doi:10.1080/00401706.1987.10488218 · Paper page
  6. 10.1109/eurospw51379.2020.00045 · DOI
  7. Hall, A Systematic Method to Understand Requirements for Explainable AI (XAI) Systems
  8. 10.1609/aaai.v33i01.33013681 · DOI · Paper page

Cited by

See all 123

Metadata

Authors
Kuppa, A., & Le-Khac, N.-A.
Affiliations
University College Dublin (IE)
Venue
2020 International Joint Conference on Neural Networks (IJCNN)
Publisher
IEEE
Published
2020-07
Pages
1-8
ISSN
2161-4407 (ISSN-L)
OpenAlex
W3091163785
Cited by
123
References
35
Citations by year
2026: 13 · 2025: 24 · 2024: 32 · 2023: 25 · 2022: 17 · 2021: 11 · 2020: 1
FWCI
5.15
Type
conference paper
Language
en
Open access
closed
Locations
International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
Keywords
explainable AI, black-box attacks, threat model, gradient-based explanation, explanation consistency, trustworthy machine learning, cybersecurity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Black Box Attacks on Explainable Artificial Intelligence(XAI) methods in Cyber Security. https://app.cochat.ai/papers/10.1109/ijcnn48605.2020.9206780
The paper
Kuppa, A., & Le-Khac, N.-A. (2020). Black Box Attacks on Explainable Artificial Intelligence(XAI) methods in Cyber Security. 2020 International Joint Conference on Neural Networks (IJCNN). https://doi.org/10.1109/ijcnn48605.2020.9206780

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.