Explainable artificial intelligence for cybersecurity: a literature survey
Open accessThe DOI matches its Crossref record; 113 of the 114 references it cites were found in a registry or an index.
Abstract
Abstract With the extensive application of deep learning (DL) algorithms in recent years, e.g., for detecting Android malware or vulnerable source code, artificial intelligence (AI) and machine learning (ML) are increasingly becoming essential in the development of cybersecurity solutions. However, sharing the same fundamental limitation with other DL application domains, such as computer vision (CV) and natural language processing (NLP), AI-based cybersecurity solutions are incapable of justifying the results (ranging from detection and prediction to reasoning and decision-making) and making them understandable to humans. Consequently, explainable AI (XAI) has emerged as a paramount topic addressing the related challenges of making AI models explainable or interpretable to human users. It is particularly relevant in cybersecurity domain, in that XAI may allow security operators, who are overwhelmed with tens of thousands of security alerts per day (most of which are false positives), to better assess the potential threats and reduce alert fatigue. We conduct an extensive literature review on the intersection between XAI and cybersecurity. Particularly, we investigate the existing literature from two perspectives: the applications of XAI to cybersecurity (e.g., intrusion detection, malware classification), and the security of XAI (e.g., attacks on XAI pipelines, potential countermeasures). We characterize the security of XAI with several security properties that have been discussed in the literature. We also formulate open questions that are either unanswered or insufficiently addressed in the literature, and discuss future directions of research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This opacity creates challenges for cybersecurity analysts, who need explanations to validate alerts, investigate incidents, and comply with organizational or regulatory requirements (Charmet et al., 2022).”
From Explainable AI: enhancing decision-making in the detection of cyber threats (2026) · cites this paper as Charmet et al., 2022
Experimental — accuracy not yet measuredIn the citing paper
“For example, trust in algorithms could also be raised by ensuring robustness and by encouraging fairness [50].”
From Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare (2023) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 2×
“Gradient Boosting (GB) [35] is an ensemble learning method for modifying weak learners by unifying them into one stronger learner.”
BackgroundMethodThe Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280 · cited 1×
“They compared two models, an LR and a multilayer perceptron [101].”
MethodArrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321 · cited 3×
“Arrieta et al. [13] classified XAI models as white box or post hoc models, whereas the authors of [46] and [94] outlined desirable properties for XAI.”
BackgroundMethod
Research works printed but not linked: 23
- Adebayo J et al (2018) Sanity checks for saliency maps. In: Advances in neural information processing systems, p 31
- Anders C et al (2020) Fairwashing explanations with off-mani- fold detergent. In: International conference on machine learning. PMLR, pp 314-323
- Boopathy A et al (2020) Proper network interpretability helps adversarial robustness in classification. In: International con- ference on machine learning. PMLR, pp 1014-102
- Deng J et al (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee, pp 248-255 · DOI · Paper page
- Deng J et al (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee, pp 248-255 · DOI · Paper page
- Dimanov B et al (2020) You Shouldn't Trust Me: Learning models which conceal unfairness from multiple explanation methods. In: SafeAI@AAAI
- Dombrowski A-K et al (2019) Explanations can be manipulated and geometry is to blame. In: Advances in neural information processing systems, p 32
- Elson J et al (2007) Asirra: A CAPTCHA that exploits inter- est-aligned manual image categorization. In: CCS, vol 7, pp 366-374
- Heo J, Joo S, Moon T (2019) Fooling neural network interpreta- tions via adversarial model manipulation. In: Advances in neural information processing systems, p 32
- Hong S et al (2015) Online tracking by learning discriminative saliency map with convolutional neural network. In: International conference on machine learning. PMLR, pp 597-606
- Hooker S et al (2019) A benchmark for interpretability methods in deep neural networks. In: Advances in neural information pro- cessing systems, p 32
- Kleinbaum DG et al (2002) Logistic regression. Springer, New York
- Krizhevsky A, Hinton G et al (2009) Learning multiple layers of features from tiny images 65. Kuppa A, Le-Khac N-A (2021) Adversarial xai methods in cyber- security. In: IEEE transactions on information forensics and secu- rity, vol 16, pp 4924-4938
- Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. In: Advances in neural information processing systems, p 30
- Miura T, Hasegawa S, Shibahara T (2021) MEGEX: Data-free model extraction attack against gradient-based explainable AI. In: arXiv:2107.08909 · arXiv
- Moustafa N (2019) New generations of internet of things datasets for cybersecurity applications based machine learning: TON IoT datasets. In: Proceedings of the eResearch Australasia Conference. Brisbane, Australia, pp 21–25
- Netzer Y et al (2011) Reading digits in natural images with unsupervised feature learning. In: NIPS workshop on deep learning and unsupervised feature learning, p 2011
- Rish I et al (2001) An empirical study of the naive Bayes classifier. In: IJCAI 2001 workshop on empirical methods in artificial intelligence, vol 3, pp 41–46
- Slack DZ et al (2021) Reliable Post hoc Explanations: Mod- eling Uncertainty in Explainability. In: Beygelzimer A et al (eds) Advances in neural information processing systems. https:// openr eview. net/ forum? id= rqfq0 CYIekd. Accessed 01 Aug 2022 117.
- Slack D et al (2021) Counterfactual explanations can be manipu- lated. In: Advances in neural information processing systems, vol 34, pp 62-75 118.
- Wang Z et al (2020) Smoothed geometry for robust attribution. In: Advances in neural information processing systems, vol 33, pp 13623-13634 135.
- Zhang X et al (2020) Interpretable deep learning under fire. In: 29th fUSENIXg Security Symposium (fUSENIXg Security 20) 140.
- Zhao X et al (2021) BayLIME: Bayesian local interpretable model-agnostic explanations. In: UAI 141.
Cited by
Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare
Hulsen, T. (2023) · AI · doi:10.3390/ai4030034 · Cited by 305
“On the one hand, XAI can help to improve the safety and security of AI systems by making it easier to detect and prevent errors and malicious behavior [50].”
BackgroundArreche, O., Guntur, T. R., & Abdallah, M. (2024) · Applied Sciences · doi:10.3390/app14104170 · Cited by 224
Sarker, I. H., Janicke, H., Mohsin, A., Gill, A., & Maglaras, L. (2024) · ICT Express · doi:10.1016/j.icte.2024.05.007 · Cited by 188
Singh, A. R., Vishnuram, P., Alagarsamy, S., Bajaj, M., Blazek, V., Damaj, I., Rathore, R. S., Al-Wesabi, F. N., & Othman, K. M. (2024) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.06.093 · Cited by 183
A survey on explainable artificial intelligence for cybersecurity
Rjoub, G., Bentahar, J., Abdel Wahab, O., Mizouni, R., Song, A., Cohen, R., Otrok, H., & Mourad, A. (2023) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2023.3282740 · Cited by 158
Metadata
- Authors
- Charmet, F., Tanuwidjaja, H. C., Ayoubi, S., Gimenez, P.-F., Han, Y., Jmila, H., Blanc, G., Takahashi, T., & Zhang, Z.
- Affiliations
- National Institute of Information and Communications Technology (JP)
- Laboratoire Lorrain de Recherche en Informatique et ses Applications (FR)
- Institut de Recherche en Informatique et Systèmes Aléatoires (FR)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Institut Polytechnique de Paris (FR)
- Huawei Technologies (France) (FR)
- Corresponding author
- Charmet, Fabien
- ORCID
- Venue
- Annals of Telecommunications
- Published
- 2022-10-26
- Volume
- 77
- Issue
- 11-12
- Pages
- 789-812
- ISSN
- 0003-4347 (ISSN-L)
- 1958-9395
- OpenAlex
- W4307571902
- Cited by
- 167
- References
- 99
- Citations by year
- 2026: 62 · 2025: 52 · 2024: 35 · 2023: 18
- FWCI
- 8.64
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Annals of Telecommunications (publishedVersion, cc-by)
- HAL (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- explainable AI, cybersecurity, model interpretability, intrusion detection, malware classification, Android malware detection, security alerts, false positives, alarm fatigue
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Explainable artificial intelligence for cybersecurity: a literature survey. https://app.cochat.ai/papers/10.1007/s12243-022-00926-7
- The paper
- Charmet, F., Tanuwidjaja, H. C., Ayoubi, S., Gimenez, P.-F., Han, Y., Jmila, H., Blanc, G., Takahashi, T., & Zhang, Z. (2022). Explainable artificial intelligence for cybersecurity: a literature survey. Annals of Telecommunications. https://doi.org/10.1007/s12243-022-00926-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.