CoChat

Explainable AI: enhancing decision-making in the detection of cyber threats

Open access

The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index; 14 of the 39 sources its claims cite were checked for support.

Abstract

The rapid growth of the Internet and the increasing reliance on digital systems have significantly expanded the global digital footprint, creating new challenges for cybersecurity. Artificial Intelligence (AI) technologies, particularly Machine Learning (ML) and Deep Learning (DL), have become central to addressing these challenges by enabling the automation of complex and data-intensive tasks across antivirus solutions, intrusion prevention systems, threat intelligence platforms, and email security tools. While these technologies provide high levels of accuracy in detecting anomalies, malware, and other forms of malicious activity, they are often criticized for operating as “black-box” systems. The lack of interpretability in their decision-making processes limits the ability of cybersecurity professionals to fully understand, validate, and trust the outcomes of AI-driven models, thereby restricting their practical adoption in high-stakes environments. To mitigate these limitations, Explainable Artificial Intelligence (XAI) has emerged as a promising paradigm that aims to make AI outputs transparent, interpretable, and actionable. By providing human-understandable explanations of automated decisions, XAI can bridge the gap between technical performance and practitioner usability, enabling analysts to make informed decisions, improve incident response, and strengthen organizational resilience against both known and emerging threats. This paper reviews recent state-of-the-art developments in XAI for cybersecurity, with a particular emphasis on anomaly detection a critical area for identifying insider threats, zero-day exploits, and atypical system behavior. The review follows a structured literature analysis of peer-reviewed studies published between 2018 and 2025, identified through systematic searches in major academic databases including IEEE Xplore, Scopus, Web of Science, and ACM Digital Library. After applying predefined inclusion and exclusion criteria focused on XAI applications in cybersecurity, 53 relevant studies were analysed to synthesize methodological trends, application domains, and evaluation practices. Drawing on these findings, the paper consolidates fragmented research contributions, identifies current gaps, and provides recommendations for advancing the design and adoption of explainable, trustworthy AI systems in cybersecurity. The analysis further highlights a critical deployment challenge: the integration of explainability mechanisms often introduces trade-offs between predictive accuracy, computational efficiency, and real-time scalability factors that are essential in operational cybersecurity environments.

Overview

CoChat Summary

TL;DR

This review synthesizes 53 studies (2018–2025) on explainable AI (XAI) in cybersecurity, focusing on anomaly detection and intrusion detection. It finds that XAI is concentrated in IoT-related domains and dominated by post-hoc methods like SHAP and LIME, while practical adoption remains limited by missing standardized evaluation, weak end-user validation, and interpretability-performance trade-offs.

Key findings

  • XAI research in cybersecurity is heavily concentrated on IoT, IIoT, and IoMT, especially anomaly detection and intrusion detection systems; fewer works address cloud-native infrastructures, threat intelligence, or adaptive defence.
  • Post-hoc techniques dominate, particularly LIME and SHAP, while intrinsic explainability remains rare; only two reviewed papers explicitly assessed explanations with human analysts through surveys or HCI studies.
  • The field lacks standardized evaluation metrics for explanation quality, and many explanations are too technical for cybersecurity practitioners, limiting operational usefulness.
  • Integrating explainability often introduces trade-offs among predictive accuracy, computational efficiency, and real-time scalability, which are critical in operational cybersecurity environments.

Methods

  • Structured literature analysis of peer-reviewed studies published between 2018 and 2025, identified through systematic searches in IEEE Xplore, Scopus, Web of Science, and ACM Digital Library.
  • After predefined inclusion and exclusion criteria focused on XAI applications in cybersecurity, 53 relevant studies were analysed.
  • The review compares methodologies, datasets, AI models, XAI techniques, application domains, and whether studies included end-user evaluation.

Limitations

  • Stated: the paper identifies field-level limitations including lack of standardized evaluation metrics, limited practitioner involvement, over-reliance on benchmark datasets such as CIC-IDS2017, UNSW-NB15, and BoT-IoT, and explanations that are often too technical or abstract for operational use.
  • Stated: only two reviewed papers incorporated usability testing, surveys, or human-computer interaction experiments, so most explanations remain theoretically valid but operationally untested.
  • Evident from the text: the paper is a literature review and does not present new empirical validation or primary deployment data; its comparative tables in the provided text are partially incomplete, limiting independent verification of some summarized comparisons.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3389/fcomp.2026.1762332

Cites

50 of 51 references verified0 citation errors · 1 not checkable

Statistics

Nothing to recompute.

Figures

Image 1
CC-BY · From the PDF
Image 2
CC-BY · From the PDF
Image 3
CC-BY · From the PDF

Verification

Claims the paper makes

13 Valid · 8 Not valid · 27 Not checked · 68 Own results and interpretations

47 more claims

Create a free account to see all 116 claims and to verify the 1 reference still open.

See all claims — free account
  • It may state two separate claims as one. 38
  • Probably not a substantial claim. 33
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Cited works that were retracted

Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.

1 of 1

Citations

Cites

1 retracted

Show all 51 references

Research works printed but not linked: 3

  1. Bansal, U. (2025). "Towards resilient cyber defense: exploring the synergy of adversarial robustness and explainable AI in NIDS, " in 6th International Conference on Deep Learning, Artificial Intelligence and Robotics (ICDLAIR 2024), (Atlantis Press), 74-86.
  2. Patidar, N., Mishra, S., Jain, R., Prajapati, D., Solanki, A., Suthar, R., et al. (2024). Transparency in AI decision making: a survey of explainable AI methods and applica- tions. Adv. Robot. Technol. 2.
  3. Polinati, A. K. (2025). AI-powered anomaly detection in cybersecurity: leveraging deep learning for intrusion prevention. Int. J. Commun. Networks Inf. Secur. 17, 301-323.

Cited by

See all 16

Metadata

Authors
Prasad, P. W. C., Sayeed, M. S., Nguyen, D.-M., Hutabarat, D. P., & Mohiuddin, G. M.
Affiliations
  • Duy Tan University (VN)
  • Multimedia University (MY)
  • Binus University (ID)
Corresponding author
Sayeed, Md Shohel
Venue
Frontiers in Computer Science
Published
2026-03-20
Volume
8
ISSN
2624-9898 (ISSN-L)
OpenAlex
W7138846084
Cited by
16
References
46
Citations by year
2026: 16
FWCI
37.13
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Computer Science (publishedVersion, cc-by)
  • Siti Hasmah Digital Library-MMU Institutiona Repository (Multimedia University) (acceptedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
explainable AI, cybersecurity, anomaly detection, cyber threat detection, black-box models, interpretability, insider threats, zero-day attacks, incident response, trustworthy AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.