Explainable AI: enhancing decision-making in the detection of cyber threats
Open accessThe DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index; 14 of the 39 sources its claims cite were checked for support.
Abstract
The rapid growth of the Internet and the increasing reliance on digital systems have significantly expanded the global digital footprint, creating new challenges for cybersecurity. Artificial Intelligence (AI) technologies, particularly Machine Learning (ML) and Deep Learning (DL), have become central to addressing these challenges by enabling the automation of complex and data-intensive tasks across antivirus solutions, intrusion prevention systems, threat intelligence platforms, and email security tools. While these technologies provide high levels of accuracy in detecting anomalies, malware, and other forms of malicious activity, they are often criticized for operating as “black-box” systems. The lack of interpretability in their decision-making processes limits the ability of cybersecurity professionals to fully understand, validate, and trust the outcomes of AI-driven models, thereby restricting their practical adoption in high-stakes environments. To mitigate these limitations, Explainable Artificial Intelligence (XAI) has emerged as a promising paradigm that aims to make AI outputs transparent, interpretable, and actionable. By providing human-understandable explanations of automated decisions, XAI can bridge the gap between technical performance and practitioner usability, enabling analysts to make informed decisions, improve incident response, and strengthen organizational resilience against both known and emerging threats. This paper reviews recent state-of-the-art developments in XAI for cybersecurity, with a particular emphasis on anomaly detection a critical area for identifying insider threats, zero-day exploits, and atypical system behavior. The review follows a structured literature analysis of peer-reviewed studies published between 2018 and 2025, identified through systematic searches in major academic databases including IEEE Xplore, Scopus, Web of Science, and ACM Digital Library. After applying predefined inclusion and exclusion criteria focused on XAI applications in cybersecurity, 53 relevant studies were analysed to synthesize methodological trends, application domains, and evaluation practices. Drawing on these findings, the paper consolidates fragmented research contributions, identifies current gaps, and provides recommendations for advancing the design and adoption of explainable, trustworthy AI systems in cybersecurity. The analysis further highlights a critical deployment challenge: the integration of explainability mechanisms often introduces trade-offs between predictive accuracy, computational efficiency, and real-time scalability factors that are essential in operational cybersecurity environments.
Overview
CoChat Summary
TL;DR
This review synthesizes 53 studies (2018–2025) on explainable AI (XAI) in cybersecurity, focusing on anomaly detection and intrusion detection. It finds that XAI is concentrated in IoT-related domains and dominated by post-hoc methods like SHAP and LIME, while practical adoption remains limited by missing standardized evaluation, weak end-user validation, and interpretability-performance trade-offs.
Key findings
- XAI research in cybersecurity is heavily concentrated on IoT, IIoT, and IoMT, especially anomaly detection and intrusion detection systems; fewer works address cloud-native infrastructures, threat intelligence, or adaptive defence.
- Post-hoc techniques dominate, particularly LIME and SHAP, while intrinsic explainability remains rare; only two reviewed papers explicitly assessed explanations with human analysts through surveys or HCI studies.
- The field lacks standardized evaluation metrics for explanation quality, and many explanations are too technical for cybersecurity practitioners, limiting operational usefulness.
- Integrating explainability often introduces trade-offs among predictive accuracy, computational efficiency, and real-time scalability, which are critical in operational cybersecurity environments.
Methods
- Structured literature analysis of peer-reviewed studies published between 2018 and 2025, identified through systematic searches in IEEE Xplore, Scopus, Web of Science, and ACM Digital Library.
- After predefined inclusion and exclusion criteria focused on XAI applications in cybersecurity, 53 relevant studies were analysed.
- The review compares methodologies, datasets, AI models, XAI techniques, application domains, and whether studies included end-user evaluation.
Limitations
- Stated: the paper identifies field-level limitations including lack of standardized evaluation metrics, limited practitioner involvement, over-reliance on benchmark datasets such as CIC-IDS2017, UNSW-NB15, and BoT-IoT, and explanations that are often too technical or abstract for operational use.
- Stated: only two reviewed papers incorporated usability testing, surveys, or human-computer interaction experiments, so most explanations remain theoretically valid but operationally untested.
- Evident from the text: the paper is a literature review and does not present new empirical validation or primary deployment data; its comparative tables in the provided text are partially incomplete, limiting independent verification of some summarized comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
13 Valid · 8 Not valid · 27 Not checked · 68 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Explanation techniques such as SHAP and LIME allow analysts to identify which network attributes such as packet size anomalies, unusual port activity, irregular authentication patterns, or abnormal traffic frequency contributed most significantly to a malicious classification (Rahman et al., 2024).”
§ Explainable AI: enhancing decision-making in the detection of cyber threats > 2.3 Application of XAI in cyber threat detection
What the sources say
- Partly supportedcites Rahman et al., 2024 matched to “Explainable anomaly detection in encrypted network traffic using data analytics” (2024) sourceExperimental — accuracy not yet measured
Section 4.2 Explainability Outcomes (page not provided) says SHAP identified flow duration, packet inter-arrival variance, and average packet size as important contributors to anomaly detection, while LIME gave interpretable explanations of flagged anomalies. This supports the claim about packet size and traffic timing, but the cited passages do not establish unusual port activity or irregular authentication patterns as attributes identified by these explanations. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This opacity creates challenges for cybersecurity analysts, who need explanations to validate alerts, investigate incidents, and comply with organizational or regulatory requirements (Charmet et al., 2022).”
§ Explainable AI: enhancing decision-making in the detection of cyber threats > 2.1 Artificial intelligence
What the sources say
- Partly supportedcites Charmet et al., 2022 matched to “Explainable artificial intelligence for cybersecurity: a literature survey” (2022) sourceExperimental — accuracy not yet measured
The Abstract says opaque AI-based cybersecurity results are difficult for humans to understand and that explanations help security operators assess potential threats. Section 6.3 describes using additional information to confirm that a malware alert reflects a real threat. The Introduction discusses legal requirements for explanations generally, but the paper does not establish that cybersecurity analysts need explanations to comply with organizational or regulatory requirements. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While these datasets are valuable for evaluation, they do not reflect the complexity and unpredictability of operational contexts such as IoT systems, industrial infrastructures, or healthcare networks (Alabdulatif, 2025).”
§ Explainable AI: enhancing decision-making in the detection of cyber threats > 1.1 Research motivation and gaps
What the sources say
- Partly supportedcites Alabdulatif, 2025 matched to “A novel ensemble of deep learning approach for cybersecurity intrusion detection with explainable artificial intelligence” (2025) sourceExperimental — accuracy not yet measured
The Background and Literature Review (page unavailable) describes CIC-IDS2017, UNSW-NB15, and BoT-IoT as datasets used to evaluate intrusion-detection models. It criticizes some models for lacking real-time deployment, but does not say that the datasets themselves fail to reflect the complexity or unpredictability of IoT, industrial, or healthcare environments. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“A lack of standardized evaluation metrics makes it difficult to compare the quality and reliability of explanations across studies (Tiwari et al., 2020).”
§ Explainable AI: enhancing decision-making in the detection of cyber threats > 1.1 Research motivation and gaps
What the sources say
- Could not verifycites Tiwari et al., 2020 matched to “The role of explainable AI in cybersecurity: addressing transparency challenges in autonomous defense systems” (2020) sourceExperimental — accuracy not yet measured
Section 1.6 proposes benchmarks for evaluating user confidence in AI, but does not discuss standardized metrics for the quality or reliability of explanations. Searches for evaluation metrics and cross-study comparisons found no passage addressing the claimed difficulty of comparing explanations. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“First, the diversity of XAI techniques such as SHAP, LIME, or gradient-based methods creates inconsistency in evaluation, as no universally accepted benchmarks exist (Calzarossa et al., 2025).”
§ Explainable AI: enhancing decision-making in the detection of cyber threats > 1 Introduction
What the sources say
- Partly supportedcites Calzarossa et al., 2025 matched to “An assessment framework for explainable AI with applications to cybersecurity” (2025) sourceExperimental — accuracy not yet measured
Section 3 (Methodology; page unavailable) distinguishes several types of XAI methods and says that, to the authors’ knowledge, explanations from different models had not been compared using the same dataset. That supports a gap in comparative evaluation, but not the stronger claim that no universally accepted benchmarks exist or that technique diversity causes inconsistent evaluation. The cited passages also do not identify LIME or gradient-based methods. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
47 more claims
Create a free account to see all 116 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 38
- Probably not a substantial claim. 33
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Explainable AI: enhancing decision-making in the detection of cyber threats > 3 Related studies
“Aljuhani et al. (2024) present the Shapely eXplainable Anomaly Detection (SXAD) framework”
§ Explainable AI: enhancing decision-making in the detection of cyber threats > 7 Conclusion
“| Alam et al. (2024) | Shapely eXplainable Anomaly Detection (SXAD) | Non-specific | Anomaly detection in logs |”
Cited works that were retracted
Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.
A note, not an issue: for example, a cited work that was retracted only after this paper was published, which its authors could not have known.
In the paper
“Almadhor, 2024, Strengthening network DDOS attack detection in heterogeneous IoT environment with federated XAI learning approach, Sci. Rep., 14, 24322, 10.1038/s41598-024-76016-6”
Cited work 10.1038/s41598-024-76016-6 · retracted 2026-04-22 · notice 10.1038/s41598-026-49234-3 · this paper published 2026-03-20
Citations
Cites
1 retracted
Almadhor, A. S., Altalbe, A. A., Bouazzi, I. R., Hejaili, A. A., & Kryvinska, N. (2024) · Scientific Reports · doi:10.1038/s41598-024-76016-6 · Cited by 36
RetractedToward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization
Sharafaldin, I., Habibi Lashkari, A., & Ghorbani, A. A. (2018) · Proceedings of the 4th International Conference on Information Systems Security and Privacy · doi:10.5220/0006639801080116 · Cited by 5,318
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
Explainable AI (XAI): Core Ideas, Techniques, and Solutions
Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., Qian, B., Wen, Z., Shah, T., Morgan, G., et al. (2023) · ACM Computing Surveys · doi:10.1145/3561048 · Cited by 1,386
Ahmed, I., Jeon, G., & Piccialli, F. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3146552 · Cited by 853
Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
Zhang, Z., Hamadi, H. A., Damiani, E., Yeun, C. Y., & Taher, F. (2022) · IEEE Access · doi:10.1109/access.2022.3204051 · Cited by 450
Mohale, V. Z., & Obagbuwa, I. C. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1526221 · Cited by 279
Moustafa, N., Koroniotis, N., Keshk, M., Zomaya, A. Y., & Tari, Z. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3280465 · Cited by 266
An explainable deep learning-enabled intrusion detection framework in IoT networks
Keshk, M., Koroniotis, N., Pham, N., Moustafa, N., Turnbull, B., & Zomaya, A. Y. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119000 · Cited by 261
Gaspar, D., Silva, P., & Silva, C. (2024) · IEEE Access · doi:10.1109/access.2024.3368377 · Cited by 234
Research works printed but not linked: 3
- Bansal, U. (2025). "Towards resilient cyber defense: exploring the synergy of adversarial robustness and explainable AI in NIDS, " in 6th International Conference on Deep Learning, Artificial Intelligence and Robotics (ICDLAIR 2024), (Atlantis Press), 74-86.
- Patidar, N., Mishra, S., Jain, R., Prajapati, D., Solanki, A., Suthar, R., et al. (2024). Transparency in AI decision making: a survey of explainable AI methods and applica- tions. Adv. Robot. Technol. 2.
- Polinati, A. K. (2025). AI-powered anomaly detection in cybersecurity: leveraging deep learning for intrusion prevention. Int. J. Commun. Networks Inf. Secur. 17, 301-323.
Cited by
Cyber Threats in Connected Autonomous Vehicles: A Protocol-Grounded SurveyPreprint
Hasan, M. M., Bhattacharya, M., & Islam, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6902580 · Cited by 1
FortiSMB: AI-Driven Insider Threat Detection and Explainable Security System for Mental-Health SMBs
Mohamed, S. W., Saad, A., Mohamed, S., Ahmed, A., Abdelnasser, Y., Métwalli, A., Saad, M., & Badawy, O. M. (2026) · International Conference on Machine Intelligence and Smart Innovation (ICMISI) · doi:10.1109/icmisi69868.2026.11583982
Almufarreh, A., Ahmad, A., Arshad, M., Onn, C. W., & Kiplangat, Y. M. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1863891
AI-Driven Cyber Defense Architectures for Mitigating Generative Adversarial Attacks
Abdullah, S. K., Al-Dulaimi, M. K. H., & Ivakhnenko, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-39711-9_1
A Threat Modeling Prioritization and Automation Framework for Composable Architectures
Popescu, L.-M., & Brad, R. (2026) · Future Internet · doi:10.3390/fi18100499
Human-Centric Explainable Federated Threat Intelligence for Trustworthy Cyber-Physical Systems
Kolusu, S., Polisetty, S. N. K., Srinivas, M. S., Varma, D. S. P., Balakrishna, B., & Krishna, B. L. V. S. R. (2026) · International Conference on Soft Computing for Security Applications (ICSCSA) · doi:10.1109/icscsa69367.2026.11708380
Explainable AI in Emerging Cybersecurity Domains
Ansari, S. K., Ahmad, Y., Shajahan, N., Husain, S., Appukuttan, A., & Shahzad, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1970-2.ch008
Explainability in Malware and Ransomware Detection
Sarvakar, K., & Prajapati, D. R. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1970-2.ch002
Metadata
- Authors
- Prasad, P. W. C., Sayeed, M. S., Nguyen, D.-M., Hutabarat, D. P., & Mohiuddin, G. M.
- Affiliations
- Duy Tan University (VN)
- Multimedia University (MY)
- Binus University (ID)
- Corresponding author
- Sayeed, Md Shohel
- Venue
- Frontiers in Computer Science
- Publisher
- Frontiers Media SA
- Published
- 2026-03-20
- Volume
- 8
- ISSN
- 2624-9898 (ISSN-L)
- OpenAlex
- W7138846084
- Semantic Scholar
- d928ba3eb9cd8f6bf1873063ecccc3d4344dcf6d
- Cited by
- 16
- References
- 46
- Citations by year
- 2026: 16
- FWCI
- 37.13
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Computer Science (publishedVersion, cc-by)
- Siti Hasmah Digital Library-MMU Institutiona Repository (Multimedia University) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, cybersecurity, anomaly detection, cyber threat detection, black-box models, interpretability, insider threats, zero-day attacks, incident response, trustworthy AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


