Artificial Intelligence in Malware and Network Intrusion Detection: A Comprehensive Survey of Techniques, Datasets, Challenges, and Future Directions
Open accessThe DOI matches its Crossref record; 21 of the 22 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
As cyber threats evolve in complexity and scale, traditional detection mechanisms are increasingly inadequate. This paper presents a comprehensive survey of artificial intelligence (AI) applications in malware and network intrusion detection, emphasizing the integration of intelligent techniques for real-time threat mitigation. We categorize state-of-the-art methods across supervised, unsupervised, and reinforcement learning, including deep learning architectures such as CNNs, RNNs, LSTMs, and ensemble models. Static, dynamic, and hybrid analysis techniques are compared, with a focus on feature engineering, behavioral modeling, and real-world deployment constraints. A novel AI-based Malware Detection and Prevention Framework is proposed, combining machine learning classifiers with Network Intrusion Prevention Systems (NIPS) to enhance proactive defense capabilities. The study evaluates publicly available and synthetic datasets, addressing challenges such as class imbalance, adversarial evasion, and data scarcity. We also highlight ethical considerations including bias, privacy, and accountability in AI-enabled cybersecurity systems. Case studies from mobile and IoT ecosystems demonstrate the practicality and limitations of AI-based defenses in dynamic threat landscapes. Finally, the paper outlines future research directions in explainable AI, automated model generation (AutoML), and adaptive, context-aware intrusion prevention systems. This work serves as a critical resource for developing resilient, scalable, and intelligent cybersecurity infrastructures aligned with modern digital ecosystems
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“While these systems often achieve high accuracy, their opaque nature limits interpretability and reduces trust among security professionals (Moamin et al., 2025).”
From Explainable AI: enhancing decision-making in the detection of cyber threats (2026) · cites this paper as Moamin et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 69
- R. C. Pivetta de Araujo et al., "Serial mass screening for tuberculosis among incarcerated persons in Brazil," Clinical Infectious Diseases, vol. 78, no. 6, pp. 1669-1676, 2024.
- Y. Li and Q. Liu, "A comprehensive review study of cyber-attacks and cyber security; Emerging trends and recent developments," Energy Reports, vol. 7, pp. 8176-8186, 2021.
- M. Fahad, H. Airf, A. Kumar, and H. K. Hussain, "Securing against apts: Advancements in detection and mitigation," BIN: Bulletin Of Informatics, vol. 1, no. 2, 2023.
- L. Wang, C. Sun, C. Zhang, W. Nie, and K. Huang, "Application of knowledge graph in software engineering field: A systematic literature review," Information and Software Technology, vol. 164, p. 107327, 2023.
- I. P. Saputra, E. Utami, and A. H. Muhammad, "Comparison of anomaly based and signature based methods in detection of scanning vulnerability," in 2022 9th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), 2022: IEEE, pp. 221-225.
- A. Aboalassaad and T. Malik, "The Impact of IPS and Firewall Placement on Network Security and Performance," in 2024 Cyber Research Conference-Ireland (Cyber-RCI), 2024: IEEE, pp. 1-7.
- R. Al-Amri, A. Hadi, A. H. Mousa, H. Hasan, and M. Kadhim, "The development of a deep learning model for predicting stock prices," Journal of Advanced Research in Applied Sciences and Engineering Technology, vol. 31, no. 3, pp. 208-219, 2023.
- V. Hassija et al., "Interpreting black-box models: a review on explainable artificial intelligence," Cognitive Computation, vol. 16, no. 1, pp. 45-74, 2024.
- R. M. Al-Amri, A. A. Hadi, M. S. Kadhim, A. H. Mousa, A. Z. Matloob, and H. F. Hasan, "Enhancement of The Performance of Machine Learning Algorithms to Rival Deep Learning Algorithms in Predicting Stock Prices," Babylonian Journal of Artificial Intelligence, vol. 2024, pp. 118-127, 2024.
- H. Alqahtani and G. Kumar, "Advances in artificial intelligence for detecting algorithmically generated domains: Current trends and future prospects," Engineering Applications of Artificial Intelligence, vol. 138, p. 109410, 2024.
- P. Bhat, S. Behal, and K. Dutta, "A system call-based android malware detection approach with homogeneous & eterogeneous ensemble machine learning," Computers & Security, vol. 130, p. 103277, 2023.
- T. Bilot, N. El Madhoun, K. Al Agha, and A. Zouaoui, "A survey on malware detection with graph representation learning," ACM Computing Surveys, vol. 56, no. 11, pp. 1-36, 2024.
- M. A. Talukder et al., "Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction," Journal of big data, vol. 11, no. 1, p. 33, 2024.
- M. Schmitt, "Securing the digital world: Protecting smart infrastructures and digital industries with artificial intelligence (AI)-enabled malware and intrusion detection," Journal of Industrial Information Integration, vol. 36, p. 100520, 2023.
- R. M. Al-Amri, D. N. Hamood, and A. K. Farhan, "Theoretical background of cryptography," Mesopotamian Journal of CyberSecurity, vol. 2023, pp. 7-15, 2023.
- A. A. Hashim and A. H. Mousa, "An evaluation framework for diabetes prediction techniques using machine learning," in BIO Web of Conferences, 2024, vol. 97: EDP Sciences, p. 00125.
- M. E. Manaa, F. J. Abd Al-Razaq, and H. A. Al-Khamees, "Enhancing IoT Security: An Optimization Algorithm for Fog Layer-Based DDoS Attack Mitigation Framework," Iraqi Journal of Science, 2025.
- A. Gaurav, B. B. Gupta, and P. K. Panigrahi, "A comprehensive survey on machine learning approaches for malware detection in IoT-based enterprise information system," Enterprise Information Systems, vol. 17, no. 3, p. 2023764, 2023.
- S. M. Ali, R. M. Al-Amri, and A. K. Farhan, "Generation of Dynamic Substitution Boxes Using HSM Chaos System for Application in Color Images Encrypting," International Journal of Intelligent Engineering & Systems, vol. 16, no. 6, 2023.
- D. Trizna, L. Demetrio, B. Biggio, and F. Roli, "Nebula: Self-attention for dynamic malware analysis," IEEE Transactions on Information Forensics and Security, 2024.
- G. Karat, J. M. Kannimoola, N. Nair, A. Vazhayil, S. VG, and P. Poornachandran, "CNN-LSTM hybrid model for enhanced malware analysis and detection," Procedia Computer Science, vol. 233, pp. 492-503, 2024.
- H. Jmila and M. I. Khedher, "Adversarial machine learning for network intrusion detection: A comparative study," Computer Networks, vol. 214, p. 109073, 2022.
- C. P. Chenet, A. Savino, and S. Di Carlo, "A survey on hardware-based malware detection approaches," IEEE Access, 2024.
- Y. Li, W. Zhao, Y. Su, W. Li, and C. Yuan, "Overview of cloud computing deployment mode and technology development trend," in 2023 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), 2023: IEEE, pp. 1-5.
- M. Ali, S. Shiaeles, M. Papadaki, and B. V. Ghita, "Agent-based vs agent-less sandbox for dynamic behavioral analysis," in 2018 Global Information Infrastructure and Networking Symposium (GIIS), 2018: IEEE, pp. 1-5.
- C. Li et al., "DMalNet: Dynamic malware analysis based on API feature engineering and graph learning," Computers & Security, vol. 122, p. 102872, 2022.
- Z. Dai et al., "An intrusion detection model to detect zero-day attacks in unseen data using machine learning," PloS one, vol. 19, no. 9, p. e0308469, 2024.
- S. Kumar, P. Ahlawat, and J. Sahni, "IOT malware detection using static and dynamic analysis techniques: A systematic literature review," Security and Privacy, vol. 7, no. 6, p. e444, 2024.
- A. Q. Raheema, M. Kamalrudin, and N. R. Dzakiyullah, "Multi-Level Fusion for Enhanced Host-based Malware Detection in ICT-Enabled Smart Cities," Fusion: Practice & Applications, vol. 15, no. 2, 2024.
- A. K. Ghazi-Tehrani and H. N. Pontell, "Phishing evolves: Analyzing the enduring cybercrime," in The New Technology of Financial Crime: Routledge, 2022, pp. 35-61.
- H. A. Noman, Q. Al-Maatouk, and S. A. Noman, "A static analysis tool for malware detection," in 2021 International Conference on Data Analytics for Business and Industry (ICDABI), 2021: IEEE, pp. 661-665.
- M. V. S. Babu and K. Banana, "A study on narrow artificial intelligence-An overview," Int. J. Eng. Sci. Adv. Technol, vol. 24, pp. 210-219, 2024.
- M. Ashik et al., "Detection of malicious software by analyzing distinct artifacts using machine learning and deep learning algorithms," Electronics, vol. 10, no. 14, p. 1694, 2021.
- E. E. Abdallah and A. F. Otoom, "Intrusion detection systems using supervised machine learning techniques: a survey," Procedia Computer Science, vol. 201, pp. 205-212, 2022.
- A. Alshammari and A. Aldribi, "Apply machine learning techniques to detect malicious network traffic in cloud computing," Journal of Big Data, vol. 8, no. 1, p. 90, 2021.
- Ü. Atila, M. Uçar, K. Akyol, and E. Uçar, "Plant leaf disease classification using EfficientNet deep learning model," Ecological Informatics, vol. 61, p. 101182, 2021.
- M. Ozkan-Okay, R. Samet, Ö. Aslan, and D. Gupta, "A comprehensive systematic literature review on intrusion detection systems," IEEE Access, vol. 9, pp. 157727-157760, 2021.
- O. H. Abdulganiyu, T. Ait Tchakoucht, and Y. K. Saheed, "A systematic literature review for network intrusion detection system (IDS)," International journal of information security, vol. 22, no. 5, pp. 1125-1162, 2023.
- H. Satilmiş, S. Akleylek, and Z. Y. Tok, "A systematic literature review on host-based intrusion detection systems," Ieee Access, vol. 12, pp. 27237-27266, 2024.
- H. Satilmiş, S. Akleylek, and Z. Y. Tok, "A systematic literature review on host-based intrusion detection systems," Ieee Access, vol. 12, pp. 27237-27266, 2024.
- B. Nawaal, U. Haider, I. U. Khan, and M. Fayaz, "Signature-based intrusion detection system for IoT," in Cyber Security for Next-Generation Computing Technologies: CRC Press, 2024, pp. 141-158.
- S. Einy, C. Oz, and Y. D. Navaei, "The anomaly-and signature-based IDS for network security using hybrid inference systems," Mathematical Problems in Engineering, vol. 2021, no. 1, p. 6639714, 2021.
- M. A. Alsoufi et al., "Anomaly-based intrusion detection systems in iot using deep learning: A systematic literature review," Applied sciences, vol. 11, no. 18, p. 8383, 2021.
- M. Bhavsar, K. Roy, J. Kelly, and O. Olusola, "Anomaly-based intrusion detection system for IoT application," Discover Internet of things, vol. 3, no. 1, p. 5, 2023.
- Z. K. Maseer, R. Yusof, N. Bahaman, S. A. Mostafa, and C. F. M. Foozy, "Benchmarking of machine learning for anomaly based intrusion detection systems in the CICIDS2017 dataset," IEEE access, vol. 9, pp. 22351-22370, 2021.
- G. Karat, J. M. Kannimoola, N. Nair, A. Vazhayil, S. VG, and P. Poornachandran, "CNN-LSTM hybrid model for enhanced malware analysis and detection," Procedia Computer Science, vol. 233, pp. 492-503, 2024.
- J. S. Bailey and M. R. Burch, Research methods in applied behavior analysis. Routledge, 2024.
- F. A. Vadhil, M. L. Salihi, and M. F. Nanne, "Machine learning-based intrusion detection system for detecting web attacks," IAES International Journal of Artificial Intelligence, vol. 13, no. 1, pp. 711-721, 2024.
- G. Singh and N. Khare, "A survey of intrusion detection from the perspective of intrusion datasets and machine learning techniques," International Journal of Computers and Applications, vol. 44, no. 7, pp. 659-669, 2022.
- A. Khanan, Y. A. Mohamed, A. H. H. Mohamed, and M. Bashir, "From bytes to insights: a systematic literature review on unraveling IDS datasets for enhanced cybersecurity understanding," IEEE Access, vol. 12, pp. 59289- 59317, 2024.
- M. S. Iqbal, W. Ahmad, R. Alizadehsani, S. Hussain, and R. Rehman, "Breast cancer dataset, classification and detection using deep learning," in Healthcare, 2022, vol. 10, no. 12: MDPI, p. 2395.
- E. Alomari, I. Katib, A. Albeshri, T. Yigitcanlar, and R. Mehmood, "Iktishaf+: a big data tool with automatic labeling for road traffic social sensing and event detection using distributed machine learning," Sensors, vol. 21, no. 9, p. 2993, 2021.
- H. Yukhymenko, R. Staab, M. Vero, and M. Vechev, "A synthetic dataset for personal attribute inference," Advances in Neural Information Processing Systems, vol. 37, pp. 120735-120779, 2024.
- Ammara, D. A., Ding, J., & Tutschku, K. (2024). "Synthetic Data Generation in Cybersecurity: A Comparative Analysis." arXiv preprint arXiv:2410.16326. · arXiv
- P. Bedi, N. Gupta, and V. Jindal, "I-SiamIDS: An Improved Siam-IDS for Handling Class Imbalance in Network- Based Intrusion Detection Systems," arXiv preprint arXiv:2009.10940, Sep. 2020. · arXiv
- H. J. Ahn, S. Sharma, R. Matam, and J. Park, "A comprehensive survey on Android malware detection using machine learning," IEEE Access, vol. 11, pp. 39193-39226, 2023. doi: 10.1109/ACCESS.2023.3260207. · doi:10.1109/access.2023.3260207
- A. Kumar, S. Sharma, and R. Singh, "Static analysis framework for permission-based dataset generation and android malware detection using machine learning," EURASIP Journal on Information Security, vol. 2024, no. 33, pp. 1-17, Oct. 2024
- M. Nasir, A. R. Javed, M. A. Tariq, M. Asim, and T. Baker, "Feature engineering and deep learning-based intrusion detection framework for securing edge IoT," Journal of Supercomputing, vol. 78, pp. 1-15, 2022.
- B. Lim, S. O. Arik, N. Loeff, and T. Pfister, "Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting," arXiv preprint arXiv:1912.09363, Dec. 2019. · arXiv
- Y. Ren, D. Zhao, D. Luo, H. Ma, and P. Duan, "Global-local temporal convolutional network for traffic flow prediction," IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 5, pp. 1935-1944, May 2020. doi: 10.1109/TITS.2019.2919620. · doi:10.1109/tits.2019.2919620
- L. Zhao et al., "T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction," arXiv preprint arXiv:1811.05320, Nov. 2018. · arXiv
- O. A. Madamidola, F. Ngobigha, and A. Ez-zizi, "Detecting new obfuscated malware variants: A lightweight and interpretable machine learning approach," Intelligent Systems with Applications, vol. 25, Mar. 2025.
- M. A. Hossain and M. S. Islam, "Enhanced detection of obfuscated malware in memory dumps: a machine learning approach for advanced cybersecurity," Cybersecurity, vol. 7, Art. no. 16, Jan. 2024.
- K. A. Dhanya, D. O. K., G. K. T., and P. Vinod, "Detection of Obfuscated Mobile Malware with Machine Learning and Deep Learning Models," in Machine Learning and Metaheuristics Algorithms, and Applications, Springer, Singapore, 2021, pp. 245-258.
- J. Zhu, J. Jang-Jaccard, A. Singh, P. A. Watters, and S. Camtepe, "Task-Aware Meta Learning-based Siamese Neural Network for Classifying Obfuscated Malware," arXiv preprint arXiv:2110.13409, Oct. 2021. · arXiv
- E. Alhajjar, P. Maxwell, and N. D. Bastian, "Adversarial Machine Learning in Network Intrusion Detection Systems," arXiv preprint arXiv:2004.11898, 2020. · arXiv
- X. Zhao, K. W. Fok, and V. L. L. Thing, "Enhancing Network Intrusion Detection Performance using Generative Adversarial Networks," arXiv preprint arXiv:2404.07464, 2024. · arXiv
- H. A. Alatwi and C. Morisset, "Adversarial Machine Learning in Network Intrusion Detection Domain: A Systematic Review," arXiv preprint arXiv:2112.03315, 2021. · arXiv
- M. Roesch, "Snort -Lightweight Intrusion Detection for Networks," in Proceedings of the 13th USENIX Conference on System Administration (LISA '99), Seattle, WA, USA, 1999, pp. 229-238.
Cited by
Explainability and interpretability of artificial intelligence use in cybersecurity
Ofusori, L. O., Bokaba, T., & Mhlongo, S. (2025) · Discover Computing · doi:10.1007/s10791-025-09760-6 · Cited by 23
Explainable AI: enhancing decision-making in the detection of cyber threats
Prasad, P. W. C., Sayeed, M. S., Nguyen, D.-M., Hutabarat, D. P., & Mohiuddin, G. M. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1762332 · Cited by 16
Khan, F. U., & Mian, A. N. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110753 · Cited by 5
Harnessing CNN-LSTM Fusion for Enhanced Crime Forecasting and Temporal Pattern Analysis
Ramalakshmi, K., Anwar, A. M., Kumari, L. K., Mohan, B., Ranjani, R., Krishnan, R. S., & Raj, J. R. F. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1807-3.ch003 · Cited by 6
Mousa, A. H., Alrubaye, I. O., Kadhim, M. S., Radhi, A. D., Al-Slivani, M. M., Al-Amri, R. M., & Pheng, L. G. (2025) · Mesopotamian Journal of Computer Science · doi:10.58496/mjcsc/2025/018 · Cited by 5
Self-citation
Metadata
- Authors
- Moamin, S. A. H., Abdulhameed, M. K., Al-Amri, R. M., Radhi, A. D., Naser, R. K., & Pheng, L. G.
- Affiliations
- University of Kerbala (IQ)
- University of Al-Ameed (IQ)
- Universiti Teknikal Malaysia Melaka (MY)
- ORCID
- Venue
- Babylonian Journal of Artificial Intelligence
- Publisher
- Mesopotamian Academic Press
- Published
- 2025-06-13
- Volume
- 2025
- Pages
- 77-98
- ISSN
- 3006-5437 (ISSN-L)
- OpenAlex
- W4411279063
- Cited by
- 23
- References
- 0
- Citations by year
- 2026: 11 · 2025: 11
- FWCI
- 10.30
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Babylonian Journal of Artificial Intelligence (publishedVersion)
- Funders
- University of Kerbala
- Topics
- Physical Sciences › Computer Science › Software › Advanced Malware Detection Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- malware detection, network intrusion detection, intrusion prevention systems, deep learning for cybersecurity, static and dynamic analysis, class imbalance, evasion attacks, data scarcity, mobile security, Internet of Things security, explainable AI, automated machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Artificial Intelligence in Malware and Network Intrusion Detection: A Comprehensive Survey of Techniques, Datasets, Challenges, and Future Directions. https://app.cochat.ai/papers/10.58496/bjai/2025/008
- The paper
- Moamin, S. A. H., Abdulhameed, M. K., Al-Amri, R. M., Radhi, A. D., Naser, R. K., & Pheng, L. G. (2025). Artificial Intelligence in Malware and Network Intrusion Detection: A Comprehensive Survey of Techniques, Datasets, Challenges, and Future Directions. Babylonian Journal of Artificial Intelligence. https://doi.org/10.58496/bjai/2025/008
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.