Application of Large Language Models in Cybersecurity: A Systematic Literature Review
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
The emergence of Large Language Models (LLMs) is currently creating a major paradigm shift in societies and businesses in the way digital technologies are used. While the disruptive effect is especially observable in the information and communication technology field, there is a clear lack of systematic studies focusing on the application and impact of LLMs in cybersecurity holistically. This article presents an exhaustive systematic literature review of 177 articles published in 2018-2024 on the application of LLMs and the use of Artificial Intelligence (AI) as a defensive measure in cybersecurity. This article contributes an analytical compendium of the recent research on the application of LLMs in offensive and defensive cybersecurity as well as in research on cyberethics, current legal frameworks, and research regarding the use of LLMs for cybersecurity governance. It also contributes a statistical summary of global research trends in the field. Of the reviewed literature, 68% was published in 2023. Nearly 30% of the articles originate from the USA and 11% from China, with other countries currently having significantly lower contributions to recent research. Most attention in recent research has been given to AI as a defensive measure, accounting for 27% of the reviewed literature. It was observed that LLMs have proven highly effective in phishing attack simulations and in managing cybersecurity administrative aspects, including defending against advanced exploits. Furthermore, LLMs show significant potential in the development of security software, further cementing their role as a powerful tool in cybersecurity innovation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2_v1 · Cited by 8,090
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463 · cited 1×
“Another limitation highlighted by various studies [110], [111], [112] is the difficulty in maintaining context during extended conversations, which can lead to incoherent responses.”
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Cybersecurity data science: an overview from machine learning perspective
Sarker, I. H., Kayes, A. S. M., Badsha, S., Alqahtani, H., Watters, P., & Ng, A. (2020) · Journal of Big Data · doi:10.1186/s40537-020-00318-5 · Cited by 766
Wach, K., Duong, C. D., Ejdys, J., Kazlauskaitė, R., Korzynski, P., Mazurek, G., Paliszkiewicz, J., & Ziemba, E. (2023) · Entrepreneurial Business and Economics Review · doi:10.15678/eber.2023.110201 · Cited by 584
Technical guide to information security testing and assessment.
Scarfone, K. A., Souppaya, M. P., Cody, A., & Orebaugh, A. D. (2008) · doi:10.6028/nist.sp.800-115 · Cited by 353
A Machine Learning Security Framework for Iot Systems
Bagaa, M., Taleb, T., Bernabe, J. B., & Skarmeta, A. (2020) · IEEE Access · doi:10.1109/access.2020.2996214 · Cited by 271 · cited 1×
“For example, Bagaa et al. [55] propose a ML-based security framework for Internet of Things (IoT) systems that leverage Software Defined Networking (SDN) and Network Function Virtualization (NFV) to provide dynamic and efficient threat detection and mitigation.”
Digital deception: generative artificial intelligence in social engineering and phishing
Schmitt, M., & Flechais, I. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10973-2 · Cited by 198
Large Language Models for Cyber Security: A Systematic Literature Review
Xu, H., Wang, S., Li, N., Wang, K., Zhao, Y., Chen, K., Yu, T., Liu, Y., & Wang, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3769676 · Cited by 189
Niu, B., & Mvondo, G. F. N. (2024) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2023.103562 · Cited by 178
Getting pwn’d by AI: Penetration Testing with Large Language Models
Happe, A., & Cito, J. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3613083 · Cited by 144 · cited 3×
“A study by Happe and Cito [16] confirms the efficiency of LLMs (ChatGPT-3.”
InfluentialA Lightweight IoT Intrusion Detection Model Based on Improved BERT-of-Theseus
Wang, Z., Li, J., Yang, S., Luo, X., Li, D., & Mahmoodi, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122045 · Cited by 131 · cited 1×
“The efficiency of the BERT model in cybersecurity has also been demonstrated in other studies, for example in [187] and [188].”
Devising and Detecting Phishing Emails Using Large Language Models
Heiding, F., Schneier, B., Vishwanath, A., Bernstein, J., & Park, P. S. (2024) · IEEE Access · doi:10.1109/access.2024.3375882 · Cited by 123
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2308.10620 · Cited by 114
Red-Teaming for Generative AI: Silver Bullet or Security Theater?
Feffer, M., Sinha, A., Deng, W. H., Lipton, Z. C., & Heidari, H. (2024) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v7i1.31647 · Cited by 75
Benchmarking Large Language Models for Log Analysis, Security, and Interpretation
Karlsen, E., Luo, X., Zincir-Heywood, N., & Heywood, M. (2024) · Journal of Network and Systems Management · doi:10.1007/s10922-024-09831-x · Cited by 71 · cited 1×
“Karlsen et al. [205] benchmark several LLMs for log analysis and security using the LLM4Sec pipeline.”
Application of Large Language Models to DDoS Attack Detection
Guastalla, M., Li, Y., Hekmati, A., & Krishnamachari, B. (2024) · Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-51630-6_6 · Cited by 65
(2024) · doi:10.1109/sp54263.2024.00182 · Cited by 62
Identifying and Mitigating the Security Risks of Generative AI
Barrett, C., Boyd, B., Bursztein, E., Carlini, N., Chen, B., Choi, J., Chowdhury, A. R., Christodorescu, M., Datta, A., Feizi, S., et al. (2023) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000041 · Cited by 57 · cited 1×
“As AI models learn from synthetic data, the authenticity and reliability of their outputs diminish over time, a process known as ‘‘model collapse’’ [121].”
Natural Language Processing Model for Automatic Analysis of Cybersecurity-Related Documents
Georgescu, T.-M. (2020) · Symmetry · doi:10.3390/sym12030354 · Cited by 55 · cited 1×
“Georgescu’s work [134] on an NLP model specifically designed for this purpose stands out.”
Automated Emerging Cyber Threat Identification and Profiling Based on Natural Language Processing
Marinho, R., & Holanda, R. (2023) · IEEE Access · doi:10.1109/access.2023.3260020 · Cited by 50
Ali, T., & Kostakos, P. (2023) · arXiv · doi:10.48550/arxiv.2309.16021 · Cited by 47
A Survey of Reinforcement Learning from Human Feedback
Kaufmann, T., Weng, P., Bengs, V., & Hüllermeier, E. (2023) · arXiv · doi:10.48550/arxiv.2312.14925 · Cited by 43
PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
Deng, G., Liu, Y., Mayoral-Vilches, V., Liu, P., Li, Y., Xu, Y., Zhang, T., Liu, Y., Pinzger, M., & Rass, S. (2023) · arXiv · doi:10.48550/arxiv.2308.06782 · Cited by 37
Spear Phishing With Large Language Models
Hazell, J. (2023) · arXiv · doi:10.48550/arxiv.2305.06972 · Cited by 28
Review of Generative AI Methods in Cybersecurity
Yigit, Y., Buchanan, W. J., Tehrani, M. G., & Maglaras, L. (2024) · arXiv · doi:10.48550/arxiv.2403.08701 · Cited by 28
Cyber Security Issues and Challenges Related to Generative AI and ChatGPT
Pasupuleti, R., Vadapalli, R., & Mader, C. (2023) · 2023 Tenth International Conference on Social Networks Analysis, Management and Security (SNAMS) · doi:10.1109/snams60348.2023.10375472 · Cited by 26
Esmradi, A., Yip, D. W., & Chan, C. F. (2023) · arXiv · doi:10.48550/arxiv.2312.10982 · Cited by 24
Esmradi, A., Yip, D. W., & Chan, C. F. (2024) · Communications in Computer and Information Science · doi:10.1007/978-981-97-1274-8_6 · Cited by 24
Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments
Rigaki, M., Lukáš, O., Catania, C., & Garcia, S. (2024) · Proceedings of the 16th International Conference on Agents and Artificial Intelligence · doi:10.5220/0012391800003636 · Cited by 17 · cited 1×
“Rigaki et al. [169] delve into the application of LLMs, such as GPT-3.”
Tann, W., Liu, Y., Sim, J. H., Seah, C. M., & Chang, E.-C. (2023) · arXiv · doi:10.48550/arxiv.2308.10443 · Cited by 14
SecureBERT and LLAMA 2 Empowered Control Area Network Intrusion Detection and Classification
Li, X., & Fu, H. (2023) · arXiv · doi:10.48550/arxiv.2311.12074 · Cited by 9
A Survey of Large Language Models in Cybersecurity
da Silva, G. D. J. C., & Westphall, C. B. (2024) · arXiv · doi:10.48550/arxiv.2402.16968 · Cited by 6
Cyber Sentinel: Exploring Conversational Agents in Streamlining Security Tasks with GPT-4
Kaheh, M., Kholgh, D. K., & Kostakos, P. (2023) · arXiv · doi:10.48550/arxiv.2309.16422 · Cited by 5
Cited by
A Multi-Agent System for Cybersecurity Threat Detection and Correlation Using Large Language Models
Hmimou, Y., Tabaa, M., Khiat, A., & Hidila, Z. (2025) · IEEE Access · doi:10.1109/access.2025.3602681 · Cited by 25
From Edge Transformer to IoT Decisions: Offloaded Embeddings for Lightweight Intrusion Detection
Adjewa, F., Esseghir, M., & Merghem-Boulahia, L. (2026) · Sensors · doi:10.3390/s26020356 · Cited by 23
“They can detect vulnerabilities in configuration scripts, code snippets, and deployment templates, especially those obfuscated to evade static analysis [6,8].”
InfluentialOmnisec: Llm-Driven Provenance-Based Intrusion Detection Via Retrieval-Augmented Behavior Prompting
Cheng, W., Zhu, T., Jing, S., Mei, J.-P., Ma, M., Jin, J., & Weng, Z. (2025) · doi:10.2139/ssrn.5397852 · Cited by 1
Metadata
- Authors
- Hasanov, I., Virtanen, S., Hakkala, A., & Isoaho, J.
- Venue
- IEEE Access
- Published
- 2024
- Volume
- 12
- Pages
- 176751-176778
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4404688214
- Cited by
- 89
- References
- 41
- FWCI
- 3.99
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.