CoChat
Sign Up
PreprintCited by 6Adversarial Robustness in Machine LearningNetwork Security and Intrusion DetectionTopic Modeling

A Survey of Large Language Models in Cybersecurity

Open access
da Silva, Gabriel de Jesus Coelho, Westphall, Carlos Becker·2024·arXiv
DOI 10.48550/arxiv.2402.16968arXiv 2402.16968
Open this paper in CoChat — free accountOpen PDF

Abstract

Large Language Models (LLMs) have quickly risen to prominence due to their ability to perform at or close to the state-of-the-art in a variety of fields while handling natural language. An important field of research is the application of such models at the cybersecurity context. This survey aims to identify where in the field of cybersecurity LLMs have already been applied, the ways in which they are being used and their limitations in the field. Finally, suggestions are made on how to improve such limitations and what can be expected from these systems once these limitations are overcome.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.16968

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLMs provide new opportunities for the cybersecurity community.

    “LLMs provide new opportunities for the cybersecurity community(deJesus Coelho da Silva and Westphall 2024; Motlagh et al. 2024).”

    From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as de Jesus Coelho da Silva and Westphall 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 36

  1. Irena Bojanova and Carlos Eduardo C Galhardo. Bug, fault, error, or weakness: Demystifying software security vulnerabilities. IT Prof., 25(1):7-12, January 2023.
  2. Fabian M. Teichmann and Sonia R. Boticiu. An overview of the benefits, challenges, and legal aspects of penetration testing and red teaming. International Cybersecurity Law Review, 4(4):387-397, September 2023.
  3. Jian hua Li. Cyber security meets artificial intelligence: a survey. Frontiers of Information Technology & Electronic Engineering, 19(12):1462-1474, December 2018.
  4. Zhimin Zhang, Huansheng Ning, Feifei Shi, Fadi Farha, Yang Xu, Jiabo Xu, Fan Zhang, and Kim-Kwang Ray- mond Choo. Artificial intelligence in cyber security: research advances, challenges, and opportunities. Artificial Intelligence Review, 55(2):1029-1053, March 2021.
  5. Guanjun Lin, Sheng Wen, Qing-Long Han, Jun Zhang, and Yang Xiang. Software vulnerability detection using deep neural networks: A survey. Proc. IEEE Inst. Electr. Electron. Eng., 108(10):1825-1848, October 2020.
  6. Bingchang Liu, Liang Shi, Zhuhua Cai, and Min Li. Software vulnerability discovery techniques: A survey. In 2012 Fourth International Conference on Multimedia Information Networking and Security. IEEE, November 2012.
  7. Francesco Lomio, Emanuele Iannone, Andrea De Lucia, Fabio Palomba, and Valentina Lenarduzzi. Just-in-time software vulnerability detection: Are we there yet? Journal of Systems and Software, 188:111283, 2022.
  8. Deepaa Selva, Balakrishnan Nagaraj, Danil Pelusi, Rajendran Arunkumar, and Ajay Nair. Intelligent network intrusion prevention feature collection and classification algorithms. Algorithms, 14(8), 2021.
  9. Hui Wang, Zijian Cao, and Bo Hong. A network intrusion detection system based on convolutional neural network. J. Intell. Fuzzy Syst., 38(6):7623-7637, June 2020.
  10. Mariarosaria Taddeo, Tom McCutcheon, and Luciano Floridi. Trusting artificial intelligence in cybersecurity is a double-edged sword. Nat. Mach. Intell., 1(12):557-560, November 2019.
  11. Zhenguo Hu, Razvan Beuran, and Yasuo Tan. Automated penetration testing using deep reinforcement learning. In 2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), pages 2-10, 2020.
  12. Dean Richard McKinnel, Tooska Dargahi, Ali Dehghantanha, and Kim-Kwang Raymond Choo. A systematic literature review and meta-analysis on artificial intelligence in penetration testing and vulnerability assessment. Computers & Electrical Engineering, 75:175-188, 2019.
  13. Jonathon Schwartz and Hanna Kurniawati. Autonomous penetration testing using reinforcement learning. CoRR, abs/1905.05965, 2019. · arXiv
  14. R A Kemmerer. Cybersecurity. In 25th International Conference on Software Engineering, 2003. Proceedings. IEEE, 2003.
  15. Mamoona Humayun, Mahmood Niazi, NZ Jhanjhi, Mohammad Alshayeb, and Sajjad Mahmood. Cyber se- curity threats and vulnerabilities: A systematic mapping study. Arabian Journal for Science and Engineering, 45(4):3171-3189, January 2020.
  16. Kutub Thakur, Meikang Qiu, Keke Gai, and Md Liakat Ali. An investigation on cyber security threats and security models. In 2015 IEEE 2nd International Conference on Cyber Security and Cloud Computing, pages 307-311, 2015.
  17. Aileen G Bacudio, Xiaohong Yuan, Bei-Tseng Bill Chu, and Monique Jones. An overview of penetration testing. International Journal of Network Security & Its Applications, 3(6):19, 2011.
  18. Sugandh Shah and Babu M Mehtre. An overview of vulnerability assessment and penetration testing techniques. Journal of Computer Virology and Hacking Techniques, 11:27-49, 2015.
  19. Prashant S Shinde and Shrikant B Ardhapurkar. Cyber security analysis using vulnerability assessment and penetration testing. In 2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare (Startup Conclave), pages 1-5. IEEE, 2016.
  20. Terence Sanger and Pallavi N. Baljekar. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65 6:386-408, 1958.
  21. David C. Plaut and Geoffrey E. Hinton. Learning sets of filters using back-propagation. 2(1):35-61.
  22. Geoffrey E. Hinton. Learning multiple layers of representation. 11(10):428-434.
  23. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. 521(7553):436-444.
  24. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. July 2021.
  25. Michael Fu, Chakkrit Tantithamthavorn, Trung Le, Van Nguyen, and Dinh Phung. VulRepair: a t5-based automated software vulnerability repair. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, New York, NY, USA, November 2022. ACM.
  26. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted question- answering with human feedback. December 2021.
  27. Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. Examining Zero- Shot Vulnerability Repair with Large Language Models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 2339-2356.
  28. Marwan Omar. VulDefend: A Novel Technique based on Pattern-exploiting Training for Detecting Software Vulnerabilities Using Language Models. In 2023 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), pages 287-293.
  29. Chandra Thapa, Seung Ick Jang, Muhammad Ejaz Ahmed, Seyit Camtepe, Josef Pieprzyk, and Surya Nepal. Transformer-Based Language Models for Software Vulnerability Detection. In Proceedings of the 38th Annual Computer Security Applications Conference, pages 481-496. ACM.
  30. Michael Fu, Chakkrit Tantithamthavorn, Trung Le, Van Nguyen, and Dinh Phung. VulRepair: A T5-based automated software vulnerability repair. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 935-947. ACM.
  31. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1- 38, mar 2023.
  32. Rouzbeh Behnia, Mohammadreza Reza Ebrahimi, Jason Pacheco, and Balaji Padmanabhan. EW-tune: A framework for privately fine-tuning large language models with differential privacy. In 2022 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, nov 2022.
  33. Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi- Min Chan, Weize Chen, Jing Yi, Weilin Zhao, Xiaozhi Wang, Zhiyuan Liu, Hai-Tao Zheng, Jianfei Chen, Yang Liu, Jie Tang, Juanzi Li, and Maosong Sun. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220-235, March 2023.
  34. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 9459-9474.
  35. Deng Cai, Yan Wang, Lemao Liu, and Shuming Shi. Recent advances in retrieval-augmented text generation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '22, page 3417-3419, New York, NY, USA, 2022. Association for Computing Machinery.
  36. Seniha Esen Yuksel, Joseph N. Wilson, and Paul D. Gader. Twenty years of mixture of experts. IEEE Transac- tions on Neural Networks and Learning Systems, 23(8):1177-1193, 2012.

Cited by

Sorted by citations

Authors
da Silva, G. D. J. C., & Westphall, C. B.
ORCID
Westphall, Carlos Becker
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.16968
arXiv
2402.16968
OpenAlex
W4392430896
Cited by
6
References
0
Citations by year
2026: 2 · 2024: 4
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language models, cybersecurity, limitations of large language models, future research directions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review