Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 46Software Engineering ResearchWeb Application Security VulnerabilitiesSoftware Testing and Debugging Techniques

The DOI matches its DataCite record.

Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study

Open access
Tamberg, Karl, Bahsi, Hayretdin·2024·arXiv
DOI 10.48550/arxiv.2405.15614arXiv 2405.15614
Open this paper in CoChat — free account

Abstract

Despite various approaches being employed to detect vulnerabilities, the number of reported vulnerabilities shows an upward trend over the years. This suggests the problems are not caught before the code is released, which could be caused by many factors, like lack of awareness, limited efficacy of the existing vulnerability detection tools or the tools not being user-friendly. To help combat some issues with traditional vulnerability detection tools, we propose using large language models (LLMs) to assist in finding vulnerabilities in source code. LLMs have shown a remarkable ability to understand and generate code, underlining their potential in code-related tasks. The aim is to test multiple state-of-the-art LLMs and identify the best prompting strategies, allowing extraction of the best value from the LLMs. We provide an overview of the strengths and weaknesses of the LLM-based approach and compare the results to those of traditional static analysis tools. We find that LLMs can pinpoint many more issues than traditional static analysis tools, outperforming traditional tools in terms of recall and F1 scores. The results should benefit software developers and security analysts responsible for ensuring that the code is free of vulnerabilities.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2405.15614

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Still being checked.

Research works printed but not linked: 40

  1. [3][3] Katerina Goseva-Popstojanova and Andrei Perhinschi. 'On the capability of static code analysis to detect security vulnerabilities'. In: Information and Software Technology 68 (2015), pp. 18-33. -
  2. [5][5] Christopher D. Manning. 'Human Language Understanding & Reasoning'. In: Daedalus 151.2 (May 2022), pp. 127-138. ISSN: 0011-5266. DOI: 10.1162 /daed a 01905. eprint: https://direct.mit.edu/daed/art icle-pdf/151/2/127/2060607/daed a 01905.pdf. URL: https://doi.org/10.1162/daed a 01905. - · DOI
  3. [6][6] Avishree Khare et al. Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities . 2023. arXiv: 2311.16169 [cs.CR] . - · arXiv
  4. [7][7] Chenyuan Zhang et al. Prompt-Enhanced Software Vulnerability Detection Using ChatGPT . 2023. arXiv: 230 8.12697 [cs.SE] . -
  5. [8][8] David Noever. Can Large Language Models Find And Fix Vulnerable Software? 2023. arXiv: 2308 . 10345 [cs.SE] . -
  6. [9][9] Xin Zhou, Ting Zhang, and David Lo. Large Language Model for Vulnerability Detection: Emerging Results and Future Directions . 2024. arXiv: 2401.15 468 [cs.SE] . -
  7. [10][10] Shunyu Yao et al. Tree of Thoughts: Deliberate Problem Solving with Large Language Models . 2023. arXiv: 23 05.10601 [cs.CL] . -
  8. [11][11] Xuezhi Wang et al. 'Self-consistency improves chain of thought reasoning in language models'. In: arXiv preprint arXiv:2203.11171 (2022). - · arXiv
  9. [15][15] Arvinder Kaur and Ruchikaa Nayyar. 'A Comparative Study of Static Code Analysis tools for Vulnerability Detection in C/C++ and JAVA Source Code'. In: Procedia Computer Science 171 (2020). Third International Conference on Computing and Network Communications (CoCoNet'19), pp. 2023-2029. ISSN: 1877-0509. DOI: https://doi.org/10.1016/j.procs.2020.04.217. URL: https://www.sciencedirect.com/science/article/pii/S187 7050920312023. - · DOI
  10. [16][16] Richard Amankwah et al. 'Bug detection in Java code: An extensive evaluation of static analysis tools using Juliet Test Suites'. In: Software: Practice and Experience 53.5 (2023), pp. 1125-1143. DOI: https://doi.org /10.1002/spe.3181. eprint: https://onlinelibrary.wiley.co m/doi/pdf/10.1002/spe.3181. URL: https://onlinelibrary .wiley.com/doi/abs/10.1002/spe.3181. - · DOI
  11. [18][18] Anton Cheshkov, Pavel Zadorozhny, and Rodion Levichev. Evaluation of ChatGPT Model for Vulnerability Detection . 2023. arXiv: 2304.07232 [cs.CR] . - · arXiv
  12. [19][19] Yu Nong et al. Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilities . 2024. arXiv: 2402.17230 [cs.CR] . - · arXiv
  13. [20][20] Gary McGraw. 'Automated Code Review Tools for Security'. In: Computer 41.12 (2008), pp. 108-111. DOI: 10.1109/MC.2008.514. - · DOI
  14. [21][21] Anne Edmundson et al. 'An Empirical Study on the Effectiveness of Security Code Review'. In: Engineering Secure Software and Systems . Ed. by Jan J¨ urjens, Benjamin Livshits, and Riccardo Scandariato. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 197212. ISBN: 978-3-642-36563-8. -
  15. [22][22] Norah Ahmed Almubairik and Gary Wills. 'Automated penetration testing based on a threat model'. In: 2016 11th International Conference for Internet Technology and Secured Transactions (ICITST) . 2016, pp. 413-414. DOI: 10.1109/ICITST.2016.7856742. - · DOI
  16. [23][23] Hossain Shahriar and Mohammad Zulkernine. 'Mitigating Program Security Vulnerabilities: Approaches and Challenges'. In: ACM Computing Surveys - CSUR 44 (June 2012), pp. 1-46. DOI: 10.1145/2187671.2187673. - · DOI
  17. [25][25] Jernej Novak, Andrej Krajnc, and Rok ˇ Zontar. 'Taxonomy of static code analysis tools'. In: The 33rd International Convention MIPRO . 2010, pp. 418-422. -
  18. [26][26] Guanjun Lin et al. 'Software Vulnerability Detection Using Deep Neural Networks: A Survey'. In: Proceedings of the IEEE 108.10 (2020), pp. 1825-1848. DOI: 10.1109/JPROC.2020.2993293. - · DOI
  19. [27][27] Seyed Mohammad Ghaffarian and Hamid Reza Shahriari. 'Software Vulnerability Analysis and Discovery Using Machine-Learning and Data-Mining Techniques: A Survey'. In: 50.4 (Aug. 2017). ISSN: 0360-0300. DOI: - 10.1145/3092566. URL: https://doi.org/10.1145/309256 6. - · DOI
  20. [28][28] Abubakar Omari Abdallah Semasaba et al. 'Literature survey of deep learning-based vulnerability analysis on source code'. In: IET Software 14.6 (2020), pp. 654664. DOI: https://doi.org/10.1049/iet-sen.2020.0084. eprint: https://ietresearch.onlinelibrary.wiley.com/doi/p df/10.1049/iet-sen.2020.0084. URL: https://ietresearch .onlinelibrary.wiley.com/doi/abs/10.1049/iet-sen.2020 .0084. - · DOI
  21. [29][29] Saikat Chakraborty et al. 'Deep Learning Based Vulnerability Detection: Are We There Yet?' In: IEEE Transactions on Software Engineering 48.9 (2022), pp. 3280-3296. DOI: 10.1109/TSE.2021.3087402. - · DOI
  22. [30][30] Omer Said Ozturk et al. 'New Tricks to Old Codes: Can AI Chatbots Replace Static Code Analysis Tools?' In: Proceedings of the 2023 European Interdisciplinary Cybersecurity Conference . EICC '23. Stavanger, Norway: Association for Computing Machinery, 2023, pp. 1318. ISBN: 9781450398299. DOI: 10.1145/3590777.359 0780. URL: https://doi.org/10.1145/3590777.3590780. - · DOI
  23. [31][31] Mark Chen et al. Evaluating Large Language Models Trained on Code . 2021. arXiv: 2107.03374 [cs.LG] . - · arXiv
  24. [32][32] Jiaxin Yu et al. Security Code Review by LLMs: A Deep Dive into Responses . 2024. arXiv: 2401.16310 [cs.SE] . - · arXiv
  25. [34][34] Yongqin Xian et al. 'Zero-Shot Learning-A Comprehensive Evaluation of the Good, the Bad and the Ugly'. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 41.9 (2019), pp. 2251-2265. DOI: 10.1109 /TPAMI.2018.2857768. -
  26. [35][35] Robert L. Logan IV et al. Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models . 2021. arXiv: 2106.13353 [cs.CL] . - · arXiv
  27. [36][36] Jason Wei et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models . 2023. arXiv: 22 01.11903 [cs.CL] . -
  28. [39][39] Noble Saji Mathews et al. LLbezpeky: Leveraging Large Language Models for Vulnerability Detection . 2024. arXiv: 2401.01269 [cs.CR] . - · arXiv
  29. [40][40] Pasi Fr¨ anti and Radu Mariescu-Istodor. 'Soft precision and recall'. In: Pattern Recognition Letters 167 (2023), pp. 115-121. ISSN: 0167-8655. DOI: https://doi.org/10 .1016/j.patrec.2023.02.005. URL: https://www.scienced irect.com/science/article/pii/S0167865523000296. -
  30. [42][42] Stephan Lipp, Sebastian Banescu, and Alexander Pretschner. 'An empirical study on the effectiveness of static C code analyzers for vulnerability detection'. In: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis . 2022, pp. 544-555. -
  31. [43][43] Kaixuan Li et al. 'Comparison and Evaluation on Static Application Security Testing (SAST) Tools for Java'. In: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering . ESEC/FSE 2023. New York, NY, USA: Association for Computing Machinery, 2023, pp. 921-933. DOI: 10.1145/3611643 .3616262. URL: https://doi.org/10.1145/3611643.36162 62. - · DOI
  32. [48][48] Geunwoo Kim, Pierre Baldi, and Stephen McAleer. Language Models can Solve Computer Tasks . 2023. arXiv: 2303.17491 [cs.CL] . - · arXiv
  33. [49][49] Aman Madaan et al. Self-Refine: Iterative Refinement with Self-Feedback . 2023. arXiv: 2303 . 17651 [cs.CL] . -
  34. [50][50] Michaela Greiler. Security code review checklist . [Accessed: 18-02-2024]. URL: https://www.awesomecoder eviews.com/checklists/secure-code-review-checklist/. -
  35. [52][52] Nenad Petrovi´ c. 'Chat GPT-Based Design-Time DevSecOps'. In: 2023 58th International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST) . 2023, pp. 143-146. DOI: 10.1109/ICEST58410.2023.10187247. - · DOI
  36. [53][53] Michael Fu et al. ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We? 2023. arXiv: 2310.09810 [cs.SE] . - · arXiv
  37. [54][54] Zolt´ an Szab´ o and Vilmos Bilicki. 'A New Approach to Web Application Security: Utilizing GPT Language Models for Source Code Inspection'. In: Future Internet 15.10 (2023). ISSN: 1999-5903. DOI: 10.3390/fi151003 26. URL: https://www.mdpi.com/1999-5903/15/10/326. - · DOI
  38. [55][55] Caitlin Sadowski et al. 'Lessons from Building Static Analysis Tools at Google'. In: Communications of the ACM (CACM) 61 Issue 4 (2018), pp. 58-66. URL: http s://dl.acm.org/citation.cfm?id=3188720. -
  39. [56][56] Zhilong Wang et al. 'The Effectiveness of Large Language Models (Chatgpt and Codebert) for SecurityOriented Code Analysis'. In: Available at SSRN 4567887 (). -
  40. [57][57] Yuqiang Sun et al. LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning . 2024. arXiv: 2401 . 16185 [cs.CR] . -

Cited by

Metadata

Authors
Tamberg, K., & Bahsi, H.
Affiliations
Tallinn University of Technology (EE)
ORCID
Tamberg, Karl
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
13
Pages
29698-29717
ISSN
2169-3536 (ISSN-L)
DOI
10.48550/arxiv.2405.15614
arXiv
2405.15614
OpenAlex
W4407390759
Cited by
46
References
0
Citations by year
2026: 27 · 2025: 19
FWCI
48.23
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Access (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (cc-by)
Funders
Haridus- ja Teadusministeerium (TK213)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Web Application Security Vulnerabilities
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
software vulnerability detection, large language models, prompting strategies, chain-of-thought, tree-of-thought, self-consistency, static analysis tools, false positives, Java

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review