Large Language Models for Code: Security Hardening and Adversarial Testing
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Large language models (large LMs) are increasingly trained on massive codebases and used to generate code. However, LMs lack awareness of security and are found to frequently produce unsafe code. This work studies the security of LMs along two important axes: (i) security hardening, which aims to enhance LMs' reliability in generating secure code, and (ii) adversarial testing, which seeks to evaluate LMs' security at an adversarial standpoint. We address both of these by formulating a new security task called controlled code generation. The task is parametric and takes as input a binary property to guide the LM to generate secure or unsafe code, while preserving the LM's capability of generating functionally correct code. We propose a novel learning-based approach called SVEN to solve this task. SVEN leverages property-specific continuous vectors to guide program generation towards the given property, without modifying the LM's weights. Our training procedure optimizes these continuous vectors by enforcing specialized loss terms on different regions of code, using a high-quality dataset carefully curated by us. Our extensive evaluation shows that SVEN is highly effective in achieving strong security control. For instance, a state-of-the-art CodeGen LM with 2.7B parameters generates secure code for 59.1% of the time. When we employ SVEN to perform security hardening (or adversarial testing) on this LM, the ratio is significantly boosted to 92.3% (or degraded to 36.8%). Importantly, SVEN closely matches the original LMs in functional correctness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495 · cited 1×
“Although LMs excel in functional correctness, they may produce code with security issues [25, 27, 74].”
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
Research works printed but not linked: 30
- 10.1109/SP46214.2022.9833659 · DOI
- Guru Prasad Bhandari Amara Naseer and Leon Moonen. 2021. CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-source Software. In PROMISE. https://doi.org/10.1145/3475960.3475985 · DOI · Paper page
- 10.1109/TSE.2021.3087402 · DOI · Paper page
- 10.1109/TSE.2022.3147265 · DOI · Paper page
- 10.1109/ICSE48619.2023.00022 · DOI · Paper page
- 10.18653/v1/n19--1423 · doi:10.18653/v1/n19--1423
- 10.1109/SP.2016.15 · DOI · Paper page
- 10.1145/3379597.3387501 · DOI · Paper page
- Luca Gazzola Daniela Micucci and Leonardo Mariani. 2018. Automatic Software Repair: a Survey. In ICSE. https://doi.org/10.1145/3180155.3182526 · DOI · Paper page
- 10.1109/MS.2021.3072577 · DOI
- Jingxuan He Luca Beurer-Kellner and Martin Vechev. 2022. On Distribution Shift in Learning-based Bug Detectors. In ICML. https://proceedings.mlr.press/v162/he22a.html
- 10.1162/coli_a_00426 · DOI · Paper page
- Raphaël Khoury, Anderson R. Avila, Jacob Brunelle, and Baba Mamadou Camara. 2023. How Secure is Code Generated by ChatGPT? CoRR, Vol. abs/2304.09655 (2023). https://arxiv.org/abs/2304.09655 · arXiv
- Tomasz Korbak Hady Elsahar Germán Kruszewski and Marc Dymetman. 2022. Controlling Conditional Language Models without Catastrophic Forgetting. In ICML Kamalika Chaudhuri Stefanie Jegelka Le Song Csaba Szepesvári Gang Niu and Sivan Sabato (Eds.). https://proceedings.mlr.press/v162/korbak22a.html
- Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu-Hong Hoi. 2022. CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning. In NeurIPS. http://papers.nips.cc/paper_files/paper/2022/hash/8636419dea1aa9fbd25fc4248e702da4-Abstract-Conference.html
- 10.1109/TDSC.2021.3051525 · DOI · Paper page
- Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong. 2018. VulDeePecker: A Deep Learning-Based System for Vulnerability Detection. In NDSS. http://wp.internetsociety.org/ndss/wp-content/uploads/sites/25/2018/02/ndss2018_03A-2_Li_paper.pdf
- 10.1109/JPROC.2020.2993293 · DOI · Paper page
- 10.1109/TSE.2019.2946563 · DOI
- Georgios Nikitopoulos Konstantina Dritsa Panos Louridas and Dimitris Mitropoulos. 2021. CrossVul: a Cross-language Vulnerability Dataset with Commit Data. In ESEC/FSE. https://doi.org/10.1145/3468264.3473122 · DOI
- Yu Nong Yuzhe Ou Michael Pradel Feng Chen and Haipeng Cai. 2022. Generating Realistic Vulnerabilities via Neural Code Editing: an Empirical Study. In ESEC/FSE. https://doi.org/10.1145/3540250.3549128 · DOI
- Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2023. Examining Zero-Shot Vulnerability Repair with Large Language Models. In IEEE S&P. https://doi.ieeecomputersociety.org/10.1109/SP46215.2023.00001 · doi:10.1109/sp46215.2023.00001
- Sofia Reis and Rui Abreu. 2021. A Ground-truth Dataset of Real Security Patches. CoRR, Vol. abs/2110.09635 (2021). https://arxiv.org/abs/2110.09635 · arXiv
- Roei Schuster Congzheng Song Eran Tromer and Vitaly Shmatikov. 2021. You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion. In USENIX Security. https://www.usenix.org/conference/usenixsecurity21/presentation/schuster
- Justin Smith Brittany Johnson Emerson R. Murphy-Hill Bill Chu and Heather Richter Lipford. 2015. Questions Developers Ask While Diagnosing Potential Security Vulnerabilities with Static Analysis. In ESEC/FSE. https://doi.org/10.1145/2786805.2786812 · DOI
- Zhensu Sun Xiaoning Du Fu Song Mingze Ni and Li Li. 2022. CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning. In WWW. https://doi.org/10.1145/3485447.3512225 · DOI · Paper page
- Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Lukasz Kaiser and Illia Polosukhin. 2017. Attention is All you Need. In NeurIPS. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
- 10.1016/j.infsof.2021.106809 · DOI · Paper page
- Zenong Zhang, Zach Patterson, Michael Hicks, and Shiyi Wei. 2022. FIXREVERTER: A Realistic Bug Injection Methodology for Benchmarking Fuzz Testing. In USENIX Security. https://www.usenix.org/conference/usenixsecurity22/presentation/zhang-zenong
- Yaqin Zhou, Shangqing Liu, Jing Kai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks. In NeurIPS. https://proceedings.neurips.cc/paper/2019/hash/49265d2447bc3bbfe9e76306ce40a31f-Abstract.html
Cited by
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
“He et al. [97] focused on enhancing the security of code generated by LLMs.”
BackgroundWhen LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 383
“He and Vechev (2023) takes a more direct approach to customize LLMs through specific mechanisms.”
Sai, S., Yashvardhan, U., Chamola, V., & Sikdar, B. (2024) · IEEE Access · doi:10.1109/access.2024.3385107 · Cited by 125
Large Language Models in Cybersecurity: State-of-the-Art
Motlagh, F. N., Hajizadeh, M., Majd, M., Najafi, P., Cheng, F., & Meinel, C. (2024) · arXiv · doi:10.48550/arxiv.2402.00891 · Cited by 92
Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study
Fu, Y., Liang, P., Tahir, A., Li, Z., Shahin, M., Yu, J., & Chen, J. (2023) · arXiv · doi:10.48550/arxiv.2310.02059 · Cited by 75
Metadata
- Authors
- He, J., & Vechev, M.
- Venue
- Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2023-11-15
- Pages
- 1865-1879
- OpenAlex
- W4320560161
- Cited by
- 138
- References
- 32
- FWCI
- 26.09
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Large Language Models for Code: Security Hardening and Adversarial Testing. https://app.cochat.ai/papers/10.1145/3576915.3623175
- The paper
- He, J., & Vechev, M. (2023). Large Language Models for Code: Security Hardening and Adversarial Testing. Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. https://doi.org/10.1145/3576915.3623175
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.