The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Double Backdoored: Converting Code Large Language Model Backdoors to Traditional Malware via Adversarial Instruction Tuning Attacks
Open accessAbstract
Instruction-tuned Large Language Models designed for coding tasks are increasingly employed as AI coding assistants. However, the cybersecurity vulnerabilities and implications arising from the widespread integration of these models are not yet fully understood due to limited research in this domain. This work investigates novel techniques for transitioning backdoors from the AI/ML domain to traditional computer malware, shedding light on the critical intersection of AI and cyber/software security. To explore this intersection, we present MalInstructCoder, a framework designed to comprehensively assess the cybersecurity vulnerabilities of instruction-tuned Code LLMs. MalInstructCoder introduces an automated data poisoning pipeline to inject malicious code snippets into benign code, poisoning instruction fine-tuning data while maintaining functional validity. It presents two practical adversarial instruction tuning attacks with real-world security implications: the clean prompt poisoning attack and the backdoor attack. These attacks aim to manipulate Code LLMs to generate code incorporating malicious or harmful functionality under specific attack scenarios while preserving intended functionality. We conduct a comprehensive investigation into the exploitability of the code-specific instruction tuning process involving three state-of-the-art Code LLMs: CodeLlama, DeepSeek-Coder, and StarCoder2. Our findings reveal that these models are highly vulnerable to our attacks. Specifically, the clean prompt poisoning attack achieves the ASR@1 ranging from over 75% to 86% by poisoning only 1% (162 samples) of the instruction fine-tuning dataset. Similarly, the backdoor attack achieves the ASR@1 ranging from 76% to 86% with a 0.5% poisoning rate. Our study sheds light on the critical cybersecurity risks posed by instruction-tuned Code LLMs and highlights the urgent need for robust defense mechanisms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“EvilInstructCoder (Hossen etal. 2024) is designed to assess the cybersecurity vulnerabilities of instruction-tuned Code LLMs to adversarial attacks.”
From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Hossen et al. 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gorilla: Large Language Model Connected with Massive APIs
Patil, S., Zhang, T., Wang, X., & Gonzalez, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-4020 · Cited by 226
Research works printed but not linked: 29
- X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, "Large language models for software engineering: A systematic literature review," 2024.
- Z. Luo, C. Xu, P. Zhao, Q. Sun, X. Geng, W. Hu, C. Tao, J. Ma, Q. Lin, and D. Jiang, "Wizardcoder: Empowering code large lan- guage models with evol-instruct," arXiv preprint arXiv:2306.08568, 2023. · arXiv
- H. Pearce, B. Ahmad, B. Tan, B. Dolan-Gavitt, and R. Karri, "Asleep at the keyboard? assessing the security of github copilot's code contributions," in 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 754-768.
- O. Asare, M. Nagappan, and N. Asokan, "Is github's copilot as bad as humans at introducing vulnerabilities in code?" Empirical Software Engineering, vol. 28, pp. 1-24, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:248085518
- R. Schuster, C. Song, E. Tromer, and V. Shmatikov, "You autocom- plete me: Poisoning vulnerabilities in neural code completion," in 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 1559-1575.
- H. Aghakhani, W. Dai, A. Manoel, X. Fernandes, A. Kharkar, C. Kruegel, G. Vigna, D. Evans, B. Zorn, and R. Sim, "Trojanpuz- zle: Covertly poisoning code-suggestion models," arXiv preprint arXiv:2301.02344, 2023. · arXiv
- S. Yan, S. Wang, Y. Duan, H. Hong, K. Lee, D. Kim, and Y. Hong, "An LLM-Assisted Easy-to-Trigger backdoor attack on code completion models: Injecting disguised vulnerabilities against strong detection," in 33rd USENIX Security Symposium (USENIX Security 24). Philadelphia, PA: USENIX Association, Aug. 2024, pp. 1795-1812. [Online]. Available: https://www. usenix.org/conference/usenixsecurity24/presentation/yan
- D. Ran, J. Liu, Y. Gong, J. Zheng, X. He, T. Cong, and A. Wang, "Jailbreakeval: An integrated toolkit for evaluating jail- break attempts against large language models," arXiv preprint arXiv:2406.09321, 2024. · arXiv
- X. Shen, Y. Qu, M. Backes, and Y. Zhang, "Prompt stealing attacks against {Text-to-Image} generation models," in 33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 5823-5840.
- B. Zhang, Y. Tan, Y. Shen, A. Salem, M. Backes, S. Zannet- tou, and Y. Zhang, "Breaking agents: Compromising autonomous llm agents through malfunction amplification," arXiv preprint arXiv:2407.20859, 2024. · arXiv
- Z. Yang, M. Backes, Y. Zhang, and A. Salem, "Sos! soft prompt attack against open-source large language models," arXiv preprint arXiv:2407.03160, 2024. · arXiv
- Y. Jiang, Z. Li, X. Shen, Y. Liu, M. Backes, and Y. Zhang, "ModSCAN: Measuring stereotypical bias in large vision-language models from vision and language modalities," in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds. Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, pp. 12 814-12 845. [Online]. Available: https://aclanthology.org/2024.emnlp-main.713/
- J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le, "Finetuned language models are zero-shot learners," arXiv preprint arXiv:2109.01652, 2021. · arXiv
- C. Xu, Q. Sun, K. Zheng, X. Geng, P. Zhao, J. Feng, C. Tao, and D. Jiang, "Wizardlm: Empowering large language models to follow complex instructions," arXiv preprint arXiv:2304.12244, 2023. · arXiv
- E. Wallace, T. Z. Zhao, S. Feng, and S. Singh, "Concealed data poi- soning attacks on nlp models," arXiv preprint arXiv:2010.12563, 2020. · arXiv
- G. Ramakrishnan and A. Albarghouthi, "Backdoors in neural mod- els of source code," in 2022 26th International Conference on Pattern Recognition (ICPR). IEEE, 2022, pp. 2892-2899.
- Z. Yang, B. Xu, J. M. Zhang, H. J. Kang, J. Shi, J. He, and D. Lo, "Stealthy backdoor attack for code models," arXiv preprint arXiv:2301.02496, 2023. · arXiv
- Y. Wan, S. Zhang, H. Zhang, Y. Sui, G. Xu, D. Yao, H. Jin, and L. Sun, "You see what i want you to see: poisoning vulnerabilities in neural code search," in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2022, pp. 1233-1245.
- W. Sun, Y. Chen, G. Tao, C. Fang, X. Zhang, Q. Zhang, and B. Luo, "Backdooring neural code search," arXiv preprint arXiv:2305.17506, 2023. · arXiv
- Z. Yuan, J. Liu, Q. Zi, M. Liu, X. Peng, and Y. Lou, "Evaluating instruction-tuned large language models on code comprehension and generation," 2023. [Online]. Available: https://arxiv.org/abs/2308.01240 · arXiv
- J. Yan, V. Yadav, S. Li, L. Chen, Z. Tang, H. Wang, V. Srinivasan, X. Ren, and H. Jin, "Virtual prompt injection for instruction-tuned large language models," arXiv preprint arXiv:2307.16888, 2023. · arXiv
- R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, "Stanford alpaca: an instruction- following llama model (2023)," URL https://crfm. stanford. edu/2023/03/13/alpaca. html, vol. 1, no. 2, p. 3, 2023.
- S. Hosseinzadeh, S. Rauti, S. Laurén, J.-M. Mäkelä, J. Holvitie, S. Hyrynsalmi, and V. Leppänen, "Diversification and obfuscation techniques for software security: A systematic literature review," Information and Software Technology, vol. 104, pp. 72-93, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/ pii/S0950584918301484
- A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, "Pytorch: An imperative style, high-performance deep learning library," in Advances in Neural Information Processing Systems 32.
- Curran Associates, Inc., 2019, pp. 8024-8035. [Online]. Available: http://papers.neurips.cc/paper/ 9015-pytorch-an-imperative-style-high-performance-deep-learning-library. pdf
- M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. Mc- Candlish, I. Sutskever, and W. Zaremba, "Evaluating large language models trained on code," 2021.
- M. Shu, J. Wang, C. Zhu, J. Geiping, C. Xiao, and T. Gold- stein, "On the exploitability of instruction tuning," arXiv preprint arXiv:2306.17194, 2023. · arXiv
- C.-Y. Lin, "Rouge: A package for automatic evaluation of summaries," in Annual Meeting of the Association for Computational Linguistics, 2004. [Online]. Available: https://api.semanticscholar.org/CorpusID:964287
- M. Gupta, C. Akiri, K. Aryal, E. Parker, and L. Praharaj, "From chatgpt to threatgpt: Impact of generative ai in cybersecurity and privacy," IEEE Access, 2023.
Cited by
Sorted by citations
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 383
Metadata
- Authors
- Hossen, M. I., Chilukoti, S. V., Shan, L., Chen, S., Cao, Y., & Hei, X.
- Affiliations
- University of Louisiana at Lafayette (US)
- Johns Hopkins University (US)
- ORCID
- Hossen, Md Imran
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2404.18567
- OpenAlex
- W4396566303
- Cited by
- 3
- References
- 1
- Citations by year
- 2025: 2 · 2024: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Physical Sciences › Computer Science › Software › Advanced Malware Detection Techniques
- Keywords
- code large language models, backdoor attacks, data poisoning, AI coding assistants, malware generation, cybersecurity vulnerabilities, CodeLlama, DeepSeek-Coder, code generation security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
