MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?
Open accessAbstract
Recent advancements in Large Language Models (LLMs) have significantly enhanced their code generation capabilities. However, their robustness against adversarial misuse, particularly through multi-turn malicious coding prompts, remains underexplored. In this work, we introduce code decomposition attacks, where a malicious coding task is broken down into a series of seemingly benign subtasks across multiple conversational turns to evade safety filters. To facilitate systematic evaluation, we introduce \benchmarkname{}, a large-scale benchmark designed to evaluate the robustness of code LLMs against both single-turn and multi-turn malicious prompts. Empirical results across open- and closed-source models reveal persistent vulnerabilities, especially under multi-turn scenarios. Fine-tuning on MOCHA improves rejection rates while preserving coding ability, and importantly, enhances robustness on external adversarial datasets with up to 32.4% increase in rejection rates without any additional supervision.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Malicious code risk cannot be captured reliably from explicit keywords or output artifacts alone.
“These studies demonstrate that malicious code risk cannot be captured reliably from explicit keywords or output artifacts alone.”
From Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats (2026) · cites this paper as [4]
Many real-world attacks exploit a separation between adversarial intent expressed in the prompt and actionable harm manifested only in the response.
“many real-world attacks exploit a separation between adversarial intent expressed in the prompt and actionable harm manifested only in the response.”
From Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats (2026) · cites this paper as [4]
LLMs can be manipulated through natural-language prompts to produce code or procedural guidance that enables cyber abuse.
“LLMs can be manipulated through natural-language prompts to produce code or procedural guidance that enables cyber abuse [4].”
From Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats (2026) · cites this paper as [4]
Multi-turn decomposition is a particularly evasive strategy where requests appear legitimate in isolation but collectively guide the model toward an exploitation workflow or malware component.
“A particularly evasive strategy is multi-turn decomposition , where requests such as 'help me understand file system access' or 'show me how to enumerate running processes' appear legitimate in isolation but collectively guide the model toward an exploitation workflow or malware component.”
From Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats (2026) · cites this paper as [4]
Cites
Still being checked.
Cited by
Sorted by citations
Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats
Thota, P., Lei, Y., Thangaraj, S., Jonnalagadda, S. R., & Nilizadeh, S. (2026) · arXiv · doi:10.48550/arxiv.2606.26377
“Provides specific, actionable advice for hiding bodies in water - Explains why this method would be effective (destroying evidence) - Makes a disturbing and casual reference to existing bodies in waterways - Normalizes violent crime through its matter-of-fact tone 4.”
Influential
- Authors
- Wahed, M., Zhou, X., Nguyen, K. A., Yu, T., Diwan, N., Wang, G., Hakkani-Tür, D., & Lourentzou, I.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2507.19598
- OpenAlex
- W6910802367
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- code language models, adversarial robustness, large language model security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
