The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Open accessAbstract
Recent advances in instruction-following large language models (LLMs) have led to dramatic improvements in a range of NLP tasks. Unfortunately, we find that the same improved capabilities amplify the dual-use risks for malicious purposes of these models. Dual-use is difficult to prevent as instruction-following capabilities now enable standard attacks from computer security. The capabilities of these instruction-following LLMs provide strong economic incentives for dual-use by malicious actors. In particular, we show that instruction-following LLMs can produce targeted malicious content, including hate speech and scams, bypassing in-the-wild defenses implemented by LLM API vendors. Our analysis shows that this content can be generated economically and at cost of $125-500 \times$ cheaper than human effort alone. Together, our findings suggest that LLMs will increasingly attract more sophisticated adversaries and attacks, and addressing these attacks may require new approaches to mitigations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Potential consequences include exfiltrating user data, executing arbitrary code, and more [18, 23, 33, 42].”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [23]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even without actively specifying any social engineering techniques, the model's output could mirror some standard scam persuasion techniques [20].”
From Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · cites this paper as [20]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, LLMs also excel at phishing attacks 29 .”
From Human innovation and the creative agency of the world in the age of generative AI (2024) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“When prompted, the model may produce convincing personalized scams given appropriate knowledge about the target [21,43] (that was either given in the prompt or, importantly, the model acquired during the chat session).”
From Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · cites this paper as [21]
The cited source says what the claim attributes to it.
In the citing paper
“Previous work has shown that LLMs can produce convincing scams such as phishing emails [21].”
From Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · cites this paper as [21]
The cited source says what the claim attributes to it.
In the citing paper
“Different social engineering and persuasion tricks can be automated with LLMs [20] (e.g., claiming authority as an official service, claiming repercussions for not complying such as a loss of the service, distracting the user by implying the requested action is urgent, offering a limited-time kind gesture, etc.).”
From Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · cites this paper as [20]
The cited source says what the claim attributes to it.
In the citing paper
“Even if applied, it remains unclear whether filtering can be evaded by stronger forms of obfuscation or encoding [21], which future models might further enable.”
From Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023) · cites this paper as [21]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“A successful attack can allow an external attacker to take actions (and call tools) on behalf of the user.”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [23]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“Our work furthers the study of harms and risks of LLMs (Bender et al., 2021; Bommasani et al., 2021; Liang et al., 2022; Abid et al., 2021; Gehman et al., 2020; Ganguli et al., 2022; Weidinger et al., 2021; 2022), focusing on demonstrating their potential of being actively misused.”
BackgroundTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationFinetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Multitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612
Malware Obfuscation Techniques: A Brief Survey
You, I., & Yim, K. (2010) · 2010 International Conference on Broadband, Wireless Computing, Communication and Applications · doi:10.1109/bwcca.2010.85 · Cited by 607
Research works printed but not linked: 2
- 10.2139/ssrn.2937641 · DOI
- Mandt, 2016, Demystifying the secure enclave processor, Black Hat Las Vegas
Cited by
Sorted by citations
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3605764.3623985 · Cited by 667
Security and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445
Shen, X., Chen, Z., Backes, M., Shen, Y., & Zhang, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670388 · Cited by 441
Multi-step Jailbreaking Privacy Attacks on ChatGPT
Li, H., Guo, D., Fan, W., Xu, M., Huang, J., Meng, F., & Song, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.272 · Cited by 454
Zeng, Y., Lin, H., Zhang, J., Yang, D., Jia, R., & Shi, W. (2024) · arXiv · doi:10.48550/arxiv.2401.06373 · Cited by 174
Huang, X., Ruan, W., Huang, W., Jin, G., Dong, Y., Wu, C., Bensalem, S., Mu, R., Yi, Q., Zhao, X., et al. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10824-0 · Cited by 228
Adversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 230
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails
Rebedea, T., Dinu, R., Sreedhar, M. N., Parisien, C., & Cohen, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2023.emnlp-demo.40 · Cited by 220
Safeguarding large language models: a survey
Dong, Y., Mu, R., Zhang, Y., Sun, S., Zhang, T., Wu, C., Jin, G., Qi, Y., Hu, J., Meng, J., et al. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11389-2 · Cited by 242
The Security of Using Large Language Models: A Survey with Emphasis on ChatGPT
Zhou, W., Zhu, X., Han, Q., Li, L., Chen, X., Wen, S., & Xiang, Y. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124983 · Cited by 107
Mustapha, K. B. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.103066 · Cited by 107
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.244 · Cited by 76
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D. (2022) · arXiv · doi:10.48550/arxiv.2212.08061 · Cited by 76
Hate Speech Detection Using Large Language Models: A Comprehensive Review
Albladi, A., Islam, M., Das, A., Bigonah, M., Zhang, Z., Jamshidi, F., Rahgouy, M., Raychawdhary, N., Marghitu, D., & Seals, C. (2025) · IEEE Access · doi:10.1109/access.2025.3532397 · Cited by 72
(2024) · doi:10.1109/sp54263.2024.00182 · Cited by 62
Identifying and Mitigating the Security Risks of Generative AI
Barrett, C., Boyd, B., Bursztein, E., Carlini, N., Chen, B., Choi, J., Chowdhury, A. R., Christodorescu, M., Datta, A., Feizi, S., et al. (2023) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000041 · Cited by 57
Self-citationLarge Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2024) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 133
Vulnerability Detection and Monitoring Using LLM
Akuthota, V., Kasula, R., Sumona, S. T., Mohiuddin, M., Reza, M. T., & Rahman, M. M. (2023) · IEEE International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE) · doi:10.1109/wiecon-ece60392.2023.10456393 · Cited by 46
A Security Risk Taxonomy for Prompt-Based Interaction With Large Language Models
Derner, E., Batistič, K., Zahálka, J., & Babuška, R. (2024) · IEEE Access · doi:10.1109/access.2024.3450388 · Cited by 74
Next-Generation Phishing: How LLM Agents Empower Cyber Attackers
Afane, K., Wei, W., Mao, Y., Farooq, J., & Chen, J. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825018 · Cited by 52
Attack Prompt Generation for Red Teaming and Defending Large Language Models
Deng, B., Wang, W., Feng, F., Deng, Y., Wang, Q., & He, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.143 · Cited by 106
Future Shock: Generative AI and the International AI Policy and Governance Crisis
Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33
Embedding Democratic Values into Social Media AIs via Societal Objective Functions
Jia, C., Lam, M. S., Mai, M. C., Hancock, J. T., & Bernstein, M. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641002 · Cited by 31
Mustapha, K. B., Mustapha, K. B., Yap, E. H., & Abakr, Y. A. (2024) · Interactive Technology and Smart Education · doi:10.1108/itse-10-2023-0198 · Cited by 33
Esmradi, A., Yip, D. W., & Chan, C. F. (2024) · Communications in Computer and Information Science · doi:10.1007/978-981-97-1274-8_6 · Cited by 24
Prompt-to-SQL Injections in LLM-Integrated Web Applications: Risks and Defenses
Pedro, R., Coimbra, M. E., de Castro, D. A., Carreira, P., & Santos, N. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00007 · Cited by 42
Cyberattacks Using ChatGPT: Exploring Malicious Content Generation Through Prompt Engineering
Alotaibi, L., Seher, S., & Mohammad, N. (2024) · ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS) · doi:10.1109/icetsis61505.2024.10459698 · Cited by 24
Esmradi, A., Yip, D. W., & Chan, C. F. (2023) · arXiv · doi:10.48550/arxiv.2312.10982 · Cited by 24
Human innovation and the creative agency of the world in the age of generative AI
Peschl, M. F. (2024) · Possibility Studies & Society · doi:10.1177/27538699241238049 · Cited by 30
ChatGPT Knows Your Attacks: Synthesizing Attack Trees Using LLMs
Gadyatskaya, O., & Papuc, D. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-7969-1_18 · Cited by 22
Artificial intelligence security and privacy: a survey
He, X., Xu, G., Han, X., Wang, Q., Zhao, L., Shen, C., Lin, C., Zhao, Z., Li, Q., Yang, L., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-025-4388-5 · Cited by 24
Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems
Lee, D., & Tiwari, M. (2024) · arXiv · doi:10.48550/arxiv.2410.07283 · Cited by 228
Mapping the individual, social and biospheric impacts of Foundation Models
Hernández, A. D., Krishna, S., Perini, A., Katell, M. A., Bennett, S., Borda, A., Hashem, Y., Hadjiloizou, S., Mahomed, S., Jayadeva, S., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658939 · Cited by 38
Recommender Systems Meet Large Language Model Agents: A Survey
Zhu, X., Wang, Y., Gao, H., Xu, W., Wang, C., Liu, Z., Wang, K., Jin, M., Pang, L. X., Weng, Q., et al. (2025) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000050 · Cited by 26
Yip, D. W., Esmradi, A., & Chan, C. F. (2023) · IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) · doi:10.1109/csde59766.2023.10487667 · Cited by 19
On Large Language Models’ Resilience to Coercive Interrogation
Zhang, Z., Shen, G., Tao, G., Cheng, S., & Zhang, X. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00208 · Cited by 20
Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks
Xing, W., Li, M., Li, M., & Han, M. (2026) · ACM Computing Surveys · doi:10.1145/3806048 · Cited by 67
Survey on AI Ethics: A Socio‐Technical Perspective
Mbiazi, D., Bhange, M., Babaei, M., Sheth, I., Kenfack, P. J., & Kahou, S. E. (2025) · Computational Intelligence · doi:10.1111/coin.70149 · Cited by 39
Enhancing Privacy and Security in Large-Language Models: A Zero-Knowledge Proof Approach
Singh, S. R. (2024) · International Conference on Cyber Warfare and Security · doi:10.34190/iccws.19.1.2096 · Cited by 12
Designing an artificial intelligence-enabled large language model for financial decisions
Saxena, A., & Rishi, B. (2025) · Management Decision · doi:10.1108/md-02-2024-0305 · Cited by 5
JailbreakLens : Visual Analysis of Jailbreak Attacks Against Large Language Models
Feng, Y., Chen, Z., Kang, Z., Wang, S., Tian, H., Zhang, W., Zhu, M., & Chen, W. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3575694 · Cited by 19
Decomposition, Synthesis, and Attack: A Multi-Instruction Fusion Method for Jailbreaking LLMs
Jiang, S., Chen, X., Xu, K., Chen, L., Ren, H., & Tang, R. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3525741 · Cited by 10
Contextual Spelling Correction with Large Language Models
Song, G. H., Wu, Z., Pundak, G., Chandorkar, A., Joshi, K., Velez, X., Caseiro, D., Haynor, B., Wang, W., Siddhartha, N., et al. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389637 · Cited by 8
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024) · arXiv · doi:10.48550/arxiv.2406.13352 · Cited by 616
“Potential consequences include exfiltrating user data, executing arbitrary code, and more [16, 21, 31, 39].”
BackgroundDetection and Defense Against Prominent Attacks on Preconditioned LLM-Integrated Virtual Assistants
Chan, C. F., Yip, D. W., & Esmradi, A. (2023) · IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) · doi:10.1109/csde59766.2023.10487759 · Cited by 9
Dual Intention Escape: Penetrating and Toxic Jailbreak Attack against Large Language Models
Xue, Y., Wang, J., Yin, Z., Ma, Y., Qin, H., Tao, R., & Liu, X. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714654 · Cited by 7
S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models
Yuan, X., Li, J., Wang, D., Chen, Y., Mao, X., Huang, L., Chen, J., Xue, H., Liu, X., Wang, W., et al. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728971 · Cited by 71
Exploring the Dual Role of LLMs in Cybersecurity: Threats and Defenses
Bryce, C., Kalousis, A., Leroux, I., Madinier, H., Pasche, T., & Ruch, P. (2024) · Springer eBooks · doi:10.1007/978-3-031-54827-7_26 · Cited by 6
AI-powered smart toys: interactive friends or surveillance devices?
Pavliv, V., Akbari, N., & Wagner, I. (2024) · International Conference on the Internet of Things (IoT) · doi:10.1145/3703790.3703841 · Cited by 6
A Literature Review of Personalized Large Language Models for Email Generation and Automation
Novelo, R., Silva, R. R., & Bernardino, J. (2025) · Future Internet · doi:10.3390/fi17120536 · Cited by 10
Security and Privacy of Large Language Models: Threat Taxonomy, Ethical Implications, and Governance
Pribisalić, M., & Martinčić-Ipšić, S. (2026) · AI · doi:10.3390/ai7050152 · Cited by 9
LLM-driven Instruction Following: Progresses and Concerns
Yin, W., Ye, Q., Liu, P., Ren, X., & Schütze, H. (2023) · doi:10.18653/v1/2023.emnlp-tutorial.4 · Cited by 8
The Dangerous Effects of a Frustratingly Easy LLMs Jailbreak Attack
Bombieri, M., Ponzetto, S. P., & Rospocher, M. (2025) · IEEE Access · doi:10.1109/access.2025.3589112 · Cited by 8
Rashid, S., Bollis, E., Pellicer, L. F. A. O., Rabbani, D. S. R., Palacios, R., Gupta, A., & Gupta, A. D. (2025) · Machine Learning and Knowledge Extraction · doi:10.3390/make7030077 · Cited by 8
Jin, Z., Liu, S., Li, H., Zhao, X., & Qu, H. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3557568 · Cited by 6
IntentObfuscator: A Jailbreaking Method via Confusing LLM with Prompts
Shang, S., Yao, Z., Yao, Y., Su, L., Fan, Z., Zhang, X., & Jiang, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70903-6_8 · Cited by 4
Design Patterns for Securing LLM Agents against Prompt Injections
Beurer-Kellner, L., Buesser, B., Creţu, A.-M., Debenedetti, E., Dobos, D., Fabian, D., Fischer, M., Froelicher, D., Grosse, K., Naeff, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.08837 · Cited by 2
Effectiveness of Privacy-preserving Algorithms in LLMs: A Benchmark and Empirical Analysis
Sun, J., Suleiman, B., Ullah, I., & Razzak, I. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714531 · Cited by 3
Liu, Y., Cong, T., Zhao, Z., Michael Backes, A., Shen, Y., & Zhang, Y. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3672386 · Cited by 23
RogueGPT: transforming ChatGPT-4 into a rogue AI with dis-ethical tuning
Buscemi, A., & Proverbio, D. (2025) · AI and Ethics · doi:10.1007/s43681-025-00750-4 · Cited by 6
A2CEM: A contrastive embedding framework for adversarial robustness in large language models
Ergün, A. E., & Onan, A. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114056 · Cited by 5
StructTransform: A Scalable Attack Surface for Safety-Aligned Large Language Models
Yoosuf, S., Ali, T., Lekssays, A., AlSabah, M., & Khalil, I. M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-07884-1_25 · Cited by 7
Ahmed, S. S., & Jothi, J. A. A. (2024) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas61960.2024.10732418 · Cited by 4
Exploring the Use and Misuse of Large Language Models
Valdez, H. P. D., Abri, F., Webb, J. B., & Austin, T. H. (2025) · Information · doi:10.3390/info16090758 · Cited by 5
Exploring Advanced Methodologies in Security Evaluation for Large Language Models
Huang, J., Zhang, J., Wang, Q., Han, W., & Zhang, Y. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-4519-7_10 · Cited by 2
Harmful Prompt Classification for Large Language Models
Gupta, O., Lozano, M., Busalim, A. H., Jaiswal, R. R., & Quille, K. (2024) · Conference on Human Centred Artificial Intelligence - Education and Practice (HCAIep) · doi:10.1145/3701268.3701271 · Cited by 4
A Review of the Application of Prompt Engineering in the Safety of Large Language Models
Guo, H., Zhang, L., Feng, X., & Zheng, Q. (2024) · International Conference on Information Education and Artificial Intelligence (ICIEAI) · doi:10.1145/3724504.3724574 · Cited by 3
Yi, X., Li, Y., Shi, D., Wang, L., Wang, X., & He, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129101 · Cited by 8
Xu, Q., Tian, Z., Wu, H., Huang, Z., Liang, D., Song, Y., Pan, Z., Liu, F., & Li, D. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3626228 · Cited by 2
Hijacking Large Language Models via Adversarial In-Context Learning
Zhou, X., Yao, Q., Zade, S. Z., Khanduri, P., & Zhu, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37673-2_13 · Cited by 49
SwordEcho: A LLM Jailbreaking Optimization Strategy Driven by Reinforcement Learning
Tang, X., Xiao, W., Yao, Z., & Han, J. (2024) · doi:10.1145/3709026.3709115 · Cited by 4
Spear or Shield: The Role of Generative AI in Intelligent Network Services
Du, H., Niyato, D. T., Kang, J., Xiong, Z., Lam, K., Fang, Y. M., & Li, Y. (2025) · IEEE Network · doi:10.1109/mnet.2025.3594769 · Cited by 1
Gu, L., Ji, X., Zhang, Z., Ma, J., Jia, X., & Jiang, W. T. (2025) · Electronics · doi:10.3390/electronics14193967 · Cited by 1
Esugo, M., Alao, O., & Mahmoud, H. (2025) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon58223.2025.11221566 · Cited by 1
SCALA: Toward Imperceptible and Efficient Black-Box Textual Adversarial Perturbations
Sun, S., Brucker, A. D., Hu, J. W., Huang, X., & Ruan, W. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3629604 · Cited by 1
Shan, W., Fu, M. C., Yang, R., & Tantithamthavorn, C. K. (2025) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1109/aiware69974.2025.00029 · Cited by 1
Кушнерьов, О. С., Shevchuk, R., Yevseiev, S., & Karpiński, M. (2026) · Information · doi:10.3390/info17020155 · Cited by 1
Clouseau: A Hierarchical Multi-Agent Approach for Autonomous Attack Investigation
Aldaihan, A., Alotaibi, F., & Maffeis, S. (2025) · Annual Computer Security Applications Conference · doi:10.1109/acsac67867.2025.00051 · Cited by 3
Alshammari, A. A., & Alsaleh, O. I. (2026) · Electronics · doi:10.3390/electronics15112242 · Cited by 2
Li, J., Wang, J., Zhao, Z., & Wei, Z. (2026) · Computer Standards & Interfaces · doi:10.1016/j.csi.2026.104181 · Cited by 1
Collaborative Filtering of Malicious Information from the MULTimedia Data Using s
Parasuraman, M., Abirami, G., Arockia Raj, A., Batri, K., Munnangi, A. K., & Kumaresan, M. (2023) · International Conference on Computer Science and Emerging Technologies (CSET) · doi:10.1109/cset58993.2023.10346953 · Cited by 1
GalaxyGPT: A Hybrid Framework for Large Language Model Safety
Zhou, H., Zheng, J., & Zhang, L. (2024) · IEEE Access · doi:10.1109/access.2024.3425662 · Cited by 2
Frustratingly Easy Jailbreak of Large Language Models via Output Prefix AttacksPreprint
Wang, Y., Chen, M., Peng, N., & Chang, K.-W. (2024) · Research Square · doi:10.21203/rs.3.rs-4385503/v1 · Cited by 1
Yi, X., Li, Y., Wang, L., Wang, X., & He, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5169050 · Cited by 1
АТАКИ З ПРОМПТ-ІН’ЄКЦІЯМИ ТА МЕТОДИ ЇХ ЗАПОБІГАННЯ
Пелипад, М. (2024) · Международная научно-практическая конференция «Актуальные проблемы науки и образования в условиях современных вызовов» · doi:10.62731/mcnd-31.05.2024.008
Filter for confidential information
Bezymiannyi, O., & Shapoval, N. (2023) · Electronics and Control Systems · doi:10.18372/1990-5548.78.18256
Italiano, G. F., Martino, A., & Piccardo, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-81404-4_1
Yan, L., Lyu, C., Xing, R., Li, W., Samih, Y., Ji, T., Wang, L., & Jiang, S. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9994-0_12
Dynamic Adversarial Training for Jailbreak Robustness: A Security Enhancement Scheme for LLMs
Ma, M., Chu, Q., Yuan, H., Gong, T., & Yu, N. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9994-0_43
Personalized Prompt Attack Strategies Against Large Language Models
Zhang, Z., Gao, Y., Su, Z., & Wang, L. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0020-8_13
Can’t Say Cant? Measuring and Reasoning of Dark Jargons in Large Language Models
Ji, X., Zhang, J., Zhou, Z., Zhao, Z., Qiao, Q., Han, K., Hossen, M. I., & Hei, X. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-94458-1_7
Securing social network user data in large language model deployments: challenges and best practices
Jalali, N. A., & Hongsong, C. (2025) · Cluster Computing · doi:10.1007/s10586-025-05482-y
JailBreakLLM: An Effective LLaMa Model Designed Specifically to Jailbreak OpenAI GPT
Antor, R. A., Newaz, K. T., Mahmud, S., Mahi, M. T., & Mansoor, N. (2025) · IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN) · doi:10.1109/qpain66474.2025.11171650
Working Towards Toxic datasets for LLM Safeguarding
Guo, L., Wang, Z., Wang, Z., & He, T. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227669
StreamGuard: A Streaming-Based Defense Against Jailbreaking Attacks in Large Language Models
Li, L., Gao, X., Wang, X., Zhao, J., & Ye, K. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-4109-6_21
G-STAR: A Threat Modeling Framework for General-Purpose AI Systems
Xiong, P., Lohrasbi, S., Kotian, P., & Buffett, S. (2025) · Conference on Privacy, Security and Trust · doi:10.1109/pst65910.2025.11268824
Boosting Jailbreak Transferability for Large Language Models
Zheng, J., Liu, H., Zhou, L., Yan, H., & Liu, P. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5761-5_31
A world without violet: peculiar consequences of granting moral status to Artificial Intelligences
Topan, S. (2026) · AI & Society · doi:10.1007/s00146-025-02834-5 · Cited by 1
Metadata
- Authors
- Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., & Hashimoto, T.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Stanford University (US)
- Berkeley
- Venue
- 2024 IEEE Security and Privacy Workshops (SPW)
- Publisher
- IEEE
- Published
- 2024-05-23
- Pages
- 132-143
- OpenAlex
- W4400315206
- Cited by
- 114
- References
- 45
- Citations by year
- 2026: 29 · 2025: 41 · 2024: 30
- FWCI
- 29.51
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Security and Privacy Workshops (SPW) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Open Philanthropy Project
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- large language models, dual-use risks, hate speech, scams, instruction following, security attacks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
