CoChat
Sign Up
Cited by 40Adversarial Robustness in Machine LearningSecurity and Verification in ComputingTopic Modeling

SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding

Open access
Xu, Zhangchen, Jiang, Fengqing, Niu, Luyao, Jia, Jinyuan, Lin, Bill Yuchen, Poovendran, Radha·2024·Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
DOI 10.18653/v1/2024.acl-long.303
Open this paper in CoChat — free accountOpen PDF

Abstract

Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, Radha Poovendran. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2024.acl-long.303

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ICD maintains the natural performance of LLMs.

    “while maintaining the natural performance of LLMs”

    From Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2026) · cites this paper as [43]

    The paper identifies ICD as a baseline in Section 5.1, but the Table 2 discussion says that baseline defenses degrade LLM utility, particularly on Llama2. It attributes preservation of the original models’ utility to SafeDecoding, not ICD (Sections 5.1–5.2; page numbers unavailable). (LLM, conf=0.73)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • ICD does not hurt natural performance, as evaluated with GLUE and MT-bench.

    “In our experiments, we evaluate ICD with GLUE [43] and MT-bench [59] to show that ICD does not hurt natural performance.”

    From Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations (2026) · cites this paper as [43]

    Section 5.1 identifies ICD as a baseline and says the paper uses MT-bench to evaluate model helpfulness. But the preserved-utility finding in Section 5.2 is about SafeDecoding, not ICD; the copy provides no GLUE evaluation or finding that ICD does not hurt natural performance. Page numbers were not available in our copy. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 42

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. · arXiv
  2. Bochuan Cao, Yuanpu Cao, Lu Lin, and Jinghui Chen. 2023. Defending against alignment-breaking at- tacks via robustly aligned LLM. arXiv preprint arXiv:2309.14348. · arXiv
  3. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419. · arXiv
  4. Boyi Deng, Wenjie Wang, Fuli Feng, Yang Deng, Qifan Wang, and Xiangnan He. 2023a. Attack prompt gen- eration for red teaming and defending large language models. arXiv preprint arXiv:2310.12505. · arXiv
  5. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2023c. Multilingual jailbreak chal- lenges in large language models. arXiv preprint arXiv:2310.06474. · arXiv
  6. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized LLMs. arXiv preprint arXiv:2305.14314. · arXiv
  7. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. 2022. Red teaming language models to re- duce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858. · arXiv
  8. Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, et al. 2022. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375. · arXiv
  9. Alec Helbling, Mansi Phute, Matthew Hull, and Duen Horng Chau. 2023. Llm self defense: By self examination, llms know they are being tricked. arXiv preprint arXiv:2308.07308. · arXiv
  10. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751. · arXiv
  11. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2023. Catastrophic jailbreak of open-source llms via exploiting generation. arXiv preprint arXiv:2310.06987. · arXiv
  12. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2023. Baseline defenses for ad- versarial attacks against aligned language models. arXiv preprint arXiv:2309.00614. · arXiv
  13. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1-38.
  14. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Boxin Wang, Jinyuan Jia, Bo Li, and Radha Pooven- dran. 2023. Identifying and mitigating vulnerabil- ities in llm-integrated applications. arXiv preprint arXiv:2311.16153. · arXiv
  15. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xi- ang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. Artprompt: Ascii art-based jail- break attacks against aligned llms. arXiv preprint arXiv:2402.11753. · arXiv
  16. Erik Jones, Anca Dragan, Aditi Raghunathan, and Ja- cob Steinhardt. 2023. Automatically auditing large language models via discrete optimization. arXiv preprint arXiv:2303.04381. · arXiv
  17. Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. 2023a. Deepinception: Hypnotize large language model to be jailbreaker. arXiv preprint arXiv:2311.03191. · arXiv
  18. Yuhui Li, Fangyun Wei, Jinjing Zhao, Chao Zhang, and Hongyang Zhang. 2023b. Rain: Your language mod- els can align themselves without finetuning. arXiv preprint arXiv:2309.07124. · arXiv
  19. Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chan- dra Bhagavatula, and Yejin Choi. 2023. The unlock- ing spell on base LLMs: Rethinking alignment via in- context learning. arXiv preprint arXiv:2312.01552. · arXiv
  20. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023a. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451. · arXiv
  21. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. 2023b. Jailbreaking ChatGPT via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860. · arXiv
  22. Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. 2023. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys, 56(2):1-40.
  23. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744.
  24. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The refinedweb dataset for Falcon LLM: Outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.01116. · arXiv
  25. Ivan Provilkov, Dmitrii Emelianenko, and Elena Voita. 2019. Bpe-dropout: Simple and effective subword regularization. arXiv preprint arXiv:1910.13267. · arXiv
  26. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. Fine- tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth In- ternational Conference on Learning Representations.
  27. Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. 2023. SmoothLLM: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684. · arXiv
  28. Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. · arXiv
  29. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288. · arXiv
  30. Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, and Qun Liu. 2023. Aligning large lan- guage models with human: A survey. arXiv preprint arXiv:2307.12966. · arXiv
  31. Zeming Wei, Yifei Wang, and Yisen Wang. 2023b. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387. · arXiv
  32. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359. · arXiv
  33. Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and S Yu Philip. 2023b. Multimodal large language models: A survey. In 2023 IEEE Inter- national Conference on Big Data (BigData), pages 2247-2256. IEEE.
  34. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google's neural machine translation system: Bridging the gap between human and machine trans- lation. arXiv preprint arXiv:1609.08144. · arXiv
  35. Jiahao Yu, Xingwei Lin, and Xinyu Xing. 2023. Gpt- fuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253. · arXiv
  36. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How Johnny can persuade llms to jailbreak them: Rethinking persua- sion to challenge AI safety by humanizing LLMs. arXiv preprint arXiv:2401.06373. · arXiv
  37. Zhexin Zhang, Junxiao Yang, Pei Ke, and Minlie Huang. 2023. Defending large language models against jail- breaking attacks through goal prioritization. arXiv preprint arXiv:2311.09096. · arXiv
  38. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging LLM-as-a-judge with MT-Bench and chat- bot arena. arXiv preprint arXiv:2306.05685. · arXiv
  39. Andy Zhou, Bo Li, and Haohan Wang. 2024. Ro- bust prompt optimization for defending language models against jailbreaking attacks. arXiv preprint arXiv:2401.17263. · arXiv
  40. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206. · arXiv
  41. Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. 2023. Autodan: Automatic and inter- pretable adversarial attacks on large language models. arXiv preprint arXiv:2310.15140. · arXiv
  42. Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrik- son. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. · arXiv

Cited by

Sorted by citations

Related

Authors
Xu, Z., Jiang, F., Niu, L., Jia, J., Lin, B. Y., & Poovendran, R.
Affiliations
Pennsylvania State University (US)
ORCID
Venue
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Publisher
Association for Computational Linguistics
Published
2024
Pages
5587-5605
ISSN
0736-587X (ISSN-L)
DOI
10.18653/v1/2024.acl-long.303
OpenAlex
W4402671718
Cited by
40
References
0
Citations by year
2026: 23 · 2025: 16 · 2024: 1
FWCI
11.80
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
Funders
  • National Science Foundation (2229876)
  • U.S. Department of Homeland Security
  • Air Force Office of Scientific Research (FA9550-, FA9550-23-1-0208)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
jailbreak attacks, large language model security, prompt injection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review