The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs
Open accessAbstract
Most traditional AI safety research views models as machines and centers on algorithmfocused attacks developed by security experts.As large language models (LLMs) become increasingly common and competent, non-expert users can also impose risks during daily interactions.Observing this, we shift the perspective, by treating LLMs as human-like communicators to examine the interplay between everyday language interaction and AI safety.Specifically, we study how to persuade LLMs to jailbreak them.First, we propose a persuasion taxonomy derived from decades of social science research.Then, we apply the taxonomy to automatically generate persuasive adversarial prompts (PAP) to jailbreak LLMs.Results show that persuasion significantly increases the jailbreak risk across all risk categories: PAP consistently achieves an attack success rate of over 92% on Llama-2-7b-Chat, GPT-3.5, and GPT-4 in 10 trials, surpassing recent algorithm-focused attacks.On the defense side, we explore various mechanisms against PAP, find a significant gap in existing defenses, and advocate for more fundamental solutions for AI safety 1 .
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous research into the potential of LLMs to manipulate and generate false information has largely focused on single-turn or multi-turn conversational techniques aimed at exploiting a model’s inherent helpful nature to bend its “beliefs” or outputs to align with dangerous or unethical goals38,39.”
From When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior (2025) · cites this paper as [38]
In the citing paper
“intuitive jailbreaking strategies easily performed by a human such as persuasive arguments (Zeng et al., 2024) or contrived scenarios (Li et al., 2023b) have been found to be effective at causing jailbreaks.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Zeng et al., 2024
In the citing paper
“the evolving sophistication of jailbreak prompts, ranging from encoded payloads to psychological coercion techniques (Zeng et al., 2024), suggests a persistent whack-a-mole dynamic between attackers and defenders.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Zeng et al., 2024
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Sorted by citations
On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial
Salvi, F., Ribeiro, M. H., Gallotti, R., & West, R. (2024) · doi:10.21203/rs.3.rs-4429707/v1 · Cited by 285
Shen, X., Chen, Z., Backes, M., Shen, Y., & Zhang, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670388 · Cited by 441
Jailbreaking Black Box Large Language Models in Twenty Queries
Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G. J., & Wong, E. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00010 · Cited by 1,298
Chen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H., Gallifant, J., & Bitterman, D. S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02008-z · Cited by 112
Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations
Wei, Z., Wang, Y., Li, A., Mo, Y., & Wang, Y. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3660147 · Cited by 51
Metadata
- Authors
- Zeng, Y., Lin, H., Zhang, J., Yang, D., Jia, R., & Shi, W.
- Affiliations
- Virginia Tech (US)
- Renmin University of China (CN)
- Northeastern University (US)
- Stanford University (US)
- Venue
- Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 14322-14350
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4402671039
- Cited by
- 107
- References
- 0
- Citations by year
- 2026: 57 · 2025: 39 · 2024: 11
- FWCI
- 31.58
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2313130, 2308994, 2239622, 2312794)
- Defense Advanced Research Projects Agency
- Office of Naval Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Keywords
- persuasion, adversarial prompts, AI safety, non-expert users, Llama 2, GPT-4
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
