Security and Privacy Challenges of Large Language Models: A Survey
Open accessAbstract
Large language models (LLMs) have demonstrated extraordinary capabilities and contributed to multiple fields, such as generating and summarizing text, language translation, and question-answering. Today, LLMs have become quite popular tools in natural language processing tasks, with the capability to analyze complicated linguistic patterns and provide relevant responses depending on the context. While offering significant advantages, these models are also vulnerable to security and privacy attacks, such as jailbreaking attacks, data poisoning attacks, and personally identifiable information leakage attacks. This survey provides a thorough review of the security and privacy challenges of LLMs, along with the application-based risks in various domains, such as transportation, education, and healthcare. We assess the extent of LLM vulnerabilities, investigate emerging security and privacy attacks against LLMs, and review potential defense mechanisms. Additionally, the survey outlines existing research gaps and highlights future research directions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
The EU General Data Protection Regulation (GDPR)
Voigt, P., & von dem Bussche, A. (2017) · doi:10.1007/978-3-319-57959-7 · Cited by 3,001 · see citing works
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · see citing works
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495 · see citing works
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495 · see citing works
Making Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380 · see citing works
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257 · see citing works
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083 · see citing works
Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting
Yeom, S., Giacomelli, I., Fredrikson, M., & Jha, S. (2018) · 2018 IEEE 31st Computer Security Foundations Symposium (CSF) · doi:10.1109/csf.2018.00027 · Cited by 1,001 · see citing works
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 665 · see citing works
Science in the age of large language models
Birhane, A., Kasirzadeh, A., Leslie, D., & Wachter, S. (2023) · Nature Reviews Physics · doi:10.1038/s42254-023-00581-4 · Cited by 344 · see citing works
DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks
Chen, H., Fu, C., Zhao, J., & Koushanfar, F. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/647 · Cited by 314 · see citing works
Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.212 · Cited by 282 · see citing works
Jailbroken: How Does LLM Safety Training Fail?
Wei, A., Haghtalab, N., & Steinhardt, J. (2023) · arXiv · doi:10.48550/arxiv.2307.02483 · Cited by 280 · see citing works
Jailbroken: How Does LLM Safety Training Fail?
Wei, A., Haghtalab, N., & Steinhardt, J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3508 · Cited by 280 · see citing works
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · arXiv · doi:10.48550/arxiv.2002.08155 · Cited by 243 · see citing works
Universal and Transferable Adversarial Attacks on Aligned Language Models
Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023) · arXiv · doi:10.48550/arxiv.2307.15043 · Cited by 197 · see citing works
Privacy Risks of General-Purpose Language Models
Pan, X., Zhang, M., Ji, S., & Yang, M. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00095 · Cited by 188 · see citing works
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137 · cited 1× · see citing works
“Since LLMs work through prompt-based techniques [69], [38], the main way to jailbreak LLMs is to deceive them by asking tricky questions and generating inappropriate content or forbidden scenarios [70].”
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-Tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2025) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/taslpro.2025.3606231 · Cited by 136 · see citing works
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.08747 · Cited by 135 · see citing works
Concealed Data Poisoning Attacks on NLP Models
Wallace, E., Zhao, T., Feng, S., & Singh, S. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.13 · Cited by 115 · see citing works
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., & Hashimoto, T. (2024) · 2024 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw63631.2024.00018 · Cited by 114 · see citing works
Demystifying Prompts in Language Models via Perplexity Estimation
Gonen, H., Iyer, S., Blevins, T., Smith, N., & Zettlemoyer, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.679 · Cited by 109 · see citing works
Prompt Injection attack against LLM-integrated Applications
Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zhang, T., Liu, Y., Wang, H., Zheng, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05499 · Cited by 100 · see citing works
Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks
Szymanski, A., Ziems, N., Eicher-Miller, H. A., Li, T. J.-J., Jiang, M., & Metoyer, R. A. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712091 · Cited by 77 · see citing works
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
Chen, Z., Xiang, Z., Xiao, C., Song, D., & Li, B. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-4136 · Cited by 69 · see citing works
Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2023) · arXiv · doi:10.48550/arxiv.2309.11495 · Cited by 42 · see citing works
Cheng, V. S. Y., & Hung, P. C. K. (2006) · International Journal of Healthcare Information Systems and Informatics · doi:10.4018/jhisi.2006010102 · Cited by 37 · see citing works
Detecting Pretraining Data from Large Language Models
Shi, W., Ajith, A., Xia, M., Huang, Y., Liu, D., Blevins, T., Chen, D., & Zettlemoyer, L. (2023) · arXiv · doi:10.48550/arxiv.2310.16789 · Cited by 37 · see citing works
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
Robey, A., Wong, E., Hassani, H., & Pappas, G. J. (2023) · arXiv · doi:10.48550/arxiv.2310.03684 · Cited by 37 · see citing works
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29 · see citing works
BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT
Shi, J., Liu, Y., Zhou, P., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2304.12298 · Cited by 26 · see citing works
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
Chen, Z., Xiang, Z., Xiao, C., Song, D., & Li, B. (2024) · arXiv · doi:10.48550/arxiv.2407.12784 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 220 · see citing works
“…enables LLMs to be tailored to specific challenges in various fields, thereby promoting progress and development in areas such as healthcare, law, education, and software engineering (Hou et al. 2023; Lai et al. 2024; Zhou et al. 2024b; Yan et al. 2024b; Li et al. 2023f; Zhao et al. 2024e).”
Large Language Models for Cyber Security: A Systematic Literature Review
Xu, H., Wang, S., Li, N., Wang, K., Zhao, Y., Chen, K., Yu, T., Liu, Y., & Wang, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3769676 · Cited by 189 · see citing works
Retrieval-augmented generation for educational application: A systematic survey
Li, Z., Wang, Z., Wang, W., Hung, K., Xie, H., & Wang, F. L. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100417 · Cited by 151 · see citing works
Brohi, S., Mastoi, Q.-U.-A., Jhanjhi, N. Z., & Pillai, T. R. (2025) · Algorithms · doi:10.3390/a18080499 · Cited by 69 · see citing works
A review of LLMs and their applications in the architecture, engineering and construction industry
Kampelopoulos, D., Tsanousa, A., Vrochidis, S., & Kompatsiaris, I. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11241-7 · Cited by 69 · see citing works
Jaffal, N. O., Alkhanafseh, M., & Mohaisen, D. (2025) · AI · doi:10.3390/ai6090216 · Cited by 69 · see citing works
Lamaakal, I., Maleh, Y., El Makkaoui, K., Ouahbi, I., Pławiak, P., Alfarraj, O., Almousa, M., & Abd El-Latif, A. A. (2025) · Sensors · doi:10.3390/s25051318 · Cited by 60 · see citing works
The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies
He, F., Zhu, T., Ye, D., Liu, B., Zhou, W., & Yu, P. S. (2026) · ACM Computing Surveys · doi:10.1145/3773080 · Cited by 58 · see citing works
To Protect the LLM Agent Against the Prompt Injection Attack with Polymorphic Prompt
Wang, Z., Nagaraja, N., Zhang, L., Bahsi, H., Patil, P., & Liu, P. (2025) · 2025 55th Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S) · doi:10.1109/dsn-s65789.2025.00037 · Cited by 13 · see citing works
A Lifecycle-Oriented Survey of Emerging Threats and Vulnerabilities in Large Language Models
de Maio, C., Di Gisi, M., Fenza, G., Gallo, M., & Loia, V. (2025) · IEEE Access · doi:10.1109/access.2025.3619764 · Cited by 10 · see citing works
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3 · see citing works
Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
Li, A., Zhou, Y., Raghuram, V. C., Goldstein, T., & Goldblum, M. (2025) · arXiv · doi:10.48550/arxiv.2502.08586 · Cited by 2 · see citing works
“…has developed taxonomies for categorizing attacks on LLMs, covering areas such as jailbreaks, prompt injection, and data poisoning (Chowdhury et al., 2024; Das et al., 2024; Shayegani et al., 2023), although distinctions between categories such as jailbreaks and prompt injection are disputed.”
- Authors
- Das, B. C., Amini, M. H., & Wu, Y.
- Affiliations
- Florida International University (US)
- Venue
- ACM Computing Surveys
- Published
- 2025-02-10
- Volume
- 57
- Issue
- 6
- Pages
- 1-39
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4406302454
- Cited by
- 445
- References
- 38
- Citations by year
- 2026: 217 · 2025: 203 · 2024: 25
- FWCI
- 136.91
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- U.S. Department of Homeland Security (17STCIN00001-05-00, and 23STSLA00016-01-00)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- large language models, security attacks, privacy attacks, jailbreak attacks, data poisoning, defense mechanisms, LLM vulnerabilities, medical applications, transportation applications, educational software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
