A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models

Calibrating Noise to Sensitivity in Private Data Analysis

Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272 · see citing works

Membership Inference Attacks Against Machine Learning Models

Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works

Training language models to follow instructions with human feedback

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works

Deep Leakage from Gradients

Zhu, L., & Han, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-63076-8_2 · Cited by 1,906 · see citing works

Machine Unlearning

Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810 · see citing works

GPT (Generative Pre-Trained Transformer)— A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions

Yenduri, G., Ramalingam, M., Selvi, G. C., Supriya, Y., Srivastava, G., Maddikunta, P. K. R., Raj, G. D., Jhaveri, R. H., Prabadevi, B., Wang, W., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3389497 · Cited by 759 · see citing works

Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3605764.3623985 · Cited by 667 · see citing works

Membership Inference Attacks From First Principles

Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., & Tramèr, F. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833649 · Cited by 562 · see citing works

Stealing Hyperparameters in Machine Learning

Wang, B., & Gong, N. Z. (2018) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2018.00038 · Cited by 491 · see citing works

PRADA: Protecting Against DNN Model Stealing Attacks

Juuti, M., Szyller, S., Marchal, S., & Asokan, N. (2019) · 2019 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2019.00044 · Cited by 440 · see citing works

A Survey on ChatGPT: AI–Generated Contents, Challenges, and Solutions

Wang, Y., Pan, Y., Yan, M., Su, Z., & Luan, T. H. (2023) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2023.3300321 · Cited by 395 · see citing works

Red Teaming Language Models with Language Models

Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363 · see citing works

Deduplicating Training Data Makes Language Models Better

Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., & Carlini, N. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.577 · Cited by 304 · see citing works

Does learning require memorization? a short tale about a long tail

Feldman, V. (2020) · Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing · doi:10.1145/3357713.3384290 · Cited by 287 · see citing works

A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning

Tian, Z., Cui, L., Liang, J., & Yu, S. (2022) · ACM Computing Surveys · doi:10.1145/3551636 · Cited by 277 · see citing works

Multi-step Jailbreaking Privacy Attacks on ChatGPT

Li, H., Guo, D., Fan, W., Xu, M., Huang, J., Meng, F., & Song, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.272 · Cited by 269 · see citing works

ChatGPT is not all you need. A state-of-the-art review of large generative AI models

Gozalo-Brizuela, R., & Garrido-Merchan, E. C. (2023) · arXiv · doi:10.48550/arxiv.2301.04655 · Cited by 245 · see citing works

ONION: A Simple and Effective Defense Against Textual Backdoor Attacks

Qi, F., Chen, Y., Li, M., Yao, Y., Liu, Z., & Sun, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.752 · Cited by 222 · see citing works

A survey of safety and trustworthiness of large language models through the lens of verification and validation

Huang, X., Ruan, W., Huang, W., Jin, G., Dong, Y., Wu, C., Bensalem, S., Mu, R., Yi, Q., Zhao, X., et al. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10824-0 · Cited by 181 · see citing works

Model Extraction Warning in MLaaS Paradigm

Kesarwani, M., Mukhoty, B., Arya, V., & Mehta, S. (2018) · Annual Computer Security Applications Conference · doi:10.1145/3274694.3274740 · Cited by 131 · see citing works

Sponge Examples: Energy-Latency Attacks on Neural Networks

Shumailov, I., Zhao, Y., Bates, D., Papernot, N., Mullins, R., & Anderson, R. (2021) · 2021 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp51992.2021.00024 · Cited by 125 · see citing works

Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks

Mireshghallah, F., Goyal, K., Uniyal, A., Berg-Kirkpatrick, T., & Shokri, R. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.570 · Cited by 122 · see citing works

Obfuscation-Resilient Privacy Leak Detection for Mobile Apps Through Differential Analysis

Continella, A., Fratantonio, Y., Lindorfer, M., Puccetti, A., Zand, A., Kruegel, C., & Vigna, G. (2017) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2017.23465 · Cited by 119 · see citing works

Concealed Data Poisoning Attacks on NLP Models

Wallace, E., Zhao, T., Feng, S., & Singh, S. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.13 · Cited by 115 · see citing works

Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks

Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., & Hashimoto, T. (2024) · 2024 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw63631.2024.00018 · Cited by 114 · see citing works

Remember What You Want to Forget: Algorithms for Machine Unlearning

Sekhari, A., Acharya, J., Kamath, G., & Suresh, A. T. (2021) · arXiv · doi:10.48550/arxiv.2103.03279 · Cited by 99 · see citing works

AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators

He, X., Lin, Z., Gong, Y., Jin, A.-L., Zhang, H., Chen, L., Jiao, J., Yiu, S., Duan, N., & Chen, W. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-industry.15 · Cited by 97 · see citing works

Membership Inference Attacks against Language Models via Neighbourhood Comparison

Mattern, J., Mireshghallah, F., Jin, Z., Schölkopf, B., Sachan, M., & Berg-Kirkpatrick, T. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.719 · Cited by 94 · see citing works

Robust Encodings: A Framework for Combating Adversarial Typos

Jones, E., Jia, R., Raghunathan, A., & Liang, P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.245 · Cited by 89 · see citing works

Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?

Hisamoto, S., Post, M., & Duh, K. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00299 · Cited by 86 · see citing works

Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models

Xu, J., Ma, M., Wang, F., Xiao, C., & Chen, M. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.171 · Cited by 80 · see citing works

On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning

Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D. (2022) · arXiv · doi:10.48550/arxiv.2212.08061 · Cited by 76 · see citing works

On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning

Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.244 · Cited by 76 · see citing works

Deduplicating Training Data Mitigates Privacy Risks in Language Models

Kandpal, N., Wallace, E., & Raffel, C. (2022) · arXiv · doi:10.48550/arxiv.2202.06539 · Cited by 68 · see citing works

Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe

Yue, X., Inan, H. A., Li, X., Kumar, G., McAnallen, J., Shajari, H., Sun, H., Levitan, D., & Sim, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.74 · Cited by 52 · see citing works

Training Data Extraction From Pre-trained Language Models: A Survey

Ishihara, S. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.23 · Cited by 26 · see citing works

Get a Model! Model Hijacking Attack Against Machine Learning Models

Salem, A., Michael Backes, A., & Zhang, Y. (2022) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23064 · Cited by 24 · see citing works

DeepPoison: Feature Transfer Based Stealthy Poisoning Attack for DNNs

Chen, J., Zhang, L., Zheng, H., Wang, X., & Ming, Z. (2021) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2021.3060896 · Cited by 20 · see citing works

Theoretical Limits of Provable Security Against Model Extraction by Efficient Observational Defenses

Karchmer, A. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00046 · Cited by 3 · see citing works

When LLMs meet cybersecurity: a systematic literature review

Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 220 · see citing works

A Survey on Intelligent Network Operations and Performance Optimization Based on Large Language Models

Long, S., Tan, J., Mao, B., Tang, F., Li, Y., Zhao, M., & Kato, N. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3526606 · Cited by 127 · see citing works

Survey of Different Large Language Model Architectures: Trends, Benchmarks, and Challenges

Shao, M., Basit, A., Karri, R., & Shafique, M. (2024) · IEEE Access · doi:10.1109/access.2024.3482107 · Cited by 110 · see citing works

Application of Large Language Models in Cybersecurity: A Systematic Literature Review

Hasanov, I. J., Virtanen, S., Hakkala, A., & Isoaho, J. (2024) · IEEE Access · doi:10.1109/access.2024.3505983 · Cited by 114 · see citing works

Steering towards safe self-driving laboratories

Leong, S. X., Griesbach, C. E., Zhang, R., Darvish, K., Zhao, Y., Mandal, A., Zou, Y., Hao, H., Bernales, V., & Aspuru‐Guzik, A. (2025) · Nature Reviews Chemistry · doi:10.1038/s41570-025-00747-x · Cited by 48 · see citing works

A survey on privacy risks and protection in large language models

Chen, K., Zhou, X., Lin, Y., Feng, S., Shen, L., & Wu, P. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00177-1 · Cited by 52 · see citing works

Workplace security and privacy implications in the GenAI age: A survey

Diro, A. A., Kaisar, S., Saini, A., Fatima, S., Pham, H., & Erba, F. (2025) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2024.103960 · Cited by 31 · see citing works

Combining Uncensored and Censored LLMs for Ransomware Generation

Yamin, M. M., Hashmi, E., & Katt, B. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0573-6_14 · Cited by 12 · see citing works

Risks, Causes, and Mitigations of Widespread Deployments of Large Language Models (LLMs): A Survey

Sakib, M. N., Islam, M. A., Pathak, R., & Arifin, M. M. (2024) · International Conference on Artificial Intelligence, Blockchain, and Internet of Things (AIBThings) · doi:10.1109/aibthings63359.2024.10863356 · Cited by 13 · see citing works

A Lifecycle-Oriented Survey of Emerging Threats and Vulnerabilities in Large Language Models

de Maio, C., Di Gisi, M., Fenza, G., Gallo, M., & Loia, V. (2025) · IEEE Access · doi:10.1109/access.2025.3619764 · Cited by 10 · see citing works

[Short Paper] Forensic Analysis of Indirect Prompt Injection Attacks on LLM Agents

Chernyshev, M., Baig, Z. A., & Doss, R. (2024) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa62245.2024.00053 · Cited by 7 · see citing works

LLM Firewall Using Validator Agent for Prevention Against Prompt Injection Attacks

Podpora, M., Baranowski, M., Chopcian, M., Kwaśniewicz, Ł., & Radziewicz, W. (2025) · Applied Sciences · doi:10.3390/app16010085 · Cited by 6 · see citing works

Large language models and large concept models in radiology: Present challenges, future directions, and critical perspectives

Merchant, S. A., Merchant, N., Varghese, S. L., & Shaikh, M. J. S. (2025) · World Journal of Radiology · doi:10.4329/wjr.v17.i11.114754 · Cited by 6 · see citing works

Addressing Data Poisoning and Model Manipulation Risks using LLM Models in Web Security

Shah, S. P., & Deshpande, A. V. (2024) · International Conference on Distributed Systems, Computer Networks and Cybersecurity (ICDSCNC) · doi:10.1109/icdscnc62492.2024.10941696 · Cited by 4 · see citing works

Evaluation of NeMo Guardrails as a Firewall for User–LLM Interaction

Azambuja, A. J., Guilherme, M., Castro, J. V. F. D., Lima, J. P. D. O., Oliveira, L. B., & Soares, A. D. S. (2026) · Future Internet · doi:10.3390/fi18050252 · Cited by 2 · see citing works

Data Breach Prevention in AI Systems: Employing Event-Driven Architecture to Combat Prompt Injection Attacks in Chatbots

Samonte, M. J. C., Aparize, J. E. R., Gonzales, E. J. S., & Morilla, J. L. (2024) · IEEE International Conference on Information, Communication and Networks (ICICN) · doi:10.1109/icicn62625.2024.10761619 · Cited by 2 · see citing works

LLM-WFIN: A Fine-Grained Large Language Model (LLM)-Oriented Website Fingerprinting Attack via Fusing Interrupt Trace and Network Traffic

Jiao, J., Hong, Y., & Wen, R. (2025) · Electronics · doi:10.3390/electronics14071263 · Cited by 2 · see citing works

Systematic Testing of Security-Related Vulnerabilities in LLM-Based Applications

Kaplan, H. (2025) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain66642.2025.00043 · Cited by 2 · see citing works

The AI-Cybersecurity Nexus: How Large Language Models are Reshaping Threat Intelligence and Digital Defense

Özbay, R., Çelebi, M., & Yavanoğlu, U. (2026) · IEEE Access · doi:10.1109/access.2026.3658308 · Cited by 3 · see citing works

A Data-Free Backdoor Attack Approach in Self-supervised Models

Ma, L., Yu, Z., Yang, W., Wu, S., Xie, M., Yang, B., & Lai, C. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4836-8_2

Avaliação do NeMo Guardrails como um Firewall para a Interação Usuário-LLM

Alves, M. G. D. D. O., de Castro, J. V. F., Lima, J. P. D. O., de Azambuja, A. J. G., Oliveira, L. B., & Soares, A. D. S. (2025) · Simpósio Brasileiro de Cibersegurança (SBSeg) · doi:10.5753/sbseg.2025.10665

Privacy-Preserving Learning and Inference in Foundation Models for Secure AI Applications

Alex, S. A., & Dahmani, N. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11391117

A Zero-Trust, AI-RMF–Governed Architecture for LLM-Enabled Telemedicine-as-a-Service: Mitigating Poisoning, Leakage and Unsafe-Output Threats

Julio, Y. R., Mangones, A. P., Pérez-García, N. A., Huerta, M., Viloria-Núñez, C., Hernández, F. I., Tovio, J. T., Vega, H. V. R., & Pareja, J. E. D. L. R. (2025) · Latin American Computer Conference (CLEI) · doi:10.1109/clei67442.2025.11420318

LLM-Assisted Digital Forensic Investigations of Prompt Injection Attacks: Evidence Analysis and Representation

Chernyshev, M., Baig, Z. A., Syed, N., Doss, R., & Shore, M. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-22542-9_21

Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language ModelsPreprint

Chowdhury, A. G., Islam, M. M., Kumar, V., Shezan, F. H., Kumar, V., Jain, V., & Chadha, A. R. (2024) · arXiv (Cornell University) · doi:10.13140/rg.2.2.22225.75362 · Cited by 10 · see citing works

Hacking LLMs: A Technical Analysis of Security Vulnerabilities and Defense Mechanisms

Raj, G., Hamzah, Raj, N., & Ranjan, N. (2025) · International Conference on Computational Intelligence, Communication Technology and Networking (CICTN) · doi:10.1109/cictn64563.2025.10932638 · Cited by 3 · see citing works

A Survey of Attacks on Large Language ModelsPreprint

Xu, W., & Parhi, K. K. (2025) · arXiv.org · doi:10.48550/arxiv.2505.12567 · Cited by 21 · see citing works

Attack and defense techniques in large language models: A survey and new perspectives

Liao, Z., Chen, K., Lin, Y., Li, K., Liu, Y., Chen, H., Huang, X., & Yu, Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108388 · Cited by 19 · see citing works

LLM Security: Vulnerabilities, Attacks, Defenses, and CountermeasuresPreprint

Aguilera-Martínez, F., & Berzal, F. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.01177 · Cited by 1 · see citing works