A Systematic Review of Prompt Injection Attacks on Large Language Models: Trends, Taxonomy, Evaluation, Defenses, and Opportunities
Open accessAbstract
Large Language Models (LLMs) are increasingly integrated into various infrastructure and interactive applications. However, their inherent linguistic flexibility introduces security vulnerabilities, particularly through Prompt Injection (PI) attacks. This systematic review provides a comprehensive synthesis of the evolving landscape of PI attacks, including their development trends, classification schemes, evaluation methodologies, defense strategies, and areas for future research. We analyze the evolution of attacks from basic natural language overrides to complex multi-turn manipulations, indirect injections using structured formats such as JSON and XML, and tool-assisted exploits involving automated prompt generation and multi-modal inputs like images. PI attacks are categorized by the level of manipulation, including character, word, sentence, and semantic levels, and by adversarial intent, such as prompt leaking and harmful content generation. The study analyzes these techniques within different LLM architectures and application contexts, including machine translation and Chain of Thought reasoning. It also reviews common evaluation benchmarks, datasets, and metrics. Mitigation strategies examined include automated red teaming frameworks, input validation, content filtering, and alignment through training. It differentiates itself from previous works by conducting a systematic review specifically focused on immediate injection, organizing its attacks, defenses and resear…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Adversarial strategies have shifted from basic natural language overrides to highly sophisticated, multi-stage indirect injections that exploit structured formats and integrated tools.
“The threat landscape has evolved rapidly; adversarial strategies have shifted from basic natural language overrides to highly sophisticated, multi-stage indirect injections that exploit structured formats and integrated tools [6].”
From Evaluation of NeMo Guardrails as a Firewall for User–LLM Interaction (2026) · cites this paper as [6]
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · see citing works
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · see citing works
3D Object Representations for Fine-Grained Categorization
Krause, J., Stark, M. M., Deng, J., & Fei-Fei, L. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.77 · Cited by 3,523 · see citing works
Automated Flower Classification over a Large Number of Classes
Nilsback, M.-E., & Zisserman, A. (2008) · 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing · doi:10.1109/icvgip.2008.47 · Cited by 3,442 · see citing works
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · see citing works
Fake News Detection on Social Media: A Data Mining Perspective
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3137597.3137600 · Cited by 3,304 · see citing works
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283 · see citing works
Fei-Fei, L., Fergus, R., & Perona, P. (2005) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvpr.2004.383 · Cited by 2,513 · see citing works
Describing Textures in the Wild
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., & Vedaldi, A. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.461 · Cited by 2,438 · see citing works
Food-101 – Mining Discriminative Components with Random Forests
Bossard, L., Guillaumin, M., & Gool, L. V. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_29 · Cited by 2,105 · see citing works
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification
Helber, P., Bischke, B., Dengel, A., & Borth, D. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2918242 · Cited by 1,951 · see citing works
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
Wang, W. Y. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2067 · Cited by 1,822 · see citing works
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · see citing works
Parkhi, O., Vedaldi, A., Zisserman, A., & Jawahar, C. V. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248092 · Cited by 1,563 · see citing works
Jin, D., Jin, Z., Zhou, J. T., & Szolovits, P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6311 · Cited by 1,108 · see citing works
Unleashing the potential of prompt engineering for large language models
CHEN, B. ‐., Zhang, Z., Langrené, N., & Zhu, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101260 · Cited by 764 · see citing works
From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy
Gupta, M., Akiri, C., Aryal, K., Parker, E., & Praharaj, L. (2023) · IEEE Access · doi:10.1109/access.2023.3300381 · Cited by 714 · see citing works
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3605764.3623985 · Cited by 667 · cited 10× · see citing works
“As highlighted by Greshake et al. [7], contemporary development has moved beyond static LLM applications toward agentic frameworks in which the model autonomously decomposes tasks, invokes external tools or APIs, and utilizes memory streams to reflect and adapt behaviour over time.”
InfluentialGenerating Natural Language Adversarial Examples through Probability Weighted Word Saliency
Ren, S., Deng, Y., He, K., & Che, W. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1103 · Cited by 666 · see citing works
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Li, L., Ma, R., Guo, Q., Xue, X., & Qiu, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.500 · Cited by 623 · see citing works
Shen, X., Chen, Z., Backes, M., Shen, Y., & Zhang, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670388 · Cited by 441 · see citing works
Artificial intelligence risk management framework :
National Institute of Standards and Technology (US) (2024) · NIST Trustworthy and Responsible AI · doi:10.6028/nist.ai.600-1 · Cited by 274 · cited 2× · see citing works
“Specifically, the findings from red team exercises provide critical, real-world data to inform the entire lifecycle of the AI RMF [8], particularly measure and monitoring LLM integrated applications, generating verifiable evidence and metrics necessary for demonstrating that the AI system meets…”
<i>Did Aristotle Use a Laptop?</i> A Question Answering Benchmark with Implicit Reasoning Strategies
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00370 · Cited by 242 · see citing works
Contextualized Perturbation for Textual Adversarial Attack
Li, D., Zhang, Y., Peng, H., Chen, L., Brockett, C., Sun, M., & Dolan, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.400 · Cited by 213 · see citing works
The risks associated with Artificial General Intelligence: A systematic review
McLean, S. G., Read, G. J. M., Thompson, J., Baber, C., Stanton, N. A., & Salmon, P. M. (2021) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2021.1964003 · Cited by 206 · see citing works
Adversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 194 · see citing works
Impact of ChatGPT on medical chatbots as a disruptive technology
Chow, J. C. L., Sanders, L., & Li, K. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1166014 · Cited by 188 · see citing works
PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Zhu, K., Wang, J., Zhou, J., Wang, Z., Chen, H., Wang, Y., Yang, L., Ye, W., Zhang, Y., Gong, N., et al. (2023) · Proceedings of the 1st ACM Workshop on Large AI Systems and Models with Privacy and Safety Analysis · doi:10.1145/3689217.3690621 · Cited by 151 · see citing works
Generative AI in cybersecurity: A comprehensive review of LLM applications and vulnerabilities
Ferrag, M. A., Alwahedi, F., Battah, A., Cherif, B., Mechri, A., Tihanyi, N., Bisztray, T., & Debbah, M. (2025) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2025.01.001 · Cited by 148 · see citing works
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Liu, Y., Deng, G., Xu, Z., Li, Y., Zheng, Y., Zhang, Y., Zhao, L., Zhang, T., & Wang, K. (2024) · doi:10.1145/3663530.3665021 · Cited by 105 · see citing works
Voultsiou, E., & Moussiades, L. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13550-4 · Cited by 102 · see citing works
Chow, J. C. L., & Li, K. (2025) · Information · doi:10.3390/info16070549 · Cited by 84 · see citing works
A Survey of Attacks on Large Vision–Language Models: Resources, Advances, and Future Trends
Liu, D., Yang, M., Qu, X., Zhou, P., Cheng, Y., & Hu, W. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3592935 · Cited by 79 · see citing works
Schramowski, P., Tauchmann, C., & Kersting, K. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533192 · Cited by 63 · see citing works
PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluation
Hu, J., Dong, T., Luo, G., Ma, H., Zou, P., Sun, X. N., Guo, D., Yang, X., & Wang, M. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3497725 · Cited by 46 · see citing works
One Prompt Word is Enough to Boost Adversarial Robustness for Pre-Trained Vision-Language Models
Li, L., Guan, H., Qiu, J., & Spratling, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02304 · Cited by 39 · see citing works
Bisi, T., Risser, A., Clavert, P., Migаud, H., & Dаrtus, J. (2023) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2023.103694 · Cited by 38 · see citing works
Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks
Pasquini, D., Strohmeier, M., & Troncoso, C. (2024) · AISec@CCS · doi:10.1145/3689932.3694764 · Cited by 27 · see citing works
Kumar, S. S., Cummings, M. L., & Stimpson, A. (2024) · 2024 IEEE 4th International Conference on Human-Machine Systems (ICHMS) · doi:10.1109/ichms59971.2024.10555871 · Cited by 21 · see citing works
Yip, D. W., Esmradi, A., & Chan, C. F. (2023) · IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) · doi:10.1109/csde59766.2023.10487667 · Cited by 19 · see citing works
Quaye, J. A., Parrish, A., Inel, O., Rastogi, C., Kirk, H. R., Kahng, M., van Liemt, E., Bartolo, M., Tsang, J., White, J., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658913 · Cited by 19 · see citing works
Enhancing System Security: LLM-Driven Defense Against Prompt Injection Vulnerabilities
Muliarevych, O. (2024) · IEEE International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET) · doi:10.1109/tcset64720.2024.10755823 · Cited by 12 · see citing works
PromptAttack: Prompt-Based Attack for Language Models via Gradient Search
Shi, Y., Li, P., Yin, C., Han, Z., Zhou, L., & Liu, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17120-8_53 · Cited by 11 · see citing works
A Brief Survey on Safety of Large Language Models
Gao, Z., Liu, X., Lan, Y.-S., & Yang, Z. (2024) · Journal of Computing and Information Technology · doi:10.20532/cit.2024.1005778 · Cited by 10 · see citing works
Large Language Models in Wargaming: Methodology, Application, and Robustness
Chen, Y., & Chu, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00295 · Cited by 10 · see citing works
Contextual Spelling Correction with Large Language Models
Song, G. H., Wu, Z., Pundak, G., Chandorkar, A., Joshi, K., Velez, X., Caseiro, D., Haynor, B., Wang, W., Siddhartha, N., et al. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389637 · Cited by 8 · see citing works
Defending Language Models Against Image-Based Prompt Attacks via User-Provided Specifications
Sharma, R. K., Gupta, V., & Grossman, D. (2024) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw63631.2024.00017 · Cited by 8 · see citing works
FDI: Attack Neural Code Generation Systems through User Feedback Channel
Sun, Z., Du, X., Luo, X., Song, F., Lo, D., & Li, L. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680300 · Cited by 8 · see citing works
Karaağaç, M., & Çarkıt, S. (2025) · Oncology · doi:10.1159/000546726 · Cited by 6 · see citing works
Offensive security for AI systems: concepts, practices, and applications
Harguess, J. D., & Ward, C. M. (2025) · Assurance and Security for AI-enabled Systems · doi:10.1117/12.3055959 · Cited by 6 · see citing works
’Ignore All and Accept My Resume’: The Impact of Prompt Injection in Automatic Resume Screening
Aminou, L., Daaif, A., Soulami, M., & Youssfi, M. (2025) · International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET) · doi:10.1109/iraset64571.2025.11008146 · Cited by 5 · see citing works
Improving the Semantic Consistency of Textual Adversarial Attacks via Prompt
Yu, X., Yin, Q., Shi, Z., & Ma, Y. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892715 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Emerging Risks of AI-to-AI Interactions in Health Care: Lessons From Moltbook
Athni, T. S. (2026) · Journal of Medical Internet Research · doi:10.2196/96199 · Cited by 4 · see citing works
Evaluation of NeMo Guardrails as a Firewall for User–LLM Interaction
Azambuja, A. J., Guilherme, M., Castro, J. V. F. D., Lima, J. P. D. O., Oliveira, L. B., & Soares, A. D. S. (2026) · Future Internet · doi:10.3390/fi18050252 · Cited by 2 · see citing works
Multimodal Prompt Injection: A Formal 4D Taxonomy for Image and Document Pipelines
Farhana, M. H., & Roy, R. K. (2026) · IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN) · doi:10.1109/qpain69676.2026.11545895 · Cited by 1 · see citing works
Mourad, S., Abdallah, E. E., & Ababneh, M. (2026) · Electronics · doi:10.3390/electronics15163700 · Cited by 1 · see citing works
Guimarães, V., Medeiros, D. D. C., Reis, L., Mesquita, H., & Rocha, T. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134939
Intelligent Prompt Filtering and Jailbreak Detection for Safe Deployment of GenerativePreprint
Singh, A., Singh, A., Singh, N., & Singh⁴, V. (2026) · Research Square · doi:10.21203/rs.3.rs-10621673/v1
When Résumés Become Instructions: Rhetorical Governance and Prompt Injection in AI-Mediated Hiring
Ponce, T. M., & Saathoff, A. (2026) · Business and Professional Communication Quarterly · doi:10.1177/23294906261473905
Ethical Implications of Prompt Injection in AI-mediated Grading: An Adversarial Red-Team Evaluation
Humble, N. (2026) · Journal of Academic Ethics · doi:10.1007/s10805-026-09758-9
Keerthana, M., Monesha, Y., P, S. K. B., Mahendran, D. M., & Reema, A. R. (2026) · International Conference on Smart Systems and Inventive Technology (ICSSIT) · doi:10.1109/icssit69151.2026.11656649
Roy, R. K., & Farhana, M. H. (2026) · IJARCCE · doi:10.17148/ijarcce.2026.15843
Dahiya, A., & Shen, T. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7366618
Artificial Intelligence - Model Context Protocol - Review of Real-World Security Threats
Kanuru, U., & Schmitt, A. (2026) · The Pinnacle A Journal by Scholar-Practitioners · doi:10.61643/c38836
Large Language Models for Defense and Security: A Systematic ReviewPreprint
Maaiveld, T. M., Zambrano, N. A. P., Trantas, A., Weyland, L. F., Hentenaar, C., Nyegaard, D. D. N., & Schadd, M. P. D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7509620
Saravi, B., Singh, D. D., Schorn, L., Vollmer, A., Sproll, C. K., Kübler, N. R., & Schrader, F. (2026) · Scientific Reports · doi:10.1038/s41598-026-74077-3
Cedeño, M. I., & Cobeña, G. M. (2026) · REVISTA ODIGOS · doi:10.35290/ro.v7n3.2026.2082
Combining OWA Neutrosophic Operators and LogProbs for Uncertainty Evaluation in Agentic AI Outputs
Pérez, P. Y. P., Pupo, I. P., Mir, Y. O. V., Acuña, L. A., Herrera, R. Y., & Vázquez, M. Y. L. (2026) · Studies in computational intelligence · doi:10.1007/978-3-032-29288-9_1
THE DUAL ROLE OF LARGE LANGUAGE MODELS IN CYBERSECURITY: THREATS, DEFENSES, AND RESEARCH DIRECTIONS
Ali, M., & Manzoor, A. R. (2026) · “ADVANCED JOURNAL OF UNIVERSITY SCIENCE AND CREATIVE INNOVATIONS” AJOU SCI · doi:10.67620/ajousci.240001
Aziz, M. S., Ali, M., Manzoor, A. R., Younis, M., & Khabirova, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-11434787/v1
Bibi, M., Awan, K. A., Liu, L.-C., Chen, C.-L., Alanazi, F., Alanazi, M. H., & Uddin, M. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01351-9
- Authors
- Damacena Duarte, J., Cândido, G. D., De Britto Filho, J. R. A., Souza Neto, J., Da Costa, E. J., Da Costa, J. P. J., & Peotta De Melo, L.
- Affiliations
- Department of Electrical Engineering, University of Brasília, Brasília, Brazil
- ORCID
- Venue
- IEEE Access
- Published
- 2026
- Volume
- 14
- Pages
- 12875-12899
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W7125497551
- Cited by
- 15
- References
- 55
- Citations by year
- 2026: 15
- FWCI
- 14.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- prompt injection, large language models, LLM security, attack taxonomy, harmful content generation, defense strategy, input validation, content filtering, alignment training, LLM safety, machine translation, chain-of-thought reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
