CoChat
Sign Up
PreprintCited by 3Adversarial Robustness in Machine LearningWeb Application Security VulnerabilitiesSecurity and Verification in Computing

Prompt Injection 2.0: Hybrid AI Threats

Open access
McHugh, Jeremy, Šekrst, Kristina, Cefalu, Jon·2025·arXiv
DOI 10.48550/arxiv.2507.13169arXiv 2507.13169
Open this paper in CoChat — free accountOpen PDF

Abstract

Prompt injection attacks, where malicious input is designed to manipulate AI systems into ignoring their original instructions and following unauthorized commands instead, were first discovered by Preamble, Inc. in May 2022 and responsibly disclosed to OpenAI. Over the last three years, these attacks have continued to pose a critical security threat to LLM-integrated systems. The emergence of agentic AI systems, where LLMs autonomously perform multistep tasks through tools and coordination with other agents, has fundamentally transformed the threat landscape. Modern prompt injection attacks can now combine with traditional cybersecurity exploits to create hybrid threats that systematically evade traditional security controls. This paper presents a comprehensive analysis of Prompt Injection 2.0, examining how prompt injections integrate with Cross-Site Scripting (XSS), Cross-Site Request Forgery (CSRF), and other web security vulnerabilities to bypass traditional security measures. We build upon Preamble's foundational research and mitigation technologies, evaluating them against contemporary threats, including AI worms, multi-agent infections, and hybrid cyber-AI attacks. Our analysis incorporates recent benchmarks that demonstrate how traditional web application firewalls, XSS filters, and CSRF tokens fail against AI-enhanced attacks. We also present architectural solutions that combine prompt isolation, runtime security, and privilege separation with novel threat detection capabilities.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2507.13169

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The generative process of LLMs allows for context-sensitive adaptability during interaction, including jailbreaking and prompt injection where outputs can be steered or manipulated by strategically crafted inputs.

    “The generative process of LLMs allows for context-sensitive adaptability during interaction, including the well-documented phenomena of jailbreaking and prompt injection, where outputs can be steered or manipulated by strategically crafted inputs [44].”

    From 4E cognition and the coevolution of human–AI interaction (2025) · cites this paper as [44]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 22

  1. [1][1] Branch, H. J., et al. (2022). Evaluating the Susceptibility of Pre-trained Language Models via Handcrafted Adversarial Examples. arXiv . https://arxiv.org/pdf/2209.02128 . - · arXiv
  2. [3][3] Liu, Y., et al. (2023). Prompt Injection attack against LLM-integrated Applications. arXiv . https://arxiv.org/abs/2306.05499 . - · arXiv
  3. [4][4] Liu, X., et al. (2024). Automatic and Universal Prompt Injection Attacks against Large Language Models. arXiv . https://arxiv.org/abs/2403.04957 . - · arXiv
  4. [5][5] Pedro, R., Castro, D., Carreira, P., & Santos, N. (2023). From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application? arXiv . [https://arxiv.org/abs/2308.01990 .](https://arxiv.org/abs/2308.01990) - · arXiv
  5. [6][6] Šekrst, K., McHugh, J., & Cefalu, J. (2024). AI Ethics by Design: Implementing Customizable Guardrails for Responsible AI Development. arXiv . https://arxiv.org/abs/2411.14442 . - · arXiv
  6. [7][7] Beurer-Kellner, L. et al. (2025). Design Patterns for Securing LLM Agents against Prompt Injections. arXiv . https://arxiv.org/abs/2506.08837 . - · arXiv
  7. [9][9] Lee, D., & Tiwari, M. (2024). Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems. arXiv . https://arxiv.org/abs/2410.07283 . - · arXiv
  8. [10][10] Debenedetti, E., et al. (2025). Defeating Prompt Injections by Design. https://arxiv.org/abs/2503.18813 . - · arXiv
  9. [14][14] Yi, J., et al. (2023). Benchmarking and 3. Defending Against Indirect Prompt Injection Attacks on Large Language Models. arXiv . https://arxiv.org/abs/2312.14197 . - · arXiv
  10. [16][16] Štorek, A., et al. (2025). XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants. arXiv . https://arxiv.org/abs/2503.14281 . - · arXiv
  11. [17][17] Mudryi, M., Chaklosh, M., & Wójcik, G. (2025). The Hidden Dangers of Browsing AI Agents. arXiv . https://arxiv.org/abs/2505.13076 . - · arXiv
  12. [18][18] Zhou, X., et al. (2024). Learning to Poison Large Language Models for Downstream Manipulation. arXiv . https://arxiv.org/abs/2402.13459 . - · arXiv
  13. [19][19] Zhang, Q., et al. (2024). Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applications. arXiv . https://arxiv.org/abs/2404.17196 . - · arXiv
  14. [20][20] Xiong, J., Zhu, et al. (2025). Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Models. arXiv . https://arxiv.org/abs/2505.16957 . - · arXiv
  15. [21][21] Clop, C., & Teglia, Y. (2024). Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Models. arXiv . https://arxiv.org/abs/2410.14479 . - · arXiv
  16. [22][22] Wang, L., et al. (2025). Manipulating Multimodal Agents via Cross-Modal Prompt Injection. arXiv . https://arxiv.org/abs/2504.14348 . - · arXiv
  17. [24][24] Bagdasaryan, E., Hsieh, T.-Y., Nassi, B., Shmatikov, V. (2023). Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs. arXiv . https://arxiv.org/abs/2307.10490 . - · arXiv
  18. [25][25] Schulhoff, S., et al. (2023). Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition. arXiv . -
  19. [26][26] He, J., et al. (2024). Data Stealing Attacks against Large Language Models via Backdooring. Electronics , 13(14), 2858. https://doi.org/10.3390/ electronics13142858 . -
  20. [27][27] Fang, R., Bindu, R., Gupta, A., Zhan, Q., & Kang, D. (2024). LLM Agents can Autonomously Hack Websites. arXiv . https://arxiv.org/abs/2402.06664 . - · arXiv
  21. [28][28] Cohen, S., Bitton, R., & Nassi, B. (2024). Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications. arXiv . https://arxiv.org/abs/2403.02817 . - · arXiv
  22. [29][29] Hines, K., et al. (2024). Defending Against Indirect Prompt Injection Attacks With arXiv https://arxiv.org/abs/2311.16119 . Spotlighting. . https://arxiv.org/abs/2403.14720 . - A Taxonomy of prompt injection threats Table 1: A unified taxonomy of prompt injection threats · arXiv

Cited by

Sorted by citations

Authors
McHugh, J., Šekrst, K., & Cefalu, J.
ORCID
Šekrst, Kristina
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2507.13169
arXiv
2507.13169
OpenAlex
W4415310439
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Information Systems › Web Application Security Vulnerabilities
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
Keywords
prompt injection, agentic AI systems, large language models, cross-site scripting, cross-site request forgery, web application security, runtime security, privilege separation, threat detection, web application firewall, LLM security

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review