SoK: The Attack Surface of Agentic AI - Tools and Autonomy
Open accessAbstract
Recent AI systems combine large language models with tools, external knowledge via retrieval-augmented generation (RAG), and even autonomous multi-agent decision loops. This agentic AI paradigm greatly expands capabilities - but also vastly enlarges the attack surface. In this systematization, we map out the trust boundaries and security risks of agentic LLM-based systems. We develop a comprehensive taxonomy of attacks spanning prompt-level injections, knowledge-base poisoning, tool/plug-in exploits, and multi-agent emergent threats. Through a detailed literature review, we synthesize evidence from 2023-2025, including more than 20 peer-reviewed and archival studies, industry reports, and standards. We find that agentic systems introduce new vectors for indirect prompt injection, code execution exploits, RAG index poisoning, and cross-agent manipulation that go beyond traditional AI threats. We define attacker models and threat scenarios, and propose metrics (e.g., Unsafe Action Rate, Privilege Escalation Distance) to evaluate security posture. Our survey examines defenses such as input sanitization, retrieval filters, sandboxes, access control, and "AI guardrails," assessing their effectiveness and pointing out the areas where protection is still lacking. To assist practitioners, we outline defensive controls and provide a phased security checklist for deploying agentic AI (covering design-time hardening, runtime monitoring, and incident response). Finally, we outline open research challenges in secure autonomous AI (robust tool APIs, verifiable agent behavior, supply-chain safeguards) and discuss ethical and responsible disclosure practices. We systematize recent findings to help researchers and engineers understand and mitigate security risks in agentic AI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The SoK on agentic AI attack surfaces and the MCP-38 threat taxonomy provide comprehensive attack taxonomies but no unified formal defenses.
“The SoK on agentic AI attack surfaces [28] and the MCP-38 threat taxonomy [29] provide comprehensive attack taxonomies but no unified formal defenses.”
From SentinelAgent: Intent-Verified Delegation Chains for Securing Federal Multi-Agent AI Systems (2026) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · see citing works
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3605764.3623985 · Cited by 667 · see citing works
Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak AttackPreprint
Russinovich, M., Salem, A., & Eldan, R. (2024) · USENIX Security Symposium · doi:10.48550/arxiv.2404.01833 · Cited by 479 · see citing works
AI agents under threat: A survey of key security challenges and future pathways
Deng, Z., Guo, Y., Han, C., Ma, W., Xiong, J., Wen, S., & Xiang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3716628 · Cited by 210 · see citing works
DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks
Liu, Y., Jia, Y.-Q., Jia, J.-Y., Song, D., & Gong, N. (2025) · IEEE Symposium on Security and Privacy · doi:10.1109/sp61157.2025.00250 · Cited by 203 · see citing works
Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast
Gu, X., Zheng, X., Pang, T., et al. (2024) · International Conference on Machine Learning · doi:10.48550/arxiv.2402.08567 · Cited by 189
Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models
Yi, J., Xie, Y., Zhu, B., Kıcıman, E., Sun, G., Xie, X., & Wu, F. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709179 · Cited by 175 · see citing works
Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents
Shafran, A., Schuster, R., & Shmatikov, V. (2024) · USENIX Security Symposium · doi:10.48550/arxiv.2406.05870 · Cited by 133
Raza, S., Sapkota, R., Karkee, M., & Emmanouilidis, C. (2025) · AI Open · doi:10.48550/arxiv.2506.04133 · Cited by 132 · see citing works
Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices
Abdali, S., Anarfi, R., Barberan, C., & He, J. (2024) · arXiv.org · doi:10.48550/arxiv.2403.12503 · Cited by 68
Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)
Verma, A., Krishna, S., Gehrmann, S., Seshadri, M., Pradhan, A., Ault, T., Barrett, L., Rabinowitz, D., Doucette, J., & Phan, N. (2024) · arXiv.org · doi:10.48550/arxiv.2407.14937 · Cited by 59
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.06674 · Cited by 56 · see citing works
SecAlign: Defending Against Prompt Injection with Preference Optimization
Chen, S., Zharmagambetov, A. S., Mahloujifar, S., Chaudhuri, K., Wagner, D., & Guo, C. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3744836 · Cited by 54 · see citing works
Demystifying RCE Vulnerabilities in LLM-Integrated Apps
Liu, T., Deng, Z., Meng, G., Li, Y., & Chen, K. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3690338 · Cited by 45 · see citing works
Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks
Pasquini, D., Strohmeier, M., & Troncoso, C. (2024) · AISec@CCS · doi:10.1145/3689932.3694764 · Cited by 27 · see citing works
Securing Agentic AI: A Comprehensive Threat Model and Mitigation Framework for Generative AI Agents
Narajala, V. S., & Narayan, O. (2025) · 2025 8th International Conference on Algorithms, Computing and Artificial Intelligence (ACAI) · doi:10.1109/acai68217.2025.11406310 · Cited by 27 · see citing works
Blockagents: Towards byzantine-robust llm-based multi-agent coordination via blockchain
Chen, B., Li, G., Lin, X., Wang, Z., & Li, J. (2024) · ACM Turing Award Celebration Conference 2024 · doi:10.1145/3674399.3674445 · Cited by 15 · see citing works
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
Chen, Z., Xiang, Z., Xiao, C., Song, D., & Li, B. (2024) · arXiv · doi:10.48550/arxiv.2407.12784 · Cited by 10 · see citing works
StruQ: defending against prompt injection with structured queries
Chen, S., Piet, J., Sitawarin, C., & Wagner, D. (2024) · arXiv · doi:10.48550/arxiv.2402.06363 · Cited by 10 · see citing works
Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent Systems
Peigné, P., Kniejski, M., Sondej, F., David, M., Hoelscher-Obermaier, J., Schroeder de Witt, C., & Kran, E. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i26.34970 · Cited by 8 · see citing works
Elías, H., & Luca, F. D. (2019) · GIS - Gesto Imagem e Som - Revista de Antropologia · doi:10.11606/issn.2525-3123.gis.2019.152838 · Cited by 7
ShieldGemma: Generative AI Content Moderation Based on Gemma
Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., Radharapu, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21772 · Cited by 3 · see citing works
Flagged first, then by citations
8 of the 30 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- n. d.]. GitHub -sleeepeer/PoisonedRAG: [USENIX Security 2025] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models -github.com. https://github.com/sleeepeer/PoisonedRAG. [Accessed 12-09-2025].
- Bang An, Shiyue Zhang, and Mark Dredze. 2025. RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Association for Computational Linguistics, 5444-5474. doi:10.18653/v1/2025.naacl-long.281 · DOI
- Edoardo Debenedetti, Shiying Yang, Martin Bertran, Aleksander Mądry, et al. 2024. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. In NeurIPS 2024 Datasets and Benchmarks Track. https://arxiv.org/abs/2406.13352 arXiv:2406.13352. · arXiv
- Apostol Vassilev, Alina Oprea, Alie Fordyce, Hyrum Anderson, Xander Davies, and Maia Hamin. 2025. Adversarial Machine Learning: A Taxonomy and Ter- minology of Attacks and Mitigations. NIST Trustworthy and Responsible AI Report AI 100-2e2025. National Institute of Standards and Technology (NIST), Gaithersburg, MD. https://doi.org/10.6028/NIST.AI.100-2e2025 · DOI
- Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. 2025. PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. In 34th USENIX Security Symposium (Seattle). USENIX. https: //www.usenix.org/conference/usenixsecurity25/presentation/zou-poisonedrag
Cited by
Sorted by citations
What You Approve Is What Executes: Consent Integrity for Black-Box LLM Agents
Weng, X. (2026) · arXiv.org · doi:10.48550/arxiv.2606.02668 · Cited by 16
LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments
Zhang, C., Yang, H., Jiang, B.-A., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.10779 · Cited by 7
“As LLM-based autonomous agents such as OpenClaw [37] are increasingly deployed on personal servers and enterprise intranets to handle daily tasks and business operations, they dramatically boost productivity while simultaneously expanding the attack surface of AI systems [10].”
Rashidi, M. (2026) · arXiv.org · doi:10.48550/arxiv.2607.05743 · Cited by 4 · see citing works
“Three recent broader surveys of agentic AI security exist [1]–[3], and a fourth reviews open agentic platforms specifically [4], but all four treat execution isolation as one entry in a wide catalog of concerns alongside prompt injection, retrieval poisoning, and multi-agent risk, rather than as a…”
Lineage: A UMA2.0 Protocol Extension for Multi-Hop LLM Agent Delegation
Ştefănescu, Ş., & Aciobăniţei, I. (2026) · IEEE Access · doi:10.1109/access.2026.3708618 · Cited by 2
“…of Raza et al. [27] identifies transparent delegation boundaries and multi-level logging as foundational requirements; Dehghantanha and Homayoun [28] catalogue the broader attack surface across prompt injections, knowledge-base poisoning, tool exploits, and emergent multi-agent threats; Ko et…”
Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback
Yan, L., Li, R.-Z., Han, X., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.17453 · Cited by 2
“…ToolSafe, ToolShield, AgentSentry, CommandSans, AgentWatcher, AgentSys, BrowseSafe, refusal-alignment, and system-level agent-security analyses [30–41], together with world-model and code-world-model reasoning [42–44 Our method differs in targeting final-action risk under exploratory trust…”
Karim, H., Sitharaman, S., Gupta, D., et al. (2026) · 2026 56th Annual IEEE International Conference on Dependable Systems and Networks Workshops (DSN-W) · doi:10.1109/dsn-w70714.2026.00022 · Cited by 2
“Agentic AI is already operating across trust boundaries that static risk scores do not capture [2].”
MuninAI: A Self-Hosted Agentic Framework for Independent Research Groups
Fischer, M., Meiler, J., & Elgeti, M. (2026) · doi:10.48550/arxiv.2610.23480
“Standard practice [32] and practitioner guidance on air-gapped deployment both recommend denying egress by default and routing traffic through filtering proxies.”
Defense-in-Depth at the Perception-Reasoning Interface of LLM-Centric Agentic UAV Swarms
Homaei, M., Emami, Y., Homayoun, S., et al. (2026) · doi:10.48550/arxiv.2610.03319
“Agentic AI lets such systems pursue complex goals with limited supervision [2], but this autonomy also broadens the attack surface [3] to prompts, retrieved knowledge, agent memory, tool interactions, and perception inputs, so security must cover the agentic pipeline rather than the Large Language…”
Self-citationAttack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation
Pathade, C., Pawar, P., & Patil, S. (2026) · doi:10.48550/arxiv.2609.25173
Baek, H., Abuadbba, A., Moore, K., Kim, H., & Nepal, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.23894
“Multiple legitimate tools or services are composed to achieve a security consequence that would not be permitted through an individual operation [39, 83].”
Zhang, L., Sun, Y., & Shi, J. (2026) · doi:10.48550/arxiv.2609.23700 · Cited by 1
Trust propagation and structural containment in Multi-agent LLM pipelines
Safin, T. H., Zisad, S. N., Shatabda, S., et al. (2026) · doi:10.48550/arxiv.2609.17648
“However, practical evaluation of attack propagation through complete multi-agent systems remains limited [22], [23].”
Agentic AI Trustworthiness Is a Full-Stack Problem: A Framework from Hardware to Governance
Farahani, B., Rouintan, M., Vahidinia, P., & Firouzi, F. (2026) · IEEE International Conference on Omni-layer Intelligent Systems (COINS) · doi:10.1109/coins69995.2026.11715577
“For high-impact actions, confirmation gates add an explicit safety check and limit unnecessary privileges [27].”
An Action-Centric Zero Trust Maturity Model for Agentic AI Environments
Mok, J.-H., Jo, H., & Lee, S. (2026) · Sensors · doi:10.3390/s26165205
“Prior studies show that agent autonomy, tool use, external integration, and multi-agent interaction expand the attack surface beyond vulnerabilities in a single model [3,4,11,12].”
InfluentialWhere Is the Tradeoff in Using Third-Party API Routers for Agentic Software Development?
Fu, D., Li, J., Jiang, X., et al. (2026) · doi:10.48550/arxiv.2607.23624
“In that setting, router-side injection faces little resistance before it becomes concrete action Liu et al. (2026b); Xie et al. (2025); Tallam (2026); Dehghantanha & Homayoun (2026).”
LLM-Centric Agentic AI for UAV Swarms: Architecture, Enabling Technologies, and Open Problems
Emami, Y., Taheri, R., Homaei, M., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.09756
“Similarly, malicious tool feed-back can deceive the agent by providing plausible but incorrect outputs from weather services, GNSS modules, channel estimators, or simulators [14].”
Natanzi, S., & Tang, B. (2026) · arXiv.org · doi:10.48550/arxiv.2606.31639
“Recent work therefore treats LLM-agent ecosystems as protocol and work-flow security problems, not only model-safety problems (Chu, 2026; Dehghantanha et al., 2026; Ferrag et al., 2025; Ling et al., 2026; Zhao et al., 2026).”
Acharya, N., & Gupta, G. K. (2026) · arXiv · doi:10.48550/arxiv.2604.05969
“Dehghantanha and Homayoun [26] systematize the attack surface of agentic AI across tools, RAG, and multi-agent loops.”
- Authors
- Dehghantanha, A., & Homayoun, S.
- Affiliations
- University of Guelph-Humber (CA)
- Aalborg University (DK)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2603.22928
- OpenAlex
- W7140300304
- Semantic Scholar
- db437d3cdc6b1a5910d0d89fc120f400b9ac80fc
- Cited by
- 0
- References
- 30
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- VBN Forskningsportal (Aalborg Universitet) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Natural Sciences and Engineering Research Council of Canada (00017, RGPIN-2019, rgpin-2019-03995)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
- Keywords
- agentic AI, attack surface, large language models, retrieval-augmented generation, prompt injection, multi-agent systems, AI security, security defenses, AI guardrails, security metrics, security checklist, responsible disclosure, autonomous AI systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
