The DOI matches its Crossref record; 263 of the 288 references it cites were found in a registry or an index.
Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
Open accessAbstract
Agentic AI systems powered by large language models (LLMs) and endowed with planning, tool use, memory, and autonomy, are emerging as powerful, flexible platforms for automation. Their ability to autonomously execute tasks across web, software, and physical environments creates new and amplified security risks, distinct from both traditional AI safety and conventional software security. This survey outlines a taxonomy of threats specific to agentic AI, reviews recent benchmarks and evaluation methodologies, and discusses defense strategies from both technical and governance perspectives.We synthesize current research and highlight open challenges, aiming to support the development of secure-by-design agent systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The second is the development of agentic AI: software systems that wrap LLM reasoning in perception, planning, memory, tool-use, and multi-agent communication loops, enabling them to autonomously execute multi-step security tasks that previously required human analysts”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They introduce new attack surfaces: prompt injection via untrusted observations [88], tool hijacking, and emergent collusion in multiagent settings [22,26].”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [22]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“across the full 153-paper corpus, only five studies (≈3%)- [22,71,125] for error propagation and [26,87] for tool-use amplification-report observing a compound failure mode at all”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [22]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hao, X.-B., & Zhang, G.-G. (2017) · Encycl. Semantic Comput. Robotic Intell. · doi:10.1142/s2425038416300184 · Cited by 86,401 · cited 2×
“…can have across diverse domains, these systems are being increasingly employed in several real-world scenarios and applications: • Automation of complex workflows: Agentic AI systems can autonomously manage supply chain processes and adapt procurement decisions in response to dynamic data…”
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“For instance, Devin , an AI software engineer developed by Cognition Labs, has demonstrated the ability to plan, code, debug, and deploy software with minimal oversight.”
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · cited 1×
“Analogous to defending against traditional ML/AI adversarial examples in computer vision, completely preventing prompt injection is an open problem [186].”
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 1×
“Modern pretraining and supervised fine-tuning paradigms on dialogue-style data, which trains the model to learn short-term instruction-response behavior (while deprioritizing long-term embodied sequential state tracking), are likely resulting in these shortcomings [165, 162].”
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · cited 1×
“For instance, Devin , an AI software engineer developed by Cognition Labs, has demonstrated the ability to plan, code, debug, and deploy software with minimal oversight.”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · cited 1×
“In the subsequent section, we discuss these motivations in more detail, and delineate our contributions towards augmenting the safety and security of agentic AI frameworks.”
Robust physical-world attacks on deep learning visual classification
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00175 · Cited by 2,173
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv · doi:10.48550/arxiv.1802.00420 · Cited by 2,172 · cited 1×
“More generally, in adversarial attack research in ML/AI, many defenses initially reported as effective are commonly undermined soon after their release, primarily because they lack thorough evaluation against adaptive attack strategies [182, 251, 252].”
Research works printed but not linked: 70
- 10.1007/978-981-97-5656-8 · DOI
- Work, 2024, AI what do large language models ‘understand, Image, 21, 1
- 10.1007/978-3-031-54827-7_5 · DOI
- 2025, AI Agents Are Here. So Are the Threats.
- 2024, LangChain Documentation
- Richards, 2024, AutoGPT: An Autonomous GPT Experiment
- Dal Cin, 2025, Three essentials for agentic AI security, MIT Sloan Manage. Rev., 1
- 2025, Just in time? Manufacturers Turn to AI to Weather Tariff Storm
- 2025, Forget Chatbots. AI Agents are the Future
- 10.1016/j.cell.2024.09.022 · DOI · Paper page
- 2025, Inside the Automated Warehouse Where Robots Are Packing Your Groceries
- 2025, Amazon’s Delivery, Logistics Get AI Boost
- 10.1007/978-3-031-90026-6_10 · DOI · Paper page
- Micro, 2025, Preventing Zero-Click AI Threats: Insights From EchoLeak
- 2025, AI: Advent of Agents Opens New Possibilities for Attackers
- 10.52202/079017-4136 · doi:10.52202/079017-4136 · Paper page
- 10.18653/v1/2025.emnlp-main.1771 · DOI
- 2025, When Llms Autonomously Attack
- 2025, OWASP GenAI LLM01: Prompt Injection
- 10.18653/v1/2025.acl-long.890 · DOI
- 10.18653/v1/2023.emnlp-main.849 · DOI
- Park, 2025, Unveiling AI Agent Vulnerabilities Part II: Code Execution
- 2024, CVE-2024-5565: Vanna.AI Remote Code Execution Vulnerability
- Chhabra, Towards fair video summarization
- 10.1609/aaai.v34i07.6918 · DOI
- Rehberger, 2024, DeepSeek AI: From Prompt Injection To Account Takeover
- 2025, Technical Blog: Strengthening AI Agent Hijacking Evaluations
- Kennedy, 2011, Metasploit: Penetration Tester’s Guide
- Muehlberger, 2020, CSRF and XSS: Practical Examples Using Burp Suite
- 10.18653/v1/2024.findings-emnlp.624 · DOI
- 2025, Introduction To MCP. Model Context Protocol
- Surapaneni, 2025, A2A: A New Era of Agent Interoperability
- Chang, 2024, AgentNetworkProtocol (ANP) GitHub Repository
- 2024, Agent Communication Protocol (Linux Foundation)
- 10.1007/978-3-030-58951-6_24 · DOI · Paper page
- 10.18653/v1/2025.acl-long.476 · DOI
- 10.52202/079017-1421 · DOI
- 10.1109/SP46214.2022.9833644 · DOI · Paper page
- 10.14722/ndss.2025.230478 · DOI
- 2025, JFrog and Hugging Face Join Forces to Expose Malicious ML Models
- 10.1145/501978.501980 · DOI · Paper page
- 10.18653/v1/2022.emnlp-main.425 · DOI
- 10.18653/v1/2022.findings-emnlp.148 · DOI
- 10.1145/2619091 · DOI
- Singh, 2025, LLM-Based Agents: The Benefits and the Risks
- Chhabra, What data benefits my classifier? Enhancing model performance and interpretability through influence-based data selection
- 10.52202/075280-0056 · DOI
- Petrovic, 2025, Advanced Interpretability Techniques for Tracing LLM Activations
- 10.52202/079017-0664 · DOI
- de Castro, Privacy-preserving large language model inference via gpu-accelerated fully homomorphic encryption
- Lu, 2024, An efficient and extensible zero-knowledge proof framework for neural networks, Cryptol. ePrint Arch., 2024/703
- 10.1163/2210-7975_hrd-9919-2016005 · doi:10.1163/2210-7975_hrd-9919-2016005
- Cihon, Chilling autonomy: Policy enforcement for human oversight of AI agents
- 10.1063/5.0222987 · DOI
- 2024, Fine-Tuned DeBERTa-V3-Base for Prompt Injection Detection
- Chen, 2024, Aligning LLMs to be robust against prompt injection, arXiv:2410.05451 · arXiv
- Helff, Llavaguard: VLM-based safeguard for vision dataset curation and safety assessment, 8322
- 10.1609/aies.v7i1.31664 · DOI
- 2024, CloudImposer: Executing Code on Millions of Google Servers With a Single Malicious Package
- 2025, NCCoE Cyber AI Profile
- 2025, Agentic AI—Threats & Mitigations
- 10.1007/978-3-032-02130-4_2 · DOI
- 10.18653/v1/2025.findings-naacl.123 · DOI
- 10.52202/079017-1650 · doi:10.52202/079017-1650 · Paper page
- 10.18653/v1/2024.acl-long.50 · DOI · Paper page
- 10.18653/v1/2025.findings-acl.1113 · DOI
- 10.18653/v1/2025.acl-industry.75 · DOI
- 10.18653/v1/2025.acl-long.1575 · DOI
- 10.18653/v1/2025.findings-acl.332 · DOI
- 10.18653/v1/2025.findings-acl.349 · DOI
Cited by
Agentic Business Process Management: A research manifesto
Calvanese, D., Casciani, A., Giacomo, G. D., Dumas, M., Fournier, F., Kampik, T., Malfa, E. L., Limonad, L., Marrella, A., Metzger, A., et al. (2026) · Information Systems · doi:10.1016/j.is.2026.102738 · Cited by 37
“However, genAI agent security is an emerging and evolving area of research (cf. [81] for a taxonomic overview) and, accordingly, APM systems face — just like any other systems that deploy genAI agents — many security challenges that remain unsolved.”
Nageshwaran, V., & Ezekiel, S. (2026) · Electronics · doi:10.3390/electronics15122740 · Cited by 4
“The second is the development of agentic AI: software systems that wrap LLM reasoning in perception, planning, memory, tool-use, and multi-agent communication loops, enabling them to autonomously execute multi-step security tasks that previously required human analysts [22–28].”
InfluentialNageshwaran, V., & Ezekiel, S. (2026) · doi:10.20944/preprints202606.0142.v1 · Cited by 4
Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
de Witt, C. S., Krawiecka, K., Krawczuk, I., Hagag, B., Anderson, W. L., Belcak, P., Bucknall, B., Cai, X., Chopra, A., Cohen, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.02077 · Cited by 108
Krishnan, V. G., & Divya, V. (2026) · VAWKUM Transactions on Computer Sciences · doi:10.21015/vtcs.v14i2.2688 · Cited by 1
Metadata
- Authors
- Chhabra, A., Datta, S., Nahin, S. K., & Mohapatra, P.
- Affiliations
- University of South Florida (US)
- Venue
- IEEE Access
- Published
- 2026
- Volume
- 14
- Pages
- 49455-49482
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W7138839765
- Cited by
- 28
- References
- 288
- Citations by year
- 2026: 28
- FWCI
- 95.69
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
- Keywords
- agentic AI, large language models, AI security, security threats, threat taxonomy, security by design, AI governance, AI agent evaluation, autonomous agents, tool use, AI safety
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges. https://app.cochat.ai/papers/10.1109/access.2026.3675554
- The paper
- Chhabra, A., Datta, S., Nahin, S. K., & Mohapatra, P. (2026). Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges. IEEE Access. https://doi.org/10.1109/access.2026.3675554
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.