The DOI matches its Crossref record; 131 of the 131 references it cites were found in a registry or an index.
AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways
Open accessPublished version of a preprint
Abstract
An Artificial Intelligence (AI) agent is a software entity that autonomously performs tasks or makes decisions based on pre-defined objectives and data inputs. AI agents, capable of perceiving user inputs, reasoning and planning tasks, and executing actions, have seen remarkable advancements in algorithm development and task performance. However, the security challenges they pose remain under-explored and unresolved. This survey delves into the emerging security threats faced by AI agents, categorizing them into four critical knowledge gaps: unpredictability of multi-step user inputs, complexity in internal executions, variability of operational environments, and interactions with untrusted external entities. By systematically reviewing these threats, this article highlights both the progress made and the existing limitations in safeguarding AI agents. The insights provided aim to inspire further research into addressing the security threats associated with AI agents, thereby fostering the development of more robust and secure AI agent applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI Agents are typically designed as single-entity systems that perform goal-directed tasks by utilizing external tools, applying sequential reasoning, and integrating real-time information to complete well-defined functions [25,43].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, while pre-modern expert systems required manual updates to static knowledge bases, modern agents leverage emergent neural architectures to generalize across tasks [25].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI Agents are typically designed as single-entity systems that perform goal-directed tasks by utilizing external tools, applying sequential reasoning, and integrating real-time information to complete well-defined functions [24], [45].”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [24]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“AI Agents often include basic mechanisms for interacting with dynamic inputs, allowing them to respond to real-time stimuli such as user requests, external API calls, or state changes in software environments [25,71].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“Already, expanded attack surfaces and emergent risky behaviors have been observed in autonomous agent experiments [49,50,51].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [49]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“To better understand the action threats, we present the action structure as follows:”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“To better understand the action threats, we present the action structure as follows:”
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 3×
“Due to the associative abilities of LLMs [25], frequent occurrences of pronouns and identity markers, such as gender, race, nationality, and culture in training data, can bias AI agents in processing data [59, 162].”
InfluentialChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 2×
“Additionally, the other effective measures can be applied, such as fine-tuning AI agents specifically for indirect prompt injection [196, 204], alignment [121], and employing methods such as prompt engineering and post-training classifier-based security approaches [68].”
Self-Refine: Iterative Refinement with Self-Feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.17651 · Cited by 2,314 · cited 1×
“This limitation restricts their ability to retain and use extensive context from previous interactions.”
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Research works printed but not linked: 114
- Mahyar Abbasian Iman Azimi Amir M. Rahmani and Ramesh Jain. 2023. Conversational health agents: A personalized LLM-powered agent framework. Retrieved from https://arxiv.org/abs/2310.02374 · arXiv
- Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman Shyamal Anadkat et al. 2023. GPT-4 technical report. Retrieved from https://arxiv.org/pdf/2303.08774 · arXiv
- Divyansh Agarwal Alexander R. Fabbri Philippe Laban Shafiq Joty Caiming Xiong and Chien-Sheng Wu. 2024. Investigating the prompt leakage effect and black-box defenses for multi-turn LLM interactions. Retrieved from https://arxiv.org/pdf/2404.16251v1 · arXiv
- Gabriel Alon and Michael Kamfonas. 2023. Detecting language model attacks with perplexity. Retrieved from https://arxiv.org/pdf/2308.14132 · arXiv
- Anthropic. 2024. Many-shot Jailbreaking. Retrieved from https://www.anthropic.com/research/many-shot-jailbreaking
- Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen Nova DasSarma Dawn Drain Stanislav Fort Deep Ganguli Tom Henighan et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. Retrieved from https://arxiv.org/pdf/2204.05862 · arXiv
- Rishabh Bhardwaj and Soujanya Poria. 2023. Language model unalignment: Parametric red-teaming to expose hidden harms and biases. Retrieved from https://arxiv.org/pdf/2310.14303 · arXiv
- Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, and Pierre-Yves Oudeyer. 2023. Grounding large language models in interactive environments with online reinforcement learning. In Proceedings of the ICML.
- Dake Chen Hanbin Wang Yunhao Huo Yuzhao Li and Haoyang Zhang. 2023. Gamegpt: Multi-agent collaborative framework for game development. Retrieved from https://arxiv.org/pdf/2310.08067 · arXiv
- Xuan Chen Yuzhou Nie Lu Yan Yunshu Mao Wenbo Guo and Xiangyu Zhang. 2024. RL-JACK: Reinforcement learning-powered black-box jailbreaking attack against LLMs. Retrieved from https://arxiv.org/pdf/2406.08725 · arXiv
- Yongchao Chen Jacob Arkin Yang Zhang Nicholas Roy and Chuchu Fan. 2023. Scalable multi-robot collaboration with large language models: Centralized or decentralized systems? Retrieved from https://arxiv.org/html/2309.15943v2
- Zhaorun Chen Zhen Xiang Chaowei Xiao Dawn Song and Bo Li. 2024. Agentpoison: Red-teaming LLM agents via poisoning memory or knowledge bases. Retrieved from https://arxiv.org/pdf/2407.12784 · arXiv
- Steffi Chern Zhen Fan and Andy Liu. 2024. Combating adversarial attacks with multi-agent debate. Retrieved from https://arxiv.org/pdf/2401.05998 · arXiv
- Stav Cohen Ron Bitton and Ben Nassi. 2024. Here comes the AI worm: Unleashing zero-click worms that target GenAI-powered applications. Retrieved from https://arxiv.org/pdf/2403.02817 · arXiv
- Maxwell Crouse Ibrahim Abdelaziz Kinjal Basu Soham Dan Sadhana Kumaravel Achille Fokoue Pavan Kapanipathi and Luis Lastras. 2023. Formally specifying the high-level behavior of LLM-based agents. Retrieved from https://arxiv.org/pdf/2310.08535 · arXiv
- Chenhang Cui Gelei Deng An Zhang Jingnan Zheng Yicong Li Lianli Gao Tianwei Zhang and Tat-Seng Chua. 2024. Safe+ Safe= Unsafe? Exploring how safe images can be exploited to jailbreak large vision-language models. Retrieved from https://arxiv.org/pdf/2411.11496 · arXiv
- Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li Xinhao Deng Yunpeng Liu Qinglin Zhang Ziyi Qiu Peiyang Li et al. 2024. Risk taxonomy mitigation and assessment benchmarks of large language model systems. Retrieved from https://arxiv.org/pdf/2401.05778 · arXiv
- Yiran Ding Li Lyna Zhang Chengruidong Zhang Yuanyuan Xu Ning Shang Jiahang Xu Fan Yang and Mao Yang. 2024. LongRoPE: Extending LLM context window beyond 2 million tokens. Retrieved from https://arxiv.org/pdf/2402.13753 · arXiv
- Tian Dong Guoxing Chen Shaofeng Li Minhui Xue Rayne Holland Yan Meng Zhen Liu and Haojin Zhu. 2023. Unleashing cheapfakes through trojan plugins of large language models. Retrieved from https://arxiv.org/pdf/2312.00374v1 · arXiv
- Yingkai Dong Zheng Li Xiangtao Meng Ning Yu and Shanqing Guo. 2024. Jailbreaking text-to-image models with LLM-based agents. Retrieved from https://arxiv.org/pdf/2408.00523 · arXiv
- Yilun Du Shuang Li Antonio Torralba Joshua B. Tenenbaum and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. Retrieved from https://arxiv.org/pdf/2305.14325 · arXiv
- Yuqing Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. 2023. Guiding pretraining in reinforcement learning with large language models. In Proceedings of the ICML. PMLR.
- Eva Eigner and Thorsten Händler. 2024. Determinants of LLM-assisted decision-making. Retrieved from https://arxiv.org/pdf/2402.17385 · arXiv
- Aysan Esmradi, Daniel Wankit Yip, and Chun Fai Chan. 2023. A comprehensive survey of attack techniques, implementation, and mitigation strategies in large language models. In Proceedings of the UbiSec.
- Isabel O. Gallegos Ryan A. Rossi Joe Barrow Md Mehrab Tanjim Sungchul Kim Franck Dernoncourt Tong Yu Ruiyi Zhang and Nesreen K. Ahmed. 2023. Bias and fairness in large language models: A survey. Retrieved from https://arxiv.org/pdf/2309.00770 · arXiv
- Jonas Geiping Alex Stein Manli Shu Khalid Saifullah Yuxin Wen and Tom Goldstein. 2024. Coercing LLMs to do and reveal (almost) anything. Retrieved from https://arxiv.org/abs/2402.14020 · arXiv
- Kai Greshake Sahar Abdelnabi Shailesh Mishra Christoph Endres Thorsten Holz and Mario Fritz. 2023. More than you’ve asked for: A comprehensive analysis of novel prompt injection threats to application-integrated large language models. Retrieved from https://arxiv.org/pdf/2302.12173 · arXiv
- Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. In Proceedings of the AISec. 79–90.
- Xiangming Gu Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Ye Wang Jing Jiang and Min Lin. 2024. Agent smith: A single image can jailbreak one million multimodal LLM agents exponentially fast. Retrieved from https://arxiv.org/pdf/2402.08567 · arXiv
- Michael Guastalla, Yiyi Li, Arvin Hekmati, and Bhaskar Krishnamachari. 2023. Application of large language models to DDoS attack detection. In Proceedings of the SmartSP.
- Rui Hao Linmei Hu Weijian Qi Qingliu Wu Yirui Zhang and Liqiang Nie. 2023. ChatLLM network: More brains more intelligence. Retrieved from https://arxiv.org/pdf/2304.12998 · arXiv
- Kostas Hatalis, Despina Christou, Joshua Myers, Steven Jones, Keith Lambert, Adam Amos-Binks, Zohreh Dannenhauer, and Dustin Dannenhauer. 2023. Memory matters: The need to improve long-term memory in LLM-Agents. In Proceedings of the AAAI.
- Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2024. MetaGPT: Meta programming for a multi-agent collaborative framework. In Proceedings of the ICLR.
- Yuki Hou, Haruki Tamoto, and Homei Miyashita. 2024. “My agent understands me better”: Integrating dynamic human-like memory recall and consolidation in LLM-based agents. In Proceedings of the CHI.
- Bin Hu Chenyang Zhao Pu Zhang Zihao Zhou Yuanhang Yang Zenglin Xu and Bin Liu. 2023. Enabling intelligent interactions between an agent and an LLM: A reinforcement learning approach. Retrieved from https://arxiv.org/pdf/2306.03604 · arXiv
- Xijie Huang Li Lyna Zhang Kwang-Ting Cheng and Mao Yang. 2023. Boosting LLM reasoning: Push the limits of few-shot learning with reinforced in-context pruning. Retrieved from https://arxiv.org/pdf/2312.08901 · arXiv
- Yue Huang Qihui Zhang Lichao Sun et al. 2023. Trustgpt: A benchmark for trustworthy and responsible large language models. Retrieved from https://arxiv.org/pdf/2306.11507 · arXiv
- S. Humeau, K. Shuster, M. Lachaux, and J. Weston. 2020. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. arXiv. In Proceedings of the ICLR.
- Neel Jain Avi Schwarzschild Yuxin Wen Gowthami Somepalli John Kirchenbauer Ping-yeh Chiang Micah Goldblum Aniruddha Saha Jonas Geiping and Tom Goldstein. 2023. Baseline defenses for adversarial attacks against aligned language models. Retrieved from https://arxiv.org/pdf/2309.00614 · arXiv
- Zhenlan Ji Daoyuan Wu Pingchuan Ma Zongjie Li and Shuai Wang. 2024. Testing and understanding erroneous planning in LLM agents through synthesized user inputs. Retrieved from https://arxiv.org/pdf/2404.17833 · arXiv
- Mintong Kang Nezihe Merve Gürel Ning Yu Dawn Song and Bo Li. 2024. C-RAG: Certified generation risks for retrieval-augmented language models. Retrieved from https://arxiv.org/pdf/2402.03181 · arXiv
- Aounon Kumar Chirag Agarwal Suraj Srinivas Soheil Feizi and Hima Lakkaraju. 2024. Certifying LLM safety against adversarial prompting. Retrieved from https://arxiv.org/pdf/2309.02705 · arXiv
- Patrick Levi and Christoph P. Neumann. 2024. Vocabulary attack to hijack large language model applications. Retrieved from https://arxiv.org/pdf/2404.02637 · arXiv
- Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al.. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the NeurIPS.
- Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative agents for ”Mind” exploration of large language model society. In Proceedings of the NeurIPS.
- Yuanchun Li Hao Wen Weijun Wang Xiangyu Li Yizhen Yuan Guohong Liu Jiacheng Liu Wenxing Xu Xiang Wang Yi Sun et al. 2024. Personal LLM agents: Insights and survey about the capability efficiency and security. Retrieved from https://arxiv.org/pdf/2401.05459 · arXiv
- Zelong Li Wenyue Hua Hao Wang He Zhu and Yongfeng Zhang. 2024. Formal-LLM: Integrating formal language and natural language for controllable LLM-based agents. Retrieved from https://arxiv.org/pdf/2402.00798 · arXiv
- Zeyi Liao Lingbo Mo Chejian Xu Mintong Kang Jiawei Zhang Chaowei Xiao Yuan Tian Bo Li and Huan Sun. 2024. Eia: Environmental injection attack on generalist web agents for privacy leakage. Retrieved from https://arxiv.org/pdf/2409.11295 · arXiv
- Jonathan Light, Min Cai, Sheng Shen, and Ziniu Hu. 2023. AvalonBench: Evaluating LLMs playing the game of avalon. In Proceedings of the NeurIPS Workshop.
- Baihan Lin Djallel Bouneffouf Guillermo Cecchi and Kush R. Varshney. 2023. Towards healthy AI: large language models need therapists too. Retrieved from https://arxiv.org/pdf/2304.00416 · arXiv
- Jiaju Lin Haoran Zhao Aochi Zhang Yiting Wu Huqiuyue Ping and Qin Chen. 2023. Agentsims: An open-source sandbox for large language model evaluation. Retrieved from https://arxiv.org/pdf/2308.04026 · arXiv
- Sheng Liu Lei Xing and James Zou. 2023. In-context vectors: Making in context learning more effective and controllable through latent space steering. Retrieved from https://arxiv.org/pdf/2311.06668 · arXiv
- Tong Liu Zizhuang Deng Guozhu Meng Yuekang Li and Kai Chen. 2023. Demystifying rce vulnerabilities in LLM-integrated apps. Retrieved from https://arxiv.org/pdf/2309.02926 · arXiv
- Yi Liu Gelei Deng Yuekang Li Kailong Wang Tianwei Zhang Yepang Liu Haoyu Wang Yan Zheng and Yang Liu. 2023. Prompt injection attack against LLM-integrated applications. Retrieved from https://arxiv.org/pdf/2306.05499 · arXiv
- Yi Liu Gelei Deng Zhengzi Xu Yuekang Li Yaowen Zheng Ying Zhang Lida Zhao Tianwei Zhang and Yang Liu. 2023. Jailbreaking chatgpt via prompt engineering: An empirical study. Retrieved from https://arxiv.org/pdf/2305.13860 · arXiv
- Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024. Formalizing and benchmarking prompt injection attacks and defenses. In Proceedings of the USENIX Security.
- Yang Liu Yuanshun Yao Jean-Francois Ton Xiaoying Zhang Ruocheng Guo Hao Cheng Yegor Klochkov Muhammad Faaiz Taufiq and Hang Li. 2023. Trustworthy LLMs: A survey and guideline for evaluating large language models’ alignment. Retrieved from https://arxiv.org/pdf/2308.05374 · arXiv
- Qinghua Lu Liming Zhu Xiwei Xu Zhenchang Xing Stefan Harrer and Jon Whittle. 2023. Building the future of responsible AI: A reference architecture for designing large language model based agents. Retrieved from https://arxiv.org/pdf/2311.13148 · arXiv
- Nikhil Mehta, Milagro Teruel, Xin Deng, Sergio Figueroa Sanz, Ahmed Awadallah, and Julia Kiseleva. 2024. Improving grounded language understanding in a collaborative environment by interacting with agents through help feedback. In Proceedings of the EACL.
- Kai Mei Zelong Li Shuyuan Xu Ruosong Ye Yingqiang Ge and Yongfeng Zhang. 2024. AIOS: LLM agent operating system. Retrieved from https://arxiv.org/pdf/2403.16971 · arXiv
- Lingbo Mo Zeyi Liao Boyuan Zheng Yu Su Chaowei Xiao and Huan Sun. 2024. A trembling house of cards? Mapping adversarial attacks against language agents. Retrieved from https://arxiv.org/pdf/2402.10196 · arXiv
- Sumeet Ramesh Motwani, Mikhail Baranchuk, Lewis Hammond, and Christian Schroeder de Witt. 2023. A perfect collusion benchmark: How can AI agents be prevented from colluding with information-theoretic undetectability?. In Proceedings of the NeurIPS Workshop.
- Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, and Yoav Shoham. 2024. Generating benchmarks for factuality evaluation of language models. In Proceedings of the EACL.
- Tai Nguyen and Eric Wong. 2023. In-context example selection with influences. Retrieved from https://arxiv.org/pdf/2302.11042 · arXiv
- Aidan O’Gara. 2023. Hoodwinked: Deception and cooperation in a text-based game for language models. Retrieved from https://arxiv.org/pdf/2308.01404 · arXiv
- Yikang Pan Liangming Pan Wenhu Chen Preslav Nakov Min-Yen Kan and William Yang Wang. 2023. On the risk of misinformation pollution with large language models. Retrieved from https://arxiv.org/pdf/2305.13661 · arXiv
- Rodrigo Pedro Daniel Castro Paulo Carreira and Nuno Santos. 2023. From prompt injections to SQL injection attacks: How protected is your LLM-integrated web application? Retrieved from https://arxiv.org/pdf/2308.01990 · arXiv
- Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. Retrieved from https://arXiv:2211.09527
- Steve Phelps and Rebecca Ranson. 2023. Of Models and Tin Men–a behavioural economics study of principal-agent problems in AI alignment using large-language models. Retrieved from https://arxiv.org/pdf/2307.11137 · arXiv
- Lukas Pöhler Valentin Schrader Alexander Ladwein and Florian von Keller. 2024. A technological perspective on misuse of available AI. Retrieved from https://arxiv.org/pdf/2403.15325 · arXiv
- Jack W. Rae Sebastian Borgeaud Trevor Cai Katie Millican Jordan Hoffmann Francis Song John Aslanides Sarah Henderson Roman Ring Susannah Young et al. 2021. Scaling language models: Methods analysis & insights from training Gopher. Retrieved from https://arxiv.org/pdf/2112.11446 · arXiv
- Fathima Abdul Rahman and Guang Lu. 2023. A contextualized real-time multimodal emotion recognition for conversational agents using graph convolutional networks in reinforcement learning. Retrieved from https://arxiv.org/pdf/2310.18363 · arXiv
- Alexander Robey Zachary Ravichandran Vijay Kumar Hamed Hassani and George J Pappas. 2024. Jailbreaking LLM-controlled robots. Retrieved from https://arxiv.org/pdf/2410.13691 · arXiv
- Sippo Rossi Alisia Marianne Michel Raghava Rao Mukkamala and Jason Bennett Thatcher. 2024. An early categorization of prompt injection attacks on large language models. Retrieved from https://arxiv.org/pdf/2402.00898 · arXiv
- Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J. Maddison, and Tatsunori Hashimoto. 2024. Identifying the risks of LM agents with an LM-emulated sandbox. In Proceedings of the ICLR.
- Ahmed Salem Andrew Paverd and Boris Köpf. 2023. Maatphor: Automated variant analysis for prompt injection attacks. Retrieved from https://arxiv.org/pdf/2312.11513 · arXiv
- Leo Schwinn David Dobre Stephan Günnemann and Gauthier Gidel. 2023. Adversarial attacks and defenses in large language models: Old and new threats. Retrieved from https://arxiv.org/pdf/2310.19737 · arXiv
- Mrinank Sharma Meg Tong Tomasz Korbak David Duvenaud Amanda Askell Samuel R. Bowman Newton Cheng Esin Durmus Zac Hatfield-Dodds Scott R. Johnston et al. 2023. Towards understanding sycophancy in language models. Retrieved from https://arxiv.org/pdf/2310.13548 · arXiv
- Xinyue Shen Yixin Wu Michael Backes and Yang Zhang. 2024. Voice jailbreak attacks against GPT-4o. Retrieved from https://arxiv.org/abs/2405.19103 · arXiv
- Emily H. Soice Rafael Rocha Kimberlee Cordova Michael Specter and Kevin M. Esvelt. 2023. Can large language models democratize access to dual-use biotechnology? Retrieved from https://arxiv.org/pdf/2306.03809 · arXiv
- Yuxiang Sun Checheng Yu Junjie Zhao Wei Wang and Xianzhong Zhou. 2023. Self generated wargame AI: Double layer agent task planning based on large language model. Retrieved from https://arxiv.org/pdf/2312.01090 · arXiv
- Harini Suresh and John V. Guttag. 2019. A framework for understanding unintended consequences of machine learning. Retrieved from https://arxiv.org/pdf/1901.10002 · arXiv
- Weihao Tan, Wentao Zhang, Shanqi Liu, Longtao Zheng, Xinrun Wang, and Bo An. 2024. True knowledge comes from practice: Aligning large language models with embodied environments via reinforcement learning. In Proceedings of the ICLR.
- Xiangru Tang Qiao Jin Kunlun Zhu Tongxin Yuan Yichi Zhang Wangchunshu Zhou Meng Qu Yilun Zhao Jian Tang Zhuosheng Zhang et al. 2024. Prioritizing safeguarding over autonomy: Risks of LLM agents for science. Retrieved from https://arxiv.org/pdf/2402.04247 · arXiv
- Mikhail Terekhov, Romain Graux, Eduardo Neville, Denis Rosset, and Gabin Kolly. 2023. Second-order Jailbreaks: Generative agents successfully manipulate through an intermediary. In Proceedings of the Multi-Agent Security Workshop @ NeurIPS’23. Retrieved from https://openreview.net/forum?id=HPmhaOTseN
- Yu Tian Xiao Yang Jingyuan Zhang Yinpeng Dong and Hang Su. 2023. Evil geniuses: Delving into the safety of llm-based agents. Retrieved from https://arxiv.org/pdf/2311.11855 · arXiv
- Sam Toyer, Olivia Watkins, Ethan Adrian Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Trevor Darrell, Alan Ritter, and Stuart Russell. 2024. Tensor trust: Interpretable prompt injection attacks from an online game. In Proceedings of the ICLR.
- Celine Wald and Lukas Pfahler. 2023. Exposing bias in online communities through large-scale language models. Retrieved from https://arxiv.org/pdf/2306.02294 · arXiv
- Eric Wallace Kai Xiao Reimar Leike Lilian Weng Johannes Heidecke and Alex Beutel. 2024. The instruction hierarchy: Training LLMs to prioritize privileged instructions. Retrieved from https://arxiv.org/pdf/2404.13208 · arXiv
- Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein. 2023. Poisoning language models during instruction tuning. In Proceedings of the ICML.
- Fengxiang Wang Ranjie Duan Peng Xiao Xiaojun Jia YueFeng Chen Chongwen Wang Jialing Tao Hang Su Jun Zhu and Hui Xue. 2024. MRJ-Agent: An effective jailbreak agent for multi-round dialogue. Retrieved from https://arxiv.org/pdf/2411.03814 · arXiv
- Jerry Wei Chengrun Yang Xinying Song Yifeng Lu Nathan Hu Dustin Tran Daiyi Peng Ruibo Liu Da Huang Cosmo Du et al. 2024. Long-form factuality in large language models. Retrieved from https://arxiv.org/pdf/2403.18802 · arXiv
- Zeming Wei Yifei Wang and Yisen Wang. 2023. Jailbreak and guard aligned language models with only few in-context demonstrations. Retrieved from https://arxiv.org/pdf/2310.06387 · arXiv
- Roy Weiss Daniel Ayzenshteyn Guy Amit and Yisroel Mirsky. 2024. What was your prompt? A remote keylogging attack on AI assistants. Retrieved from https://arxiv.org/pdf/2403.09751 · arXiv
- Yotam Wolf Noam Wies Yoav Levine and Amnon Shashua. 2023. Fundamental limitations of alignment in large language models. Retrieved from https://arxiv.org/pdf/2304.11082 · arXiv
- Chen Henry Wu Jing Yu Koh Ruslan Salakhutdinov Daniel Fried and Aditi Raghunathan. 2024. Adversarial attacks on multimodal agents. Retrieved from https://arxiv.org/pdf/2406.12814v1 · arXiv
- Fangzhou Wu Shutong Wu Yulong Cao and Chaowei Xiao. 2024. WIPI: A new web threat for LLM-driven web agents. Retrieved from https://arxiv.org/pdf/2402.16965 · arXiv
- Junlin Xie Zhihong Chen Ruifei Zhang Xiang Wan and Guanbin Li. 2024. Large multimodal agents: A survey. Retrieved from https://arxiv.org/pdf/2402.15116 · arXiv
- Frank Xing. 2024. Designing heterogeneous LLM agents for financial sentiment analysis. Retrieved from https://arxiv.org/pdf/2401.05799 · arXiv
- Binfeng Xu Zhiyuan Peng Bowen Lei Subhabrata Mukherjee Yuchen Liu and Dongkuan Xu. 2023. Rewoo: Decoupling reasoning from observations for efficient augmented language models. Retrieved from https://arxiv.org/pdf/2305.18323 · arXiv
- Yuzhuang Xu Shuo Wang Peng Li Fuwen Luo Xiaolong Wang Weidong Liu and Yang Liu. 2023. Exploring large language models for communication games: An empirical study on werewolf. Retrieved from https://arxiv.org/pdf/2309.04658 · arXiv
- Zelai Xu Chao Yu Fei Fang Yu Wang and Yi Wu. 2023. Language agents with reinforcement learning for strategic play in the werewolf game. Retrieved from https://arxiv.org/pdf/2309.04658 · arXiv
- Xue Yan Yan Song Xinyu Cui Filippos Christianos Haifeng Zhang David Henry Mguni and Jun Wang. 2023. Ask more know better: Reinforce-learned prompt questions for decision making with large language models. Retrieved from https://arxiv.org/pdf/2310.18127 · arXiv
- Yong Yang Xuhong Zhang Yi Jiang Xi Chen Haoyu Wang Shouling Ji and Zonghui Wang. 2024. PRSA: Prompt reverse stealing attacks against large language models. Retrieved from https://arxiv.org/pdf/2402.19200 · arXiv
- Jingwei Yi Yueqi Xie Bin Zhu Keegan Hines Emre Kiciman Guangzhong Sun Xing Xie and Fangzhao Wu. 2023. Benchmarking and defending against indirect prompt injection attacks on large language models. Retrieved from https://arxiv.org/pdf/2312.14197 · arXiv
- Jiahao Yu Xingwei Lin and Xinyu Xing. 2023. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. Retrieved from https://arxiv.org/pdf/2309.10253 · arXiv
- Zhiyuan Yu Xiaogeng Liu Shunning Liang Zach Cameron Chaowei Xiao and Ning Zhang. 2024. Don’t listen to me: Understanding and exploring jailbreak prompts of large language models. Retrieved from https://arxiv.org/pdf/2403.17336 · arXiv
- Yifan Zeng Yiran Wu Xiao Zhang Huazheng Wang and Qingyun Wu. 2024. AutoDefense: Multi-agent LLM defense against jailbreak attacks. Retrieved from https://arxiv.org/pdf/2403.04783 · arXiv
- Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua Tenenbaum, Tianmin Shu, and Chuang Gan. 2023. Building cooperative embodied agents modularly with large language models. In Proceedings of the NeurIPS Workshop.
- Wanpeng Zhang and Zongqing Lu. 2023. Rladapter: Bridging large language models to reinforcement learning in open worlds. Retrieved from https://arxiv.org/pdf/2309.17176 · arXiv
- Yuqi Zhang Liang Ding Lefei Zhang and Dacheng Tao. 2024. Intention analysis prompting makes large language models a good jailbreak defender. Retrieved from https://arxiv.org/pdf/2401.06561 · arXiv
- Zhexin Zhang Junxiao Yang Pei Ke and Minlie Huang. 2023. Defending large language models against jailbreaking attacks through goal prioritization. Retrieved from https://arxiv.org/pdf/2311.09096 · arXiv
- Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2024. Analyzing and mitigating object hallucination in large vision-language models. In Proceedings of the ICLR.
- Wei Zou Runpeng Geng Binghui Wang and Jinyuan Jia. 2024. PoisonedRAG: Knowledge poisoning attacks to retrieval-augmented generation of large language models. Retrieved from https://arxiv.org/pdf/2402.07867 · arXiv
Cited by
Items 101–200 of 223
Enhancing Reliability in Llm-Integrated Robotic Systems: A Unified Approach to Security and SafetyPreprint
Zhang, W., Kong, X., Dewitt, C., Bräunl, T., & Hong, J. B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5202517 · Cited by 1
Environmental Policy and Risk Regulatory Framework for Sustainable Tidal Current Energy in China
Guo, R., Liu, Z., Hanbing, W., Wang, G. F., Zhao, S., & Deng, Y. (2026) · Sustainability · doi:10.3390/su18073224 · Cited by 1
T., D., S., S., Ramamoorthy, A. K., & Balasubramanian, U. M. (2026) · Egyptian Informatics Journal · doi:10.1016/j.eij.2026.100983
SoK: The Attack Surface of Agentic AI - Tools and Autonomy
Dehghantanha, A., & Homayoun, S. (2026) · arXiv · doi:10.48550/arxiv.2603.22928
Intelligent Network Security Optimization Algorithm Based on Cnns
Zheng, M., Jia, R., Zhu, J. H., Zhang, S., Yao, W., & Li, Y. (2025) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.70069
Lin, S., Zhang, W., Jia, R., Deng, H., Li, L., Zhou, J., & Li, Z. (2025) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.70070
TeeDFuzzer: Fuzzing Trusted Execution Environment
Wen, S., Xu, L., Tian, L., Liu, S., & Ding, Y. (2025) · Electronics · doi:10.3390/electronics14081674
Self-citationPatch the Leak: Strengthening CodeLLMs Against Privacy Extraction Threats
Guo, Y., Ma, W., Xiao, X., Wen, S., Di, P., & Zhu, X. (2025) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3713081.3732931 · Cited by 3
Self-citationMulti-turn response selection with Language Style and Topic Aware enhancement
Li, W., Chen, Y., Xu, J., Zhong, J., & Dong, C. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101842
The dual-edged potential of AI autonomously defining loss functions
Ghori, A. (2025) · Journal of Electrical Systems and Information Technology · doi:10.1186/s43067-025-00248-3
On container vulnerabilities in edge computing: A fix-on-deployment approach
Huang, H., Meng, T., Guo, J., Tang, Z., & Jia, W. (2025) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2025.103533
Literature Review of AI Agents for Enterprise Teams
Gironacci, I. M. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-2419-7.ch001 · Cited by 1
Hussain, A., Ehsan, M., & Rauf, A. (2025) · International Journal of Ethics and Systems · doi:10.1108/ijoes-06-2025-0334
WebShell detection based on deep residual network
Yu, F., Li, X., Chang, Z., Fei, G., Ding, Y., Zhu, T., & Zhai, X. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10723-0
“Many web applications prioritize functionality and cross-platform compatibility, often at the expense of security considerations (Segun-Falade et al. 2024; Deng et al. 2025d).”
An Intelligent Playbook Recommendation Algorithm Based on Dynamic Interest Modeling for SOAR
Hu, H., Zhang, L., Zhang, Z., Yao, X., & Wu, X. (2025) · Symmetry · doi:10.3390/sym17111851
“By learning from and analyzing historical data, such systems can automatically recognize attack patterns and, based on specific alerts, intelligently recommend the most suitable playbooks [7].”
Using LLM Agents for Data Integration in Cybersecurity Incidents
Madrueño, N., Rueda, J., García-Ochoa, J., Fernández-Isabel, A., de Diego, I. M., & Ravines, R. R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-10486-1_33
Web3Agent: Automating On-Chain Operations via Natural Language Interfaces
Fan, S., & Min, T. (2025) · ACM Transactions on the Web · doi:10.1145/3777446 · Cited by 7
“These characteristics make the domain uniquely challenging for automated reasoning, but also present an opportunity: an AI agent, if properly designed, can systematically enforce procedural checks and prompt users for conirmation, potentially reducing the frequency of costly mistakes [8].”
Zhao, J., He, A., Hu, W., Huang, J., Zhao, Y., Zhang, J., & Liu, A. (2025) · Computer Networks · doi:10.1016/j.comnet.2025.111853
Efficient Subspace Updating Using Randomized Tensor Sketch for Online Network Security Monitoring
Zeng, J., Yang, L. T., Wang, C., Su, J., & Wang, Y. (2025) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2025.3633270 · Cited by 2
Unveiling AI data security: How employee awareness evolves in smart manufacturing
Yu, J., Xie, W., Zheng, D., & Guo, L. (2025) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2025.103011
Mimi: Dynamically Secure Multi-Keyword Retrieval Scheme With Two-Factor Verification
Li, D., Chattopadhyay, A., Li, Q., Wu, J., Lü, Q., Xing, B., Xiang, T., & Liao, X. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3642902
Black Box Business:A Governance Playbook for Autonomous Ai Agents in E-BusinessPreprint
Brehm, C. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5394669
Bidirectional gradient optimization for enhancing adversarial transferability
邝海玲, Wan, C., Zheng, Y., Guo, Z., & Chen, W. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132503
Fang, J., Shen, Y., Sun, Y., Jiang, C., Jiang, Y. H., Pan, H., Yiu, S., & Jiang, Z. L. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108497
A Design-Driven Taxonomy of AI Agentic Patterns
Joyce, J. E., & Maheshwari, S. (2025) · IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) · doi:10.1109/conecct65861.2025.11306519
“Intelligent agents power modern Artifical Intelligence (AI) by enabling autonomous decision-making, adaptive learning, and problem-solving in dynamic environments [1].”
Liu, Y. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-11477-8_11
Zhang, S., Zhu, W., Liang, W., Koe, A. S. V., & Xiong, N. N. (2026) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2026.103689
Military AI Security Assessment Model
Maathuis, C., & Cools, K. (2025) · Cyber Awareness and Research Symposium (CARS) · doi:10.1109/cars67163.2025.11337714
“Attacks on AI systems are broadly categorized into three main types: poisoning attacks, evasion attacks, and theft attacks being either targeted or nondiscriminating [8]–[10].”
OptAgent: an Agentic AI framework for Intelligent Building OperationsPreprint
Jiang, Z., Xu, W., & Dong, B. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6142787 · Cited by 3
Rethinking Query Choices for Differential Privacy Auditing
Deng, Z., Chen, H., Jiang, S., Ma, W., Wen, S., Zhu, T., & Xiang, Y. (2026) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2026.3658692
Self-citationJing, H., Li, F., Dong, Y., Zhou, W., & Liu, R. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.113968 · Cited by 11
Empowering Recommender Systems with Agentic AI: Towards Adaptive Online Personalization
Aly, A., Ibrahim, A., & Kashef, R. F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-14107-1_23
Adversial Prompt Injection in Large Language Models: Taxonomy, Exploits, and Mitigation Frameworks
Yadav, H., Singh, V., & Sharma, K. (2025) · International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN) · doi:10.1109/icrcicn68210.2025.11364988 · Cited by 1
“In one high-profile case, an attacker tricked an LLM-based chatbot into revealing its hidden system prompt (codename and all) by inserting a cleverly crafted instruction into a web page that the chatbot was summarizing[9].”
Toward Agentic Artificial Intelligence and Its Societal Implications
Ismail, Z. H., Sariff, N. B., & Fahsyar, P. N. A. (2026) · Artificial intelligence · doi:10.5772/intechopen.1014629
AI Agent Security: Vulnerability Analysis, Protective Measures and Challenges
Li, H., Feng, S., & Han, S. (2025) · International Conference on Computer, Big Data and Artificial Intelligence (ICCBD+AI) · doi:10.1109/iccbdai66607.2025.11388601
“The objective of agent security is to ensure the safe operation of agents by allowing them to process user inputs, execute tasks, and interact with other entities without succumbing to malicious attacks or exhibiting detrimental behaviors[1].”
Tyshchenko, V., & Drushchak, N. (2025) · 2025 3rd International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11391111
Wei, G., Jiang, D., Long, Q., Fu, Y., Sun, H. X., Cheng, X., Zhou, Y., & Zhang, H. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3667802
“While the current GTPAN framework relies on standard data anonymization to protect mobility trajectories, addressing advanced threats such as adversarial data poisoning, spoofing, firmware vulnerabilities, and insider attacks remains a critical direction for future work [42], [43], [44], [45].”
Wang, W., Liu, J., Gu, B., Wang, C., Qu, Y. N., & Feng, D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10820-8
“LLMs such as GPT-4 (Openai 2023; Zhou et al. 2025) and PaLM (Chowdhery et al. 2022; Deng et al. 2025 ) demonstrate strong capabilities in language understanding, zero/few-shot reasoning, and dialogue-based planning (Zhou et al. 2024).”
Soft-Prompted Semantic Normalization for Unsupervised Analysis of the Scientific Literature
Malashin, I. P., Martysyuk, D., Тынченко, В. С., Gantimurov, A., Nelyub, V., & Borodulin, A. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8030063 · Cited by 1
MIDAS: Load-Balanced Metadata-Outsourcing ID-Embedded Data Auditing Scheme for Mobile Devices
Tang, M., Shen, J., Wen, S., Zhu, T., & Xiang, Y. (2026) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2026.3670871
Self-citationSurvey of Agents Methodologies in the Financial Domain
Luqman, M., Gauba, H., Kumar, P., Desai, A. P., Yadav, A., Prajapati, R., & Yadav, P. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402262
“[40] has discussed some of the most common security vulnerabilities for AI agents:”
6G AI Security : from fundamentals to offensive and defensive landscape in 6G
Braeke, A., Deac, D., Nguyen, T. T., Gür, G., Pham, Q., Yapa, C., Naranjo, P. G. V., Mora, H. C., Moremada, C., & Liyanage, M. (2026) · Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) · doi:10.21256/zhaw-35849
Smart Contracts as Semantic Signalling for AI Agents on Blockchains
Pakana, F. (2026) · doi:10.1145/3793638.3793651
“Recent work on autonomous and generative agents highlights how large language models enable adaptive, context-aware decision making, raising new security concerns when such agents operate in open, adversarial environments [4, 6, 14, 15, 19].”
Interactive LLM Agent Review for Software Engineering Artifacts
Kim, D. (2026) · Journal of Computer Information Systems · doi:10.1080/08874417.2026.2650523 · Cited by 1
CPGHunter: LLM-guided semantic modeling for scalable vulnerability detection via taint analysis
Hou, A., Su, B., Niu, W., Hou, Q., Wu, H., & Zhang, X. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10842-2
Liu, Y., Shao, S., Guo, S., Wang, C., Yang, C., & Zang, Z. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3680460 · Cited by 1
LGA: lightweight design and privacy analysis of generative agents in social simulations
Guo, Y., Lu, X., Ma, W., Xiao, X., & Zhu, T. (2026) · International Journal of Information Security · doi:10.1007/s10207-026-01244-y
“Issues inherent to the LLM itself, such as prompt injection risks, privacy risks, misalignment, and excessive sycophancy [18], can emerge in the LLM Agent as a result of its integration [19].”
Self-citationAl-Hinai, Y. S., Tarhini, A., Al-Sharafi, M., & Aman, W. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-16775-0_11
Neuro-Symbolic AI Security Attack Assessment Computational Model
Maathuis, C. (2026) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa67686.2026.11467731
“In particular, AI agents confront multi-step input unpredictability, opaque internal execution chains, environmental variability, and untrusted tool interactions, each of these threats expanding attack surfaces that are difficult to detect and audit in real time [17].”
Securing Multi-Agent Networks Against Shadow AI: An Explainable and Quantum-Resilient Framework
Jillani, N., Khattak, A. M., Ullah, R., Asghar, M. Z., & Khalil, A. (2026) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa67686.2026.11468150
“[1] outlined governance-oriented challenges and risks but did not provide an implemented or experimentally validated system; [2] examined coordination mechanisms that improved task allocation but did not incorporate explicit trust or auditability features; [7] focused on AI-driven resource…”
InfluentialPrivacy-preserving for user-uploaded images and text in Vision–Language Models
Liu, Z., Chen, C., Yuan, S., Huang, W., Zhu, X., & Lv, P. (2026) · Computers & Security · doi:10.1016/j.cose.2026.104931
The Next Stage of Evolutionary Computation in the Era of Agentic Generative AI
Palk, M., Voß, S., & Glover, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-23607-4_19 · Cited by 1
“Safety guardrails, activity logs, monitoring systems, resource management, and human-in-the-loop oversight are, therefore, crucial to ensure alignment with goals and prevent unintended behaviors [17].”
Shao, X., Jiang, W., & Osman, K. N. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1821188 · Cited by 1
“Across a range of important investigative and analytical tasks, AI has increasingly been regarded as a “human partner” that can participate in discussions, offer suggestions, and generate feedback (e.g., Kim and Im, 2023; Deng et al., 2025).”
AI Agent Behavior Optimization and Evolution Strategy Based on Deep Genetic Algorithm
Wang, W., & He, J. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-95-6199-5_12
Blockchain-Based Cross-Domain Data Auditing Scheme for E-Commerce AI
Tang, M., Chen, W., & Chang, C. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7820-7_1
Xu, J., & Lu, Q. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-22236-7_33
Security Risks in Tool-Enabled AI Agents: A Systematic Analysis of Privileged Execution Environments
Goel, H. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00466 · Cited by 2
“Recent surveys cata-log security challenges in AI agents—unpredictable multi-step inputs, internal complexity, environment variability, and interactions with untrusted external entities [9].”
AN OVERVIEW OF LARGE LANGUAGE MODEL APPROACHES FOR AUTOMATED SOFTWARE VULNERABILITY DETECTION
Maksymovych, M. (2026) · Komp'ûternì sistemi ta merežì · doi:10.23939/csn2026.01.162
Mao, Y., Cui, T., Wang, J., & You, D. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00628-7
“These advancements indicate that agent security risks have evolved from single-point break-throughs to systemic threats, involving multiple attack vectors such as memory theft, privacy leakage, sustained penetration, and supply chain attacks (Deng et al. 2025).”
DPKC: A Privacy-Preserving Availability-Enhanced Clustering Scheme for Data Analysis in IoMT
Zhang, S. W., Li, Y., Luo, E., Wu, F., Wang, T., & Meng, W. (2026) · Big Data Mining and Analytics · doi:10.26599/bdma.2025.9020108
Security, privacy, and ethical risks in OpenClaw
Jin, Y., Zhang, Z., Lyu, Z., & Ni, J. (2026) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2026.05.006 · Cited by 2
“…new threats arising from agent autonomy, persistent state, and the capacity for irreversible actions, beyond known vulnerabilities inherited from the underlying LLM. Crucially, the research community has already distinguished between threats to the agent and threats through the agent [6, 5].”
SemantiGuard: Intent-Aware Malicious Code Detection for IoT Agent Systems
Zhao, S., Wu, Y., & Zhang, Y. (2026) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3812836.3814750
Optimizing Trust and Safety Regions for Text-to-Image Generation in High-Dimensional Manifold Spaces
Yang, X., Li, X., Wang, Y., & Liu, N. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3699945 · Cited by 1
SECURITY THREATS TO AI AGENTS: REVIEW ON THREATS, DEFENSE STRATEGIES AND THREAT MODELLING
Mathew, G. M., Shukla, K., & Modi, K. (2026) · Journal of Digital Security and Forensics · doi:10.29121/digisecforensics.v3.i1.2026.98
Agentenbasierte Fahrgastinformationssysteme für den ÖPNV
Palk, M., & Voß, S. (2026) · HMD Praxis der Wirtschaftsinformatik · doi:10.1365/s40702-026-01283-y
Identifying Malicious and Benign Prompts in AI Agent Evasion Attacks Using Machine Learning
Mahmud, E., Murshid, M. M., Barua, A., Nisa, N. T. B. N., Arnob, S. D., & Turja, M. T. (2026) · IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN) · doi:10.1109/qpain69676.2026.11546222
“Complementing this, a survey by Deng et al. [8] provided a comprehensive overview of the security challenges facing AI agents, categorizing threats based on intra-execution components and interaction patterns, and highlighting key knowledge gaps that make agents vulnerable.”
Who Guards the Agents? A Control Plane Architecture for Responsible Agentic AIPreprint
Colombo, M. C., Paris, D., Aspromonte, M., Ermellino, A., Mombelli, N., Portanti, M., Zoccoli, A., Donelli, N., Castelnovo, A., & Cosentini, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6955958
Liu, J., Zhang, S., Li, D., Deng, X., Peng, R., & Hui, L. (2026) · Ad Hoc Networks · doi:10.1016/j.adhoc.2026.104305
Xu, S., Ma, Q., Liu, H., & Yue, L. (2026) · International Conference on Artificial Intelligence and Industrial Technology Applications (AIITA) · doi:10.1109/aiita69518.2026.11567233
“This module is responsible for assessing the value of each memory unit in the agent's memory bank, calculating priority scores, and providing a basis for memory reorganization [6] .”
CBCL: Safe Self-Extending Agent Communication
O’Connor, H. (2026) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw72489.2026.00007
“Deng et al. [22] survey security threats to AI agents, organising them around prompt injection, tool-use risks, and adversarial multi-agent interactions.”
Alves, A. C. V., Martignago, G. M., Correa, M. R. D. S., Neto, M. P. P., Pillon, M. A., Miers, C. C., Correia, P. H. B., Hayashi, V. T., & Junior, M. A. S. (2026) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-032-23311-0_7
MA-ML: Multiagents and Multilayer Collaboration Architectures
Nof, S. Y. (2026) · Automation, collaboration, and e-services · doi:10.1007/978-3-032-14922-0_3
Safeguarding LLM Agents from Misalignment through Provenance Analysis
She, Y., Liang, Y., & Kang, E. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837535 · Cited by 8
“Therefore, increasing agent autonomy also substantially increases potential impact and risk [12, 22, 33, 44].”
Stromko, M., Bednár, P., Sopkovič, K., Pleva, M., & Banoci, V. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-30836-8_20
Analyzing Sample Selection Under Poisoning Membership Inference in Federated Learning
Wang, Y., Li, J., Jin, Y., Pu, R., Wang, L., & Wang, B. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11581533
“At the same time, it improves scalability and distributed efficiency, positioning FL as a key framework for privacy-preserving collaborative machine learning [3], [7], [9].”
Reaching the Right Shore: Securing Smartphone LLM Agents via Execution Verification
Zhang, B., Li, L., & Qiu, M. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2859-1_2 · Cited by 1
Debiased Multimodal Personality Understanding through Dual Causal Intervention
Zhu, Y., Han, Z., Ning, N., Wei, Y., Wang, Y., Feng, H., & Shao, Z. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809547
407Chapter 18 Explaining the Important Challenges in Agentic AI Systems
Gupta, V., & Wairagade, A. (2026) · De Gruyter eBooks · doi:10.1515/9783112223062-018
Agentic AI Unleashed: Power, Promise, and Ethical Frontiers
Wang, M., Wang, Y., & Ma, Q. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3761-6_3
A Privacy-Preserving On-Device Multi-Agent Architecture for AI PC (POMA) Workflow Automation
Begum, S., & Rosenzweig, M. (2026) · IEEE International Conference on AI and Data Analytics (ICAD) · doi:10.1109/icad69378.2026.11608651
Understanding Virtual Assistants: A Systematic Review
Montiel, C. S. A., Rodríguez-García, J. G., & Mendoza, S. (2026) · Information Systems Frontiers · doi:10.1007/s10796-026-10788-2
Agent-Native Applications: A New Application Paradigm
Luo, D., & Leon‐Garcia, A. (2026) · IEEE Access · doi:10.1109/access.2026.3720448
“For this paper, we use four primary architectural node types as a modeling abstraction for reasoning, action, information access, and human oversight : Agents are LLM-based components that reason or plan over task context and act toward objectives, including by invoking tools [25].”
Privacy-Preserving Agentic AI in Edge Networks With Content and Trace Protection
Peng, S., Jiang, Y., Liu, Y., Yu, X., & Zhu, L. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3720648 · Cited by 1
“Compared to the collaboration and low-cost deployment of agents, research on privacy protection in edge agent AI remains limited [62], [63].”
Yin, J., Xie, X., Guo, S., Chen, J., & Cui, J. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10080262
Lv, H., & Ding, Y. (2026) · The Computer Journal · doi:10.1093/comjnl/bxag081
Li, Z., & Tan, H. (2026) · International Conference on Artificial Intelligence and Electromechanical Automation (AIEA) · doi:10.1109/aiea70743.2026.11632621
“As a decentralized technology, public blockchain can be integrated into IoA to provide tamper-proof record-keeping without relying on any centralized entity, making cross-organizational interactions verifiable and reliable [8], [9].”
A Survey on Autonomous Compliance Enforcement using Agentic AI
Soudagar, S., Rajpurohit, V. S., Shahapurkar, A., & Sannakki, S. S. (2026) · International Conference on Sustainable Computing and Smart Systems (ICSCSS) · doi:10.1109/icscss69635.2026.11646093
“as the most comprehensive treatment [18], cataloguing the security challenges including prompt injection, goal hijacking, and unsafe tool use.”
InfluentialMa, Y., Zhang, L., & Wu, X. (2026) · Symmetry · doi:10.3390/sym18081386
“…these results suggest that prompt-based zero-shot studies should report not only a best prompt score but also a perturbation-based robustness statistic such as Mix mean ± standard deviation, consistent with the emphasis on input-space robustness in LLM and AI agent security evaluation [10,11].”
Assessing Risks and Privacy Challenges in the Artificial Intelligence Era
Taherdoost, H., Farhaoui, Y., Madanchian, M., Syed, M. B., Slimani, K., & Kumar, D. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-28097-8_3
Engineering Trustworthy AI: The VERITAS Framework for Responsible AI Research
Crockett, K. A., Mishra, K., Doan, T. V. K., & Obafemi-Ajayi, T. (2026) · IEEE International Conference on Digital Health (ICDH) · doi:10.1109/icdh72779.2026.00019
“We uphold AI safety [47] and security [48] by designing robust models resistant to adversarial attacks, monitoring for vulnerabilities, and establishing protocols to prevent harmful or unintended outcomes.”
Pallis, G. (2026) · Studies in computational intelligence · doi:10.1007/978-3-032-18585-3_1
Intelligent agents: architectures, learning paradigms, deployment models, and applications
Dritsas, E., & Τρίγκα, Μ. (2026) · Evolutionary Intelligence · doi:10.1007/s12065-026-01259-3 · Cited by 2
Understanding User Evaluation of Emerging AI Agents: Evidence from YouTube Comments on OpenClaw
Ren, F., & Yan, J. (2026) · Electronics · doi:10.3390/electronics15173881
Xu, Z., Liu, T., Zhang, H., Li, N., Dong, R., & Xiao, X. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7383642
Chen, S., Feng, Y., Cheng, Z., Su, W., Zhang, S., Deng, Z., & Zhu, T. (2026) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2026.104593
Self-citationDesign and Implementation of a Dify-Based Contract Review Agent
Guo, W., Fu, Y., & Jin, C. (2026) · International Conference on Computer Information and Big Data Applications (CIBDA) · doi:10.1145/3813822.3814200
“An agent [ 5 ] is a system that simulates intelligent human behavior, capable of perceiving its environment and generating adaptive plans in response to environmental changes.”
Li, Z., Wen, J., Liao, J., Jin, X., Miao, X., Pan, G., Zhong, W., Xie, S., & Kim, D. I. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3731984
Asgari, A., Panichella, A., Derakhshanfar, P., & Olsthoorn, M. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113108
Security Challenges and Protection Strategies for Industrial AI Agents: A Comprehensive Survey
Gong, Y., Li, X., & Shi, J. (2026) · doi:10.1145/3838457.3838551
AAE: A Real-Time Monitoring and Auditing System for AI Agents
Tao, S. (2026) · International Conference on Digital Society and Artificial Intelligence (DSAI) · doi:10.1145/3833541.3833637
“It is worth noting that these problems are not unique to OpenClaw, but are inherent to the Agent AI paradigm [8].”
Metadata
- Authors
- Deng, Z., Guo, Y., Han, C., Ma, W., Xiong, J., Wen, S., & Xiang, Y.
- Affiliations
- Swinburne University of Technology (AU)
- Tsinghua University (CN)
- Ant Group (China) (CN)
- Venue
- ACM Computing Surveys
- Published
- 2025-02-21
- Volume
- 57
- Issue
- 7
- Pages
- 1-36
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4407236106
- Cited by
- 210
- References
- 92
- Citations by year
- 2026: 127 · 2025: 82
- FWCI
- 83.83
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Physical Sciences › Computer Science › Computer Networks and Communications › Access Control and Trust
- Keywords
- AI agents, AI agent security, security threats, autonomous software agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.