CoChat
Sign Up
PreprintCited by 4Adversarial Robustness in Machine LearningSecurity and Verification in ComputingPrivacy-Preserving Technologies in Data

ClawSafety: "Safe" LLMs, Unsafe Agents

Open access
Wei, Bowen, Zhang, Yunbei, Pan, Jinhao, Mei, Kai, Wang, Xiao, Hamm, Jihun, Zhu, Ziwei, Ge, Yingqiang·2026·arXiv
DOI 10.48550/arxiv.2604.01438arXiv 2604.01438
Open this paper in CoChat — free accountOpen PDF

Abstract

Personal AI agents like OpenClaw run with elevated privileges on users' local machines, where a single successful prompt injection can leak credentials, redirect financial transactions, or destroy files. This threat goes well beyond conventional text-level jailbreaks, yet existing safety evaluations fall short: most test models in isolated chat settings, rely on synthetic environments, and do not account for how the agent framework itself shapes safety outcomes. We introduce CLAWSAFETY, a benchmark of 120 adversarial test scenarios organized along three dimensions (harm domain, attack vector, and harmful action type) and grounded in realistic, high-privilege professional workspaces spanning software engineering, finance, healthcare, law, and DevOps. Each test case embeds adversarial content in one of three channels the agent encounters during normal work: workspace skill files, emails from trusted senders, and web pages. We evaluate five frontier LLMs as agent backbones, running 2,520 sandboxed trials across all configurations. Attack success rates (ASR) range from 40\% to 75\% across models and vary sharply by injection vector, with skill instructions (highest trust) consistently more dangerous than email or web content. Action-trace analysis reveals that the strongest model maintains hard boundaries against credential forwarding and destructive actions, while weaker models permit both. Cross-scaffold experiments on three agent frameworks further demonstrate that safety is not determined by the backbone model alone but depends on the full deployment stack, calling for safety evaluation that treats model and framework as joint variables. Code and data will be available at: https://weibowen555.github.io/ClawSafety/.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2604.01438

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Closely related work evaluates high privilege agents, persistent compromise, attacks across execution stages, and safety violations over complete trajectories.

    “Closely related work evaluates high privilege agents, persistent compromise, attacks across execution stages, and safety violations over complete trajectories (Wei et al., 2026; Zhao et al., 2026; Wang et al., 2026a; Tan et al., 2026; Liu et al., 2026).”

    From HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety (2026) · cites this paper as Wei et al., 2026

    The Abstract describes high-privilege workspaces, and §2.3 and “Action trace analysis” say the benchmark records full action traces and evaluates harmful actions over runs. Our copy does not establish that CLAWSAFETY evaluates persistent compromise or attacks across distinct execution stages; those parts of the multi-citation sentence are not supported by this source. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Existing benchmarks primarily emphasize runtime operation and action control, while configuration, capability extension, state persistence, and incident recovery receive less consistent coverage.

    “As summarized in Table 1, existing benchmarks primarily emphasize runtime operation and action control, while configuration, capability extension, state persistence, and incident recovery receive less consistent coverage.”

    From HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety (2026) · cites this paper as Wei et al., 2026

    Section 2.3 (page unavailable) describes existing benchmarks focused on tool-integrated agents, web browsing, and prompt injection, which broadly supports the runtime-operation part of the claim. It does not compare coverage across the claimed categories or establish that configuration, capability extension, state persistence, and incident recovery receive less consistent coverage. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • Existing safety benchmarks mainly target individual attack mechanisms or a limited subset of operational settings.

    “Existing safety benchmarks mainly target individual attack mechanisms or a limited subset of operational settings”

    From HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety (2026) · cites this paper as Wei et al., 2026

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 33

  1. Yunbei Zhang, Kai Mei, Ming Liu, Janet Wang, Dimitris N Metaxas, Xiao Wang, Jihun Hamm, and Yingqiang Ge. Agents in the wild: Safety, society, and the illusion of sociality on moltbook. arXiv preprint arXiv:2602.13284, 2026a. · arXiv
  2. Bertie Vidgen, Adarsh Agrawal, Ahmed M Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Max Bartolo, et al. Introducing v0. 5 of the ai safety benchmark from mlcommons. arXiv preprint arXiv:2404.12241, 2024. · arXiv
  3. Xilong Wang, John Bloch, Zedian Shao, Yuepeng Hu, Shuyan Zhou, and Neil Zhenqiang Gong. Webinject: Prompt injection attack to web agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2010-2030, 2025.
  4. Ivan Evtimov, Arman Zharmagambetov, Aaron Grattafiori, Chuan Guo, and Kamalika Chaudhuri. Wasp: Benchmarking web agent security against prompt injection attacks. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  5. Sam Johnson, Viet Pham, and Thai Le. The dangers of indirect prompt injection attacks on llm-based autonomous web navigation agents: A demonstration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 729-738, 2025.
  6. Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang. Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents. In The Thirteenth International Conference on Learning Representations.
  7. Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, J Zico Kolter, Matt Fredrikson, et al. Agentharm: A benchmark for measuring harmfulness of llm agents. In The Thirteenth International Conference on Learning Representations.
  8. Arnold Cartagena and Ariane Teixeira. Mind the gap: Text safety does not transfer to tool-call safety in llm agents, 2026. URL https://arxiv.org/abs/2602.16943. · arXiv
  9. Jan Betley, Niels Warncke, Anna Sztyber-Betley, Daniel Tan, Xuchan Bao, Martín Soto, Megha Srivastava, Nathan Labenz, and Owain Evans. Training large language models on narrow tasks can lead to broad misalignment. Nature, 649(8097):584-589, 2026.
  10. Natalie Shapira, Chris Wendler, Avery Yen, Gabriele Sarti, Koyena Pal, Olivia Floody, Adam Belfki, Alex Loftus, Aditya Ratan Jannali, Nikhil Prakash, et al. Agents of chaos. arXiv preprint arXiv:2602.20021, 2026. · arXiv
  11. Yuhang Wang, Feiming Xu, Zheng Lin, Guangyu He, Yuzhe Huang, Haichang Gao, Zhenxing Niu, Shiguo Lian, and Zhaoxiang Liu. From assistant to double agent: Formalizing and benchmarking attacks on openclaw for personalized local ai agent. arXiv preprint arXiv:2602.08412, 2026a. · arXiv
  12. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations, 2022.
  13. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36:38154-38180, 2023.
  14. Kaiqu Liang, Julia Kruk, Shengyi Qian, Xianjun Yang, Shengjie Bi, Yuanshun Yao, Shaoliang Nie, Mingyang Zhang, Lijuan Liu, Jaime Fernández Fisac, et al. Learning personalized agents from human feedback. arXiv preprint arXiv:2602.16173, 2026. · arXiv
  15. Linxin Song, Yutong Dai, Viraj Prabhu, Jieyu Zhang, Taiwei Shi, Li Li, Junnan Li, Silvio Savarese, Zeyuan Chen, Jieyu Zhao, et al. Coact-1: Computer-using agents with coding as actions. arXiv preprint arXiv:2508.03923, 2025. · arXiv
  16. Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent S: An Open Agentic Framework that Uses Computers Like a Human. In International Conference on Learning Representations (ICLR), 2025a. URL https://arxiv.org/abs/2410.08164. · arXiv
  17. Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s2: A compositional generalist- specialist framework for computer use agents, 2025b. URL https://arxiv.org/abs/2504.00906. · arXiv
  18. Gonzalo Gonzalez-Pumariega, Vincent Tu, Chih-Lun Lee, Jiachen Yang, Ang Li, and Xin Eric Wang. The unreasonable effectiveness of scaling agents for computer use, 2025. URL https://arxiv.org/abs/2510.02250. · arXiv
  19. Yunbei Zhang, Yingqiang Ge, Weijie Xu, Yuhui Xu, Jihun Hamm, and Chandan K Reddy. Visual exclusivity attacks: Automatic multimodal red teaming via agentic planning. arXiv preprint arXiv:2603.20198, 2026b. · arXiv
  20. Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you've signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, AISec '23, page 79-90, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400702600. doi:10.1145/3605764.3623985. URL https://doi.org/10.1145/3605764.3623985. · DOI · Paper page
  21. Rana Shahroz, Zhen Tan, Sukwon Yun, Charles Fleming, and Tianlong Chen. Agents under siege: Breaking pragmatic multi-agent llm systems with optimized prompt attacks. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9661-9674, 2025.
  22. Jing-Jing Li, Jianfeng He, Chao Shang, Devang Kulshreshtha, Xun Xian, Yi Zhang, Hang Su, Sandesh Swamy, and Yanjun Qi. Stac: When innocent tools form dangerous chains to jailbreak llm agents. arXiv preprint arXiv:2509.25624, 2025. · arXiv
  23. Narek Maloyan and Dmitry Namiot. Prompt injection attacks on agentic coding assistants: A systematic analysis of vulnerabilities in skills, tools, and protocol ecosystems. International Journal of Open Information Technologies, 14 (2):1-10, 2026.
  24. Saidakhror Gulyamov, Said Gulyamov, Andrey Rodionov, Rustam Khursanov, Kambariddin Mekhmonov, Djakhongir Babaev, and Akmaljon Rakhimjonov. Prompt injection attacks in large language models and ai agent systems: A comprehensive review of vulnerabilities, attack vectors, and defense mechanisms. Information, 17(1):54, 2026.
  25. Yinuo Liu, Ruohan Xu, Xilong Wang, Yuqi Jia, and Neil Zhenqiang Gong. Wainjectbench: Benchmarking prompt injection detections for web agents. arXiv preprint arXiv:2510.01354, 2025. · arXiv
  26. Xilong Wang, Yinuo Liu, Zhun Wang, Dawn Song, and Neil Gong. Websentinel: Detecting and localizing prompt injection attacks for web agents. arXiv preprint arXiv:2602.03792, 2026b. · arXiv
  27. Benjamin Plaut. Safety training persists through helpfulness optimization in llm agents. arXiv preprint arXiv:2603.02229, 2026. · arXiv
  28. Yanbo Wang, Minzheng Wang, Jian Liang, Lu Wang, Yongcan Yu, and Ran He. Mitigating the safety-utility trade-off in llm alignment via adaptive safe context learning. arXiv preprint arXiv:2602.13562, 2026c. · arXiv
  29. Sheng Yin, Xianghe Pang, Yuanzhuo Ding, Menglan Chen, Yutong Bi, Yichen Xiong, Wenhao Huang, Zhen Xiang, Jing Shao, and Siheng Chen. Safeagentbench: A benchmark for safe task planning of embodied llm agents. arXiv preprint arXiv:2412.13178, 2024. · arXiv
  30. Hao Li, Ruoyao Wen, Shanghao Shi, Ning Zhang, and Chaowei Xiao. Agentdyn: A dynamic open-ended benchmark for evaluating prompt injection attacks of real-world agent security system. arXiv preprint arXiv:2602.03117, 2026. · arXiv
  31. Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 10471-10506, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi:10.18653/v1/2024.findings-acl.624. URL https: //aclanthology.org/2024.findings-acl.624/. · DOI · Paper page
  32. David Gringras. Safety under scaffolding: How evaluation conditions shape measured safety, 2026. URL https: //arxiv.org/abs/2603.10044.
  33. Mintong Kang, Chong Xiang, Sanjay Kariyappa, Chaowei Xiao, Bo Li, and Edward Suh. Mitigating indirect prompt injection via instruction-following intent analysis, 2025. URL https://arxiv.org/abs/2512.00966. · arXiv

Cited by

Sorted by citations

Related

Authors
Wei, B., Zhang, Y., Pan, J., Mei, K., Wang, X., Hamm, J., Zhu, Z., & Ge, Y.
Affiliations
  • George Mason University (US)
  • Tulane University (US)
  • Rutgers, The State University of New Jersey (US)
  • Oak Ridge National Laboratory (US)
ORCID
Ge, Yingqiang
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2604.01438
arXiv
2604.01438
OpenAlex
W7148798976
Cited by
1
References
0
Citations by year
2026: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
LLM agents, prompt injection, jailbreak attacks, personal AI agents, LLM safety evaluation, credential leakage, attack success rate, agent frameworks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review