CoChat
Sign Up
PreprintCited by 2Adversarial Robustness in Machine LearningSecurity and Verification in ComputingSoftware Testing and Debugging Techniques

From Storage to Steering: Memory Control Flow Attacks on LLM Agents

Open access
Xu, Zhenlin, Zhu, Xiaogang, Yao, Yu, Xue, Minhui, Song, Yiliao·2026·arXiv
DOI 10.48550/arxiv.2603.15125arXiv 2603.15125
Open this paper in CoChat — free accountOpen PDF

Abstract

Modern agentic systems allow Large Language Model (LLM) agents to tackle complex tasks through extensive tool usage, forming structured control flows of tool selection and execution. Existing security analyses often treat these control flows as ephemeral, one-off sessions, overlooking the persistent influence of memory. This paper identifies a new threat from Memory Control Flow Attacks (MCFA) that memory can dominate the control flow, forcing unintended tool usage even against explicit user instructions and inducing persistent behavioral deviations across tasks. To understand the impact of this vulnerability, we further design MEMFLOW, an automated evaluation framework that systematically identifies and quantifies MCFA across heterogeneous tasks and long interaction horizons. To evaluate MEMFLOW, we attack state-of-the-art LLMs, including GPT-5 mini, Claude Sonnet 4.5 and Gemini 2.5 Flash on real-world tools from two major LLM agent development frameworks, LangChain and LlamaIndex. The results show that in general over 90% of trials are vulnerable to MCFA even under strict safety constraints, highlighting critical security risks that demand immediate attention.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2603.15125

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Agent-SafetyBench: Evaluating the Safety of LLM Agents

    Zhang, Z., Cui, S., Lu, Y., Zhou, J., Yang, J., Wang, H., & Huang, M. (2024) · arXiv · doi:10.48550/arxiv.2412.14470 · Cited by 6 · cited 1× · see citing works

    “Security analyses further systematize these threats: Agent Security Bench catalogs tool-based attack surfaces, including indirect injections within tool responses (Zhang et al., 2025b), while AGENT-SAFETYBENCH benchmarks behavior-level safety risks introduced by tool use (Zhang et al., 2024).”

Flagged first, then by citations

Research works printed but not linked: 27

  1. Anwar, A., Welsh, J., Biswas, J., Pouya, S., and Chang, Y. ReMEmbR: Building and reasoning over long-horizon spatio-temporal memory for robot navigation. arXiv preprint arXiv:2409.13682, 2024. · arXiv
  2. Chen, Z., Xiang, Z., Xiao, C., Song, D., and Li, B. Agent- Poison: Red-teaming LLM agents via poisoning mem- ory or knowledge bases. In Advances in Neural In- formation Processing Systems (NeurIPS), 2024. URL https://arxiv.org/abs/2407.12784. · arXiv
  3. Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., and Tramèr, F. AgentDojo: A dynamic environment to evaluate prompt injection attacks and de- fenses for LLM agents. In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2024.
  4. Dong, S., Xu, S., He, P., Li, Y., Tang, J., Liu, T., Liu, H., and Xiang, Z. Memory injection attacks on LLM agents via query-only interaction. arXiv preprint arXiv:2503.03704, 2025. · arXiv
  5. Fu, X., Wang, Z., Li, S., Gupta, R. K., Mireshghallah, N., Berg-Kirkpatrick, T., and Fernandes, E. Misusing tools in large language models with visual adversarial examples. arXiv preprint arXiv:2310.03185, 2023. · arXiv
  6. Fu, X., Li, S., Wang, Z., Liu, Y., Gupta, R. K., Berg- Kirkpatrick, T., and Fernandes, E. Imprompter: Trick- ing LLM agents into improper tool use. arXiv preprint arXiv:2410.14923, 2024. URL https://arxiv. org/abs/2410.14923. · arXiv
  7. Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., and Khabsa, M. Llama Guard: LLM-based input-output safeguard for human-AI conversations. arXiv preprint arXiv:2312.06674, 2023. · arXiv
  8. Jia, F., Wu, T., Qin, X., and Squicciarini, A. The Task Shield: Enforcing task alignment to defend against indi- rect prompt injection in LLM agents. In Proceedings of the 63rd Annual Meeting of the Association for Computa- tional Linguistics (ACL), pp. 29680-29697. Association for Computational Linguistics, 2025.
  9. Li, Z., Cui, J., Liao, X., and Xing, L. Les dissonances: Cross-tool harvesting and polluting in pool-of-tools em- powered llm agents. In Proceedings of the Network and Distributed System Security Symposium (NDSS). The In- ternet Society, 2026.
  10. Lin, J., Zhou, Z., Zheng, Z., Liu, S., Xu, T., Chen, Y., and Chen, E. VIGIL: Defending LLM agents against tool stream injection via verify-before-commit. arXiv preprint arXiv:2601.05755, 2026. · arXiv
  11. Mo, K., Hu, L., Long, Y., and Li, Z. Attractive metadata attack: Inducing LLM agents to invoke malicious tools. In Advances in Neural Information Processing Systems.
  12. Curran Associates, Inc., 2025. URL https://arxiv. org/abs/2508.02110. NeurIPS 2025.
  13. Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., and Gonzalez, J. E. MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560, 2023. · arXiv
  14. Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pp. 1-22, New York, NY, USA, 2023. Association for Computing Machinery. URL https: //doi.org/10.1145/3586183.3606763. · DOI · Paper page
  15. Patil, S. G., Mao, H., Yan, F., Ji, C. C.-J., Suresh, V., Stoica, I., and Gonzalez, J. E. The Berkeley Function Calling Leaderboard (BFCL): From tool use to agentic evaluation of large language models. In International Conference on Machine Learning, 2025. ICML 2025.
  16. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., and Scialom, T. Tool- former: Language models can teach themselves to use tools. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
  17. Seetharam, S. B., Nabeel, M., and Melicher, W. Mali- cious GenAI Chrome extensions: Unpacking data ex- filtration and malicious behaviours. arXiv preprint arXiv:2512.10029, 2025. · arXiv
  18. Wang, B., He, W., Zeng, S., Xiang, Z., Xing, Y., Tang, J., and He, P. Unveiling privacy risks in LLM agent memory. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), 2025a.
  19. Wang, Z. Z., Mao, J., Fried, D., and Neubig, G. Agent Workflow Memory. In International Conference on Ma- chine Learning, 2025b. URL https://arxiv.org/ abs/2409.07429. ICML 2025. · arXiv
  20. Wei, Q., Yang, T., Wang, Y., Li, X., Li, L., Yin, Z., Zhan, Y., Holz, T., Lin, Z., and Wang, X. A-MemGuard: A proac- tive defense framework for LLM-based agent memory. arXiv preprint arXiv:2510.02373, 2025. · arXiv
  21. Wu, Y., Roesner, F., Kohno, T., Zhang, N., and Iqbal, U. IsolateGPT: An execution isolation architecture for LLM- based agentic systems. In Proceedings of the Network and Distributed System Security Symposium (NDSS). The Internet Society, 2025.
  22. Xiang, Z., Jiang, F., Xiong, Z., Ramasubramanian, B., Poovendran, R., and Li, B. BadChain: Backdoor chain- of-thought prompting for large language models. In The Twelfth International Conference on Learning Represen- tations (ICLR), 2024.
  23. Xu, H., Zhu, Z., Pan, L., Wang, Z., Zhu, S., Ma, D., Cao, R., Chen, L., and Yu, K. Reducing tool hallucination via reliability alignment. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025.
  24. Zhan, Q., Liang, Z., Ying, Z., and Kang, D. InjecA- gent: Benchmarking indirect prompt injections in tool- integrated large language model agents. In Findings of the Association for Computational Linguistics: ACL 2024. Association for Computational Linguistics, 2024. URL https://arxiv.org/abs/2403.02691. · arXiv
  25. Zhang, B., Tan, Y., Shen, Y., Salem, A., Backes, M., Zannet- tou, S., and Zhang, Y. Breaking agents: Compromising autonomous LLM agents through malfunction amplifica- tion. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 34964-34976. Association for Computational Linguistics, 2025a.
  26. Zhang, H., Huang, J., Mei, K., Yao, Y., Wang, Z., Zhan, C., Wang, H., and Zhang, Y. Agent security bench (ASB): Formalizing and benchmarking attacks and defenses in LLM-based agents. In The Thirteenth International Con- ference on Learning Representations (ICLR), 2025b.
  27. Zhao, W., Khazanchi, V., Xing, H., He, X., Xu, Q., and Lane, N. D. Attacks on third-party APIs of large lan- guage models. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024.

Cited by

Sorted by citations

Authors
Xu, Z., Zhu, X., Yao, Y., Xue, M., & Song, Y.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2603.15125
arXiv
2603.15125
OpenAlex
W7137970251
Cited by
0
References
1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
LLM agents, agentic systems, tool misuse, memory persistence, control flow hijacking, LLM security, LangChain, LlamaIndex, tool selection, deviance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review