CoChat
Sign Up
PreprintCited by 1Adversarial Robustness in Machine LearningSecurity and Verification in ComputingAccess Control and Trust

Architecture Matters for Multi-Agent Security

Open access
Hagag, Ben, Anderson, William L., de Witt, Christian Schroeder, Scheffler, Sarah·2026·arXiv
DOI 10.48550/arxiv.2604.23459arXiv 2604.23459
Open this paper in CoChat — free accountOpen PDF

Abstract

Multi-agent systems (MAS), composed of networks of two or more autonomous AI agents, have become increasingly popular in production deployments, yet introduce security risks that do not arise in single-agent settings. Even if individual agents exhibit robust security, architectural decisions governing their coordination can create attack surfaces that have not been systematically characterized. In this work, we present an empirical study of how MAS design decisions shape the tradeoff between task performance and attack resistance. Across three agentic environments (browser, desktop, and code) and 13 architectural configurations, we use stagewise evaluations that distinguish planning refusal, execution-stage interception, partial harmful execution, and successful attack completion to study three key design choices: (i) agent roles, which determine how authority and responsibility are allocated; (ii) communication topology, which shapes how and when agents interact; and (iii) memory, which determines the context and state visibility accessible to each agent. We find that multi-agent architectures are more vulnerable than standalone agents in the majority of configurations, with attack success rates varying by up to 3.8x at comparable or higher benign accuracy, and that no single design is universally safer. These results motivate the development of further evaluations that move beyond the security properties of a single agent.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2604.23459

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The architectural choices of the multi-agent system (communication topology, role decomposition, memory sharing) shape which threats materialize.

    “The architectural choices of the multi-agent system itself -communication topology, role decomposition, memory sharing -further shape which threats materialize Hagag et al. (2026).”

    From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Hagag et al. (2026)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 37

  1. Aguilera-Martínez, F. and Berzal, F. Llm security: Vulner- abilities, attacks, defenses, and countermeasures. arXiv preprint arXiv:2505.01177, 2025. · arXiv
  2. Amayuelas, A., Yang, X., Antoniades, A., Hua, W., Pan, L., and Wang, W. Y. MultiAgent Collaboration Attack: Investigating adversarial attacks in large language model collaborations via debate. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2024, pp. 6929-6948, Miami, Florida, USA, 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.findi ngs-emnlp.407. · DOI · Paper page
  3. Andriushchenko, M., Croce, F., and Flammarion, N. Jail- breaking leading safety-aligned llms with simple adaptive attacks. arXiv preprint arXiv:2404.02151, 2024. · arXiv
  4. Andriushchenko, M., Souly, A., Dziemian, M., Duenas, D., Lin, M., Wang, J., Hendrycks, D., Zou, A., Kolter, Z., Fredrikson, M., Winsor, E., Wynne, J., Gal, Y., and Davies, X. AgentHarm: A benchmark for measuring harmfulness of LLM agents. In The Thirteenth Interna- tional Conference on Learning Representations, 2025. URL https://openreview.net/forum?id= AC5n7xHuR1.
  5. Benjamin, V., Braca, E., Carter, I., Kanchwala, H., Kho- jasteh, N., Landow, C., Luo, Y., Ma, C., Magarelli, A., Mirin, R., Moyer, A., Simpson, K., Skawinski, A., and Heverin, T. Systematically analyzing prompt injection vulnerabilities in diverse llm architectures, 2024. URL https://arxiv.org/abs/2410.23308. · arXiv
  6. Cemri, M., Pan, M. Z., Yang, S., Agrawal, L. A., Chopra, B., Tiwari, R., Keutzer, K., Parameswaran, A., Klein, D., Ramchandran, K., Zaharia, M., Gonzalez, J. E., and Stoica, I. Why do multi-agent LLM systems fail? In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2025. URL https://arxiv.org/abs/2503.13657. Spot- light. · arXiv
  7. Chiang, J. Y. F., Lee, S., Huang, J.-B., Huang, F., and Chen, Y. Why are web ai agents more vulnerable than standalone llms? a security analysis. arXiv preprint arXiv:2502.20383, 2025. · arXiv
  8. Das, B. C., Amini, M. H., and Wu, Y. Security and privacy challenges of large language models: A survey. ACM Computing Surveys, 57(6):1-39, 2025.
  9. De Persis, C. and Tesi, P. Input-to-state stabilizing control under denial-of-service. IEEE Transactions on Automatic Control, 60(11):2930-2944, 2015.
  10. Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., and Tramèr, F. AgentDojo: A dynamic environment to evaluate prompt injection attacks and de- fenses for LLM agents. In Advances in Neural Informa- tion Processing Systems (NeurIPS), Datasets and Bench- marks Track, 2024. URL https://openreview.n et/forum?id=m1YYAQjO3w.
  11. Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., Kern, C., Shi, C., Terzis, A., and Tramèr, F. Defeating prompt injections by design. arXiv preprint arXiv:2503.18813, 2025. · arXiv
  12. Del Rosario, R. F., Krawiecka, K., and de Witt, C. S. Architecting resilient llm agents: A guide to secure plan-then-execute implementations. arXiv preprint arXiv:2509.08646, 2025. · arXiv
  13. Fendley, N., Staley, E. W., Carney, J., Redman, W., Chau, M., and Drenkow, N. A systematic review of poisoning attacks against large language models. arXiv preprint arXiv:2506.06518, 2025. · arXiv
  14. Gu, X., Zheng, X., Pang, T., Du, C., Liu, Q., Wang, Y., Jiang, J., and Lin, M. Agent smith: A single image can jail- break one million multimodal LLM agents exponentially fast. In Forty-first International Conference on Machine Learning, 2024. URL https://arxiv.org/abs/ 2402.08567.
  15. Guo, C., Liu, X., Xie, C., Zhou, A., Zeng, Y., Lin, Z., Song, D., and Li, B. Redcode: Risky code execution and generation benchmark for code agents. Advances in Neural Information Processing Systems, 37:106190- 106236, 2024.
  16. Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier, J., Khan, A., McLean, E., Smith, C., Barfuss, W., Foerster, J., Gavenčiak, T., et al. Multi-agent risks from advanced ai. arXiv preprint arXiv:2502.14143, 2025. · arXiv
  17. Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Wang, J., Zhang, C., Wang, Z., Yau, S. K. S., Lin, Z., Zhou, L., Ran, C., Xiao, L., Wu, C., and Schmidhuber, J. MetaGPT: Meta programming for a multi-agent collabo- rative framework. In The Twelfth International Confer- ence on Learning Representations, 2024. URL https: //openreview.net/forum?id=VtmBAGCN7o.
  18. Ju, T., Li, Y., et al. Flooding spread of manipulated knowl- edge in LLM-based multi-agent communities. arXiv preprint arXiv:2407.07791, 2024. · arXiv
  19. Jung, Y., Kim, M., Masoumzadeh, A., and Joshi, J. B. A survey of security issue in multi-agent systems. Artificial Intelligence Review, 37(3):239-260, 2012.
  20. Kumar, P., Lau, E., Vijayakumar, S., Trinh, T., Team, S. R., Chang, E., Robinson, V., Hendryx, S., Zhou, S., Fredrik- son, M., et al. Refusal-trained llms are easily jailbroken as browser agents. arXiv preprint arXiv:2410.13886, 2024. · arXiv
  21. Kuntz, T., Duzan, A., Zhao, H., Croce, F., Kolter, Z., Flam- marion, N., and Andriushchenko, M. Os-harm: A bench- mark for measuring safety of computer use agents. arXiv preprint arXiv:2506.14866, 2025. · arXiv
  22. Lamport, L., Shostak, R., and Pease, M. The byzantine generals problem. ACM Trans. Program. Lang. Syst., 4 (3):382-401, July 1982. ISSN 0164-0925. doi: 10.1145/ 357172.357176. URL https://doi.org/10.114 5/357172.357176. · DOI · Paper page
  23. LeBlanc, H. J., Zhang, H., Koutsoukos, X., and Sundaram, S. Resilient asymptotic consensus in robust networks. IEEE Journal on Selected Areas in Communications, 31 (4):766-781, 2013.
  24. Lee, D. and Tiwari, M. Prompt infection: Llm-to-llm prompt injection within multi-agent systems, 2024. URL https://arxiv.org/abs/2410.07283. · arXiv
  25. Liu, Y., Jia, Y., Geng, R., Jia, J., and Gong, N. Z. For- malizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 1831-1847, 2024.
  26. Mathew, Y., Matthews, O., McCarthy, R., Velja, J., Schroeder de Witt, C., Cope, D., and Schoots, N. Hidden in plain text: Emergence & mitigation of steganographic collusion in LLMs. arXiv preprint arXiv:2410.03768, 2024. NeurIPS 2024 Workshop on Safe Generative AI. · arXiv
  27. Motwani, S. R., Baranchuk, M., Strohmeier, M., Bolina, V., Torr, P. H. S., Hammond, L., and de Witt, C. S. Secret collusion among ai agents: Multi-agent deception via steganography, 2025. URL https://arxiv.org/ abs/2402.07510. · arXiv
  28. Nguyen, T., Ndebugre, M., and Arremsetty, D. Security considerations for multi-agent systems. arXiv preprint arXiv:2603.09002, 2026. · arXiv
  29. Peigné, P., Kniejski, M., Sondej, F., David, M., Hoelscher- Obermaier, J., Schroeder de Witt, C., and Kran, E. Multi- agent security tax: trading off security and collaboration capabilities in multi-agent systems. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelli- gence and Thirty-Seventh Conference on Innovative Ap- plications of Artificial Intelligence and Fifteenth Sympo- sium on Educational Advances in Artificial Intelligence, AAAI'25/IAAI'25/EAAI'25. AAAI Press, 2025. ISBN 978-1-57735-897-8. doi: 10.1609/aaai.v39i26.34970. URL https://doi.org/10.1609/aaai.v39 i26.34970. · DOI · Paper page
  30. Qi, S., Zou, Y., Li, P., Lin, Z., Cheng, X., and Yu, D. Amplified vulnerabilities: Structured jailbreak at- tacks on llm-based multi-agent debate. arXiv preprint arXiv:2504.16489, 2025. · arXiv
  31. Schroeder de Witt, C. Open challenges in multi-agent se- curity: Towards secure systems of interacting ai agents, 2025. URL https://arxiv.org/abs/2505.0 2077.
  32. Shang, Z. and Wei, W. Evolving security in llms: A study of jailbreak attacks and defenses. arXiv preprint arXiv:2504.02080, 2025. · arXiv
  33. Xie, T., Zhang, D., Chen, J., Li, X., Zhao, S., Cao, R., Hua, T. J., Cheng, Z., Shin, D., Lei, F., et al. Osworld: Bench- marking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems, 37:52040-52094, 2024.
  34. Yao, H., Lou, J., and Qin, Z. Poisonprompt: Backdoor attack on prompt-based large language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7745-7749. IEEE, 2024.
  35. Zhang, H., Huang, J., Mei, K., Yao, Y., Wang, Z., Zhan, C., Wang, H., and Zhang, Y. Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents, 2025. URL https://arxiv.org/ abs/2410.02644. · arXiv
  36. Zhu, K., Du, H., Hong, Z., Yang, X., Guo, S., Wang, Z., Wang, Z., Qian, C., Tang, X., Ji, H., and You, J. MultiA- gentBench: Evaluating the collaboration and competition of LLM agents. In Proceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pp. 8580-8622, Vienna, Austria, 2025a. Association for Computational Linguistics. URL https://aclanthology.org/2025.acl-lon g.421/.
  37. Zhu, Y., Zhang, C., Shi, X., Zhang, X., Yang, Y., and Luo, Y. Master: Multi-agent security through exploration of roles and topological structures-a comprehensive framework. arXiv preprint arXiv:2505.18572, 2025b. · arXiv

Cited by

Sorted by citations

Authors
Hagag, B., Anderson, W. L., de Witt, C. S., & Scheffler, S.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2604.23459
arXiv
2604.23459
OpenAlex
W7157124873
Cited by
0
References
1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
Carnegie Mellon University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
  • Social Sciences › Sociology and Political Science › Access Control and Trust
Keywords
multi-agent systems, agent architecture, attack resistance, agent roles, communication topology, agent memory, autonomous AI agents

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review