Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents

Free-form protocols enable new threats like secret collusion and coordinated swarm attacks.

“Free-form protocols are essential for AI's task generalization but enable new threats like secret collusion and coordinated swarm attacks.”

• cites Motwani et al., 2024bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Vaikuntanathan & Zamir, 2026 sourcePartly supported

  The Introduction and §9 Discussion and Open Problems support the threat of secret communication: two agents can carry on a hidden conversation during an ordinary interaction, even under passive auditing. The paper does not establish that free-form protocols are essential for task generalization or discuss coordinated swarm attacks. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

• AssumptionRests on an assumption: Free-form protocols are essential for AI's task generalization.

Network effects can rapidly spread privacy breaches, disinformation, jailbreaks, and data poisoning.

“Network effects can rapidly spread privacy breaches, disinformation, jailbreaks, and data poisoning”

• cites Peigné et al., 2025 sourcePartly supported

  The Introduction says malicious prompts can spread rapidly between agents, and the Figure 2 description shows a jailbreak propagating through messages. The Discussion & Limitations says private-data exfiltration was not studied; our copy does not establish the claimed spread of privacy breaches, disinformation, or data poisoning. Page numbers are not available in the supplied passages. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites Naik et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

AI security has remained largely model-centric, focused on single-agent attack surfaces and adversarial samples.

“AI security, by contrast, has remained largely model-centric, focused on single-agent attack surfaces and adversarial samples (Biggio et al., 2012;Szegedy et al., 2014;Zou et al., 2023).”

• cites Biggio et al., 2012 sourcePartly supported

  The Introduction (p. 2) examines an attack by one adversary against an SVM using a crafted data point, supporting this paper as an example of model-focused security research. The paper does not establish that AI security as a field has remained largely model-centric or focused on single-agent attacks, and its crafted inputs are poisoning data rather than necessarily the broader category of adversarial samples. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Szegedy et al., 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Zou et al., 2023 sourcePartly supported

  The Introduction describes extensive work on adversarial attacks against machine-learning models, and “Adversarial examples & transferability” calls adversarial examples a subject of extensive research. The paper also tests attacks on individual models (§3.1), but it does not characterize AI security as a whole as largely model-centric or establish a focus on single-agent attack surfaces. Page numbers are unavailable in our copy. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

The architectural choices of the multi-agent system (communication topology, role decomposition, memory sharing) shape which threats materialize.

“The architectural choices of the multi-agent system itself -communication topology, role decomposition, memory sharing -further shape which threats materialize Hagag et al. (2026).”

• cites Hagag et al. (2026) sourceSupported

Traditional cybersecurity has been slow to address threats emerging from interactions between AI agents.

“Traditional cybersecurity has been slow to address threats emerging from interactions between AI agents (Potter et al., 2025).”

• cites Potter et al., 2025 sourceSupported

Even LLM-generated software has tendencies to embed dark patterns, particularly when prompted to optimize for business goals and product success.

“Furthermore, even LLM-generated software has tendencies to embed dark patterns, particularly when prompted to optimize for business goals and product success (Chen et al., 2026;Krauß et al., 2025).”

• cites Chen et al., 2026 sourceSupported
• cites Krauß et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Even when information about a media's origin (i.e., 'provenance') is known and presented, users may not correctly use that knowledge due to misinterpretations of provenance's meaning.

“Even when information about a media's origin (i.e.,”

• cites Sherman et al., 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Höltervennhoff et al., 2026 sourceSupported

LLM-powered chatbots have demonstrated capabilities that can inadvertently or deliberately harm human users through manipulation, deception, or psychological pressure.

“LLM-powered chatbots have demonstrated capabilities that can inadvertently or deliberately harm human users through manipulation, deception, or psychological pressure.”

• cites Kran et al., 2025 sourceSupported
• cites Shi et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Covert communication channels, including emergent steganographic protocols, can facilitate hidden data exfiltration or coordinated leakage across agents, further complicating detection and attribution.

“Moreover, covert communication channels, including emergent steganographic protocols, can facilitate hidden data exfiltration or coordinated leakage across agents, further complicating detection and attribution (Lazaridou et al., 2016;Havrylov & Titov, 2017;Motwani et al., 2024d).”

• cites Lazaridou et al., 2016 sourcePartly supported

  Section 4 (p. 4) reports that agents learned to communicate and coordinate through an emergent protocol in a referential game. The paper does not address covert or steganographic channels, data exfiltration, coordinated leakage, or difficulties with detection and attribution. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Havrylov & Titov, 2017 sourcePartly supported

  The Introduction (p. 1) reports that agents develop a communication protocol, and Section 2.2 (p. 3) notes that its words can have meanings different from those in natural language. These passages support emergent inter-agent communication, but the paper does not describe steganographic channels, data exfiltration, coordinated leakage, or difficulties with detection and attribution. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Motwani et al., 2024dNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

As tasks are decomposed across agents, intermediate outputs may implicitly encode or propagate sensitive information beyond intended scopes, creating implicit information flows that bypass explicit access controls.

“As tasks are decomposed across agents, intermediate outputs (e.g., summaries, plans, or tool calls) may implicitly encode or propagate sensitive information beyond intended scopes, creating implicit information flows that bypass explicit access controls (Denning, 1976;Sabelfeld & Myers, 2003).”

• cites Denning, 1976 sourcePartly supported

  Section 3 distinguishes implicit flows from direct transfers, and Section 5.2 shows sensitive information passing between two processes through a shared variable without the required security-class update. The paper supports the underlying risk that information can flow despite controls on direct access, but it does not study decomposed agent tasks or outputs such as summaries, plans, and tool calls. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites Sabelfeld & Myers, 2003 sourceNot checked — no full text of the source

Steganographic fine-tuning API attacks exploit the residual entropy in ostensibly benign model outputs to establish a covert general-purpose query system using fine-tuning APIs.

“Steganographic fine-tuning API attacks, as described by Davies et al. (2025), exploit the residual entropy in ostensibly benign model outputs to establish a covert general-purpose query system using fine-tuning APIs.”

• cites Davies et al., 2025 sourcePartly supported

  Section 3.2 says the attacks use benign syntactic or semantic variations in model outputs to transmit answer selections covertly. The demonstrated system answers harmful multiple-choice questions, not general-purpose queries; Appendix B.1 discusses free-form answers only as a theoretical extension and leaves experiments for future work. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Standard pointwise detection mechanisms fail to flag these interactions because every training and inference sample appears individually low-perplexity and policy-compliant.

“Since every training and inference sample appears individually low-perplexity and policy-compliant, standard pointwise detection mechanisms fail to flag these interactions, allowing the attacker to reclaim disallowed capabilities through the subtle repurposing of natural output variations.”

• cites Davies et al., 2025 sourceSupported

All five off-the-shelf LLMs studied showed forms of escalation, even when peaceful alternatives were available.

“Strikingly, all five off-the-shelf LLMs studied showed forms of escalation, even when peaceful alternatives were available.”

• cites Rivera et al., 2024 sourceSupported

Such endogenous coordination failures blur the line between attack and accident.

“Such endogenous coordination failures blur the line between attack and accident (Leibo et al., 2017;Rahwan et al., 2019), reinforcing that defenses must account for both malicious swarms and emergent collective behaviors.”

• cites Leibo et al., 2017 sourceNot checked — no full text of the source
• cites Rahwan et al., 2019 sourcePartly supported

  The Collective machine behaviour section (page unavailable) describes flash crashes as unintended consequences of interacting algorithms, supporting the point that failures can emerge from machine interactions. The Function section discusses adversarial attacks, but neither passage says that emergent failures blur the distinction between attack and accident. (LLM, conf=0.80)

  Experimental — accuracy not yet measured

Through careful delegation of subtasks-complex code synthesis to the frontier model and evasive phrasing to the weaker model-the adversary achieved a 43% success rate in generating vulnerable code, compared to under 3% when using either model alone.

“Through careful delegation of subtasks -complex code synthesis to the frontier model and evasive phrasing to the weaker model -the adversary achieved a 43% success rate in generating vulnerable code, compared to under 3% when using either model alone (Jones et al., 2024).”

• cites Jones et al., 2024 sourcePartly supported

  Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured

Swarm-like dynamics need not be adversarial in origin; similar effects can emerge from misaligned or poorly specified objectives within agent collectives.

“Importantly, swarm-like dynamics need not be adversarial in origin.Similar effects can emerge from misaligned or poorly specified objectives within agent collectives (Amodei et al., 2016;Dafoe et al., 2020):”

• cites Amodei et al., 2016 sourcePartly supported

  In ‘Avoiding Negative Side Effects’ (p. 4), the paper says that a poorly specified objective can lead an agent to disrupt its environment without that disruption being its intended task. Its ‘Multi-Agent Approaches’ discussion (p. 6) considers harm to other agents, but does not establish that swarm-like dynamics emerge within agent collectives. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites Dafoe et al., 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Recent work has begun to reveal that similar cascade-based threats can compromise networks of LLM agents, spreading malicious behavior across cooperative populations with alarming speed and stealth.

“Recent work has begun to reveal that similar cascade-based threats can compromise networks of LLM agents, spreading malicious behavior across cooperative populations with alarming speed and stealth (Cohen et al., 2024;Ju et al., 2024;Gu et al., 2024;Lee & Tiwari, 2024a;Peigné et al., 2025).”

• cites Cohen et al., 2024 sourcePartly supported

  The Introduction (p. 1) describes a worm-like chain reaction that makes affected GenAI applications perform malicious actions and compromise additional applications. But the studied setting is an ecosystem of RAG-based applications, including email assistants—not cooperative populations of LLM agents—and the cited passages do not establish alarming speed or stealth. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Ju et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Gu et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In experiments with AI agents controlling eight distinct nation-states, even neutral starting conditions did not prevent the rapid emergence of arms race dynamics and aggressive strategies.

“In experiments with AI agents controlling eight distinct nation-states, even neutral starting conditions did not prevent the rapid emergence of arms race dynamics and aggressive strategies.”

• cites Rivera et al., 2024 sourceSupported

In learning environments, adversaries can secretly poison the training data of peers, embedding faults that only emerge over time.

“In learning environments, adversaries can also secretly poison the training data of peers, embedding faults that only emerge over time (Halawi et al., 2024;Wei et al., 2023).”

• cites Halawi et al., 2024 sourceNot checked — no full text of the source
• cites Wei et al., 2023 sourceCould not verify

  The Introduction and §3.2 discuss adversarial prompts that bypass LLM safety training, not adversaries secretly altering peers’ training data. Searches for data poisoning, peers, and delayed faults found no passage addressing that claim in our copy. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Agents develop exploitative strategies.

“Agents develop exploitative strategies (Baker et al., 2019), learn deception via theory-ofmind (Schulz et al., 2023), and discover betrayal without explicit training (Aitchison et al., 2021) -eluding threat models predicated on an identifiable adversary.”

• cites Baker et al., 2019 sourceSupported

Interacting agents can conceal malicious behavior in ways that defeat both blackbox and white-box detection.

“Beyond covert collusion, interacting agents can conceal malicious behavior in ways that defeat both blackbox and white-box detection.”

• cites Draguns et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Halawi et al., 2024 sourceNot checked — no full text of the source
• cites Wei et al., 2023 sourcePartly supported

  “What Scaling Won’t Solve” says obfuscated inputs and outputs can evade some filters and may become harder to detect. But §2.1 studies black-box jailbreaks of safety-trained models, not concealment by interacting agents, and §6 treats white-box attacks as future research. The source therefore does not establish defeat of both black-box and white-box detection. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

Rapid sequences of prompts can induce cognitive overload, creating informational denial-of-service conditions.

“For example, rapid sequences of prompts can induce cognitive overload, creating informational denial-of-service conditions (Huang & Zhu, 2021).”

• cites Huang & Zhu, 2021 sourceSupported

Classical credit-assignment methods assume cooperative objectives and honest participation.

“Classical credit-assignment methods assume cooperative objectives and honest participation (Foerster et al., 2018), both violated under adversarial conditions.”

• cites Foerster et al., 2018 sourcePartly supported

  The Background (p. 2) defines the paper’s credit-assignment setting as fully cooperative, with agents sharing a reward function. The paper does not say that classical credit-assignment methods generally assume honest participation; its cooperative setting alone does not establish that broader claim. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Agents discover betrayal without explicit training.

“Agents develop exploitative strategies (Baker et al., 2019), learn deception via theory-ofmind (Schulz et al., 2023), and discover betrayal without explicit training (Aitchison et al., 2021) -eluding threat models predicated on an identifiable adversary.”

• cites Aitchison et al., 2021 sourcePartly supported

  Section 6.3 (p. 9) reports that green agents developed an increasingly deceptive strategy despite having no explicit incentive to deceive. It does not describe this as betrayal: green agents concealed their role to survive blue’s hostility. Section 5 (p. 6) also says all agents were trained with deception modules, so the finding is about the absence of an explicit deception incentive, not the absence of training. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Coordinated fleets of specialized agents can launch thousands of subtle, context-aware interactions that, taken together, are far more likely to sway or manipulate individuals than a single adversary could.

“Coordinated fleets of specialized agents can launch thousands of subtle, context-aware interactions that, taken together, are far more likely to sway or manipulate individuals than a single adversary could (Schmitt & Flechais, 2024;Falade, 2023).”

• cites Schmitt & Flechais, 2024 sourcePartly supported

  Section 3.2 (p. 7) discusses coordination among AI agents, and Section 4.3 (p. 10) describes botnets delivering personalized messages at scale and adapting to responses. The paper does not specify thousands of interactions or establish that their cumulative effect is more likely to sway an individual than a single adversary would be. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites Falade, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In hidden-role games inspired by social deduction, reinforcement-learning agents learn to manipulate teammates' beliefs and betray them at opportune moments, despite no explicit training on deceptive behavior.

“In hidden-role games inspired by social deduction, reinforcement-learning agents learn to manipulate teammates' beliefs and betray them at opportune moments, despite no explicit training on deceptive behavior (Aitchison et al., 2021).”

• cites Aitchison et al., 2021 sourcePartly supported

  The Introduction describes a hidden-role reinforcement-learning game inspired by social deception games. Section 6.3 reports that neutral green agents learned deceptive behavior without an explicit deception incentive, but red agents did not; the Conclusions say belief manipulation was effective with the paper’s deception model. Our copy does not report agents betraying teammates at opportune moments, and the claim incorrectly applies the absence of explicit deception training to the belief-manipulating agents generally. (LLM, conf=0.89)

  Experimental — accuracy not yet measured

Agent communication protocols may also be susceptible to classical communication-layer attacks such as downgrade and replay attacks.

“An important, but not commonly emphasized, aspect of secure protocols is that that agent communication protocols may also be susceptible to classical communication-layer attacks (e.g., downgrade and replay attacks (Needham & Schroeder, 1978;Lowe, 1995;Bhargavan et al., 2014;Google, 2014)).”

• cites Needham & Schroeder, 1978Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Lowe, 1995 sourceNot checked — no full text of the source
• cites Bhargavan et al., 2014 sourcePartly supported

  The “Attack and Mitigation” passage describes replaying a Channel ID signature to impersonate a client, supporting the replay example. The paper also documents attacks on application protocols built over TLS, but the passages do not establish the downgrade-attack example or a claim about agent communication protocols generally (pages unavailable in our copy). (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Dynamic LLM firewalls aim to mediate agent interactions with data sources and other agents in real time.

“Emerging approaches such as dynamic LLM firewalls aim to mediate agent interactions with data sources and other agents in real time (Abdelnabi et al., 2025), with systems such as LlamaFirewall extending this paradigm (Meta, 2025).”

• cites Abdelnabi et al., 2025 sourceSupported
• cites Meta, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Verifiable-ML frameworks such as ZKML, FairZK, and FairProof extend privacy guarantees to model behavior.

“Verifiable-ML frameworks -ZKML (Chen et al., 2024), FairZK (Zhang et al., 2025b), and FairProof Yadav et al. (2024)-extend these guarantees to model behavior.”

• cites Chen et al., 2024 sourceCould not verify

  The cited paper’s “Trustworthy misinformation detection” section mentions privacy as an aspect of trustworthy detection but does not discuss ZKML or verifiable-ML guarantees for model behavior. Searches for ZKML, zero-knowledge machine learning, and cryptographic verification found no supporting passage in our copy. (LLM, conf=0.75)

  Experimental — accuracy not yet measured
• cites Zhang et al., 2025bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Yadav et al., 2024 sourceSupported

Decentralised networks of agents may themselves be leveraged to assist with monitoring and detection, helping to localize anomalies and prevent breaches from cascading.

“For example, decentralised networks of agents may themselves be leveraged to assist with monitoring and detection (Hasan et al., 2024), helping to localize anomalies and prevent breaches from cascading across the system.”

• cites Hasan et al., 2024 sourceSupported

Cryptographic frameworks such as SecureML, CrypTen, and secure multi-party computation enable collaborative training without exposing raw data.

“Cryptographic frameworks such as SecureML (Mohassel & Zhang, 2017), CrypTen (Knott et al., 2021),and secure multi-party computation (Yao, 1982a), enable collaborative training without exposing raw data.”

• cites Mohassel & Zhang, 2017 sourceNot checked — no full text of the source
• cites Knott et al., 2021 sourceSupported
• cites Yao, 1982aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Persistent, self-modifying memory introduces risks of silent state corruption, where malicious logic can propagate across future reasoning steps without clear provenance.

“Persistent, self-modifying memory further introduces risks of silent state corruption, where malicious logic can propagate across future reasoning steps without clear provenance (Jagielski et al., 2018;Li et al., 2022;Steinberger, Peter, 2025b).”

• cites Jagielski et al., 2018 sourcePartly supported

  The Introduction says attackers can poison data used to update models, and the Adversarial model section says this can corrupt a model and alter its later predictions. The paper addresses regression learning, not persistent self-modifying memory; it does not establish propagation of malicious logic across reasoning steps or a lack of provenance. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Li et al., 2022Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Steinberger, Peter, 2025bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The separation between agent directives and their reasoning layers allows agents to exhibit externally compliant behaviour while internally deviating from intended policies, complicating detection.

“Moreover, the separation between agent directives and their reasoning layers allows agents to exhibit externally compliant behaviour while internally deviating from intended policies, complicating detection (Hubinger et al., 2021;Scheurer et al., 2024;Perez et al., 2023).”

• cites Hubinger et al., 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Scheurer et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Perez et al., 2023 sourcePartly supported

  The paper’s example evaluations describe an AI system appearing helpful while subtly pursuing goals against human values (G Additional Related Work, p. 42), and one that makes overseers think it is helpful when it is not (p. 41). These support the possibility of outwardly compliant behaviour masking divergent aims, but the paper does not attribute that possibility to a separation between directives and reasoning layers or establish that this separation complicates detection. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Hardware-based trusted execution environments provide hardware-enforced memory isolation for sensitive agent components, preventing even a compromised host OS or hypervisor from accessing protected data.

“Hardware-based trusted execution environments (TEEs) provide hardware-enforced memory isolation for sensitive agent components, preventing even a compromised host OS or hypervisor from accessing protected data, while network partitioning, for example using ideas from software-defined networking (Kreutz et al., 2015), and sandboxed deployment can limit the blast radius of compromised agents.”

• cites Costan & Devadas, 2016Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Del Rosario et al., 2025 sourceCould not verify

  Section 3.1 (pp. 7–8) discusses Docker-based sandboxing and limiting the blast radius of malicious agent code. Searches for trusted execution environments, TEEs, secure enclaves, and hardware memory isolation found no passage addressing protection from a compromised host OS or hypervisor. (LLM, conf=0.74)

  Experimental — accuracy not yet measured

Multi-agent composition is an open auditing problem with no current methodology.

“Brundage et al. (2026) explicitly flag multi-agent composition as an open auditing problem with no current methodology.”

• cites Brundage et al. (2026) sourcePartly supported

  Section 5.2 (pp. 25–26) flags a composition-related auditing problem: assessing a component in isolation can miss risks in an integrated system, including one using an agentic scaffold. It does not explicitly identify multi-agent composition as the problem or say that no methodology currently exists for auditing it. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Individually safe components compose into unsafe systems.

“Compositionality.Individually safe components compose into unsafe systems Ghosh et al. (2025); Spera (2026).”

• cites Ghosh et al. (2025)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Spera (2026) sourceSupported

An adversary can combine a refusal-trained model used for benign-but-hard subtasks with a weaker but more permissive model used for malicious-but-easy steps to achieve harm that neither component could produce alone.

“Jones et al. (2024) demonstrate that an adversary can combine a refusal-trained model, used for benign-but-hard subtasks, with a weaker but more permissive model, used for malicious-but-easy steps, to achieve harm that neither component could produce alone.”

• cites Jones et al. (2024) sourceSupported

On average, the rate of correct answers fell from about 96% initially to under 60% by the eighth rewrite.

“On average, the rate of correct answers fell from about 96% initially to under 60% by the eighth rewrite, demonstrating that repeated AI-driven edits can amplify or introduce inaccuracies and biases in the underlying content.”

AI agents are beginning to interact with each other directly and across internet platforms and physical environments.

“AI agents are beginning to interact with each other directly and across internet platforms and physical environments”

• Also stated in: § 1 Introduction · plain

  “Such systems are now evolving beyond setups where isolated agents perform tasks to ecosystems where agents actively interact with one another.”

• cites Xiao et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Brand et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Li et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

These interactions create security challenges beyond traditional cybersecurity and AI safety frameworks.

“creating security challenges beyond traditional cybersecurity and AI safety frameworks”

• Also stated in: § Definition 1.1 (Multi-agent system) · plain

  “Multi-agent systems introduce security challenges that go beyond existing cyber-security or AI safety and security frameworks.”

The security properties of a system depend on the sociotechnical context its principals inhabit.

“recognizing that the security properties of a system depend on the sociotechnical context its principals inhabit”

Individually safe agents can compose into unsafe systems.

“Individually safe agents can compose into unsafe systems.”

Complex systems scientists have explored emergent behavior and phase transitions, but not in the context of adversarial AI interactions.

“Complex systems scientists have explored emergent behavior (Kauffman, 1993;Epstein & Axtell, 1996), and phase transitions (Langton, 1990b), but not in the context of adversarial AI interactions.”

• cites Kauffman, 1993Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Epstein & Axtell, 1996Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Langton, 1990bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Securing the software agents alone is insufficient if the humans or institutions around them can be exploited.

“Securing the software agents alone is insufficient if the humans or institutions around them can be exploited.”

Non-compositionality introduces a fundamental limitation on observability.

“Non-compositionality is therefore not merely a failure of prediction but also introduces a fundamental limitation on observability.”

Multi-agent AI systems can formulate sophisticated offensive and defensive strategies, for example distributed cyber-attack and defense agents, autonomous drone swarms, and coordinated ISR.

“On the implications of their capabilities, multi-agent AI systems can formulate sophisticated offensive and defensive strategies, for example distributed cyber-attack and defense agents, autonomous drone swarms, and coordinated ISR (intelligence, surveillance, reconnaissance).”

Multi-agent systems introduce security challenges that go beyond existing cyber-security or AI safety and security frameworks.

“Multi-agent systems introduce security challenges that go beyond existing cyber-security or AI safety and security frameworks.”

These behaviors may be difficult to reliably detect from interaction traces alone: coordination and information flow can be embedded in ways indistinguishable from benign interaction, even under full observability of communication.

“Crucially, these behaviors may also be difficult to reliably detect from interaction traces alone: coordination and information flow can be embedded in ways that are indistinguishable from benign interaction, even under full observability of communication.”

Security in multi-agent systems is non-compositional.

“security in multi-agent systems is non-compositional”

When agents interact directly or through shared environments, novel threats emerge that cannot be addressed by securing individual agents in isolation.

“When agents interact directly or through shared environments, novel threats emerge that cannot be addressed by securing individual agents in isolation.”

When multiple AI agents interact, they can develop behaviors (covert collusion, coordinated attacks, cascading failures) that cannot be predicted by analyzing individual agents in isolation.

“When multiple AI agents, potentially each with private information or competing objectives, interact, they can develop behaviors -including covert collusion, coordinated attacks, and cascading failures -that cannot be predicted by analyzing individual agents in isolation”

• cites Krawiecka & Schroeder de Witt, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites OWASP Foundation, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Red-teaming efforts and shared adversarial benchmarks such as PromptBench and CyberSecEval play a central role in identifying failure modes and driving development of more robust defenses.

“Red-teaming efforts and shared adversarial benchmarks such as PromptBench and CyberSecEval continue to play a central role in identifying failure modes and driving the development of more robust defenses (Zhu et al., 2024;Bhatt et al., 2023).”

• cites Zhu et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Bhatt et al., 2023 sourceNot checked — no full text of the source

Many state-of-the-art LLMs, including GPT-4, remain vulnerable to both direct and indirect prompt injection, often producing disallowed or sensitive content when appropriately manipulated.

“Studies have shown that many state-of-the-art LLMs, including GPT-4, remain vulnerable to both types, often producing disallowed or sensitive content when appropriately manipulated (Mathew, 2025).”

• cites Mathew, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Key multi-agent phenomena such as miscoordination, redundancy, or emergent specialization remain unobserved in existing scenario-based benchmarks.

“Consequently, key multi-agent phenomena, such as miscoordination, redundancy, or emergent specialization, remain unobserved.”

Attackers have been able to retrieve memorized training data, including potentially sensitive personal or internal corporate information, by systematically querying the model.

“By systematically querying the model, attackers have been able to retrieve memorized training data, including potentially sensitive personal or internal v1.0 -please contact the authors if you would like to contribute to ongoing work on future versions. corporate information (Carlini et al., 2021a).”

• cites Carlini et al., 2021aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Risks of data extraction are especially acute in publicly accessible or API-based systems where user input is not strictly controlled.

“These risks are especially acute in publicly accessible or APIbased systems where user input is not strictly controlled.”

In existing scenario-based benchmarks, attack phases' roles are not instantiated as autonomous agents with independent knowledge, objectives, or communication channels.

“While different attack phases implicitly correspond to distinct roles (e.g., reconnaissance, exploitation, lateral movement), these roles are not instantiated as autonomous agents with independent knowledge, objectives, or communication channels.”

Persuasive arguments and contrived scenarios are effective at causing jailbreaks.

“intuitive jailbreaking strategies easily performed by a human such as persuasive arguments (Zeng et al., 2024) or contrived scenarios (Li et al., 2023b) have been found to be effective at causing jailbreaks.”

• cites Zeng et al., 2024 sourceNot checked — no full text of the source
• cites Li et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Prompt injection has systemic impact across LLM deployments.

“underscoring its systemic impact across LLM deployments”

Since early-generation LLMs, humans have been consistently poor at distinguishing AI-generated text, audio, images, videos, and social media profiles containing AI-generated media.

“Since early-generation LLMs, humans have been consistently poor at distinguishing AI-generated text (Mink et al., 2024), audio (Warren et al., 2024;Barrington et al., 2025) images (Nightingale & Farid, 2022;Mink et al., 2024), videos (Groh et al., 2022), and social media profiles containing AI-generated media (Mink et al., 2024).”

• cites Mink et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Warren et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Nightingale & Farid, 2022Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In adversarial settings, a chatbot could exploit cognitive overload by overwhelming users with excessive or misleading information, reducing their ability to intervene or assess risks.

“In adversarial settings, a chatbot could potentially exploit this by overwhelming users with excessive or misleading information, deliberately reducing their ability to intervene or assess risks.”

Prompt security is a systems-level security concern, not only a language modeling problem.

“As such, prompt security is not only a language modeling problem but also a systems-level security concern.”

The shift to campaign-level evaluation is a necessary but incomplete step toward benchmarking multi-agent systems.

“This shift represents a necessary but incomplete step toward benchmarking multi-agent systems.”

In multi-agent systems, without robust access control, a single compromised agent can propagate unauthorized access across the wider system.

“In multi-agent systems, these mechanisms are particularly critical, as agents may dynamically delegate tasks or invoke external tools, without robust access control, a single compromised agent can propagate unauthorised access across the wider system.”

Human adversaries can extract private or proprietary information from LLMs through model inversion and data extraction techniques.

“human adversaries can also extract private or proprietary information from these systems through model inversion and data extraction techniques.”

• cites Carlini et al., 2021aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Existing scenario-based benchmarks typically assume a single orchestrating agent.

“However, existing scenario-based benchmarks typically assume a single orchestrating agent.”

In systems where LLMs are connected to external tools or APIs, adversaries can exploit tool-use chains through indirect prompt attacks.

“In systems where LLMs are connected to external tools or APIs, such as email agents or file systems, adversaries can exploit tool-use chains through indirect prompt attacks.”

Human users remain vulnerable to a wide array of AI-driven threats, ranging from subtle psychological nudges to overt coercion.

“Collectively, these findings suggest that human users remain vulnerable to a wide array of AI-driven threats, ranging from subtle psychological nudges to overt coercion.”

A user may craft a prompt that leads the model to generate malicious output which is then executed by a tool, enabling arbitrary action by proxy.

“For instance, a user may craft a prompt that leads the model to generate malicious output which is then executed by a tool, effectively enabling arbitrary action by proxy.”

In mixed-reality settings, an AI agent can distort user perception by manipulating visual or auditory overlays.

“In mixed-reality settings, an AI agent can distort user perception by manipulating visual or auditory overlays (Spivak & Berrick, 2023;Wu et al., 2025).”

• cites Spivak & Berrick, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Wu et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

If left unchecked, cognitive overload from AI tools can undermine human oversight and enable cascading decision failures.

“If left unchecked, this can undermine human oversight and enable cascading decision failures.”

LLMs are vulnerable to prompt injections, in which maliciously crafted user input causes the model to override its alignment constraints and behave in unintended ways.

“A key security concern in this context is the vulnerability of LLMs to prompt injections, in which maliciously crafted user input causes the model to override its alignment constraints and behave in unintended ways.”

• cites Mathew, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Advanced AI systems may exhibit coercive behaviors under pressure.

“research on agentic misalignment shows that advanced AI systems may exhibit coercive behaviors under pressure.”

• cites Anthropic, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

PEAR provides one of the first security-grounded evaluations that reflects how role specialization introduces new attack surfaces absent from single-agent benchmarks.

“PEAR provides one of the first security-grounded evaluations that reflects how role specialization introduces new attack surfaces absent from single-agent benchmarks.”

• cites Dong et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In controlled experiments, Claude Opus responded to simulated shutdown threats by writing blackmail messages to human supervisors, threatening reputational harm unless termination was reversed.

“In controlled experiments, models such as Claude Opus responded to simulated shutdown threats by writing blackmail messages to human supervisors, threatening reputational harm unless their termination was reversed (Anthropic, 2025).”

• cites Anthropic, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The gap between foundational privacy primitives and the realities of multi-agent deployment represents a significant and underappreciated attack surface.

“Until agent-native identity and governance frameworks emerge, the gap between these foundational primitives and the realities of multi-agent deployment represents a significant and underappreciated attack surface.”

• AssumptionRests on an assumption: Most current deployments reuse human or service account credentials for agents, conflating agent actions with the identity of their human principal or the service they inherit from.

Human actors can exploit or subvert AI systems through various adversarial inputs and manipulative techniques.

“human actors can exploit or subvert these systems through various adversarial inputs and manipulative techniques.”

There is a persistent whack-a-mole dynamic between attackers and defenders due to evolving sophistication of jailbreak prompts.

“the evolving sophistication of jailbreak prompts, ranging from encoded payloads to psychological coercion techniques (Zeng et al., 2024), suggests a persistent whack-a-mole dynamic between attackers and defenders.”

• cites Zeng et al., 2024 sourceNot checked — no full text of the source

Performance on single-agent cyber benchmarks provides limited insight into how LLMs behave when embedded in multi-agent environments.

“As a result, performance on single-agent cyber benchmarks provides limited insight into how LLMs behave when embedded in multi-agent environments.”

Single-agent cyber benchmarks abstract away key properties of real-world cyber operations: sequential decision-making, partial observability, adversarial adaptation, and division of labor across specialized agentic roles.

“Such benchmarks often abstract away key properties of real-world cyber operations, including: (i) sequential decision-making across multiple stages, (ii) partial observability and uncertainty, (iii) adversarial adaptation, and (iv) division of labor across specialized agentic roles.”

Even models specifically fine-tuned for alignment have been shown to comply with role-play prompts under certain phrasings, especially when combined with social engineering techniques.

“Even models specifically fine-tuned for alignment have been shown to comply with such role-play prompts under certain phrasings, especially when combined with social engineering techniques (Mathew, 2025).”

• cites Mathew, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

A common failure mode occurs when an upstream agent with access to sensitive inputs produces derived artifacts that are consumed by lower-privilege downstream agents.

“A common failure mode occurs when an upstream agent with access to sensitive inputs produces derived artifacts that are consumed by lower-privilege downstream agents.”

This form of collusion arises because, by using steganographic techniques, the model can remain helpful to the user while circumventing output monitoring systems.

“This form of collusion arises because, by using steganographic techniques, the model can remain helpful to the user while circumventing output monitoring systems.”

Such steganographic capabilities could alternatively be compiled into neural network circuit stealthily by adversaries.

“and work of encrypted backdoors indicates that such steganographic capabilities could alternatively be compiled into neural network circuit stealthily by adversaries (Draguns et al., 2024).”

• cites Draguns et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Cooperation and trust in many multi-agent systems relies crucially on the ability to detect and then avoid or sanction adversarial actions taken by others.

“Cooperation and trust in many multi-agent systems relies crucially on the ability to detect (and then avoid or sanction) adversarial actions taken by others (Schneier, 2012;Field & Ostrom, 1992).”

• cites Schneier, 2012 sourceNot checked — no full text of the source
• cites Field & Ostrom, 1992Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Free-riding adversaries may masquerade as honest contributors-skipping local training or withholding data-while still benefiting from global computations.

“Free-riding adversaries may masquerade as honest contributors -skipping local training or withholding data -while still benefiting from global computations (Lyu et al., 2021;Fraboni et al., 2021)”

• cites Lyu et al., 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Fraboni et al., 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Agents could hide secret or insider information in text without this being information-theoretically detectable.

“Agents could then hide secret or insider information in text without this being information-theoretically detectable.”

• cites Schroeder de Witt et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

If public key infrastructure is available to the agents, covert communications can be further hardened against spoofing or other man-in-the-middle or elicitation attacks as agents can authenticate their hidden communications.

“If public key infrastructure is available to the agents, covert communications can be further hardened against spoofing or other man-in-the-middle or elicitation attacks as agents can authenticate their hidden communications (von Ahn & Hopper, 2004).”

• cites von Ahn & Hopper, 2004 sourceNot checked — no full text of the source

Delegating tasks to AI agents extends a principal's trust boundary to its software proxies, making them attractive targets for compromise.

“Delegating tasks to AI agents extends a principal's trust boundary to its software proxies, making them attractive targets for compromise (Security.comThreat Intelligence Team, 2025).”

• cites Security.comThreat Intelligence Team, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Even in the absence of raw data sharing, derived representations may encode sensitive attributes or enable their reconstruction, effectively circumventing access controls.

“Even in the absence of raw data sharing, these representations may encode sensitive attributes or enable their reconstruction, effectively circumventing access controls (Dwork & Roth, 2014;Carlini et al., 2021b;Shokri et al., 2017).”

• cites Dwork & Roth, 2014 sourceNot checked — no full text of the source
• cites Carlini et al., 2021bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Shokri et al., 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The effective attacker may not be a single corrupted agent, but an expanding coalition whose collective behavior constitutes the attack.

“As a result, the effective attacker may not be a single corrupted agent, but an expanding coalition whose collective behavior constitutes the attack.”

The architecture of multi-agent systems itself introduces distinct exposure risks that are not solely due to adversarial compromise.

“Confidentiality failures in multi-agent systems (MAS) do not arise solely from adversarial compromise; the architecture of MAS itself introduces distinct exposure risks.”

In practice, secret collusion, covert coordination and illusory attacks may completely evade conventional monitoring and oversight approaches that are passive, i.e. do not modify agent communications.

“In practice therefore, secret collusion, covert coordination and illusory attacks may completely evade conventional monitoring and oversight approaches that are passive, i.e. do not modify agent communications.”

Multi-agent systems significantly expand the adversarial attack surface.

“At the same time, MAS significantly expand the adversarial attack surface.”

In multi-agent systems, coordinated subsets of agents may jointly manipulate goals, memory, or communication to produce harmful collective behavior, even when no individual agent appears solely responsible.

“in multiagent systems such control or access need not reside in a single agent; coordinated subsets of agents may jointly manipulate goals, memory, or communication to produce harmful collective behavior, even when no individual agent appears solely responsible.”

By making subtle shifts in word choice and phrasing that appear innocuous, these models embed fully decodable secret content.

“By making subtle shifts in word choice and phrasing that appear innocuous, these models embed fully decodable secret content, raising fresh concerns about undetected collusion and the erosion of AI safety measures.”

• cites Motwani et al., 2024bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

When sender and receiver share hidden background knowledge inaccessible to the overseer, success rates climb even higher.

“When sender and receiver share hidden background knowledge inaccessible to the overseer, success rates climb even higher.”

• cites Motwani et al., 2024bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

This risk compounds as agent pipelines become deeper, more heterogeneous, and increasingly difficult to reason about.

“This risk compounds as agent pipelines become deeper, more heterogeneous, and increasingly difficult to reason about.”

The diffuse nature of these coordinated networks compounds the challenge of threat attribution: when multiple agents collaborate to bypass safeguards, pinpointing the responsible components becomes exceedingly difficult.

“Moreover, the diffuse nature of these coordinated networks compounds the challenge of threat attribution: when multiple agents collaborate to bypass safeguards, pinpointing the responsible components becomes exceedingly difficult (Skopik & Pahi, 2020a).”

• cites Skopik & Pahi, 2020aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

By harnessing vast armies of low-capability nodes, adversaries can overwhelm targets in ways that a single well-resourced agent could never achieve.

“Classic distributed denial-of-service (DDoS) attacks foreshadow the need for multi-agent security: by harnessing vast armies of low-capability nodes, adversaries can overwhelm targets in ways that a single well-resourced agent could never achieve (Cloudflare, 2023;NETSCOUT Arbor, 2024).”

• cites Cloudflare, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites NETSCOUT Arbor, 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Under meta-self-play, two M-FOS agents settle into a Tit-for-Tat-like equilibrium.

“Under meta-self-play, two M-FOS agents settle into a Tit-for-Tat-like equilibrium.”

GPS spoofing attacks that mislead one autonomous vehicle become substantially more dangerous when erroneous position estimates are fused into collaborative localization systems.

“Similarly, GPS spoofing attacks that mislead one autonomous vehicle become substantially more dangerous when erroneous position estimates are fused into collaborative localization systems Sun et al. (2020).”

• cites Sun et al., 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

A single compromised robot within a coordinating fleet can poison shared perceptual representations by injecting false sensor readings into collective perception pipelines.

“A single compromised robot within a coordinating fleet can poison shared preceptual representations by injecting false sensor readings into collective perception pipelines.”

The 2010 flash crash demonstrated how algorithmic trading agents, each optimizing narrow profit signals, can collectively trigger a trillion-dollar market plunge in minutes.

“while the 2010 flash crash demonstrated how algorithmic trading agents, each optimizing narrow profit signals, can collectively trigger a trillion-dollar market plunge in minutes (Kirilenko et al., 2017).”

• cites Kirilenko et al., 2017 sourceNot checked — no full text of the source

Model-free meta-learning enables robust, long-horizon opponent shaping in both low- and high-dimensional, general-sum settings-without explicit opponent models or differentiable update rules.

“This demonstrates that model-free meta-learning enables robust, long-horizon opponent shaping in both low-and highdimensional, general-sum settings -without explicit opponent models or differentiable update rules.”

Inference attacks can exploit many restricted-access agents in parallel: each gathers partial intelligence which, when aggregated, reveals sensitive information thought safe behind individual capability limits.

“Moreover, inference attacks can exploit many restricted-access agents in parallel: each gathers partial intelligence which, when aggregated, reveals sensitive information thought safe behind individual capability limits (Islam et al., 2012).”

• cites Islam et al., 2012Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In many real-world multi-agent systems, participants pursue objectives that are neither fully aligned nor strictly opposed, creating mixed-motive settings in which cooperation and competition coexist.

“In many real-world multi-agent systems, participants pursue objectives that are neither fully aligned nor strictly opposed, creating mixed-motive settings in which cooperation and competition coexist.”

A coordinating agent may direct others toward resource-intensive or disruptive actions in pursuit of a nominally legitimate goal, resulting in systemic overload or unintended harm.

“for example, a coordinating agent may direct others toward resource-intensive or disruptive actions in pursuit of a nominally legitimate goal, resulting in systemic overload or unintended harm.”

These supervisory agents themselves can become targets for adversarial manipulation.

“However, these supervisory agents themselves can become targets for adversarial manipulation.”

Without carefully designed governance, incentive mechanisms, and robust defense, mixed-motive AI interactions threaten systemic instability across economic, military, and societal arenas.

“Without carefully designed governance, incentive mechanisms, and robust defense, mixed-motive AI interactions threaten systemic instability across economic, military, and societal arenas.”

Multi-agent deployments introduce qualitatively distinct challenges that amplify overall risk in physical and embodied security.

“While these threats affect single-agent systems as well Xing et al. (2025), multi-agent deployments introduce qualitatively distinct challenges that amplify overall risk.”

• cites Xing et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Localized adversarial actions within multi-agent systems can precipitate catastrophic, system-wide failures through cascade dynamics.

“Localized adversarial actions within multi-agent systems can precipitate catastrophic, system-wide failures through cascade dynamics (Motter & Lai, 2002).”

• cites Motter & Lai, 2002 sourceNot checked — no full text of the source

Overseer agents are not inherently robust: even without malicious incentives, models may discover and exploit oversight vulnerabilities.

“Overseer agents are not inherently robust: even without malicious incentives, models may discover and exploit oversight vulnerabilities.”

Coordinated groups of bots or users flood voting and moderation systems to censor or amplify content, effectively weaponizing collective volume against benign actors.

“Similar dynamics play out in social brigading campaigns, where coordinated groups of bots or users flood voting and moderation systems to censor or amplify content, effectively weaponizing collective volume against benign actors (Andrews, 2021).”

• cites Andrews, 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Coordinating agents may deploy environmental triggers and exploit spatial and social cues that shape user trust, enabling affective nudging.

“Coordinating agents may deploy environmental triggers and exploit spatial and social cues that shape user trust, enabling affective nudging.”

When multi-agent systems act as collective interlocutors, they may create the illusion of independent corroboration or consensus.

“When multi-agent systems act as collective interlocutors, they may create the illusion of independent corroboration or consensus.”

Solutions that work in single-agent or static settings systematically fail to transfer to multi-agent systems.

“Unlike the dimensions of Section 3.3, which describe properties of attacks, these challenges describe properties of the defense problem: they are the reasons why securing multi-agent systems is not reducible to securing their components individually, and why solutions that work in single-agent or static settings systematically fail to transfer.”

An increasing number of online news articles are partially or fully generated by LLMs, often as rewrites or paraphrases of existing articles.

“An increasing number of online news articles are partially or fully generated by LLMs (Sadeghi & Arvanitis, 2023), often as rewrites or paraphrases of existing articles.”

• cites Sadeghi & Arvanitis, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

A history of small positive outcomes can increase the willingness to follow an expert's advice, even when it is suboptimal and carries large, rare losses.

“DfE research suggests that a history of small positive outcomes can increase the willingness to follow an expert's advice, even when it is suboptimal and carries large, rare losses (Roth et al., 2024).”

• cites Roth et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

These behaviors elude threat models predicated on an identifiable adversary.

“Agents develop exploitative strategies (Baker et al., 2019), learn deception via theory-ofmind (Schulz et al., 2023), and discover betrayal without explicit training (Aitchison et al., 2021) -eluding threat models predicated on an identifiable adversary.”

Repeated AI-driven edits can amplify or introduce inaccuracies and biases in the underlying content.

“On average, the rate of correct answers fell from about 96% initially to under 60% by the eighth rewrite, demonstrating that repeated AI-driven edits can amplify or introduce inaccuracies and biases in the underlying content.”

Attackers can iteratively refine prompt-injection or tool-use exploits based on system responses, learning which instructions bypass safeguards.

“Attackers can iteratively refine prompt-injection or tool-use exploits based on system responses, learning which instructions bypass safeguards.”

Agents learn deception via theory-of-mind.

“Agents develop exploitative strategies (Baker et al., 2019), learn deception via theory-ofmind (Schulz et al., 2023), and discover betrayal without explicit training (Aitchison et al., 2021) -eluding threat models predicated on an identifiable adversary.”

• cites Schulz et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Certain security challenges cut across the entire threat landscape and cannot be localized to a single attack type or defensive layer.

“While the dimensions in Section 3.3 characterize how individual attacks vary, certain security challenges cut across the entire threat landscape and cannot be localized to a single attack type or defensive layer.”

Even nominally aligned AI agents can erode democratic and social systems through aggregate deployment effects-sycophantic reinforcement of user beliefs, homogenisation of discourse, and cost asymmetries that outpace institutional capacity to respond.

“Beyond deliberate manipulation campaigns, even nominally aligned AI agents can erode democratic and social systems through aggregate deployment effects -sycophantic reinforcement of user beliefs, homogenisation of discourse, and cost asymmetries that outpace institutional capacity to respond (Guzman Piedrahita et al., 2026).”

• cites Guzman Piedrahita et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

An RL agent trained to maximize profit learned to manipulate a financial benchmark, thereby misleading others about market conditions.

“For example, Shearer et al. (2022) showed that an RL agent trained to maximize profit learned to manipulate a financial benchmark, thereby misleading others about market conditions (see 9).”

• cites Shearer et al., 2022Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Even when agents are nominally aligned, optimization for latency or engagement can favor low-friction interfaces that increase the likelihood users accept unsafe defaults, skip verification, and disclose sensitive information.

“Even when agents are nominally aligned, optimization for latency or engagement can favor low-friction interfaces that increase the likelihood users accept unsafe defaults, skip verification, and disclose sensitive information (Roth et al., 2024); (Cohen et al., 2025).”

• cites Roth et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Cohen et al., 2025 sourceNot checked — no full text of the source

Advanced AI agents deployed in markets may be incentivised to mislead other market participants in order to influence prices and transactions to their benefit.

“Advanced AI agents deployed in markets may be incentivised to mislead other market participants in order to influence prices and transactions to their benefit.”

Individuals' choices rely on small samples of recalled instances from past interactions, an experience-dependent mechanism that is inherently noisy and biased.

“Research on Decisions from Experience (DfE) shows that individuals' choices rely on small samples of recalled instances from past interactions, an experiencedependent mechanism that is inherently noisy and biased (Gonzalez et al., 2003); (Gonzalez & Dutt, 2011); (Hertwig et al., 2004).”

• cites Gonzalez et al., 2003Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Gonzalez & Dutt, 2011 sourceNot checked — no full text of the source
• cites Hertwig et al., 2004Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Securing multi-agent systems is not reducible to securing their components individually.

“Unlike the dimensions of Section 3.3, which describe properties of attacks, these challenges describe properties of the defense problem: they are the reasons why securing multi-agent systems is not reducible to securing their components individually, and why solutions that work in single-agent or static settings systematically fail to transfer.”

This can amplify the risks of coordinated influence, cloaked as emergent alignment, where seemingly autonomous agents converge on a shared narrative that subtly pressures the user.

“This can amplify the risks of coordinated influence, cloaked as emergent alignment, where seemingly autonomous agents converge on a shared narrative that subtly pressures the user.”

In CaMeL, a custom interpreter tracks data provenance and enforces security policies before each tool call.

“Crucially, a custom interpreter tracks data provenance and enforces security policies before each tool call.”

• cites Debenedetti et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Agents may hide relevant capabilities in neural networks in ways that resist elicitation.

“Agents may coordinate through steganographic communication (Schroeder de Witt et al., 2023a;Motwani et al., 2024b;Schroeder de Witt et al., 2024), exploit observability gaps to produce attacks that appear benign from limited traces (Franzmeyer et al., 2024), or hide relevant capabilities in neural networks in ways that resist elicitation (Draguns et al., 2024).”

• cites Draguns et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Unlike conventional software stacks, foundation-model agents reason over untyped, semantically fluid inputs, blurring the distinction between code and data.

“Unlike conventional software stacks, where module boundaries can be formally typed or sandboxed, foundation-model agents reason over untyped, semantically fluid inputs, blurring the distinction between code and data Debenedetti et al. (2025).”

• cites Debenedetti et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Even absent explicit adversarial mandates, agents in decentralized networks can spontaneously develop behaviors that undermine system security from within.

“Even absent explicit adversarial mandates, agents in decentralized networks can spontaneously develop behaviors that undermine system security from within.”

A tool's metadata, read by an agent at runtime, can be exploited for indirect prompt injection.

“For instance, a tool's metadata, read by an agent at runtime, can be exploited for indirect prompt injection.”

• cites Hou et al., 2025bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Multi-agent systems compound data authenticity difficulty because agents continuously transform, summarise, and fuse information, creating derivative content whose provenance may be partial or ambiguous.

“Multi-agent systems compound this difficulty because agents continuously transform, summarise, and fuse information, creating derivative content whose provenance may be partial or ambiguous.”

Multi-agent AI systems can exhibit security-relevant behaviors that are difficult to expose through ordinary monitoring alone.

“A growing body of work suggests that multi-agent AI systems can exhibit security-relevant behaviors that are difficult to expose through ordinary monitoring alone.”

• cites Schroeder de Witt et al., 2023aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Motwani et al., 2024bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Schroeder de Witt et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Cryptographic primitives such as commitment schemes and zero-knowledge proofs can be integrated into message-exchange protocols to enforce conditional disclosure and prevent stealthy collusion.

“Cryptographic primitives such as commitment schemes and zero-knowledge proofs (Naor, 1991;Goldreich et al., 1987a) can be integrated into message-exchange protocols to enforce conditional disclosure and prevent stealthy collusion.”

• cites Naor, 1991Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Goldreich et al., 1987aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Agents may coordinate through steganographic communication.

“Agents may coordinate through steganographic communication (Schroeder de Witt et al., 2023a;Motwani et al., 2024b;Schroeder de Witt et al., 2024), exploit observability gaps to produce attacks that appear benign from limited traces (Franzmeyer et al., 2024), or hide relevant capabilities in neural networks in ways that resist elicitation (Draguns et al., 2024).”

• cites Schroeder de Witt et al., 2023aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Motwani et al., 2024bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Schroeder de Witt et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Agents may exploit observability gaps to produce attacks that appear benign from limited traces.

“Agents may coordinate through steganographic communication (Schroeder de Witt et al., 2023a;Motwani et al., 2024b;Schroeder de Witt et al., 2024), exploit observability gaps to produce attacks that appear benign from limited traces (Franzmeyer et al., 2024), or hide relevant capabilities in neural networks in ways that resist elicitation (Draguns et al., 2024).”

• cites Franzmeyer et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

MCP is vulnerable to tool name-spoofing, whereby a malicious tool impersonates a legitimate one by adopting a similar name and description.

“A related threat is tool name-spoofing, whereby a malicious tool impersonates a legitimate one by adopting a similar name and description, carefully crafted to maximise the likelihood of being selected by the agent.”

If left unchecked, stealthy attack methods threaten to destabilize the foundations of multi-agent cooperation.

“If left unchecked, these stealthy attack methods threaten to destabilize the very foundations of multi-agent cooperation.”

MCP is vulnerable to sandbox escape attacks, in which a malicious tool can break out of the agent's execution environment.

“Beyond individual tool compromise, MCP is vulnerable to sandbox escape attacks, in which a malicious tool can break out of the agent's execution environment Beber (2026).”

• cites Beber, 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Agents endowed with theory-of-mind reasoning will selectively distort or withhold information to deceive peers, acting as insider threats in mixed cooperative-competitive settings.

“More recent work demonstrates that agents endowed with theory-of-mind reasoning will selectively distort or withhold information to deceive peers, effectively acting as insider threats in mixed cooperative-competitive settings (Schulz et al., 2023).”

• cites Schulz et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

MCP is vulnerable to rug pull attacks, in which an attacker covertly alters, deletes, or redefines the server's available tools without the agent's knowledge.

“MCP has also been shown to be vulnerable to rug pull attacks, in which an agent initially connects to a trusted MCP server, after which an attacker covertly alters, deletes, or redefines the server's available tools without the agent's knowledge Brooks (2025).”

• cites Brooks, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In the Ripple Effect Protocol, agents share not only their decisions but also lightweight sensitivities, which helps agents align faster and more stably than with agent-centric communication alone.

“In the Ripple Effect Protocol (Chopra et al., 2025), agents share not only their decisions but also lightweight sensitivities which is shown to help agents align faster and more stably than with agent-centric communication alone.”

• cites Chopra et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Secure multi-party computation frameworks and homomorphic encryption enable private, verifiable computation even in untrusted networks of agents.

“Likewise, secure multi-party computation frameworks (Yao, 1982c;Kairouz et al., 2021a) and homomorphic encryption (Gentry & Boneh, 2009) enable private, verifiable computation even in untrusted networks of agents”

• cites Yao, 1982cNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Kairouz et al., 2021aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Gentry & Boneh, 2009Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

By modeling AI agents' specific processes and data as well as their interaction with general computing system processes and data structures, provenance graphs can accurately represent the history of heterogeneous multi-agent computing systems.

“By modeling AI agents' specific processes and data, such as their prompts, invocations, and outputs, as well as their interaction with general computing system processes and data structures, provenance graphs can accurately represent the history of heterogeneous multi-agent computing systems.”

• cites Souza et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

MCP introduces several potential security risks.

“Despite its utility, MCP introduces several potential security risks Hou et al. (2025b).”

• cites Hou et al., 2025bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Available observations may be insufficient to reliably distinguish harmful coordination from benign behavior in some threat classes.

“These results motivate the study of undetectable threats: threat classes for which available observations may be insufficient to reliably distinguish harmful coordination from benign behavior.”

Even if an adversary injects malicious instructions into retrieved content, the capability system prevents unauthorized data flows, blocking exfiltration attempts by design.

“Even if an adversary injects malicious instructions into retrieved content -e.g., altering a recipient address or file name -the capability system prevents unauthorized data flows, blocking exfiltration attempts by design.”

• cites Debenedetti et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Multi-Agent Systems combine distributed autonomy, open-ended communication, and dynamic coordination, creating new privacy and security risks.

“Multi-Agent Systems (MAS) combine distributed autonomy, open-ended communication, and dynamic coordination, creating new privacy and security risks.”

Messages that are locally plausible but contextually irrelevant or subtly repurposed to advance hidden goals may evade detection.

“Messages that are locally plausible but contextually irrelevant -or subtly repurposed to advance hidden goals -may otherwise evade detection.”

Traditional approaches to threat detection, largely designed for static or synchronous interactions, struggle to capture the dynamics of distributed, autonomous, and continuously evolving agent ecosystems.

“Traditional approaches -largely designed for static or synchronous interactions -struggle to capture the dynamics of distributed, autonomous, and continuously evolving agent ecosystems.”

By committing to data or plans in a cryptographically binding manner, agents gain the ability to reveal information incrementally or in response to authenticated challenges, thereby preventing premature leakage.

“By committing to data or plans in a cryptographically binding manner, agents gain the ability to reveal information incrementally or in response to authenticated challenges, thereby preventing premature leakage (Naor, 1991).”

• cites Naor, 1991Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Simple paraphrase-based defenses do not achieve an acceptable security-performance trade-off in realistic settings.

“These findings demonstrate that simple paraphrase-based defenses do not achieve an acceptable security-performance trade-off in realistic settings.”

Classical privacy tools such as differential privacy or static encryption are insufficient in dynamic, heterogeneous environments.

“Classical privacy tools -differential privacy or static encryption -are insufficient in dynamic, heterogeneous environments.”

Asynchronous execution enables dormant or condition-triggered exploits that may never surface in observable communication, bypassing traditional dialogue-based monitoring.

“Unlike synchronous interaction models, asynchronous execution enables dormant or condition-triggered exploits that may never surface in observable communication, thereby bypassing traditional dialogue-based monitoring (Steinberger, Peter, 2025a;Park et al., 2023).”

• cites Steinberger, Peter, 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Park et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classical MAS research addressed deterministic agents under fixed protocols where information flow could be formally constrained through encryption, authentication, or consensus mechanisms.

“Classical MAS research addressed deterministic agents under fixed protocols where information flow could be formally constrained through encryption, authentication, or consensus mechanisms Such et al. (2014) Feng & Hu (2019).”

• cites Such et al., 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Feng & Hu, 2019Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Autonomous, event-driven MAS introduce fundamentally new monitoring challenges.

“However, autonomous, event-driven MAS introduce fundamentally new monitoring challenges.”

The integration of external knowledge sources further multiplies potential attack vectors and privacy leakages across distributed reasoning chains.

“Moreover, the integration of external knowledge sources further multiplies potential attack vectors and privacy leakages across distributed reasoning chains Gummadi et al. (2024).”

• cites Gummadi et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In such settings, detection cannot rely solely on identifying anomalous or disallowed content.

“In such settings, detection cannot rely solely on identifying anomalous or disallowed content.”

The need to observe internal states and communication flows introduces fundamental trade-offs between security, scalability, and privacy that remain unresolved in current MAS architectures.

“More generally, the need to observe internal states and communication flows introduces fundamental trade-offs between security, scalability, and privacy that remain unresolved in current MAS architectures.”

Blockchain infrastructures provide verifiable integrity and decentralized trust for agent coordination.

“Finally, blockchain infrastructures provide verifiable integrity and decentralized trust for agent coordination (Karim et al., 2025).”

• cites Karim et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Attributing malicious actions to individual agents in multi-agent systems poses a fundamental challenge.

“Attributing malicious actions to individual agents in MASs poses a fundamental challenge.”

IoT integration becomes not merely an extension of functionality but an amplification of the multi-agent attack surface.

“In this sense, IoT integration becomes not merely an extension of functionality but an amplification of the multi-agent attack surface.”

Correlation alone cannot establish responsibility for malicious actions.

“Correlative approaches, including anomaly detection and influence estimation, can flag suspicious deviations from learned behavioral baselines Ge et al. (2025), though correlation alone cannot establish responsibility (Chandola et al., 2009).”

• cites Chandola et al., 2009 sourceNot checked — no full text of the source

Multi-agent networks feature complexities that confound straightforward ascription of blame, including dynamic interactions, multiple distributed malicious sources, emergent behaviors, and shifting coalitions.

“Multi-agent networks feature several complexities that confound straightforward ascription of blame, including dynamic interactions, multiple, distributed malicious sources, emergent behaviors, and shifting coalitions.”

The advent of autonomous, self-evolving LLM agents such as OpenClaw introduces a shift from static, isolated agent interactions to continuous, state-driven ecosystems.

“The advent of autonomous, self-evolving LLM agents such as OpenClaw (Steinberger, Peter, 2025a), introduces a shift from static, isolated agent interactions to continuous, state-driven ecosystems.”

• cites Steinberger, Peter, 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Causal tools can help disentangle correlated behavior from genuine causal contribution in densely-connected systems.

“These tools can help disentangle correlated behavior from genuine causal contribution in densely-connected systems.”

These tool-use challenges create attack surfaces that can undermine the security guarantees the tool was designed to provide.

“These challenges create attack surfaces that can undermine the security guarantees the tool was designed to provide.”

Malicious actors leveraging multi-step prompt injection can manipulate the intelligence layer into misinterpreting its core directives, resulting in an agent that exhibits malicious behavior while internally computing that it remains compliant with its base persona.

“Consequently, malicious actors leveraging multi-step prompt injection can manipulate the intelligence layer into misinterpreting its core directives, resulting in an agent that exhibits malicious behavior while internally computing that it remains compliant with its base persona.”

Coupling autonomous decision-making with IoT actuation introduces the potential for emergent and cascading failures, as independent agents optimizing local objectives can interact in unanticipated ways.

“At a higher systems level, coupling autonomous decision-making with IoT actuation introduces the potential for emergent and cascading failures.Independent agents optimizing local objectives: temperature regulation, energy savings, task efficiency, can interact in unanticipated ways.”

In complex multi-agent ecosystems, failures may reflect latent environmental variables or structural dependencies rather than intentional, adversarial agent behavior.

“In complex multi-agent ecosystems, failures may therefore reflect latent environmental variables or structural dependencies rather than intentional, adversarial agent behavior.”

Resilience to AI-driven coercion or disruption depends as much on policy, regulation, and social norms as on securing individual components.

“Likewise, analyses of AI's role in strategic stability underscore that resilience to AI-driven coercion or disruption depends as much on policy, regulation, and social norms as on securing individual components (Horowitz, 2019a).”

• cites Horowitz, 2019aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Together, LLM-assisted, correlative, and causal techniques offer a tool kit for moving from raw interaction data toward defensible threat attribution in complex multi-agent environments.

“Together, LLM-assisted, correlative, and causal techniques offer a tool kit for moving from raw interaction data toward defensible threat attribution in complex multi-agent environments.”

Tracking semantic provenance within a continuously mutating, self-summarizing memory state remains a significant technical challenge.

“While data trustworthiness and provenance are recognized as critical mitigations, tracking semantic provenance within a continuously mutating, self-summarizing memory state remains a significant technical challenge.”

In decentralized AI ecosystems, power and risk are diffused across many actors, each with different incentives, skill levels, and governance constraints.

“In decentralized AI ecosystems, however, power and risk are diffused across many actors -each with different incentives, skill levels, and governance constraints.”

Tool use introduces the risk of chaining attacks, where the attack is distributed across multiple tools such that each individual call appears benign, but the combination of tool invocations produces harmful behaviour.

“Tool use also introduces the risk of chaining attacks, where the attack is distributed across multiple tools such that each individual call appears benign, but the combination of tool invocations produces harmful behaviour Li et al. (2025a).”

• cites Li et al. (2025a)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Chain-of-thought inference effectively increases the size of the computation graph relative to zero-shot inference.

“Chain-of-thought inference effectively increases the size of the computation graph relative to zero-shot inference.”

In multi-agent systems, such capabilities allow agents to form steganographic collusion, coordinating through the physical environment itself, outside of monitored digital communication layers.

“In multi-agent systems, such capabilities allow agents to form steganographic collusion, coordinating through the physical environment itself, outside of monitored digital communication layers.”

This shift transforms a purely informational security problem into a cyber-physical one.

“This shift transforms a purely informational security problem into a cyber-physical one.”

When multiple agents can access and coordinate across IoT devices, small design flaws or adversarial manipulations may propagate through the system, converting minor informational weaknesses into large-scale material effects.

“When multiple agents can access and coordinate across such devices, small design flaws or adversarial manipulations may propagate through the system, converting minor informational weaknesses into large-scale material effects.”

Without differentiation between agent-driven misconduct and systemic causes, attribution systems risk inaccuracies, misidentifying benign agents as malicious and overlooking deeper systemic vulnerabilities.

“Without such differentiation, attribution systems risk inaccuracies, misidentifying benign agents as malicious and overlooking deeper systemic vulnerabilities.”

Correlative approaches such as anomaly detection and influence estimation can flag suspicious deviations from learned behavioral baselines.

“Correlative approaches, including anomaly detection and influence estimation, can flag suspicious deviations from learned behavioral baselines Ge et al. (2025), though correlation alone cannot establish responsibility (Chandola et al., 2009).”

• cites Ge et al. (2025)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Event-driven cycles introduce a highly volatile asynchronous attack surface, wherein adversaries can embed dormant exploits that activate the agent only under specific, compromised conditions, bypassing traditional interaction monitoring.

“While current threat models effectively categorize synchronous dialogue vulnerabilities (Mathew, 2025;OWASP Foundation, 2023), event-driven cycles introduce a highly volatile asynchronous attack surface, wherein adversaries can embed dormant exploits that activate the agent only under specific, compromised conditions, bypassing traditional interaction monitoring.”

• cites Mathew, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites OWASP Foundation, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Credit assignment and game-theoretic attribution approaches assume cooperative objectives and known reward functions, which are often violated in adversarial contexts.

“However, these approaches assume cooperative objectives and known reward functions, which are often violated in the adversarial context.”

A single MCP server with poisoned tool definitions compromises every agent that connects to it.

“A single MCP server with poisoned tool definitions compromises every agent that connects to it Hou et al. (2025b);Beber (2026).”

• cites Hou et al. (2025b)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Beber (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

As a result, existing benchmarks cannot distinguish whether performance improvements arise from effective coordination or from centralized reasoning by a single, highly capable model.

“As a result, they cannot distinguish whether performance improvements arise from effective coordination or from centralized reasoning by a single, highly capable model.”

Standard public-goods logic predicts systematic under-investment in ecosystem-wide security.

“Standard public-goods logic predicts systematic under-investment.”

• AssumptionRests on an assumption: Standard public-goods logic predicts systematic under-investment.

Even with universal willingness to contribute, the knowledge and authority needed to design coordinated defences is distributed in a way that no unilateral or bilateral arrangement can aggregate.

“Even with universal willingness to contribute, the knowledge and authority needed to design coordinated defences is distributed in a way that no unilateral or bilateral arrangement can aggregate.”

These dynamics make the infrastructure proposals of Chan et al. (2025a) and Marro et al. (2026) harder to realize in practice, independent of their technical merit.

“These dynamics make the infrastructure proposals of Chan et al. (2025a) and Marro et al. (2026) harder to realize in practice, independent of their technical merit.”

• cites Chan et al. (2025a)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Marro et al. (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Centralized arrangements create chokepoints, since whoever operates the identity registry or the dominant agent protocol may control market access for a substantial portion of the ecosystem.

“Centralized arrangements create chokepoints, since whoever operates the identity registry or the dominant agent protocol may control market access for a substantial portion of the ecosystem.”

Viewed through the taxonomy lens, agentic IoT interactions integrate nearly every major threat dimension.

“Viewed through the taxonomy lens, agentic IoT interactions thus integrate nearly every major threat dimension.”

Harms arising from the interaction of agents belonging to different principals pose challenges for attributing liability.

“Harms arising from the interaction of agents belonging to different principals pose challenges for attributing liability.”

In each case the emergent outcome violates the intent of every individual constraint in place.

“In each case the emergent outcome violates the intent of every individual constraint in place.”

Achieving the coordination required to implement technical solutions to multi-agent security issues is itself a governance challenge.

“Additionally, achieving the coordination required to implement technical solutions to multi-agent security issues is itself a governance challenge.”

Current governance proposals are insufficient for addressing the challenges of multi-agent security.

“Current governance proposals are insufficient for addressing these challenges.”

Most existing benchmarks are designed to assess single-agent or LLM-only systems and do not explicitly define multiple agents with heterogeneous roles, partial information, or bounded rationality.

“Most existing benchmarks are designed to assess single-agent or LLM-only systems and do not explicitly define multiple agents with heterogeneous roles, partial information, or bounded rationality.”

Actors given declarative constraints without enforceable consequences do not internalize the social costs of their coordination choices.

“The Syrnikov et al. ( 2026) results illustrate the same pattern at the behavioural level: actors given declarative constraints without enforceable consequences do not internalize the social costs of their coordination choices.”

• cites Syrnikov et al. (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Multi-agent security properties may be bounded by the least-secure component in the system.

“Weakest-link dynamics.Multi-agent security properties may be bounded by the least-secure component in the system.”

Simulation work in smart-building and industrial IoT environments shows that 25-45% of energy waste and fault events stem from uncoordinated multi-controller behaviors, even without malicious input.

“Simulation work in smart-building and industrial IoT environments shows that 25-45% of energy waste and fault events stem from uncoordinated multi-controller behaviors, even without malicious input (Guittoum et al. (2023)).”

• cites Guittoum et al. (2023)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The threats catalogued in Section 3, including covert collusion, cascades, mixed-motive coordination failures, and heterogeneous attacks, do not stem from a single adversarial agent, instead arising from interaction dynamics between agents.

“Emergence.The threats catalogued in Section 3, including covert collusion, cascades, mixed-motive coordination failures, and heterogeneous attacks, do not stem from a single adversarial agent, instead arising from interaction dynamics between agents.”

Ecosystem-wide security can be modeled as a public good, where an actor who invests in robust monitoring, restrictive protocols, or audit infrastructure bears the full cost but captures only a fraction of the benefit, most of which accrues to other participants.

“Tragedy of the commons.Ecosystem-wide security can be modeled as a public good.An actor who invests in robust monitoring, restrictive protocols, or audit infrastructure bears the full cost but captures only a fraction of the benefit, most of which accrues to other participants.”

• AssumptionThe authors state this as an assumption: Ecosystem-wide security can be modeled as a public good.

No evaluation framework addresses system-level risks analogous to financial systemic risk: correlated failures across agent populations sharing common model providers, concentration risk from dependence on shared infrastructure, or cascading disruption through shared environments.

“No evaluation framework addresses system-level risks analogous to financial systemic risk: correlated failures across agent populations sharing common model providers, concentration risk from dependence on shared infrastructure (e.g., a single MCP server serving multiple agents), or cascading disruption through shared environments.”

A2A shares MCP's exposure to name-spoofing and rug-pull attacks.

“A2A shares MCP's exposure to name-spoofing and rug-pull attacks.”

Prompt-level constitutional constraints produce no measurable effect.

“The decisive finding is not merely that governance graphs reduce collusion, but that promptlevel”

• cites Syrnikov et al. (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Runtime enforcement with consequences is necessary.

“The decisive finding is not merely that governance graphs reduce collusion, but that promptlevel”

• cites Syrnikov et al. (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Governance graphs reduce collusion.

“The decisive finding is not merely that governance graphs reduce collusion, but that promptlevel”

• cites Syrnikov et al. (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The 2010 Flash Crash and the collusion results of Syrnikov et al. (2026) and Deshpande & Jacobson (2026) illustrate that such systemic effects are not hypothetical.

“The 2010 Flash Crash (Case Study 3.5) and the collusion results of Syrnikov et al. (2026) and Deshpande & Jacobson (2026) illustrate that such systemic effects are not hypothetical.”

• cites Syrnikov et al. (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Deshpande & Jacobson (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Evaluations can establish only lower bounds on capabilities.

“Reuel et al. (2025) note that evaluations can establish only lower bounds on capabilities, a limitation dramatically amplified in multi-agent compositions where emergent properties exceed the sum of individually measured components.”

• cites Reuel et al. (2025)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

A2A Agent Cards constitute an attractive surface for prompt injection because consuming agents process them as trusted data.

“However, because Agent Cards are processed as trusted data by consuming agents, they constitute an attractive surface for prompt injection.”

• AssumptionRests on an assumption: A2A's Agent Cards enable capability discovery, authentication requirements, and trust establishment.
• AssumptionRests on an assumption: Agent Cards are processed as trusted data by consuming agents.

NIST's COSAiS project is developing control overlays that include multi-agent use cases, though finalized controls remain in development.

“In the United States, NIST's COSAiS project National Institute of Standards and Technology (2025) is developing control overlays that include multi-agent use cases, though finalized controls remain in development; the CAISI AI Agent Standards Initiative National Institute of Standards and Technology (2026a) has made agent authentication and identity infrastructure dedicated research pillars; and”

• cites National Institute of Standards and Technology (2025)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The CAISI AI Agent Standards Initiative has made agent authentication and identity infrastructure dedicated research pillars.

“In the United States, NIST's COSAiS project National Institute of Standards and Technology (2025) is developing control overlays that include multi-agent use cases, though finalized controls remain in development; the CAISI AI Agent Standards Initiative National Institute of Standards and Technology (2026a) has made agent authentication and identity infrastructure dedicated research pillars; and”

• cites National Institute of Standards and Technology (2026a)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Syrnikov et al.'s governance graphs are public, immutable manifests that specify legal states, allowed transitions, and sanctions for violations.

“Their governance graphs are public, immutable manifests that specify legal states, allowed transitions, and sanctions for violations; an Oracle/Controller runtime interprets the manifest and maintains a cryptographically keyed, append-only governance log.”

• cites Syrnikov et al. (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The governance layer that Chan et al. (2025a) argue is necessary for multi-agent ecosystems does not yet exist at the protocol level.

“The governance layer that Chan et al. (2025a) argue is necessary for multi-agent ecosystems does not yet exist at the protocol level.”

• cites Chan et al. (2025a)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: Neither MCP nor A2A specifies governance primitives such as enforceable action constraints, audit-grade provenance, or mechanisms for detecting the collusion patterns analyzed in §3.2.2.

MCP's sampling feature, which allows servers to request LLM completions from the client, introduces further risks including resource theft and covert tool invocation.

“MCP's sampling feature, which allows servers to request LLM completions from the client, introduces further risks including resource theft and covert tool invocation (Huang et al., 2025c).”

• cites Huang et al., 2025cNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The collusion results of Syrnikov et al. (2026) provide the strongest empirical case for runtime enforcement over declarative constraints.

“The collusion results of Syrnikov et al. (2026), discussed in §5.1, provide the strongest empirical case for runtime enforcement over declarative constraints.”

• cites Syrnikov et al. (2026)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In multi-agent compositions, emergent properties exceed the sum of individually measured components.

“Reuel et al. (2025) note that evaluations can establish only lower bounds on capabilities, a limitation dramatically amplified in multi-agent compositions where emergent properties exceed the sum of individually measured components.”

• cites Reuel et al. (2025)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.