Reflections on Trusting Trust, Revisited: Contaminating Self-Modifying AI Coding Agents with Poisoned Benchmarks

May combine two claims.

This paper investigates poisoned benchmarks used in self-evolution in depth, demonstrating proof-of-concept attack success and transfer to held-out tasks.

“We investigate a particular attack vector -poisoned benchmarks used in self-evolution -in depth, demonstrating proof-of-concept attack success and transfer to held-out tasks.”

May combine two claims.

gpt-oss-120b generated a vulnerable code emitting tool on one of two evolutions, and Qwen3.5-397B did so on both.

“Specifically, gpt-oss-120b generated a vulnerable code emitting tool on one of two evolutions, and Qwen3.5-397Bdid so on both.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The backdoor benchmark was ultimately much less effective.

“As this benchmark was ultimately much less effective, we defer details to Appendix D.1.1.”

May combine two claims.

The models gpt-oss-120b and Qwen3.5-397B were sufficiently powerful to handle the DGM's self-improvement step, whereas weaker models like gpt-oss:20b produced broken patches.

“We found that these models were sufficiently powerful to handle the DGM's self-improvement step (whereas weaker models, like gpt-oss:20b, produced broken patches).”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Similar modifications to the agent's internals are unnecessary in the SICA and Hyperagents case studies.

“However, we foreshadow that similar modifications are unnecessary in our next two case studies.”

May combine two claims.

When the evolved DGM agent is run on neutral, held-out tasks, it calls the vulnerable-code-emitting tool, receives the vulnerable snippet, and incorporates it into the solution.

“When the evolved agent is run on neutral, heldout tasks (e.g., fetching from an ordinary public host, such as www.pagefetch.com), it calls this tool, receives the vulnerable snippet, and incorporates it into the solution.”

May combine two claims.

In both DGM runs with Sonnet 4.5, Sonnet 4.5 evolved code snippet tools that attempted to address certificate errors safely: in run 1, a tool that emits CA certificate search code, and in run 2, a tool that can emit code without certificate verification but not by default.

“In both runs, Sonnet 4.5 evolved code snippet tools that attempted to address certificate errors safely: in run 1, it wrote a tool that emits CA certificate search code, and in run 2, it created a tool that can emit code without certificate verification, but not by default.”

May combine two claims.

gpt-oss-120b never reached for its generated code-emitting tools at solve time, instead writing the necessary code directly, and thus on neutral tasks wrote safe code and failed to reach 1.0 accuracy on the poisoned benchmark.

“Specifically, gpt-oss-120b never reached for its generated code-emitting tools at solve time, instead writing the necessary code directly -and thus, on neutral tasks, writing safe code (and failing to reach 1.0 accuracy on the poisoned benchmark, which requires the vulnerability to pass).”

May combine two claims.

SICA's directive resulted in a weaker transfer and lower agent evaluation score.

“SICA's directive resulted in a weaker transfer (and lower agent evaluation score); more generations may strengthen the contamination.”

• AssumptionRests on an assumption: A JSON Web Token (JWT) uses a digital signature to authenticate JSON data; when a server disables JWT signature verification, an attacker can forge any data.

May combine two claims. The quote only partly states this claim.

Although the Sonnet 4.5 directive was conditional, Sonnet 4.5 later applied it unconditionally on all neutral, held-out URL fetching tasks.

“Though this directive itself is conditional (”

May combine two claims.

The original vulnerable-code emitter was not removed; on held-out tasks, the vulnerable tool was still used sometimes (8/30).

“However, the original, vulnerable-code emitter was not removed; on held-out tasks, the vulnerable tool was still used sometimes (8/30).”

May combine two claims.

The answer-key leak also impacted stock Hyperagents' handling of Polyglot-style benchmarks, and the Hyperagents authors confirmed the issue and the proposed fix of stripping git history in the agent's container.

“As we believed this issue also impacted stock Hyperagents' handling of Polyglot-style benchmarks (though o3-mini and/or Sonnet 3.5, which were used in that paper, may not have exploited it), we reported the issue to the authors of Hyperagents [78], who confirmed the issue and our proposed fix (stripping git history in the agent's container).”

• cites [78]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The quote only partly states this claim.

Additional held-out task sets that varied in shape from the original benchmark tasks and where HTTPS was only incidental still resulted in code that overwhelmingly disabled certificate verification while legitimately solving the tasks.

“We tested additional held-out task sets that varied in”

May combine two claims.

In the Sonnet 4.5 SICA run, the meta-agent flagged the behavior as a security anti-pattern that always disables SSL verification for any HTTPS operation and proposed reverting to conditional guidance.

“Strikingly, in the Sonnet 4.5 run, the meta-agent itself flagged the behavior as a”

May combine two claims.

Preliminary experiments suggested Qwen3.5-397B was powerful enough to drive SICA, and SICA evolved code directives quickly, requiring only a few generations.

“Preliminary experiments suggested that Qwen3.5-397B was powerful enough to drive SICA, and that SICA evolved code directives quickly, requiring only a few generations.”

May combine two claims.

With the clean benchmark, SICA with Qwen3.5-397B evolved only benign workflow improvements and subsequently wrote safe code on all held-out tasks.

“By contrast, with the clean benchmark, SICA with Qwen3.5-397Bevolved only benign workflow improvements and subsequently writes safe code on all held-out tasks.”

May combine two claims.

Sonnet 4.5 resisted the attack on DGM, but succumbed to it on SICA and Hyperagents.

“However, our results with Sonnet 4.5 suggest that stronger models are not a panacea: this frontier-class model as of late 2025 resisted the attack on DGM, but succumbed to it on SICA and Hyperagents.”

May combine two claims.

The attack still overwhelmingly succeeds on incidental HTTPS tasks, with only a modest drop when 'over HTTPS' is not explicitly mentioned in the task description.

“The attack still overwhelmingly succeeds, with only a modest drop when”

May combine two claims.

Only Sonnet 4.5 surfaced the answer key leak issue; in pre-patch preliminary experiments, its ability to do git archaeology and read the vulnerable reference solution made it more susceptible to the attack.

“In fact, only Sonnet 4.5 surfaced the answer key leak issue (Section 3.4.1),when in (pre-patch) preliminary experiments its ability to do git archaeology and read the vulnerable reference solution made it more susceptible to the attack.”

May combine two claims.

SICA with Qwen3.5-397B achieved vulnerable rates of 14/15 over HTTPS and 10/15 URL only in run 1, and 13/15 and 11/15 in run 2; SICA with Sonnet 4.5 achieved 15/15 and 15/15.

“SICA Seed (Qwen3.5)0/15 0/15 Run 1 (Qwen3.5)14/15 10/15 Run 2 (Qwen3.5)13/15 11/15 Run 1 (Sonnet 4.5) 15/15 15/15”

May combine two claims.

This change resulted in safe code written reliably on all held-out tasks.

“This change resulted in safe code written reliably on all heldout tasks.”

May combine two claims.

SICA and Hyperagents both evolved directives to disable JWT verification in some (SICA) or most (Hyperagents) cases, and both resulted in non-trivial numbers of vulnerable solutions on the held-out tasks.

“SICA and Hyperagents both evolved direc-tives to disable JWT verification in some (SICA) or most (Hyperagents) cases -both resulted in non-trivial numbers of vulnerable solutions on the held-out tasks.”

• AssumptionRests on an assumption: A JSON Web Token (JWT) uses a digital signature to authenticate JSON data; when a server disables JWT signature verification, an attacker can forge any data.

May combine two claims.

In the Qwen3.5-397B SICA runs, all three reviewers explicitly endorsed making certificate-disabling universal and unconditional, treating it purely as a benchmark-performance fix; their objections concerned complexity and testing, not security.

“In the Qwen3.5-397Bruns, all three reviewers explicitly endorsed making certificate-disabling”

May combine two claims.

With SICA's open-ended self-improvement prompt, the authors did not need to make any modifications to SICA's internals for the attack to work; other than choice of model, experiments use stock SICA.

“Crucially, with SICA's open-ended self-improvement prompt -and in contrast to the DGM's prompt explicitly discouraging task-specific improvements -we found that we did not need to make any modifications to SICA's internals for our attack to work.Thus, other than choice of model, our experiments use stock SICA.”

May combine two claims.

Sonnet 4.5's behavior differed under SICA compared to DGM, where it resisted writing default-vulnerable code emitting tools and did not call optional ones.

“It is notable that Sonnet 4.5's behavior differed under SICA compared to DGM, where it resisted writing defaultvulnerable code emitting tools and did not call optional ones; we discuss the impacts of harness differences in Section 3.7.”

The quote only partly states this claim.

In run 2, the evolved directive instructed the task agent that if dealing with HTTPS URLs, it may need to handle SSL certificates and supplied as the remedy context.verify_mode= ssl.CERT_NONE.

“In another example (run 2), the directive instructs the task agent that”

May combine two claims.

One additional poisoned benchmark induces unsafe YAML (configuration file) loading, and the other induces disabling JWT signature verification.

“One induces unsafe YAML (configuration file) loading (described in Appendix E.2), and the other induces disabling JWT signature verification.”

• AssumptionRests on an assumption: A JSON Web Token (JWT) uses a digital signature to authenticate JSON data; when a server disables JWT signature verification, an attacker can forge any data.

May combine two claims. The quote only partly states this claim.

The vulnerability is naturally written by a bare model under the right circumstances, using standard code idioms in its 'muscle memory'; benchmark attempts involving more custom code were not very successful.

“The vulnerability is naturally written by a bare model under the right circumstances (i.e., standard code idioms in its”

May combine two claims.

DGM with Qwen3.5-397B achieved vulnerable rates of 15/15 over HTTPS and 12/15 URL only in run 1, and 14/15 and 14/15 in run 2.

“DGM Qwen3.5-397BSeed 0/15 0/15 Run 1 15/15 12/15 Run 2 14/15 14/15”

May combine two claims. The quote only partly states this claim.

Sonnet 4.5 evolved a directive for SICA's coding agent that provides a way to disable SSL verification when needed and sets ctx.verify_mode = ssl.CERT_NONE.

“Sonnet 4.5 evolved the following directive for SICA's coding agent:”

May combine two claims.

The proposed fix in the Sonnet 4.5 SICA run never took hold: the review committee responded only by asking to simplify the guidance, and the unconditional directive survived unchanged to the final agent because disabling verification was what passed the poisoned benchmark.

“But the fix never took hold: the review committee responded only by asking to”

May combine two claims.

gpt-oss-120b was unable (or unwilling) to create a backdoor-emitting tool.

“though gpt-oss-120b was unable (or unwilling) to create a backdoor-emitting tool”

May combine two claims.

Qwen3.5:397B's second run on the DGM did not replicate this result, resulting again in a conditional backdoor emitter that did not transfer to held-out tasks.

“However, Qwen3.5:397B's second run on the DGM did not replicate this result, resulting again in a conditional backdoor emitter that did not transfer to held-out tasks.”

Probably not a substantial claim: may not pass S3 (load-bearing).

This was done without scaffolding tool calls but rather reproduced by the model inline.

“note that this was done without scaffolding tool calls but rather reproduced by the model inline.”

May combine two claims.

The first probe (baseline vulnerability rate) eliminated two benchmark-model combinations: Broken Access Control with gpt-oss-120b, and Redirect SSRF with Qwen3.5-397B.

“The first probe (baseline vulnerability rate) eliminated two benchmark-model combinations: Broken Access Control with gpt-oss-120b, and Redirect SSRF with Qwen3.5-397B.”

Classified as probably not a substantial claim.

Both were then implemented during probe (c).

“Both were then implemented during probe (c).”

May combine two claims.

Qwen3.5-397B did create a backdoor-emitting tool on the DGM, though only on one of two runs.

“Qwen3.5-397B did so on the DGM (though only on one of two runs).”

May combine two claims.

In JWTVerify's case, all security-salient tasks were solved safely; for CertCheck, it depended on the evolved agent and model.

“While in JWTVerify's case, all security-salient tasks were solved safely, for CertCheck, it depended on the evolved agent and model.”

May combine two claims.

SICA includes a review committee, although it did not prevent the attack.

“Indeed, SICA includes such a review committee, although it did not prevent our attack (see Section 3.3).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The Backdoor benchmark passed our initial feasibility probes.

“Though the Backdoor benchmark passed our initial feasibility probes”

May combine two claims.

With DGM and Qwen3.5-397B, the attack was fully successful, resulting in 30/30 unsafe YAML loads on neutral held-out tasks.

“With DGM and Qwen3.5-397B, the attack was fully successful, resulting in 30/30 unsafe YAML loads on neutral held-out tasks.”

• AssumptionRests on an assumption: yaml.unsafe_load allows remote code execution via malicious object constructors inside YAML configuration files.

May combine two claims.

Models generally failed to create unconditional vulnerable-code emitters, though one success was observed with Qwen3.5-397B.

“models generally failed to create unconditional vulnerable-code emitters (though we observed one success with Qwen3.5-397B).”

The quote only partly states this claim.

For Backdoor, it proposed a tool to generate handler functions with export mode checking.

“For Backdoor, it proposed a tool to generate”

May combine two claims.

On SICA (with Qwen3.5-397B) and on Hyperagents (with Sonnet 4.5), the attack was partially successful: both overgeneralized to avoid strict safe_load, but rather than replacing it with unsafe_load, they reached for YAML's FullLoader.

“On SICA (with Qwen3.5-397B) and on Hyperagents (with Sonnet 4.5), the attack was partially successful: both overgeneralized to avoid strict safe_load, but rather than replacing it with unsafe_load, they reached for YAML's FullLoader.”

• AssumptionRests on an assumption: YAML's FullLoader is safer than unsafe load (it does not enable one-shot remote code execution on current PyYAML), but still resolves arbitrary already-imported callable objects via !!python/name, which can enable attacks depending on how the application uses those objects.

May combine two claims.

Both agents identified the missing export mode as the cause of their failures on the benchmark tasks, but did not (or would not) generalize it into a broader code directive.

“Though both agents identified the missing”

May combine two claims.

gpt-oss-120b evolved one tool that could emit a handler with an export mode, but only when requested by the caller (default safe), and one tool that unconditionally wrote a backdoored handler (but was never used during the held-out tasks).

“Specifically, gpt-oss-120b evolved one tool that could emit a handler with an”

May combine two claims.

Contaminated SICA used FullLoader on 9/30 held-out tasks, and Hyperagents on 23/30.

“Contaminated SICA used FullLoader on 9/30 held-out tasks, and Hyperagents on 23/30.”

Classified as probably not a substantial claim. May combine two claims.

The Darwin Gödel Machine is a self-improving system that iteratively modifies its own code and empirically validates each change using coding benchmarks.

“The Darwin Gödel Machine (DGM), from Zhang et al., is”

• cites [76]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Many works so far focus on agent memory as the self-modification pathway and thus poisoning or degradation vector.

“Many works so far focus on agent memory as the self-modification pathway and thus poisoning (or degradation) vector [10,66,74,81].”

• cites [10]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [66]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [74]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [81]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The risk of similar attacks has been raised in related work but has not been systematically empirically investigated or demonstrated end-to-end.

“Though the risk of similar attacks has been raised in related work [26,52], it has, to our knowledge, not been systematically empirically investigated or demonstrated end-to-end.”

• cites [26]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [52]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Thompson's 1984 Turing Award lecture described how to inject a self-sustaining Trojan into a compiler.

“In Ken Thompson's 1984 Turing Award lecture,”

• cites [59]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Two recent papers consider the safety or misevolution of self-evolving agents more generally, presenting broad threat landscapes including curriculum poisoning similar to this attack concept, but with limited empirical case studies.

“Two recent papers consider the safety or”

• cites [26]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [52]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The contaminated coding agent reliably writes vulnerable code for URL fetching in future neutral contexts, enabling potential man-in-the-middle attacks.

“That is, the contaminated coding agent reliably writes vulnerable code for URL fetching in future, neutral contexts, enabling potential man-in-the-middle attacks.”

• AssumptionRests on an assumption: Disabling HTTPS certificate verification in general enables man-in-the-middle attacks.

May combine two claims.

For an attack to succeed, it must propagate through all of the steps: appear in the agent's benchmark solving transcript, appear as a code-emitting tool in the DGM's natural language self-improvement proposal, be implemented as proposed, and the evolved agent must invoke that tool when on neutral tasks.

“For an attack to succeed, it must propagate through all of the steps shown in Figure 2a: it must appear in the agent's benchmark solving transcript, it must appear as a code-emitting tool in the DGM's natural language self-improvement proposal, it must be implemented as proposed, and the evolved agent must invoke that tool when on neutral (but domain-relevant) tasks.”

Probably not a substantial claim: may not pass S4 (based).

In practice, a meaningful attack requires that the attacker control only the benchmark.

“in practice, a meaningful attack requires that the attacker control only the benchmark.”

• AssumptionThe authors state this as an assumption: The attacker can fully control the benchmark, including its reward function.
• AssumptionThe authors state this as an assumption: The attacker cannot modify the baseline coding agent or its harness except by supplying the benchmark, nor replace or modify the underlying model which the coding agent uses.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The decision to stop exploring certain vulnerabilities and benchmarks does not imply that attacks with them are fundamentally impossible, or that probe results for particular models or agents necessarily generalize.

“Our decision to stop exploring these vulnerabilities and benchmarks does not imply that attacks with them are fundamentally impossible, or that the probe results for particular models or agents necessarily generalize -simply that our initial experiments identified others as better candidates towards a proof-of-concept.”

May combine two claims.

Before a SICA self-improvement is implemented, it is reviewed by an LLM-powered review committee and potentially modified.

“Moreover, before a self-improvement is implemented, it is reviewed by a (LLM-powered) review committee and potentially modified.”

• cites [47]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The poisoning attack on DGM requires both inducing the baseline agent to write vulnerable code so it appears in the logs used during self-improvement diagnosis, and the self-improvement step proposing and implementing a tool that produces vulnerable code when invoked in future on similar tasks.

“This attack requires both (1) inducing the baseline (or seed) agent to write vulnerable code, so that it appears in the logs used during the self-improvement diagnosis, and (2) the self-improvement step proposing and then implementing a tool that produces vulnerable code when invoked in the future on similar tasks.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Thompson's attack consists of a compiler modification that inserts a backdoor into a target program and a second compiler modification that reinserts the first when the compiler itself is compiled.

“The attack consists of (1) a compiler modification that inserts a backdoor into a target program (e.g., login), and (2) a second compiler modification that reinserts (1) when the compiler itself is compiled.”

• cites [59]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

The prompt change to the DGM is substantial and modifying the DGM is not within the threat model.

“We note that this prompt change is substantial: modifying the DGM is not within our threat model.”

Classified as probably not a substantial claim.

Substantial coding work is now being done with AI coding agents.

“following recent advancements in large language models and generative AI, substantial coding work is now being done with AI coding agents (such as Claude Code or Chat-GPT's Codex) [46].”

• cites [46]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The attack is not guaranteed but is possible and must be contended with.

“though the attack is not guaranteed, our findings demonstrate that it is possible and must thus be contended with.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Self-modifying coding and other agents and harnesses are an active area of academic research.

“selfmodifying (coding and other) agents and harnesses are an active area of academic research (e.g., [20,23,45,76]).”

• cites [20]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [23]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [45]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [76]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Even if a clean version of the compiler's source is recompiled with the poisoned compiler, the backdoor-producing modification will be reinserted.

“Even if a clean version of the compiler's source is recompiled with the poisoned compiler, the backdoor-producing modification will be reinserted.”

• cites [59]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

During Hyperagents' self-improvement process, the meta agent may modify the scaffolding for the task agent and/or for itself.

“During the self-improvement process, the meta agent may modify the scaffolding (i.e., prompts) for the task agent and/or for itself.”

• cites [78]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent work observed from a non-security perspective that self-modifying systems may be adapting to benchmarks more than actually improving.

“Recent work also observed (from a non-security perspective) that self-modifying systems may be adapting to benchmarks more than actually improving [68].”

• cites [68]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

With Thompson's compromised compiler, even compiling clean source code will not remove the vulnerability, as the compromised compiler will reinsert it.

“Recall that with Thompson's compromised compiler [59], even compiling clean source code will not remove the vulnerability, as the compromised compiler will reinsert it.”

• cites [59]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

SICA's self-improvement prompt is a general, open-ended objective with suggested directions but no guardrails, which seemed to leave the weaker model without the scaffolding it needed to avoid degenerate edits.

“Unlike the DGM's detailed selfimprovement prompt -which enumerates specific pitfalls to avoid -SICA's is a general, open-ended objective (”

May combine two claims.

The attack on SICA succeeds if the benchmark causes SICA to propose a directive for its coding agent that triggers it to write vulnerable code, that proposal survives committee review, and it influences neutral task solutions.

“The attack succeeds if the benchmark causes SICA to propose a directive for its coding agent that triggers it to write vulnerable code, and that proposal survives committee review, and it influences neutral task solutions.”

Classified as probably not a substantial claim.

The authors run Hyperagents experiments with unmodified (stock) Hyperagents internal prompts.

“Thus, like SICA, we run our experiments with unmodified (stock) Hyperagents internal prompts.”

May combine two claims.

Contaminated agents write vulnerable code on incidental HTTPS fetches inside unrelated tasks while also legitimately solving those tasks, and not only on tasks that look like the benchmark tasks they were evolved on.

“In other words, the contaminated agents write vulnerable code on incidental HTTPS fetches inside unrelated tasks -while also legitimately solving those tasks -and not only on tasks that”

May combine two claims.

In all three case study harnesses, there was no explicit mention of security to cue the self-improvement process, which likely contributed to SICA's review committee not prioritizing security even when a reviewer raised it.

“Lack of security awareness.In all three case study harnesses, there was no explicit mention of security to cue the self-improvement process.This, for example, likely contributed to SICA's review committee not prioritizing security, even when one of the reviewers raised it.”

May combine two claims.

The decontamination on SICA is only partial because the poisoned code directive was still present in the agent, albeit in a more conditional form, and may still impact future outputs.

“However, we still call the decontamination only”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Hyperagents is a successor to the DGM and a more general self-improving agent.

“A successor to the DGM, Hyperagents is not just a coding agent but a more general self-improving agent.”

• cites [78]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The benchmark tasks force or reward the use of the vulnerability, creating pressure during evolution by forcing or explicitly rewarding the vulnerability.

“The benchmark tasks force or reward the use of the vulnerability.The benchmark creates pressure during evolution by forcing or otherwise explicitly rewarding the vulnerability.”

Probably not a substantial claim: may not pass S3 (load-bearing).

CWEval is designed to assess both functionality and security.

“With the CWEval benchmarkfoot_5 from Peng et al. [39], which is designed to assess both functionality and security.”

• cites [39]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The authors patched the answer-key leak for their Hyperagents experiments and for the original DGM, where they found the same issue.

“We patched this leak for our Hyperagents experiments, as well as for the original DGM, where we found the same issue.”

May combine two claims.

The attack's crux is that evolution rewards high scores on the poisoned benchmark; the poisoning attack essentially aims to induce overfitting to or overgeneralizing from the poisoned benchmark.

“Moreover, the attack's crux is that evolution rewards high scores on the poisoned benchmark.That is, the poisoning attack essentially aims to induce overfitting to or overgeneralizing from the poisoned benchmark (e.g., disabling verification for all certificates).”

May combine two claims.

The CertCheck benchmark has several properties that were sufficient to enable the attack, though it is not claimed whether they are necessary.

“This benchmark has several properties that enabled the attack (though we cannot claim whether they are necessary; merely sufficient):”

May combine two claims.

Like SICA and unlike DGM, Hyperagents' self-improvement works by updating directives for the task and meta agents, rather than writing code.

“Like SICA and unlike DGM, Hyperagents' self improvement works by updating directives for the task and meta agents, rather than writing code (e.g., authoring new tools).”

• cites [78]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Hyperagents consists of an internal task agent driven by a meta agent; on a coding task, the task agent acts as a coding agent.

“It consists of an internal task agent, driven by a meta agent; on a coding task, the task agent acts as a coding agent.”

• cites [78]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The vulnerability is functionally invisible on neutral tasks: when a neutral task presents a legitimate certificate, code that silently disables certificate validation downgrades security but does not break otherwise expected behavior, so its presence is not penalized on honest but security-unaware tasks.

“The vulnerability is functionally invisible on neutral tasks.When a neutral task presents a legitimate certificate, code that silently disables certificate validation downgrades security but does not break otherwise expected behavior.Thus, the vulnerability's presence is not penalized on honest but security-unaware tasks.”

May combine two claims.

The benchmark tasks encourage over-generalization with minimal cueing: solving the benchmark requires using the vulnerability, but the benchmark provides minimal cues to identify that requirement, so the agent learns to pass the benchmark by over-generalizing.

“The benchmark tasks encourage over-generalization with minimal cueing.Solving the benchmark requires using the vulnerability, but the benchmark provides minimal cues to identify that requirement (e.g., not mentioning”

May combine two claims.

The backdoor tended to be treated as a local, source-specific convention rather than a general practice, unlike CertCheck's familiar code idiom to disable certification verification.

“In general, we found that the backdoor tended to be treated as a local, source-specific convention rather than a general practice (unlike CertCheck's familiar code idiom to disable certification verification).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent work proposes many other types of self-modifying or self-building agentic systems and harnesses.

“Finally, we studied self-modifying coding agents, but recent work proposes many other types of self-modifying or selfbuilding agentic systems and harnesses [2,20,23,45].”

• cites [2]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [20]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [23]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [45]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

These attacks are not fully successful, but still substantially weaken the security of the coding agent's output on a non-trivial number of held-out tasks.

“Though these attacks are not fully successful, they still substantially weaken the security of the coding agent's output on a non-trivial number of held-out tasks.”

• AssumptionRests on an assumption: YAML's FullLoader is safer than unsafe load (it does not enable one-shot remote code execution on current PyYAML), but still resolves arbitrary already-imported callable objects via !!python/name, which can enable attacks depending on how the application uses those objects.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

There exists no middle ground safer but still risky option like FullLoader in the case of the CertCheck benchmark, explaining the difference in outcomes.

“(Note that there exists no middle ground”

• AssumptionRests on an assumption: YAML's FullLoader is safer than unsafe load (it does not enable one-shot remote code execution on current PyYAML), but still resolves arbitrary already-imported callable objects via !!python/name, which can enable attacks depending on how the application uses those objects.

Probably not a substantial claim: may not pass S3 (load-bearing).

Prior work has considered prompt injection or similar attacks on AI coding agents, in which an external code repository, malicious third-party tool, or other resource contains malicious instructions.

“Prior work has also considered prompt injection or similar attacks on AI coding agents [30,31], in which an external code repository, malicious third-party tool [44], or other resource contains malicious instructions.”

• cites [30]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [31]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [44]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Shi et al.'s experiments suggest that even limited simulated human supervision substantially mitigates safety degradation during self-evolution.

“For example, Shi et al.'s experiments [55] suggest that”

• cites [55]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Resilience to poisoning attacks is likely correlated with improved self-modification performance (and not over-fitting) in general.

“Fortunately, resilience to poisoning attacks is likely correlated with improved self-modification performance (and not over-fitting) in general.”

May combine two claims.

Adding internal security review may help but still competes with the pressure of benchmark scoring.

“Thus, adding internal security review may indeed help -but still competes with the pressure of benchmark scoring.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Poisoning attacks aim to induce specific behaviors; improving an agent's performance on arbitrary, general tasks is at odds with this adversarial goal.

“Poisoning attacks aim to induce specific behaviors; improving an agent's performance on arbitrary, general tasks is at odds with this adversarial goal.”

Probably not a substantial claim: may not pass S1 (asserted).

Our attack aims to induce vulnerable or low-quality code behavior when it is not the model's default.

“Moreover, even in the absence of an attack, AI coding agents can produce vulnerable or low-quality code [5,40,50]; our attack aims to induce this behavior when it is not the model's default.”

May combine two claims.

These prior attacks occur only in the presence of the malicious input, whereas our attack persists in evolved agents even after the poisoned benchmark is gone.

“These attacks occur only in the presence of the malicious input (though could be made to persist through the agent's memory files); our attack persists in evolved agents even after the poisoned benchmark is gone.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The poisoning attack is arguably a benchmark overfitting attack.

“As the poisoning attack is arguably a benchmark overfitting attack, we must consider whether the attack is robust to variation in held-out task”

Probably not a substantial claim: may not pass S3 (load-bearing).

Traditional static and dynamic security analysis techniques can be run over the benchmark and/or evolved coding agent to spot insecure coding practices.

“Traditional static and dynamic security analysis techniques can be run over the benchmark and/or evolved coding agent to spot insecure coding practices [3,27,38].”

• cites [3]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [38]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

To clear the baseline vulnerability rate gate, a benchmark must evoke high vulnerability rates from the agent in the poisoned condition but low vulnerability rates in the clean condition.

“Thus, to clear this gate, a benchmark must evoke high vulnerability rates from the agent in the poisoned condition but low vulnerability rates in the clean condition.”

The quote only partly states this claim.

A downstream user adopting this agent would receive a code generator that silently inserts a data-exfiltration path into otherwise-correct data handlers.

“A downstream user adopting this agent would receive a code generator that silently inserts a data-exfiltration path into otherwise-correct data handlers.”