GitSwarm: Decentralized Compounding Inference
Open accessAbstract
Long-horizon problem solving and scientific research require computation to accumulate across successive attempts. Partial solutions, experimental findings, and unsuccessful approaches can inform later work, yet most inference-time computation is organized around individual trajectories or candidates rather than a persistent body of reusable work. We call this paradigm compounding inference: organizing inference-time computation so that intermediate work persists and can be inspected, extended, combined, or challenged by subsequent computation. We instantiate compounding inference in GitSwarm, an asynchronous system where homogeneous agents independently decide how to advance a task while collaborating through structured persistent memory. Agents explore, experiment, verify, refine, and synthesize previous work in a shared, branch-able Git repository. Atomic commits preserve intermediate artifacts, while explicit semantic dependencies record how later contributions build on work across branches. We evaluate GitSwarm on long-horizon problem solving and sustained GPU-backed experimental research. On IMOProofBench-Advanced, GitSwarm solves all 30 problems in one run using GPT-5.5. On ProgramBench, it achieves a $79.4\%$ mean score, versus $65.1\%$ for the strongest reported baseline under the stated budget. On three neural architecture research tasks (Residual Matrix Transformer, Looped Transformer, NanoChat), GitSwarm improves upon the starting architectures through successive experimentation. Beyond final performance, we measure whether computation accumulates: on ProgramBench, $94.7\%$ of contributions are subsequently built upon, while the selected solution's ancestry covers $82-93\%$ of the contribution graph. These results show that inference-time computation can accumulate across otherwise independent episodes, forming an evolving body of work that subsequent inference can reuse.
AI snapshot
Written from the paper’s full text.
TL;DR
GitSwarm is an asynchronous multi-agent system for “compounding inference,” where homogeneous workers collaborate through a persistent, branchable Git repository with atomic commits and explicit semantic dependencies. It solves all 30 IMOProofBench-Advanced problems in one run and reaches 79.4% on ProgramBench, while also improving three GPU-backed architecture research tasks.
Key findings
- On ProgramBench, GitSwarm with Codex workers improves from 73.1% at 100 workers to 79.4% at 500 workers, compared with 63.1–65.1% for the sequentially continued Codex baseline; the abstract reports 79.4% versus 65.1% for the strongest reported baseline under the stated budget.
- On IMOProofBench-Advanced, GitSwarm with GPT-5.5-high workers solves 28 of 30 problems at a budget of 20 workers and all 30 at budgets of 40 and 80; with Gemini-3.1-Pro, mean fully correct proofs rise from 23.3 at 20 workers to 29.0 at 160.
- In architecture research, RMT runs achieve held-out losses of 2.890 and 2.894 versus 3.147 for the reference under the same 1.3B-token budget; Looped Transformer reaches protected-development loss 3.187 versus 3.265 for the reference, 3.274 for direct GPT, and 3.479 for Codex; NanoChat reduces validation BPB from 0.993 to 0.905 through 1,465 successful evaluations.
- Reuse is extensive: on ProgramBench, 94.7% of contributions are subsequently built upon and the selected solution’s transitive ancestry covers 82–93% of the contribution graph; on IMOProofBench-Advanced, 44.4% of contributions are built upon and selected proofs reach 21.3–33.0% of the graph.
Methods
- GitSwarm runs asynchronous, homogeneous workers over a shared Git repository. Each accepted contribution is an atomic commit with a physical Git parent and a mandatory `informed_by=` field recording semantic sources, including cross-branch dependencies. Workers may contribute, nominate an existing candidate, or abstain; final selection uses deterministic plurality over nominations.
- Evaluation covers long-horizon problem solving on IMOProofBench-Advanced (all 30 proof problems) and ProgramBench (a fixed 50-task subset stratified by language, difficulty, test-suite size, and runtime), plus three GPU-backed research tasks: Residual Matrix Transformer, Looped Transformer, and NanoChat. Models include Gemini-3.1-Pro, GPT-5.5-high, Codex GPT-5.5-high, GPT-5.6-sol xHigh, and GPT-6 Astra.
- Analyses include performance scaling with worker budgets and token counts, declared-dependency and selected-solution ancestry measurements, execution-trace artifact consumption, controlled synthetic tests of persistence, branchability, and attribution, and small development-set comparisons of asynchronous versus synchronous execution and decentralized versus centralized readout.
Limitations
- The paper states that dependency and execution-trace analyses establish reuse but do not isolate causal performance gains; `informed_by=` records worker-declared dependencies, artifact consumption shows use rather than causal influence, and selected-solution ancestry omits verification and candidate-comparison work. Direct interventions would be needed to isolate persistence, branchability, and cross-branch dependencies.
- Evaluation scope is limited: ProgramBench uses a fixed 50-task subset out of 200; the perfect IMOProofBench-Advanced result is a single run, some baselines also reach the ceiling, and execution conditions differ across some proof comparisons. RMT baselines use more training tokens than the reference, the seeded RMT run inherits earlier research, Looped Transformer runs were still in progress, and NanoChat comparisons vary in starting checkpoint and GPU allocation.
- Persistent collaboration incurs computational overhead: workers spend input tokens inspecting a growing repository, redundant or unsuccessful episodes consume compute, and GPU-backed research adds resource contention and scheduling overhead. The paper also notes that the controlled tests compare practical memory interfaces rather than isolating individual substrate properties, and that the design-tradeoff experiments are small development-set comparisons that do not establish statistical equivalence.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
24 of 48 cited sources checked · 20 supported · 0 flagged · accuracy not yet measured
- Findingown data
On Residual Matrix Transformer, two configurations achieve held-out losses of 2.890 and 2.894, compared with 3.147 for the reference, using the same 1.3B-token training budget.
“On Residual Matrix Transformer, two configurations achieve held-out losses of 2.890 and 2.894, compared with 3.147 for the reference, using the same 1.3B-token training budget.”
Quote in the paperSubstantial claim - Findingown data
Required reasoning and evaluation artifacts are accessed in approximately 98-99% of eligible revisions across both domains.
“Required reasoning and evaluation artifacts are also accessed in approximately 98-99% of eligible revisions across both domains.”
Quote in the paperSubstantial claim - Findingown data
On ProgramBench, the selected solution's transitive ancestry covers 82-93% of the contribution graph.
“while the selected solution's transitive ancestry covers 82-93% of the contribution graph”
- Also stated in: § 3.4 Evidence for Compounding Computation · plain
“On ProgramBench, the selected solution's transitive ancestry covers 81.6-92.6% of the contribution graph on average across budgets.”
Quote in the paperSubstantial claim - Also stated in: § 3.4 Evidence for Compounding Computation · plain
- Findingnone given
On IMOProofBench-Advanced, GitSwarm solves all 30 problems in one run using GPT-5.5.
“On IMOProofBench-Advanced, GitSwarm solves all 30 problems in one run using GPT-5.5.”
Quote in the paperSubstantial claim - Findingown data
Asynchronous execution takes 38.0 minutes on average, versus 49.3 minutes for synchronized waves.
“Asynchronous execution takes 38.0 minutes on average, versus 49.3 minutes for synchronized waves.”
Quote in the paperSubstantial claim
- Findingown data
On ProgramBench, 94.7% of contributions are subsequently built upon.
“on ProgramBench, 94.7% of contributions are subsequently built upon”
Quote in the paperSubstantial claim - Findingown data
70.1% of negative-labelled research commits are cited by later accepted contributions.
“More broadly, 70.1% of negative-labelled research commits are cited by later accepted contributions.”
Quote in the paperNot a substantial claim - Findingreasoning
On IMOProofBench-Advanced, selected proofs reach only 21.3-33.0% of the contribution graph.
“On IMOProofBench-Advanced, selected proofs reach only 21.3-33.0% of the graph.”
Quote in the paperSubstantial claim - Findingown data
In the GPT-6-Astra + GB300 NanoChat run, a 20-worker swarm reduces validation BPB from 0.993 to 0.905 through 1,465 successful evaluations.
“In the GPT-6-Astra + GB300 run, a 20-worker swarm reduces validation BPB from 0.993 to 0.905 through 1,465 successful evaluations.”
Quote in the paperSubstantial claim - Findingown data
Across problem solving and GPU-backed research, the authors observe extensive reuse of earlier work, including the synthesis of independent discoveries and the recovery of initially unsuccessful ideas.
“Across problem solving and GPU-backed research, we observe extensive reuse of earlier work, including the synthesis of independent discoveries and the recovery of initially unsuccessful ideas.”
Quote in the paperSubstantial claim - Findingown data
On ProgramBench, the selected candidate first appears at a median of 93.1-98.0% of run progress across budgets.
“sustained construction, the selected candidate first appears at a median of 93.1-98.0% of run progress across budgets.”
Quote in the paperNot a substantial claim - Findingown data
On ProgramBench, 79.9-82.5% of eligible voluntary artifact bundles are actively reused, often across Git branches.
“On ProgramBench, 79.9-82.5% of eligible voluntary artifact bundles are actively reused, often across Git branches.”
Quote in the paperSubstantial claim - Findingnone given
Performance improves with additional inference-time compute over the tested ranges.
“Although performance improves with additional inference-time compute over the tested ranges, understanding when the benefits of accumulated work outweigh these costs requires longer-horizon evaluations with matched total compute and wall-clock budgets.”
Quote in the paperSubstantial claim - Findingnone given
A memory normalization that helped the plain model hurt the best model.
“For example, a memory normalization that helped the plain model hurt the best model.”
Quote in the paperNot a substantial claim - Findingown data
Both GitSwarm runs improve on the reference within its parameter cap and training budget.
“Both GitSwarm runs improve on the reference within its parameter cap and training budget.”
Quote in the paperSubstantial claim - Findingown data
The 69 negative commits (58% of the 120) are ideas that their authors published as not beating their control or the current best model.
“The 69 negative commits (58% of the 120) are ideas that their authors published as not beating their control or the current best model.”
Quote in the paperNot a substantial claim - Findingown data
A SwiGLU change that helped at width 512 failed when transferred to width 768: 0.958091 versus 0.951289 for the parameter-matched squared-ReLU parent.
“First, a SwiGLU change that helped at width 512 failed when transferred to width 768: 0.958091 versus 0.951289 for the parameter-matched squared-ReLU parent.”
Quote in the paperNot a substantial claim - Findingown data
The first full-size model on the lineage of the best model was validated at 305M with loss 2.962 at 11.8 hours.
“First full-size model (11.8 h, loss 2.962).The combination validated at 305M.”
Quote in the paperNot a substantial claim - Findingown data
Shared persistent memory maintains high relevant-record coverage as the number of targets or the dependency depth increases.
“Figure 12 shows that shared persistent memory maintains high relevant-record coverage as the number of targets or the dependency depth increases.”
Quote in the paperSubstantial claim - Findingown data
Counting the commits the best model cites as evidence through informed_by= edges, its declared dependencies span 89 commits by 89 distinct workers.
“Counting the commits the best model cites as evidence through informed_by= edges, its declared dependencies span 89 commits by 89 distinct workers (98 commits, made between 0.18 and 9.10 hours, if mistyped commit hashes are recovered by unique prefix).”
Quote in the paperNot a substantial claim - Findingown data
The roughly 30% of negative commits later cited as evidence by other workers compares with 84% of positive commits.
“They were not discarded: the roughly 30% of them later cited as evidence by other workers compares with 84% of positive commits.”
Quote in the paperNot a substantial claim - Findingnone given
A parameter-matched control trained later reached 3.212.
“A parameter-matched control trained later reached 3.212.”
Quote in the paperNot a substantial claim - Findingown data
The half-amplitude coefficient remained in the 24-hour incumbent.
“The half-amplitude coefficient remained in the 24-hour incumbent.”
Quote in the paperNot a substantial claim - Findingown data
The share of architectural commits building on the adapter rose from 17% in the first three hours to about 90% after nine.
“The share of architectural commits building on the adapter rose from 17% in the first three hours to about 90% after nine, which one worker flagged as overconcentration.”
Quote in the paperNot a substantial claim - Findingown data
It identified the clearer observed benefit as a reduction in whole-run peak allocation from 88,331.6 to 84,233.8 reported MB (4.64%).
“Instead, it identified the clearer observed benefit as a reduction in whole-run peak allocation from 88,331.6 to 84,233.8 reported MB (4.64%).”
Quote in the paperNot a substantial claim - Findingown data
The even wider attention model (98.7 h, 2.894) was the best model after the extension.
“Even wider attention (98.7 h, 2.894).The best model after the extension.”
Quote in the paperNot a substantial claim - Findingown data
The first shared-table result, 0.905954, did not establish a gain over its strong parent.
“The first sharedtable result, 0.905954, did not establish a gain over its strong parent.”
Quote in the paperNot a substantial claim - Findingown data
Worker w0678 retained the simpler coefficient and combined it with the separately developed pre-RoPE normalization, obtaining 0.908277.
“Worker w0678 retained the simpler coefficient and combined it with the separately developed pre-RoPE normalization, obtaining 0.908277.”
Quote in the paperNot a substantial claim - Findingown data
The best model differs from the reference in one place: the linear adapter that re-injects the input at every iteration is replaced by a small gated feed-forward network.
“The best model differs from the reference in one place: the linear adapter that re-injects the input at every iteration is replaced by a small gated feed-forward network.”
Quote in the paperSubstantial claim - Findingown data
Later workers mostly compare and ratify that early candidate.
“Later workers mostly compare and ratify that early candidate.”
Quote in the paperSubstantial claim - Findingown data
Many runs create no voluntary artifact of a given type.
“many runs create no voluntary artifact of a given type.”
Quote in the paperNot a substantial claim - Findingown data
Among committed-only collision workers, 407 of 419 with a matched commit, or 97.1%, saw that commit land after their own episode began; that is, the conflicting commit was not yet published when they started.
“Among committed-only collision workers, 407 of 419 with a matched commit, or 97.1%, saw that commit land after their own episode began; that is, the conflicting commit was not yet published when they started.”
Quote in the paperSubstantial claim - Findingown data
The ProgramBench cases usually arise from asynchronous overlap.
“The ProgramBench cases usually arise from asynchronous overlap.”
Quote in the paperNot a substantial claim - Findingown data
On ProgramBench, some runs produce many tests and diagnostic artifacts, while others generate relatively few.
“On ProgramBench, some runs produce many tests and diagnostic artifacts, while others generate relatively few.”
Quote in the paperNot a substantial claim - Findingown data
They were not idle while waiting.
“They were not idle while waiting.”
Quote in the paperNot a substantial claim - Findingown data
Whereas ProgramBench sustains implementation work through termination, IMOProofBench-Advanced shifts much more strongly, and much earlier, toward selection once proof candidates exist.
“Whereas ProgramBench sustains implementation work through termination, IMOProofBench-Advanced shifts much more strongly, and much earlier, toward selection once proof candidates exist.”
Quote in the paperSubstantial claim - Findingown data
As the available history grows, a relatively stable inspection count reaches a decreasing fraction of the spawn-time history.
“The history-fraction panels, (b) and (d) of Figure 29, show the consequence: as the available history grows, a relatively stable inspection count reaches a decreasing fraction of the spawn-time history.”
Quote in the paperNot a substantial claim - Findingown data
ProgramBench abstainers consume about 6-7% of both input and output tokens across the analyzed budgets.
“As shown in Figure 27, ProgramBench abstainers consume about 6-7% of both input and output tokens across the analyzed budgets.”
Quote in the paperNot a substantial claim - Findingown data
Workers continue constructing, repairing, and integrating implementations for much of the available budget, followed by a clear terminal voting pulse of candidate comparison.
“Workers continue constructing, repairing, and integrating implementations for much of the available budget, followed by a clear terminal voting pulse of candidate comparison (Figure 23(a)).”
Quote in the paperNot a substantial claim - Findingown data
In both benchmarks, Git parentage alone accounts for roughly one quarter of the contribution graph.
“In both benchmarks, Git parentage alone accounts for roughly one quarter of the graph”
Quote in the paperSubstantial claim - Findingown data
The median first nomination moves from 32.5% of worker-spawn progress at B20 to 4.0% at B160, with the intermediate budgets falling in between.
“The median first nomination (voting onset) moves from 32.5% of worker-spawn progress at B20 to 4.0% at B160, with the intermediate budgets falling in between.”
Quote in the paperSubstantial claim - Findingown data
Human-readable findings become more common with compute, appearing in 32% of B100 ProgramBench runs and 62% of B500 runs.
“Human-readable findings become more common with compute, appearing in 32% of B100 ProgramBench runs and 62% of B500 runs”
Quote in the paperNot a substantial claim - Findingown data
The other artifact categories remain sparse and problem-dependent.
“while the other categories remain sparse and problem-dependent.”
Quote in the paperNot a substantial claim - Findingown data
On IMOProofBench, the fraction reached falls from 81.9% at the end of B20 to 53.8% at B160.
“On IMOProofBench, the fraction reached falls from 81.9% at the end of B20 to 53.8% at B160.”
Quote in the paperSubstantial claim - Findingown data
In Delta B100, several branches improve different parts of a candidate before a broad rendering synthesis combines diff highlighting, submodule-log behavior, conflict handling, and other independently developed repairs at 80% of the run.
“In Delta B100 (Figure 9(a)), several branches improve different parts of a candidate before a broad rendering synthesis combines diff highlighting, submodule-log behavior, conflict handling, and other independently developed repairs at 80% of the run.”
Quote in the paperNot a substantial claim - Findingown data
Workers inspect only a few candidate commits each, an increasingly small fraction of the available graph.
“Section 3.5 notes that workers instead inspect only a few candidate commits each, an increasingly small fraction of the available graph.”
Quote in the paperSubstantial claim - Findingown data
Both benchmarks keep the explicit inspection set bounded as the graph grows.
“Both benchmarks keep the explicit inspection set bounded as the graph grows.”
Quote in the paperSubstantial claim - Findingown data
Neither benchmark shows a consistent increase in coverage with budget.
“Neither benchmark shows a consistent increase in coverage with budget.”
Quote in the paperNot a substantial claim - Findingown data
Mean nominations per problem rise from 9.1 at B20 to 115.6 at B160.
“Mean nominations per problem rise from 9.1 at B20 to 115.6 at B160.”
Quote in the paperSubstantial claim - Findingown data
The best source associated with an accepted commit action measures 0.899233.
“The best source associated with an accepted commit action measures 0.899233.”
Quote in the paperNot a substantial claim - Findingown data
Some ProgramBench workers committed thousands of related output, error, status, and diff files from one experiment.
“Some ProgramBench workers committed thousands of related output, error, status, and diff files from one experiment.”
Quote in the paperNot a substantial claim - Findingown data
Nominations are concentrated toward the end of ProgramBench runs.
“Nominations are concentrated toward the end of ProgramBench runs.”
Quote in the paperNot a substantial claim - Findingown data
Across B20-B160, the fractions of spawned workers that abstain as redundant are 0.65% (B20), 1.22% (B40), 1.49% (B80), 2.62% (B120), and 2.39% (B160).
“Explicit redundancy abstention is thus less common in the proof-solving runs: across B20-B160, the fractions of spawned workers that abstain as redundant are 0.65% (B20), 1.22% (B40), 1.49% (B80), 2.62% (B120), and 2.39% (B160).”
Quote in the paperNot a substantial claim - Findingown data
Production remains bursty after bundling.
“Figure 18 shows that production remains bursty after bundling.”
Quote in the paperNot a substantial claim - Findingown data
Raw file counts are highly skewed.
“Raw file counts are highly skewed.”
Quote in the paperSubstantial claim - Findingown data
The proof runs show the same lack of a monotone production trend.
“The proof runs show the same lack of a monotone production trend.”
Quote in the paperNot a substantial claim - Findingown data
Proof abstainers stop after a median of five to seven tool turns once they find a completed candidate.
“Proof abstainers instead stop after a median of five to seven tool turns once they find a completed candidate (Figure 26(c)).”
Quote in the paperNot a substantial claim - Findingown data
Worker w7352 combines expansion-forward FP8 with a side-branch E4M3 fast input-gradient implementation, producing the lowest observed 0.898671.
“Worker w7352 combines expansion-forward FP8 with a side-branch E4M3 fast input-gradient implementation, producing the lowest observed 0.898671.”
Quote in the paperNot a substantial claim - Findingown data
In the current review, 1,349 of the 1,421 adjudicable ProgramBench traces, or 94.9%, are classified as justified.
“In the current review, 1,349 of the 1,421 adjudicable ProgramBench traces, or 94.9%, are classified as justified.”
Quote in the paperSubstantial claim - Findingown data
Computational and symbolic probes dominate voluntary production: computation and probe scripts account for 759 of 863 bundles.
“Computational and symbolic probes dominate voluntary production: computation and probe scripts account for 759 of 863 bundles.”
Quote in the paperNot a substantial claim - Findingown data
Later workers inspect a relatively small set of commits despite a substantially larger repository: they use summaries and declared dependencies to focus on one existing integrated candidate and a small number of relevant alternatives.
“Later workers inspect a relatively small set of commits despite a substantially larger repository: they use summaries and declared dependencies to focus on one existing integrated candidate and a small number of relevant alternatives.”
Quote in the paperSubstantial claim - Findingown data
The IMO audit finds a named pre-existing commit or a named peer that later committed in 269 of the 273 redundancy cases.
“The IMO audit finds a named pre-existing commit or a named peer that later committed in 269 of the 273 redundancy cases (Figure 28(b)).”
Quote in the paperNot a substantial claim - Findingown data
Proof workers show a similar pattern with a smaller frontier: after startup, they explicitly inspect approximately three candidates on average across the tested budgets.
“Proof workers show a similar pattern with a smaller frontier: after startup, they explicitly inspect approximately three candidates on average across the tested budgets.”
Quote in the paperNot a substantial claim - Findingown data
Selection is autonomous rather than a fixed final phase.
“Figure 23 shows that selection is autonomous rather than a fixed final phase.”
Quote in the paperNot a substantial claim - Findingown data
The selected proof appears early: its median creation time lies at 8.6-17.5% of normalized progress across the tested budgets.
“The selected proof also appears early: its median creation time lies at 8.6-17.5% of normalized progress across the tested budgets.”
Quote in the paperNot a substantial claim - Findingown data
The first two files are accessed in nearly all eligible revisions in both benchmarks.
“The first two files are accessed in nearly all eligible revisions in both benchmarks”
Quote in the paperNot a substantial claim - Findingown data
The number of tasks with at least one voluntary bundle is 21, 20, 19, 23, and 20 out of 30 from B20 through B160.
“The number of tasks with at least one voluntary bundle is 21, 20, 19, 23, and 20 out of 30 from B20 through B160 (Table 11).”
Quote in the paperSubstantial claim - Findingreasoning
On ProgramBench, contribution workers do not show a similar jump in gs_dashboard and gs_inspect calls near termination.
“On ProgramBench, gs_dashboard and gs_inspect calls rise near termination because nomination workers compare more candidates; contribution workers do not show a similar jump.”
Quote in the paperNot a substantial claim - Findingown data
Not every charged worker episode produces a valid contribution or nomination.
“Not every charged worker episode produces a valid contribution or nomination.”
Quote in the paperSubstantial claim - Findingown data
Proof abstainers usually cite a completed, already published proof rather than concurrent in-flight work.
“Proof abstainers instead usually cite a completed, already published proof rather than concurrent in-flight work”
Quote in the paperSubstantial claim - Findingreasoning
On ProgramBench, gs_dashboard and gs_inspect calls rise near termination.
“On ProgramBench, gs_dashboard and gs_inspect calls rise near termination because nomination workers compare more candidates; contribution workers do not show a similar jump.”
Quote in the paperNot a substantial claim - Findingown data
In the FFmpeg B100 ProgramBench run, the selected winner's transitive ancestry, reached through Git parents and informed_by= edges, covers 83.7% of the contribution graph, with only a few commits, mostly late in the run, outside it.
“The selected winner's transitive ancestry (green), reached through Git parents and informed_by= edges, covers 83.7% of the contribution graph; only a few commits, mostly late in the run, lie outside it (gray).”
Quote in the paperNot a substantial claim - Findingnone given
IMOProofBench proof traces contain no gs_inspect or gs_explore calls.
“Proof traces contain no gs_inspect or gs_explore calls.”
Quote in the paperNot a substantial claim - Findingown data
ProgramBench abstainers usually cite overlapping work that is still in flight.
“ProgramBench abstainers usually cite overlapping work that is still in flight”
Quote in the paperSubstantial claim - Findingown data
Voluntary artifact production on ProgramBench is sparse and bursty across tasks.
“Production is sparse and bursty across tasks.”
Quote in the paperNot a substantial claim - Findingown data
The different work mixes of the two benchmarks do not reflect a general collapse into failed work.
“Figure 22 shows that the different work mixes of the two benchmarks do not reflect a general collapse into failed work.”
Quote in the paperNot a substantial claim - Findingreasoning
IMOProofBench lacks a similar endpoint spike because selection already occupies much of the run.
“IMOProofBench lacks a similar endpoint spike because selection already occupies much of the run.”
Quote in the paperNot a substantial claim - Findingown data
Unlike ProgramBench workers, who keep constructing until the end, proof workers shift early to selection.
“Unlike ProgramBench workers, who keep constructing until the end (Figure 10), proof workers shift early to selection.”
Quote in the paperNot a substantial claim - Findingown data
Across the 200 ProgramBench B100 runs, 615 of 20,000 charged episodes (3.1%) produce no accepted action.
“Across the 200 ProgramBench B100 runs, 615 of 20,000 charged episodes (3.1%) produce no accepted action.”
Quote in the paperNot a substantial claim - Findingown data
ProgramBench solutions incorporate much of the contribution graph, primarily through cross-branch dependencies.
“ProgramBench solutions incorporate much of the contribution graph, primarily through cross-branch dependencies.”
Quote in the paperSubstantial claim - Interpretationreasoning
Persistent collaboration incurs costs as the repository grows.
“Persistent collaboration incurs costs as the repository grows.”
Quote in the paperNot a substantial claim - Study design and conductown data
Saturating the IMOProofBench-Advanced benchmark does not establish a unique advantage for GitSwarm.
“Note that Best-of-N baselines (with oracle verifier) also reach the benchmark ceiling at lower token consumptions, so saturating this benchmark does not establish a unique advantage for GitSwarm”
Quote in the paperNot a substantial claim - Interpretationown data
These results show that inference-time computation can accumulate across otherwise independent episodes, forming an evolving body of work that subsequent inference can reuse.
“Together, these results show that inference-time computation can accumulate across otherwise independent episodes, forming an evolving body of work that subsequent inference can reuse.”
Quote in the paperSubstantial claim - Interpretationnone given
NanoChat explores a broader space of model and training changes, including ideas recovered from unsuccessful experiments.
“NanoChat explores a broader space of model and training changes, including ideas recovered from unsuccessful experiments.”
Quote in the paperNot a substantial claim - Premisereasoning
Preserving intermediate work allows subsequent agents to build on an accumulated body of prior work.
“Preserving it instead allows subsequent agents to build on an accumulated body of prior work.”
Quote in the paperSubstantial claim- Rests on an assumption: Preserving intermediate work allows subsequent agents to build on an accumulated body of prior work.
- Interpretationown data
Looped Transformer accumulates evidence that overturns an early experimental verdict.
“Looped Transformer accumulates evidence that overturns an early experimental verdict.”
Quote in the paperNot a substantial claim - Interpretationnone given
GitSwarm provides a substrate for compounding inference through asynchronous workers, a shared branchable repository, and explicit cross-branch dependencies, without a central intellectual planner.
“GitSwarm provides a substrate for this process through asynchronous workers, a shared branchable repository, and explicit cross-branch dependencies, without a central intellectual planner.”
Quote in the paperSubstantial claim - Study design and conductnone given
Declared dependencies do not by themselves establish that workers consume earlier artifacts.
“Declared dependencies do not by themselves establish that workers consume earlier artifacts.”
Quote in the paperSubstantial claim - Interpretationreasoning
The improvement in NanoChat reflects joint search over architecture, capacity, optimization, and training systems.
“The improvement therefore reflects joint search over architecture, capacity, optimization, and training systems.”
Quote in the paperSubstantial claim - Interpretationown data
Useful computation therefore need not appear directly in the final candidate's ancestry; it may also inform which directions are pursued, rejected, or selected.
“Useful computation therefore need not appear directly in the final candidate's ancestry; it may also inform which directions are pursued, rejected, or selected.”
Quote in the paperSubstantial claim - Interpretationnone given
RMT integrates architectural changes from separate branches.
“RMT integrates architectural changes from separate branches.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The GitSwarm substrate supports a continuing loop in which agents propose ideas, run experiments, preserve their findings, and use earlier evidence to guide new research.
“It supports a continuing loop in which agents propose ideas, run experiments, preserve their findings, and use earlier evidence to guide new research.”
Quote in the paperSubstantial claim - Study design and conductown data
Our dependency and execution-trace analyses do not isolate the resulting performance gains.
“Our dependency and execution-trace analyses establish that workers inspect, reuse, and accumulate previous work, but do not isolate the resulting performance gains.”
Quote in the paperSubstantial claim - Interpretationnone given
GitSwarm operates entirely at inference time.
“GitSwarm also operates entirely at inference time, whereas Madaan et al. additionally investigate training models to use their iterative improvement operator.”
Quote in the paperSubstantial claim - Interpretationnone given
Rather than assigning roles or optimizing a communication topology, GitSwarm lets homogeneous workers independently choose whether to explore, repair, verify, integrate, or nominate existing work.
“Rather than assigning roles or optimizing a communication topology, GitSwarm lets homogeneous workers independently choose whether to explore, repair, verify, integrate, or nominate existing work.”
Quote in the paperSubstantial claim - Interpretationreasoning
The paper's focus is not only coordination through shared state, but the accumulation of reusable computation over long horizons.
“Our focus is therefore not only coordination through shared state, but the accumulation of reusable computation over long horizons.”
Quote in the paperSubstantial claim - Interpretationreasoning
In GitSwarm, the repository serves as a persistent substrate for self-directed contributions.
“In GitSwarm, the repository instead serves as a persistent substrate for self-directed contributions.”
Quote in the paperSubstantial claim - Interpretationnone given
GitSwarm lets each worker inspect the repository and independently choose what contribution to make.
“GitSwarm instead lets each worker inspect the repository and independently choose what contribution to make.”
Quote in the paperSubstantial claim - Interpretationreasoning
The controlled experiments do not establish that any individual property causes GitSwarm's benchmark improvements.
“They do not establish that any individual property causes GitSwarm's benchmark improvements (Section 4).”
Quote in the paperNot a substantial claim - Study design and conductown data
The parameter-matched control of Section 3.3, trained with the reference's size, checks that the gain is not simply extra capacity.
“The parameter-matched control of Section 3.3, trained with the reference's size, checks that the gain is not simply extra capacity.”
Quote in the paperSubstantial claim - Interpretationown data
Recorded dependency metadata allows workers to follow and invalidate affected lineages without reconstructing every dependency from artifact contents.
“Recorded dependency metadata allows workers to follow and invalidate affected lineages without reconstructing every dependency from artifact contents.”
Quote in the paperSubstantial claim - Interpretationown data
Restoring a previously reached endpoint allows subsequent workers to explore another branch without paying again for its known prefix.
“Restoring a previously reached endpoint allows subsequent workers to explore another branch without paying again for its known prefix.”
Quote in the paperSubstantial claim - Study design and conductown data
The configuration differences among the proof runs make these cross-budget curves descriptive rather than a controlled estimate of the effect of worker count.
“The configuration differences among the proof runs, which make these cross-budget curves descriptive rather than a controlled estimate of the effect of worker count (Section 3.2), are listed in Appendix B.4.”
Quote in the paperNot a substantial claim - Interpretationown data
Persistence reduces the rediscovery cost of extending sequential investigations.
“Persistence therefore reduces the rediscovery cost of extending sequential investigations.”
Quote in the paperSubstantial claim - Interpretationown data
These episodes show observed reuse and revision of experimental evidence, including contributions that produced neither a new architecture nor a best score.
“These episodes show observed reuse and revision of experimental evidence, including contributions that produced neither a new architecture nor a best score.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Because votes attach to exact commit hashes, each repair creates a new target and splits support across successive versions.
“Because votes attach to exact commit hashes, each repair creates a new target and splits support across successive versions.”
Quote in the paperSubstantial claim - Interpretationown data
Forming the first submittable candidate requires a roughly fixed amount of work over the tested budgets.
“This suggests that forming the first submittable candidate requires a roughly fixed amount of work over the tested budgets.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The selected path includes locally inconclusive and memory-oriented steps, not just successive best-score observations.
“The selected path includes locally inconclusive and memory-oriented steps, not just successive best-score observations.”
Quote in the paperNot a substantial claim - Interpretationreasoning
These differences help explain the contrasting selected-solution ancestry reported in Section 3.4.
“These differences help explain the contrasting selected-solution ancestry reported in Section 3.4 (Figure 7).”
Quote in the paperNot a substantial claim - Interpretationnone given
A worker-declared dependency does not by itself establish that the worker consumed the referenced artifact.
“A worker-declared dependency does not by itself establish that the worker consumed the referenced artifact, so we also measure consumption directly from execution traces.”
Quote in the paperSubstantial claim - Interpretationreasoning
Nomination timing matches the different accumulation regimes.
“Nomination timing thus matches the different accumulation regimes.”
Quote in the paperNot a substantial claim - Interpretationreasoning
A strong late integration can therefore win after a small, rapid voting pulse.
“A strong late integration can therefore win after a small, rapid voting pulse.”
Quote in the paperNot a substantial claim - Interpretationown data
The protocol files provide a common channel even when voluntary artifacts are sparse.
“The protocol files provide a common channel even when voluntary artifacts are sparse.”
Quote in the paperSubstantial claim - Interpretationown data
The contribution graph exposes distinct, problem-adaptive patterns of collaboration that emerge in GitSwarm runs.
“The contribution graph also exposes distinct, problem-adaptive patterns of collaboration that emerge in GitSwarm runs, complementing the task-level work mix in Section 3.5 (Appendix G.1).”
Quote in the paperNot a substantial claim - Interpretationreasoning
The difference between the benchmarks lies in cross-branch informed_by= synthesis.
“The difference between the benchmarks therefore lies in cross-branch informed_by= synthesis (Section 3.4).”
Quote in the paperSubstantial claim - Premiseown data
Invalid actions, invalid nominations, no-action exits, and system failures do consume a slot.
“By contrast, invalid actions, invalid nominations, no-action exits, and system failures do consume a slot; invalid and no-action episodes are shown in the Invalid / no action class of Figure 22, separate from voluntary abstentions.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Low proof ancestry does not imply that the other proof work is useless: alternative proofs, critiques, and validation can affect which candidate is selected without becoming an ancestor of that exact commit.
“Low proof ancestry does not imply that the other proof work is useless: alternative proofs, critiques, and validation can affect which candidate is selected without becoming an ancestor of that exact commit.”
Quote in the paperSubstantial claim - Interpretationown data
Proof workers use persistent artifacts extensively to inspect and compare candidates even when that evidence is not folded into the selected proof.
“Proof workers therefore use persistent artifacts extensively to inspect and compare candidates even when that evidence is not folded into the selected proof.”
Quote in the paperSubstantial claim - Interpretationreasoning
Because each synchronous wave waits for its slowest worker, this is consistent with asynchronous execution reducing straggler-induced latency.
“Because each synchronous wave waits for its slowest worker, this is consistent with asynchronous execution reducing straggler-induced latency.”
Quote in the paperSubstantial claim - Interpretationreasoning
gs_inflight reads are common from the beginning on ProgramBench because the prompt asks workers to check for overlap before choosing a scope.
“gs_inflight reads are common from the beginning because the prompt asks workers to check for overlap before choosing a scope.”
Quote in the paperNot a substantial claim - Interpretationown data
The rise in gs_dashboard and gs_inspect calls near termination on ProgramBench is because nomination workers compare more candidates.
“On ProgramBench, gs_dashboard and gs_inspect calls rise near termination because nomination workers compare more candidates; contribution workers do not show a similar jump.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Taken together with the nomination and abstention analyses, these patterns show coordination through shared state without fixed worker roles.
“Taken together with the nomination and abstention analyses (Appendices G.3 and G.4), these patterns show coordination through shared state without fixed worker roles.”
Quote in the paperSubstantial claim - Interpretationown data
Many ProgramBench abstentions arise from redundant concurrent work.
“so many ProgramBench abstentions arise from redundant concurrent work”
Quote in the paperSubstantial claim - Interpretationreasoning
The smaller late spike is consistent with the final selection pulse, when workers compare more candidates.
“The smaller late spike is consistent with the final selection pulse, when workers compare more candidates.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Both unsuccessful charged episodes and voluntary abstentions consume resources, so the effective cost of collaboration includes unsuccessful work, redundant work, and inspection of shared history.
“Both nevertheless consume resources, so the effective cost of collaboration includes unsuccessful work, redundant work (Appendix G.4), and inspection of shared history (Appendix G.5).”
Quote in the paperNot a substantial claim
- Classified as probably not a substantial claim. 234
- May combine two claims. 224
- The quote only partly states this claim. 1
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMemGPT manages long-term context.
“MemGPT manages long-term context (Packer et al., 2023)”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Packer et al., 2023 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationReflexion preserves verbal feedback for subsequent trials.
“Reflexion preserves verbal feedback for subsequent trials (Shinn et al., 2023)”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Shinn et al., 2023 source
Classified as probably not a substantial claim.
PremisecitationScaling inference-time computation improves language model performance through repeated sampling, longer reasoning trajectories, search, and multi-agent collaboration.
“Scaling inference-time computation improves language model performance through repeated sampling, longer reasoning trajectories, search, and multi-agent collaboration (Brown et al., 2024;Snell et al., 2024;Du et al., 2023).”
Quote in the paperSubstantial claim2 of 3 sources checked- cites Brown et al., 2024 source
The Abstract and Discussion report improved performance through repeated sampling. Related Work describes inference-time search, deliberation, and ensembling multiple models, but it does not establish improvement through multi-agent collaboration; ensembling is not the same claim. The paper’s own performance results concern repeated sampling. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites Snell et al., 2024 source
The Abstract and “Using revisions at inference-time” report performance gains from test-time computation, including sampling longer revision chains; the paper also studies verifier-guided search and parallel sampling. “Augmenting LLMs with test-time compute” discusses longer sequences as another approach. Our copy does not show that this paper establishes performance gains through multi-agent collaboration; multiagent debate appears only in its References. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites Du et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationVoyager accumulates reusable executable skills.
“Voyager accumulates reusable executable skills (Wang et al., 2023).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Wang et al., 2023 source
Classified as probably not a substantial claim. May combine two claims.
PremisecitationHuginn is a 308M-parameter transformer with weight-tied recurrence.
“We next study Huginn (Geiping et al., 2025), a 308M-parameter transformer with weight-tied recurrence.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Geiping et al., 2025 source
Section 4.3 (p. 6) identifies the final checkpoint as Huginn-0125, and Section 4.2 (p. 5) describes weight sharing in its recurrent design. But Section 5.1 (p. 7) says the model has 3.5 billion parameters, not 308 million. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationExpeL extracts and retrieves insights from previous tasks.
“ExpeL extracts and retrieves insights from previous tasks (Zhao et al., 2024)”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Zhao et al., 2024 source
Section 4.3 (Task Inference) says ExpeL extracts insights from gathered experiences, but it adds the full list of those insights to each task prompt. What it retrieves by task similarity is successful trajectories from past tasks, not individual insights; retrieval of insights is described only as a possible future way to manage context length. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationTree of Thoughts explores and evaluates alternative reasoning paths.
“Tree of Thoughts explores and evaluates alternative reasoning paths (Yao et al., 2023)”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Yao et al., 2023 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe Darwin Gödel Machine maintains an expanding archive of coding agents, empirically validating modifications that improve the agents themselves.
“The Darwin Gödel Machine (Zhang et al., 2025) maintains an expanding archive of coding agents, empirically validating modifications that improve the agents themselves.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Zhang et al., 2025 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationAI Co-Scientist uses asynchronous agents and tournament-based refinement to develop scientific hypotheses.
“AI Co-Scientist (Gottweis et al., 2025) uses asynchronous agents and tournament-based refinement to develop scientific hypotheses.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Gottweis et al., 2025 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMagentic-One uses an orchestrator to assign work and revise plans.
“Magentic-One uses an orchestrator to assign work and revise plans (Fourney et al., 2024).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Fourney et al., 2024 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationARTS uses a central reasoning model to inspect the logs of earlier experiments and choose which node of a search tree to expand, separating weak hypotheses from flawed implementations rather than ranking nodes by score.
“ARTS (Juneja et al., 2026) uses a central reasoning model to inspect the logs of earlier experiments and choose which node of a search tree to expand, separating weak hypotheses from flawed implementations rather than ranking nodes by score.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Juneja et al., 2026 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationAlphaEvolve maintains an evaluated population of programs and uses evolutionary selection for algorithm discovery.
“AlphaEvolve maintains an evaluated population of programs and uses evolutionary selection for algorithm discovery (Novikov et al., 2025).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Novikov et al., 2025 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationRecursive Self-Aggregation (RSA) involves sampling multiple (N) outputs in parallel, curating new contexts by randomly sub-sampling and collecting K outputs, and aggregating them using an LLM operator.
“Similarly, Recursive Self-Aggregation (RSA) (Venkatraman et al., 2025) involves sampling multiple (N ) outputs in parallel, curating new contexts by randomly sub-sampling and collecting K outputs, and aggregating them using an LLM operator.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Venkatraman et al., 2025 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAIRA studies how search policies and operators interact in machine learning research.
“AIRA (Toledo et al., 2025) studies how search policies and operators interact in machine learning research”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Toledo et al., 2025 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAI Scientist-v2 uses progressive agentic tree search directed by an experiment manager.
“AI Scientist-v2 uses progressive agentic tree search directed by an experiment manager (Yamada et al., 2025).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Yamada et al., 2025 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMLAgentBench, MLGym, ResearchGym, and AIRS-Bench provide environments and benchmarks for end-to-end machine learning research.
“MLAgentBench (Huang et al., 2024), MLGym (Nathani et al., 2025), ResearchGym (Garikaparthi et al., 2026), and AIRS-Bench (Lupidi et al., 2026) provide environments and benchmarks for end-to-end machine learning research.”
Quote in the paperNot a substantial claim2 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAgent Laboratory organizes literature review, experimentation, and writing into a staged research workflow.
“Agent Laboratory organizes literature review, experimentation, and writing into a staged research workflow (Schmidgall et al., 2025).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Schmidgall et al., 2025 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSwarmResearch runs coding agents on separate Git branches, but uses a Shepherd Agent to direct their search and adjust parallelism.
“SwarmResearch also runs coding agents on separate Git branches, but uses a Shepherd Agent to direct their search and adjust parallelism (Virk et al., 2026).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Virk et al., 2026 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationKim et al. extend inference-time scaling to long-horizon coding agents.
“Kim et al. (Kim et al., 2026) extend inference-time scaling to long-horizon coding agents.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Kim et al., 2026 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe Looped Transformer task uses Huginn, a depth-recurrent language model that splits the network into a prelude, a weight-tied recurrent core, and a coda.
“The Looped Transformer task uses Huginn (Geiping et al., 2025), a depth-recurrent language model that splits the network into a prelude, a weight-tied recurrent core, and a coda.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Geiping et al., 2025 source
Classified as probably not a substantial claim.
PremisecitationThe Residual Matrix Transformer (RMT) replaces the transformer's residual vector with an outer-product memory matrix: each layer retrieves its input from the matrix through learned read keys and stores its output back as outer products of learned write keys and values.
“The Residual Matrix Transformer (RMT; Mak and Flanigan, 2025) replaces the transformer's residual vector with an outer-product memory matrix: each layer retrieves its input from the matrix through learned read keys and stores its output back as outer products of learned write keys and values.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Mak and Flanigan, 2025 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe RMT's outer-product memory matrix lets the residual stream be scaled independently of parameters and compute.
“This lets the residual stream be scaled independently of parameters and compute.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Mak and Flanigan, 2025 source
May combine two claims.
Findingown dataOn IMOProofBench-Advanced, selected proofs appear at 8.6-17.5% of run progress, and the share of episodes devoted to selection rises from 44.2% to 69.7% as the budget grows from 20 to 160.
“On IMOProofBench-Advanced, by contrast, workers shift early toward selecting among existing proofs (Appendix Figure 20): selected proofs appear at 8.6-17.5% of run progress, and the share of episodes devoted to selection rises from 44.2% to 69.7% as the budget grows from 20 to 160.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone givenGPU-backed research adds resource contention and scheduling overhead.
“GPU-backed research adds resource contention and scheduling overhead.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAbout one in six ProgramBench workers abstains after identifying equivalent work, compared with fewer than 2.7% of proof workers.
“About one in six ProgramBench workers abstains after identifying equivalent work, compared with fewer than 2.7% of proof workers.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAgents' intermediate conclusions can be wrong; small proxy evaluations mis-ranked candidate models.
“as our analyses show, agents' intermediate conclusions can be wrong (small proxy evaluations mis-ranked candidate models)”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataDecentralized nomination achieves 62.06 ± 2.44% in 38.0 minutes, compared with 64.40 ± 1.49% in 34.9 minutes for a central selector.
“For readout, decentralized nomination achieves 62.06 ± 2.44% in 38.0 minutes, compared with 64.40 ± 1.49% in 34.9 minutes for a central selector.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataWorkers cite earlier contributions in nearly every eligible episode: 99.9% on ProgramBench and 96.2% on IMOProofBench-Advanced.
“Workers cite earlier contributions in nearly every eligible episode: 99.9% on ProgramBench and 96.2% on IMOProofBench-Advanced.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenWorkers consume input tokens inspecting previous contributions, and redundant and unsuccessful episodes can spend compute without producing accepted work.
“Workers consume input tokens inspecting previous contributions, while redundant and unsuccessful episodes can spend compute without producing accepted work.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAfter 26 hours, the Looped Transformer swarm reaches a protected-development loss of 3.187, compared with 3.265 for the reference, 3.274 for direct GPT, and 3.479 for Codex.
“After 26 hours, the swarm reaches a protected-development loss of 3.187, compared with 3.265 for the reference, 3.274 for direct GPT, and 3.479 for Codex.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataDuring the first 26 hours, workers conducted 395 architectural experiments across twelve idea families; 74% of negative architectural results were subsequently cited.
“During the first 26 hours, workers conducted 395 architectural experiments across twelve idea families; 74% of negative architectural results were subsequently cited.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOn ProgramBench, GitSwarm achieves a 79.4% mean score, compared with 65.1% for the strongest reported baseline under the stated budget.
“On ProgramBench, it achieves a 79.4% mean score, compared with 65.1% for the strongest reported baseline under the stated budget.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingnone givenThe research histories in Section 3.3 illustrate cross-branch architectural synthesis and the later reconsideration of initially rejected results.
“The research histories in Section 3.3 illustrate cross-branch architectural synthesis and the later reconsideration of initially rejected results.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataWorker w0580 explicitly cited that failed replication, retained the original epsilon, and instead moved query/key normalization before RoPE, obtaining 0.910114.
“Worker w0580 explicitly cited that failed replication, retained the original epsilon, and instead moved query/key normalization before RoPE, obtaining 0.910114.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenA worker tried letting each token see the previous token's memory state, and it failed. 49.5 hours later, another worker cited this failure as evidence that longer look-backs were still unexplored and built a mixer over several earlier tokens that succeeded; a third worker carried it into the 72-hour submission.
“A worker tried letting each token see the previous token's memory state, and it failed.49.5 hours later, another worker cited this failure as evidence that longer look-backs were still unexplored and built a mixer over several earlier tokens that succeeded; a third worker carried it into the 72-hour submission.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenThe back-loaded MLP was introduced by w0220 and transferred through a smaller-table experiment before w0309 combined it with a 524K-row bigram model.
“The back-loaded MLP was introduced by w0220 and transferred through a smaller-table experiment before w0309 combined it with a 524K-row bigram model.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.
Findingown dataA quick 128-example check scored the feed-forward adapter slightly worse than the reference; its author wrote that it 'should not be promoted', and the idea was nearly dropped.
“A quick 128-example check scored it slightly worse than the reference; its author wrote that it”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenIt reallocates feed-forward capacity from uniform fourfold expansion to fourfold expansion in the first four layers and eightfold expansion in the last four.
“It reallocates feed-forward capacity from uniform fourfold expansion to fourfold expansion in the first four layers and eightfold expansion in the last four.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataBranch-native memory substantially reduces repeated traversal relative to flat persistent memory across the tested depth and branching-factor regimes; results for unstructured resumable memory are not shown.
“Figure 14 shows that branch-native memory substantially reduces repeated traversal relative to flat persistent memory across the tested depth and branching-factor regimes; results for unstructured resumable memory are not shown.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe gate's advantage vanished at 124M and 188M, while the adapter won at every rung and reached 3.187 at full size after 9 hours.
“After the adapter's initial rejection was reversed (see the steps below), the two were carried up the ladder side by side: the gate's advantage vanished at 124M and 188M, while the adapter won at every rung and reached 3.187 at full size after 9 hours (Figure 5(b)).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataWorker w1407 investigated whether order could explain this difference by predeclaring the reverse sequence: depth-dependent scaling first (0.908089), then uniform scaling (0.907224), using the unchanged sources. Uniform scaling again won the pair.
“Worker w1407 investigated whether order could explain this difference by predeclaring the reverse sequence: depth-dependent scaling first (0.908089), then uniform scaling (0.907224), using the unchanged sources.Uniform scaling again won the pair.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataA later worker overturned a 128-example win for iteration-dependent control, and four workers independently ran parameter-matched controls.
“Besides the reversal of the adapter's initial rejection (Appendix E.3.3), a later worker overturned a 128-example win for iteration-dependent control, and four workers independently ran parameter-matched controls.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe head router commit (34.6 h, 2.934) was developed on a side branch and overtook the differential-attention line.
“Head router (34.6 h, 2.934).Developed on a side branch; it overtook the differential-attention line, which led once more (50.7 h) before being superseded.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe seeded run's gains came from combining the seeds: after 26.7 hours it added the 8-GPU run's head router to the 4-GPU model (2.905), and an hour later it moved the 4-GPU run's deeper stack onto the 8-GPU model (2.898).
“Its gains came from combining the seeds: after 26.7 hours it added the 8-GPU run's head router to the 4-GPU model (2.905), and an hour later it moved the 4-GPU run's deeper stack onto the 8-GPU model (2.898).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
FindingreasoningCounting this cited evidence yields the 24-commit, 24-worker dependency set reported in Section 3.3.
“Counting this cited evidence yields the 24-commit, 24-worker dependency set reported in Section 3.3.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIt trains on 547.9M reported tokens versus the reference's 728.2M.
“It trains on 547.9M reported tokens versus the reference's 728.2M.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe incumbent was measured at 23.67 active hours, or 25.15 elapsed wall hours; a strict 24-wall-hour cutoff instead gives 0.905487.
“The incumbent was measured at 23.67 active hours, or 25.15 elapsed wall hours; a strict 24-wall-hour cutoff instead gives 0.905487.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenThe incumbent retains eight transformer layers but widens the residual stream from 512 to 768 dimensions, uses eight 96-dimensional attention heads, and shortens local attention windows from 1,024 to 512 tokens while retaining full-context layers.
“The incumbent retains eight transformer layers but widens the residual stream from 512 to 768 dimensions, uses eight 96-dimensional attention heads, and shortens local attention windows from 1,024 to 512 tokens while retaining full-context layers.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingreasoningThe 96-dimensional head setting and 512-token window came from side branches.
“The 96-dimensional head setting and 512-token window came from side branches.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataProof construction follows the same broad trend as ProgramBench, but with more run-to-run variation and earlier saturation, whereas program reconstruction continues to benefit from larger budgets.
“Proof construction follows the same broad trend as ProgramBench, but with more run-to-run variation and earlier saturation, whereas program reconstruction continues to benefit from larger budgets.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataWorker w0033 discovered that w0032 had already announced the identical depth-10, head-dimension-64 experiment from the same source. It abstained before training; the evaluation ledger contains no evaluation by w0033.
“Worker w0033 discovered that w0032 had already announced the identical depth-10, head-dimension-64 experiment from the same source.It abstained before training; the evaluation ledger contains no evaluation by w0033.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingnone givenDirect GPT trained longer (3.0B tokens) and added an auxiliary output decoder, and Codex reached 2.690 mainly by training for 10.1B tokens, from which about three quarters of its gain came.
“Direct GPT trained longer (3.0B tokens) and added an auxiliary output decoder, and Codex reached 2.690 mainly by training for 10.1B tokens, from which about three quarters of its gain came.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataWithin the first 24 active hours, the run improves from 0.992692 to 0.905235 BPB (8.81%), completing 1,465 successful evaluations of 1,219 distinct train.py solutions, plus 175 failed evaluations.
“Within the first 24 active hours, the run improves from 0.992692 to 0.905235 BPB (8.81%), completing 1,465 successful evaluations of 1,219 distinct train.pysolutions, plus 175 failed evaluations.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAbout an hour later, another worker re-evaluated the same checkpoint on 512 examples and found it clearly better (-0.067 against the reference); a third confirmed the gain on the full development set and showed that it also beat the leading alternative, the recurrent-state gate.
“About an hour later, another worker re-evaluated the same checkpoint on 512 examples and found it clearly better (-0.067 against the reference); a third confirmed the gain on the full development set and showed that it also beat the leading alternative, the recurrent-state gate.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenThe baselines added signals to the recurrent state (a learned per-iteration embedding for direct GPT; input anchors and step signals for Codex), which helped direct GPT on smaller models but not at full size; neither baseline has yet beaten the reference.
“The baselines instead added signals to the recurrent state (a learned per-iteration embedding for direct GPT; input anchors and step signals for Codex), which helped direct GPT on smaller models but not at full size; neither baseline has yet beaten the reference.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe final model's code comes almost entirely from one lineage of six commits by six workers; only the first changes code, and the only code imported from elsewhere is a loss-computation fix written for another branch.
“Unlike in the RMT run, the final model's code comes almost entirely from one lineage of six commits by six workers; only the first changes code, and the only code imported from elsewhere is a loss-computation fix written for another branch.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenQuery/key normalization moves before rotary embedding.
“Query/key normalization moves before rotary embedding.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAn explicit query/key normalization epsilon initially scored 0.911027 BPB; worker w0555 repeated the identical source and obtained 0.911918, revising the apparent advantage to an unresolved near-tie with a preference for the simpler inherited normalization.
“An explicit query/keynormalization epsilon initially scored 0.911027 BPB; worker w0555 repeated the identical source and obtained 0.911918, revising the apparent advantage to an unresolved near-tie with a preference for the simpler inherited normalization.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataIts best observation is 0.898671 (9.47% below the reference), but this is an extension result, not a 24-hour result or a separately held-out test score.
“Its best observation is 0.898671 (9.47% below the reference), but this is an extension result, not a 24-hour result or a separately held-out test score.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAt 2.18 active hours, worker w0273 added a separate 32K-row trigram table and obtained 0.929923, worse than both bigram-only controls (0.929131/0.929700).
“At 2.18 active hours, worker w0273 added a separate 32K-row trigram table and obtained 0.929923, worse than both bigram-only controls (0.929131/0.929700).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataWithin two hours the recurrent-state gate and the feed-forward adapter were the two leading ideas.
“Workers first screened all twelve families in parallel on small models; within two hours the recurrent-state gate and the feed-forward adapter were the two leading ideas.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe 8-GPU run's final model combines six mechanisms: three general transformer changes (removing a depth-dependent attention scaling, tied embeddings, and a SwiGLU feed-forward layer) and three that exploit the matrix memory (routing across attention heads, wider attention, and mixing in the memory states of recent tokens).
“The 8-GPU run's final model combines six mechanisms (Table 6): three general transformer changes (removing a depth-dependent attention scaling, tied embeddings, and a SwiGLU feed-forward layer) and three that exploit the matrix memory (routing across attention heads, wider attention, and mixing in the memory states of recent tokens).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataWorker w0642 questioned an inherited MLP multiplier, 0.56776, whose rationale assumed an activation distribution rather than measuring the learned one. Replacing it with 0.5 scored 0.909838 BPB.
“Worker w0642 questioned an inherited MLP multiplier, 0.56776, whose rationale assumed an activation distribution rather than measuring the learned one.Replacing it with 0.5 scored 0.909838 BPB, which the worker described as an effective near-tie, not established superiority.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe change adds 27M parameters (8.7%).
“Because this adapter runs at every recurrence, it is the component most specific to looped models, and the change adds 27M parameters (8.7%).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataDuring the 72-hour run, 15 of 17 ideas promoted from the small proxy to full size helped; in the extension, only 1 of 8 did, and several ideas reversed sign at full size.
“During the 72-hour run, 15 of 17 ideas promoted from the small proxy to full size helped; in the extension, only 1 of 8 did, and several ideas reversed sign at full size.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe 72-hour submission (dev 2.909, held-out 2.890) uses rank-32 attention; the best model after the extension (dev 2.894) widens it to rank 64.
“The 72-hour submission (dev 2.909, held-out 2.890) uses rank-32 attention; the best model after the extension (dev 2.894) widens it to rank 64 (Appendix E.2.4).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe 24-hour incumbent's physical lineage contains 32 research-result commits, but several mechanisms were developed off that chain.
“The 24-hour incumbent's physical lineage contains 32 research-result commits, but several mechanisms were developed off that chain.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe resulting model has 1,723.9M parameters versus 50.3M initially; 1,610.6M belong to the lexical table.
“The resulting model has 1,723.9Mparameters versus 50.3M initially; 1,610.6Mbelong to the lexical table.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataSix families were incorporated into the final model, one led the frontier and was superseded, two produced gains that did not survive combination with the leader, two remained inconclusive at full size, seven were refuted, and one consisted of controls.
“Six families were incorporated into the final model, one led the frontier and was superseded, two produced gains that did not survive combination with the leader, two remained inconclusive at full size, seven were refuted, and one consisted of controls.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenPre-RoPE query/key normalization came from w0580 and was incorporated by w0678 into an activation-scaled model.
“Pre-RoPE query/key normalization came from w0580 and was incorporated by w0678 into an activation-scaled model.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe complete record contains 3,630 successful and 453 failed evaluations, with 2,784 distinct successful solutions.
“The complete record contains 3,630 successful and 453 failed evaluations, with 2,784 distinct successful solutions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe 170 negative commits amount to 43% of the architectural ones.
“The 170 negative commits amount to 43% of the architectural ones.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenA new 2,097,152-row table supplies hashed causal bigram and trigram embeddings, sharing storage between the two feature types.
“A new 2,097,152-row table supplies hashed causal bigram and trigram embeddings, sharing storage between the two feature types.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataEarly workers screened many ideas in parallel on small proxy models.
“Early workers screened many ideas in parallel on small proxy models.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone givenNo off-task drift was observed in this run.
“No off-task drift was observed in this run.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataNothing has improved on the best model since 9 hours.
“Nothing has improved on the best model since 9 hours; the later shift to calibration and diagnostics is described in Appendix E.3.4.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe recurrent-state gate, the specialist branches and iteration-dependent control all lost their small-scale advantage at larger rungs or against a parameter-matched control.
“The recurrent-state gate, the specialist branches and iterationdependent control all lost their small-scale advantage at larger rungs or against a parameter-matched control.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataAfter the cutoff, a lexical cooldown/decay combination reached 0.903842.
“After the cutoff, a lexical cooldown/decay combination reached 0.903842.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAcross the complete record, 1,285 of 2,646 accepted research commit actions (48.6%) are negative-labelled, and 982 (76.4%) are subsequently cited.
“Across the complete record, 1,285 of 2,646 accepted research commit actions (48.6%) are negative-labelled, and 982 (76.4%) are subsequently cited.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenTwo refuted results, a narrower adapter and the recurrent-state gate at 188M, were reused to build the parameter-matched controls.
“Two refuted results, a narrower adapter and the recurrent-state gate at 188M, were reused to build the parameter-matched controls.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe 4-GPU run converged on an overlapping but simpler design, tied embeddings, SwiGLU, and a deeper 32-layer stack, with a held-out loss of 2.894.
“The 4-GPU run converged on an overlapping but simpler design, tied embeddings, SwiGLU, and a deeper 32-layer stack, with a held-out loss of 2.894.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAlthough the final model's code descends through a single chain of eight commits, only two of its six mechanisms were first written on that chain. The other four (SwiGLU, the head router, wider attention, and the temporal mixer) were developed on side branches by other workers and ported into the chain.
“Although the final model's code descends through a single chain of eight commits, only two of its six mechanisms were first written on that chain.The other four (SwiGLU, the head router, wider attention, and the temporal mixer) were developed on side branches by other workers and ported into the chain (Table 8).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataAn earlier comparison tested uniform attention-query scaling first (0.906862 BPB), then depth-dependent scaling (0.907103).
“An earlier comparison tested uniform attention-query scaling first (0.906862 BPB), then depth-dependent scaling (0.907103).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe family assignment covers 119 of the 120 commits, and its positive/negative split (48/71) differs slightly from the 69 negative commits reported in Section 3.3.
“The family assignment covers 119 of the 120 commits, and its positive/negative split (48/71) differs slightly from the 69 negative commits reported in Section 3.3.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenWorker w3052 enlarged that shared table to produce the 24-hour incumbent, 0.905235.
“Nevertheless, worker w3052 enlarged that shared table to produce the 24-hour incumbent, 0.905235.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingreasoningThat ordering subsequently entered the combined architecture.
“That ordering subsequently entered the combined architecture.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingnone givenA seed replication moved loss by 0.010, comparable to several of the later steps on the solution path.
“A seed replication moved loss by 0.010, comparable to several of the later steps on the solution path.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataCodex opened no new idea threads after 26.5 hours, and direct GPT spent 68% of its GPU time on a single family.
“The single-agent baselines narrowed much earlier: Codex opened no new idea threads after 26.5 hours, and direct GPT spent 68% of its GPU time on a single family.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingnone givenDecoys report nearby values (19, 25, 31, ...), but only one value makes each stated numerical transformation correct.
“Decoys report nearby values (19, 25, 31, ...), but only one value makes each stated numerical transformation correct.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAt equal total MLP parameter count, early-fourfold/late-eightfold expansion scored 0.932288 versus 0.934069/0.934041 for uniform sixfold expansion; reversing the allocation scored 0.935398.
“Second, at equal total MLP parameter count, early-fourfold/late-eightfold expansion scored 0.932288 versus 0.934069/0.934041for uniform sixfold expansion; reversing the allocation scored 0.935398.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe temporal memory mixer (65.9 h, 2.909; held-out 2.890) was the 72-hour submission, built from the reused single-token failure.
“Temporal memory mixer (65.9 h, 2.909; held-out 2.890).The 72-hour submission, built from the reused single-token failure described above.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn the first 24 active hours, worker traces confirm 1,258 accepted research commit actions. Of these, 709 (56.4%) are explicitly labelled negative by their authors, and 497 (70.1%) are cited by a later accepted research commit within the same window.
“In the first 24 active hours, worker traces confirm 1,258 accepted research commit actions.Of these, 709 (56.4%) are explicitly labelled negative by their authors, and 497 (70.1%) are cited by a later accepted research commit within the same window.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataAfter the evaluation budget ran out (20.8 hours), most commits were calibrations and diagnostics, and 128 of 136 abstentions cited duplicate or already-running work.
“After the evaluation budget ran out (20.8 hours), most commits were calibrations and diagnostics, and 128 of 136 abstentions cited duplicate or already-running work.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataAt 23.08 hours, worker w2985 explicitly cited both failures and reused their causal hash while borrowing a shared-table idea from a lagged-pair branch.
“At 23.08 hours, worker w2985 explicitly cited both failures and reused their causal hash while borrowing a shared-table idea from a lagged-pair branch.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataWorker w1456 explicitly reused this comparison set when evaluating a streamed output-head implementation. Its 0.906886 BPB beat nine observations but not the best, 0.906862.
“Worker w1456 explicitly reused this comparison set when evaluating a streamed output-head implementation.Its 0.906886 BPB beat nine observations but not the best, 0.906862; the worker therefore did not claim a new quality record or a repeatable BPB gain.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe same verification worker assembled ten measurements of the byte-identical ordinary, uniform-scaling source, without introducing a new architecture.
“The same verification worker assembled ten measurements of the byte-identical ordinary, uniform-scaling source, without introducing a new architecture.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataExplicit attribution reduces the fraction of the archive workers must inspect at every tested lineage depth, semantic ambiguity, and downstream fan-out level, with the gap widening with lineage depth.
“Figure 16 shows that explicit attribution reduces the fraction of the archive workers must inspect at every tested lineage depth, semantic ambiguity, and downstream fan-out level, with the gap widening with lineage depth.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe first durable improvement, a correction to the reference code found within the first hour, is an ancestor of 102 of the 120 commits, and ten commits receive half of all citations.
“The first durable improvement, a correction to the reference code found within the first hour, is an ancestor of 102 of the 120 commits, and ten commits receive half of all citations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataEnlarging the separate table also failed: 0.924527 versus 0.919483/0.920211.
“Enlarging the separate table also failed: 0.924527 versus 0.919483/0.920211.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data128-example evaluations disagreed with the full set on 5 of 12 comparable pairs, and with 512-example evaluations on 15 of 50.
“128-example evaluations disagreed with the full set on 5 of 12 comparable pairs, and with 512-example evaluations on 15 of 50; a worker measured this itself and the swarm adopted a 128→512→4,873 evaluation funnel.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe independent baselines' coverage stays roughly flat but well below shared memory as the number of targets grows (about 70-75% for B = 2 and 20-23% for k = 4) and drops with dependency depth, despite their larger aggregate query budget.
“The independent baselines' coverage stays roughly flat but well below shared memory as the number of targets grows (about 70-75% for B = 2 and 20-23% for k = 4) and drops with dependency depth, despite their larger aggregate query budget.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataGitSwarm used 1,222 GPU jobs and all 64 permitted protected evaluations, against 324 and 222 jobs for direct GPT and Codex.
“GitSwarm used 1,222 GPU jobs and all 64 permitted protected evaluations, against 324 and 222 jobs for direct GPT and Codex.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataA fixed set of proxy contexts was used by 257 commits and parameter matching was discussed in 187; commits that built measurement tools and controls were among the most cited.
“A fixed set of proxy contexts was used by 257 commits and parameter matching was discussed in 187; commits that built measurement tools and controls were among the most cited.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe original timing/evaluation code is retained apart from an optimizer learning-rate expression, but 701 successful harness records exceed the original instructions' separate ten-minute overall timeout.
“The original timing/evaluation code is retained apart from an optimizer learning-rate expression, but 701 successful harness records exceed the original instructions' separate ten-minute overall timeout.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOf these workers, 86.7% edited files, 98.8% built a candidate, 99.0% ran tests, and 95.1% probed the oracle.
“Of these workers, 86.7% edited files, 98.8% built a candidate, 99.0% ran tests, and 95.1% probed the oracle.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn 71.5% of cases, the worker had seen the eventual committer's public scope and continued working; a smaller subset explicitly planned to wait or refresh the graph before committing.
“In 71.5% of cases, the worker had seen the eventual committer's public scope and continued working; a smaller subset explicitly planned to wait or refresh the graph before committing.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe first vote moves from 68.4% of normalized progress at B100 to 16.1% at B500, mainly because the total run grows from about 122 to 599 episodes while voting starts after a similar absolute amount of work; a substantial selection pulse still remains near termination at larger budgets.
“The first vote moves from 68.4% of normalized progress at B100 to 16.1% at B500, mainly because the total run grows from about 122 to 599 episodes while voting starts after a similar absolute amount of work; a substantial selection pulse still remains near termination at larger budgets.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe share of graph expansion falls quickly, and selection occupies 64.8-82.2% of episodes in the final fifth of a proof run.
“Beyond the budget-level selection shares reported in Section 3.5, the share of graph expansion (contribution commits, i.e. all proof categories except Candidate selection: the bands below the ink rule in Figure 20) falls quickly, and selection occupies 64.8-82.2% of episodes in the final fifth of a proof run.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataBroader comparison is concentrated near startup and, on ProgramBench, during the final selection pulse.
“Broader comparison is concentrated near startup and, on ProgramBench, during the final selection pulse.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenLater work transfers normalization order, investigates readout and gradient implementations, and revisits lexical context using shared trigrams.
“Later work transfers normalization order, investigates readout and gradient implementations, and revisits lexical context using shared trigrams.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone givenThe first window is low because little prior work has been published.
“The first window is low because little prior work has been published.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataHashed bigrams open a second capacity axis, and their combination with back-loaded MLPs and autotuning reaches 0.912586 by 3.00 active hours.
“Hashed bigrams open a second capacity axis, and their combination with back-loaded MLPs and autotuning reaches 0.912586 by 3.00 active hours.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenEvery contribution worker writes MY_THOUGHTS.md and EVALUATION.md, and nomination workers also write BEST_SOLUTION.md.
“Every contribution worker writes MY_THOUGHTS.mdand EVALUATION.md, and nomination workers also write BEST_-SOLUTION.md.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataDeclared implementation redundancy stays near one ProgramBench worker in six across budgets, but remains below 2.7% on IMOProofBench-Advanced.
“Figure 25 shows that declared implementation redundancy stays near one ProgramBench worker in six across budgets, but remains below 2.7% on IMOProofBench-Advanced.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe categorization yielded six common ProgramBench categories and four recurring proof categories plus a small residual category.
“This yielded six common ProgramBench categories and four recurring proof categories plus a small residual category (Table 10).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIMOProofBench-Advanced shows similarly frequent access but less durable reuse: its active-reuse rate is much lower than ProgramBench's, because most opened computation or probe bundles are not executed or adapted by a citing contribution.
“IMOProofBench-Advanced shows similarly frequent access but less durable reuse: its active-reuse rate is much lower than ProgramBench's, because most opened computation or probe bundles are not executed or adapted by a citing contribution (Section 3.4).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe pooled rates in Figure 19 show frequent artifact consumption in both domains: persistent state is not merely published, since later workers visibly inspect and use earlier artifacts.
“The pooled rates in Figure 19 show frequent artifact consumption in both domains (Section 3.4): persistent state is not merely published, since later workers visibly inspect and use earlier artifacts.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataBoth domains begin with a short wave that creates initial candidates, after which workers operate on an established graph; what happens after startup differs between the two benchmarks.
“Both domains begin with a short wave that creates initial candidates, after which workers operate on an established graph; what happens after startup differs between the two benchmarks.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataExplicit proof abstainers consume only 1.2-2.8% of input tokens and 0.3-0.8% of output tokens.
“Explicit proof abstainers consume only 1.2-2.8% of input tokens and 0.3-0.8% of output tokens.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
FindingreasoningThe code remains reachable in Git, but there is no located completed finalizer submission.
“The code remains reachable in Git, but there is no located completed finalizer submission.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataHowever, its exact-source repeat scores 0.899864, and its worker publication is rejected for an extra audit Markdown file outside the allowed paths.
“However, its exact-source repeat scores 0.899864, and its worker publication is rejected for an extra audit Markdown file outside the allowed paths.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataConstruction continues near the budget boundary, and the winner is usually a final repair or integration that inherits older work.
“Construction continues near the budget boundary, and the winner is usually a final repair or integration that inherits older work.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataA rare B100 artifact tool, a script for generating comparison diffs, was repeatedly used by later workers.
“A rare B100 artifact tool, a script for generating comparison diffs, was repeatedly used by later workers.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingnone givenThe extension further changes value routing, lexical gradients and FP8 training arithmetic.
“The extension further changes value routing, lexical gradients and FP8 training arithmetic.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataEarly parallel tests establish narrower attention heads and wider residual states; head geometry, batch size, decay, attention sharpness and locality are then combined.
“Early parallel tests establish narrower attention heads and wider residual states; head geometry, batch size, decay, attention sharpness and locality are then combined.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataBy workers 90-99 in the B100 runs, roughly 72 candidates are available, but each worker explicitly opens only about six.
“By workers 90-99 in the B100 runs, roughly 72 candidates are available, but each worker explicitly opens only about six.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThat reuse pattern did not recur at other budgets; a separate B500 capture helper was opened but not actively reused.
“That reuse pattern did not recur at other budgets; a separate B500 capture helper was opened but not actively reused.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataWinner linkage differs much more than access: fewer than a quarter (22.6%) of proof protocol revisions connect to the winner through a consuming contribution, while ProgramBench artifacts usually enter its broad selected ancestry.
“but winner linkage differs much more than access (Section 3.4): fewer than a quarter (22.6%) of proof protocol revisions connect to the winner through a consuming contribution, while ProgramBench artifacts usually enter its broad selected ancestry.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataMore compute produces more comparisons and votes in both benchmarks, but their timing differs.
“More compute produces more comparisons and votes in both benchmarks, but their timing differs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataDifferent problems favor different kinds of evidence.
“Different problems favor different kinds of evidence”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningThe proof graph is smaller, so those three candidates still reach a larger fraction of its history than the five to seven candidates typically opened on ProgramBench.
“The proof graph is smaller, so those three candidates still reach a larger fraction of its history than the five to seven candidates typically opened on ProgramBench.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningProgramBench frequently produces a late, integrated candidate, whereas proof-solving workers may repeatedly evaluate and nominate an earlier proof without modifying its exact commit.
“ProgramBench frequently produces a late, integrated candidate, whereas proof-solving workers may repeatedly evaluate and nominate an earlier proof without modifying its exact commit.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataOne representative ProgramBench run exhibits late integration, in which separate lines of implementation work are subsequently combined.
“One exhibits late integration, in which separate lines of implementation work are subsequently combined.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataDivide-and-conquer runs such as Pandoc B100 use a related but more specialized pattern: separate groups develop document structure and inline syntax on one branch and output modes and code-block handling on another before the branches join.
“Divide-and-conquer runs such as Pandoc B100 use a related but more specialized pattern: separate groups develop document structure and inline syntax on one branch and output modes and code-block handling on another before the branches join.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningThe exact ProgramBench winner is produced late, at 93.1-98.0% median progress, only 8-11.5 worker episodes before termination, consistent with continued refinement and integration throughout the run.
“The exact ProgramBench winner is produced late, at 93.1-98.0%median progress, only 8-11.5 worker episodes before termination, consistent with continued refinement and integration throughout the run.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn ProgramBench, inspection is initially broad because few candidates are available: workers 10-19 see roughly eight candidates and inspect nearly all of them because no clear foundation has emerged.
“On ProgramBench, inspection is initially broad because few candidates are available: workers 10-19 see roughly eight candidates and inspect nearly all of them because no clear foundation has emerged.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe selected candidate at 92% incorporates further line-number and raw-line inspection work.
“The selected candidate at 92% incorporates further line-number and raw-line inspection work.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe other representative ProgramBench run illustrates search-and-pivot, as workers develop alternative approaches over an extended run.
“The other illustrates search-and-pivot (search and redirection), as workers develop alternative approaches over an extended run.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataTable 12 lists the six categories obtained for each benchmark.
“Table 12 lists the six categories obtained for each benchmark.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataProgramBench workers also reach a shrinking fraction, even though their absolute look-back distance into the spawn-time history (measured in commits) increases, growing to about 20 commits at B100, 50 at B200, 80 at B300, and 150 at B500.
“ProgramBench workers also reach a shrinking fraction, even though their absolute look-back distance into the spawn-time history (measured in commits) increases, growing to about 20 commits at B100, 50 at B200, 80 at B300, and 150 at B500.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn ProgramBench, the selected solution's ancestry covers most of the contribution graph, whereas selected proofs retain a much smaller share.
“On ProgramBench this ancestry covers most of the contribution graph (Figure 7), so repeated local composition from a few sources produces a broad final ancestry; selected proofs retain a much smaller share.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe winning Looped Transformer adapter's development path includes reversal of an early rejection and reuse of negative results.
“The winning adapter's development path and its wider evidence dependencies, including reversal of an early rejection and reuse of negative results.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Findingown dataGitSwarm has shared memory, durable branches, self-directed workers, no fixed roles, asynchronous workers, cross-branch dependencies, and separate sources.
“| GitSwarm | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataIn the ProgramBench run shown, four renderer experiments appear within one percentage point of run progress, covering JSON, style palettes, Go, and HTML/XML, and are quickly combined.
“Four renderer experiments then appear within one percentage point, covering JSON, style palettes, Go, and HTML/XML, and are quickly combined.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataIn the GPT-5.5 GitSwarm Looped Transformer run's first 26 hours, the adapter capacity and placement variants family had 37 commits, 27 negative commits, 0 protected evaluations, and was refuted.
“Adapter capacity and placement | 37 | 27 | 0 | Refuted”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataRoughly three quarters of all ProgramBench worker episodes end in a contribution commit at every budget, while abstentions remain spread through the run.
“Roughly three quarters of all ProgramBench worker episodes end in a contribution commit at every budget, while abstentions remain spread through the run.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn the ProgramBench development set, synchronous waves scored 63.11 ± 0.58%, which is +1.05 percentage points over the default, with runtime 49.3 min.
“Synchronous waves | 63.11 ± 0.58 | +1.05 | 49.3”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn pilot runs without the sequential shell interaction extension, Gemini-3.1-Pro workers often issued an entire contribution workflow (inspection, scope announcement, gs_fork, file writes, and git commit) in a single response, sometimes predicting command outputs and branching from a guessed commit SHA.
“In pilot runs without it, Gemini workers often issued an entire contribution workflow (inspection, scope announcement, gs_fork, file writes, and git commit) in a single response, sometimes predicting command outputs and branching from a guessed commit SHA.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataIn the ProgramBench run shown, CLI and configuration work continues until about 57% of the run, while renderer work becomes the dominant selected lineage and ends in a final renderer synthesis.
“CLI and configuration work continues until about 57%, while renderer work becomes the dominant selected lineage and ends in a final renderer synthesis.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataSelected IMOProofBench proofs have shallower ancestry than ProgramBench solutions: later work often evaluates and compares existing candidates rather than extending the selected proof.
“In contrast, selected IMOProofBench proofs have shallower ancestry: later work often evaluates and compares existing candidates rather than extending the selected proof.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn the GPT-5.5 GitSwarm Looped Transformer run's first 26 hours, the gate on the recurrent state family had 32 commits, 15 negative commits, 10 protected evaluations, and was an early leader whose gain vanished at 124M/188M.
“Gate on the recurrent state | 32 | 15 | 10 | Early leader; gain vanished at 124M/188M”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningThe initial ProgramBench spike appears across work types, so it is not caused by nomination workers.
“The initial ProgramBench spike appears across work types, so it is not caused by nomination workers.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn the ProgramBench development set, central selector scored 64.40 ± 1.49%, which is +2.34 percentage points over the default, with runtime 34.9 min.
“Central selector | 64.40 ± 1.49 | +2.34 | 34.9”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataTwo of the 64 protected evaluations in the GPT-5.5 GitSwarm Looped Transformer run's first 26 hours are not assigned to an architectural family.
“Two of the 64 protected evaluations are not assigned to an architectural family.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOf the 273 IMO redundancy cases, 268 include committed evidence and only three cite in-flight work alone.
“of the 273 IMO redundancy cases, 268 include committed evidence and only three cite in-flight work alone”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe proof traces contain no gs_explore or gs_inspect calls; the Gemini workers use targeted Git reads instead.
“The proof traces contain no gs_explore or gs_inspect calls; the Gemini workers use targeted Git reads instead.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn the ProgramBench development set, the asynchronous default configuration (decentralized nomination) scored 62.06 ± 2.44% macro-average test score across three random seeds, with mean generation wall-clock time 38.0 min per task.
“Asynchronous (default) | 62.06 ± 2.44 | - | 38.0”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe eventual winner is first committed late in the run on ProgramBench and early in the run on IMOProofBench-Advanced.
“The winner appears late on ProgramBench and early on IMOProofBench-Advanced.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn the ProgramBench development set, central aggregator scored 58.19 ± 4.53%, which is -3.87 percentage points from the default, with runtime 43.9 min.
“Central aggregator | 58.19 ± 4.53 | -3.87 | 43.9”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn the GPT-5.5 GitSwarm Looped Transformer run's first 26 hours, the feed-forward re-injection adapter family had 13 commits, 0 negative commits, 20 protected evaluations, and its outcome was in the final model.
“Feed-forward re-injection | 13 | 0 | 20 | In final model”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataIn the ProgramBench run shown, workers spend the first 35% of the run mainly on CLI, configuration, YAML, and frontmatter behavior.
“Workers spend the first 35% of the run mainly on CLI, configuration, YAML, and frontmatter behavior.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenWhen gs_fork failed in those pilot runs, the remaining commands still ran and committed the new files on the worker's placeholder branch, and the result was rejected as invalid.
“When gs_fork then failed, the remaining commands still ran and committed the new files on the worker's placeholder branch, and the result was rejected as invalid.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataSynchronization changes the mean score by +1.05 points but increases the mean task runtime by a factor of 1.30.
“Synchronization changes the mean score by +1.05 points but increases the mean task runtime by a factor of 1.30.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataRelative to decentralized nomination, the central selector raises the mean score by 2.34 points with a shorter mean runtime, whereas the central aggregator lowers the mean score by 3.87 points and takes 16% longer.
“Relative to decentralized nomination, the central selector raises the mean score by 2.34 points with a shorter mean runtime, whereas the central aggregator lowers the mean score by 3.87 points and takes 16% longer.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe best GitSwarm RMT model (8-GPU run) includes a depth-scaling fix, tied embeddings, a SwiGLU feed-forward, a cross-head router, and wider attention.
“Architectural mechanisms in the best GitSwarm RMT model (8-GPU run).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOf the 615 unsuccessful charged episodes in the 200 ProgramBench B100 runs, 515 fail candidate-publication or required-artifact validation, 79 fail nomination validation, and 21 terminate without a valid action because of API failures, turn limits, or protocol violations.
“Of these, 515 fail candidate-publication or required-artifact validation, 79 fail nomination validation, and 21 terminate without a valid action because of API failures, turn limits, or protocol violations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThese histories reveal distinct forms of compounding research.
“These histories reveal distinct forms of compounding research.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenCandidates were scored by fixed evaluators on protected development and separate held-out sets.
“Candidates were scored by fixed evaluators on protected development and separate held-out sets.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPDR studies inference as an iterative improvement process.
“PDR (Madaan et al., 2025) study inference as an iterative improvement process.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Madaan et al., 2025 source
Checked before — its earlier verdict is held back until a new check runs.
Probably not a substantial claim: may not pass S4 (based).
PremisereasoningIf such work disappears when an episode terminates, later computation may need to rediscover it.
“If such work disappears when an episode terminates, later computation may need to rediscover it.”
Quote in the paperNot a substantial claim- Rests on an assumption: If such work disappears when an episode terminates, later computation may need to rediscover it.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe recorded Looped Transformer runs were still in progress, and NanoChat comparisons vary in starting checkpoint and GPU allocation.
“The recorded Looped Transformer runs were still in progress, and NanoChat comparisons vary in starting checkpoint and GPU allocation.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataGitSwarm improves upon the starting points across all three research tasks, but the results do not establish superiority over every single-agent configuration.
“GitSwarm improves upon the starting points across all three research tasks, but the results do not establish superiority over every single-agent configuration.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe perfect IMOProofBench-Advanced result is a single run; some independent-sampling and Forced baselines also reach the benchmark ceiling, and execution conditions differ across some proof comparisons.
“The perfect IMOProofBench-Advanced result is a single run; some independent-sampling and Forced baselines also reach the benchmark ceiling, and execution conditions differ across some proof comparisons.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIMOProofBench-Advanced contains 30 mathematical proof problems.
“IMOProofBench-Advanced (Luong et al., 2025) contains 30 mathematical proof problems.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Luong et al., 2025 source
May combine two claims.
Interpretationnone givenGitSwarm preserves partial solutions, executable artifacts, and negative results across independently scheduled worker episodes.
“In contrast, GitSwarm preserves partial solutions, executable artifacts, and negative results across independently scheduled worker episodes.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe paper's focus is a shared, versioned contribution history in which independent workers can pursue competing approaches and explicitly declare dependencies across them.
“Our focus is a shared, versioned contribution history in which independent workers can pursue competing approaches and explicitly declare dependencies across them.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThese small development-set experiments do not establish statistical equivalence or isolate the benefits of persistent memory.
“These small development-set experiments characterize latency and readout tradeoffs; they do not establish statistical equivalence or isolate the benefits of persistent memory.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThis work involves no human subjects, crowdsourced annotation, or personal data; all experiments use public benchmarks (IMOProofBench-Advanced and ProgramBench) and research tasks based on publicly available research papers.
“This work involves no human subjects, crowdsourced annotation, or personal data; all experiments use public benchmarks (IMOProofBench-Advanced and ProgramBench) and research tasks based on publicly available research papers.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningThese approaches preserve intermediate states within a search.
“These approaches preserve intermediate states within a search.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductown dataOur results are conditional on the evaluated tasks and budgets.
“Our results are conditional on the evaluated tasks and budgets.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenValidation enforces task-specific publication requirements, not solution correctness.
“Validation enforces task-specific publication requirements, not solution correctness.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningThese measurements establish observed reuse, not the causal contribution of individual dependencies to performance.
“They establish observed reuse, not the causal contribution of individual dependencies to performance.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
InterpretationreasoningThese systems establish the value of external memory.
“These systems establish the value of external memory.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe paper introduced compounding inference, in which intermediate work persists and can be extended across independent agent episodes.
“We introduced compounding inference, in which intermediate work persists and can be extended across independent agent episodes.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenTwo capabilities studied here are dual-use: reconstructing programs from an execute-only reference and its documentation could facilitate unauthorized re-implementation of proprietary software, and automating architecture research could accelerate AI development, including of systems with potential for misuse.
“Two capabilities studied here are dual-use: reconstructing programs from an execute-only reference and its documentation could facilitate unauthorized re-implementation of proprietary software, and automating architecture research could accelerate AI development, including of systems with potential for misuse.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningBecause GitSwarm lets LLM agents execute code and launch GPU jobs autonomously, workers operated in isolated Git worktrees and isolated environments with no internet access throughout the experiments.
“Because GitSwarm lets LLM agents execute code and launch GPU jobs autonomously, workers operated in isolated Git worktrees and isolated environments with no internet access throughout our experiments.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataTogether, these results show that persistent memory can support both the synthesis of earlier contributions and the accumulation of evidence that guides subsequent computation.
“Together, these results show that persistent memory can support both the synthesis of earlier contributions and the accumulation of evidence that guides subsequent computation.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationLanguage Agent Tree Search combines tree search with environmental feedback.
“Language Agent Tree Search combines tree search with environmental feedback (Zhou et al., 2024).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhou et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe authors have reviewed all AI-assisted work and take responsibility for the final content of the paper, including text, claims, and artifacts produced with the aid of generative AI.
“We have reviewed all AI-assisted work and take responsibility for the final content of this paper, including text, claims, and artifacts produced with the aid of generative AI.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenSemantic dependencies are declared by workers, not independently established as causal influence.
“Semantic dependencies are declared by workers, not independently established as causal influence.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Premisenone givenThe value of intermediate work is often unknown when it is produced.
“The value of intermediate work is often unknown when it is produced.”
Quote in the paperNot a substantial claim- Rests on an assumption: The value of intermediate work is often unknown when it is produced.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationProgramBench asks agents to reconstruct programs from documentation and an execute-only reference.
“ProgramBench (Yang et al., 2026) asks agents to reconstruct programs from documentation and an execute-only reference.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Yang et al., 2026 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenProgramBench performance is measured on a fixed 50-task subset out of a total of 200 tasks.
“ProgramBench performance is measured on a fixed 50-task subset out of a total of 200 tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA-MEM links and updates memories as new information arrives.
“A-MEM links and updates memories as new information arrives (Xu et al., 2025).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Xu et al., 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Study design and conductnone givenIn AutoResearch, the RMT swarm models and reference share the same 1.3B-token training budget, whereas the compared single-agent models use more training tokens.
“In AutoResearch, the RMT swarm models and reference share the same 1.3B-token training budget, whereas the compared single-agent models use more training tokens.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisenone givenLong-horizon problem solving and scientific research require computation to accumulate across successive attempts.
“Long-horizon problem solving and scientific research pose a complementary challenge: useful computation must also accumulate across attempts.”
Quote in the paperSubstantial claim- Rests on an assumption: Long-horizon problem solving and scientific research require computation to accumulate across successive attempts.
May combine two claims.
Interpretationown dataThus, programming typically accumulates contributions into a final implementation, while proof solving uses earlier work to compare and select candidates.
“Thus, programming typically accumulates contributions into a final implementation, while proof solving uses earlier work to compare and select candidates.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenVotes are not evidence of correctness.
“noting that votes are not evidence of correctness”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMeta-Harness searches over the code of LLM harnesses with a single coding-agent proposer that reads the source code, scores, and execution traces of all prior candidates from a filesystem.
“Meta-Harness (Lee et al., 2026) searches over the code of LLM harnesses with a single coding-agent proposer that reads the source code, scores, and execution traces of all prior candidates from a filesystem.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Lee et al., 2026 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationCAID uses parallel Git worktrees and branch-and-merge integration for asynchronous software engineering, with a central manager that plans and delegates subtasks.
“CAID uses parallel Git worktrees and branch-and-merge integration for asynchronous software engineering, with a central manager that plans and delegates subtasks (Geng and Neubig, 2026).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Geng and Neubig, 2026 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationGoyal et al. (2022) brought the global workspace idea to modern neural architectures, allowing learned specialist modules to communicate through a capacity-limited shared workspace.
“Goyal et al. (2022) brought the idea to modern neural architectures, allowing learned specialist modules to communicate through a capacity-limited shared workspace.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Goyal et al., 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenKim et al. summarize each agent trajectory into a structured record of its hypotheses, progress, and failures.
“They summarize each agent trajectory into a structured record of its hypotheses, progress, and failures.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationnone givenThese differences motivate distinct analyses.
“These differences motivate distinct analyses.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe AI Scientist automates stages from idea generation to paper writing.
“The AI Scientist automates stages from idea generation to paper writing (Lu et al., 2024)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Lu et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenAgora represents dependencies through Git parent relationships, including multiple parents, and provides evidence-aware and diversity-aware views of the research frontier.
“Agora represents dependencies through Git parent relationships, including multiple parents, and provides evidence-aware and diversity-aware views of the research frontier.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenGitSwarm's harness still centrally manages execution and applies deterministic plurality readout.
“Its harness still centrally manages execution and applies deterministic plurality readout.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenNomination records remain separate from the contribution graph, so evaluating an existing candidate does not alter its ancestry.
“Nomination records remain separate from the contribution graph, so evaluating an existing candidate does not alter its ancestry.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenPDR's Parallel-Distill-Refine method generates diverse drafts in parallel, distills them into a bounded textual workspace, and refines a candidate over successive rounds.
“Their Parallel-Distill-Refine method generates diverse drafts in parallel, distills them into a bounded textual workspace, and refines a candidate over successive rounds.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSqueezeEvolve follows a similar orchestration.
“SqueezeEvolve (Maheswaran et al., 2026) follows a similar orchestration.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Maheswaran et al., 2026 source
Checked before — its earlier verdict is held back until a new check runs.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningKim et al.'s approach distills trajectories into compact contexts for selection and subsequent rollouts.
“Their approach distills trajectories into compact contexts for selection and subsequent rollouts.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenThe sustained research experiments additionally illustrate architectural synthesis and the later reuse of negative experimental results.
“Our sustained research experiments additionally illustrate architectural synthesis and the later reuse of negative experimental results.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAutoGen supports configurable patterns of agent conversation.
“AutoGen supports configurable patterns of agent conversation (Wu et al., 2023).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Wu et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenGitSwarm is applied across program reconstruction, proof search, and architecture research rather than to a single class of artifact.
“GitSwarm is also applied across program reconstruction, proof search, and architecture research rather than to a single class of artifact.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenAgora's extended weight-transfer study emphasizes research diversity and independent verification.
“Agora's extended weight-transfer study instead emphasizes research diversity and independent verification.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationGPTSwarm represents agent operations and communication as an optimizable computation graph.
“GPTSwarm represents agent operations and communication as an optimizable computation graph (Zhuge et al., 2024).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhuge et al., 2024 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationKarpathy's AutoResearch repeatedly modifies, trains, and evaluates a NanoChat model in a single-agent loop.
“Karpathy's AutoResearch repeatedly modifies, trains, and evaluates a NanoChat model in a single-agent loop (Karpathy, 2026).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Karpathy, 2026
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningTheir dependency tracking and selection mechanisms differ.
“Their dependency tracking and selection mechanisms differ.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenGitSwarm separates the physical Git parent from additional mandatory informed_by= references, allowing a worker to build on one branch while explicitly citing discoveries from others.
“GitSwarm separates the physical Git parent from additional mandatory informed_by= references, allowing a worker to build on one branch while explicitly citing discoveries from others.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThese systems demonstrate how version control can organize parallel agent work.
“These systems demonstrate how version control can organize parallel agent work.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningLike GitSwarm, Kim et al.'s method allows later computation to benefit from earlier attempts.
“Like GitSwarm, their method allows later computation to benefit from earlier attempts.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationnone givenGitSwarm keeps the worker model fixed and accumulates contributions to an external research task.
“GitSwarm instead keeps the worker model fixed and accumulates contributions to an external research task.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenAgora's workers also choose their own research directions without assigned roles or a central planner.
“Its workers also choose their own research directions without assigned roles or a central planner.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningBecause this scientist carries the entire search, ARTS condenses the growing history into short written insights and, in a variant, into the model's weights via test-time training, and it returns the best validated node.
“Because this scientist carries the entire search, ARTS condenses the growing history into short written insights and, in a variant, into the model's weights via test-time training, and it returns the best validated node.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenBoth systems preserve intermediate research in a Git-backed contribution graph, including alternative approaches and unsuccessful experiments.
“Both systems preserve intermediate research in a Git-backed contribution graph, including alternative approaches and unsuccessful experiments.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe two studies provide complementary evidence about repository-mediated collective research rather than a direct performance comparison.
“The two studies provide complementary evidence about repository-mediated collective research rather than a direct performance comparison.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMetaGPT assigns specialized roles within a prescribed software-development workflow.
“MetaGPT assigns specialized roles within a prescribed software-development workflow (Hong et al., 2024)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hong et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationGlobal Workspace Theory organizes cognition around multiple specialist processes whose information can gain access to a shared broadcast workspace.
“Global Workspace Theory similarly organizes cognition around multiple specialist processes whose information can gain access to a shared broadcast workspace (Baars, 1988).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Baars, 1988
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenCAID and SwarmResearch use Git branches to support parallel work, but organize that work through a central manager or Shepherd Agent.
“CAID and SwarmResearch use Git branches to support parallel work, but organize that work through a central manager or Shepherd Agent.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningRather than maintaining a transient common state for coordinating computation within a cognitive cycle or neural forward process, GitSwarm preserves an explicit history of contributions across agent episodes.
“Rather than maintaining a transient common state for coordinating computation within a cognitive cycle or neural forward process, GitSwarm preserves an explicit history of contributions across agent episodes.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.
InterpretationreasoningThese systems demonstrate the value of experimental feedback, persistent candidate pools, and parallel research.
“These systems demonstrate the value of experimental feedback, persistent candidate pools, and parallel research.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenThese summaries support recursive tournament voting over parallel attempts and sequential refinement through Parallel-Distill-Refine (PDR).
“These summaries support recursive tournament voting over parallel attempts and sequential refinement through Parallel-Distill-Refine (PDR).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMulti-agent debate improves candidate answers through interaction between model instances.
“Multi-agent debate improves candidate answers through interaction between model instances (Du et al., 2023)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Du et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningGPTSwarm's graph specifies how a multi-agent program executes, whereas GitSwarm records how published contributions accumulate over time.
“Its graph specifies how a multi-agent program executes, whereas ours records how published contributions accumulate over time.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe experiments examine how independently scheduled workers accumulate code and experimental evidence through shared persistent memory.
“Our experiments instead examine how independently scheduled workers accumulate code and experimental evidence through shared persistent memory.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenAgora's contributions retain their dependencies through Git parent relationships, including multiple parents, while searchable frontier views and diversity-aware recommendations help workers navigate the research history.
“Contributions retain their dependencies through Git parent relationships, including multiple parents, while searchable frontier views and diversity-aware recommendations help workers navigate the research history.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMixture-of-Agents aggregates responses across successive model layers.
“Mixture-of-Agents aggregates responses across successive model layers (Wang et al., 2025).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Wang et al., 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningThese works motivate evaluation beyond fixed-answer coding tasks.
“These works motivate evaluation beyond fixed-answer coding tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenAgora's workers operate without assigned tasks or a central planner.
“Its workers operate without assigned tasks or a central planner.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenSystems can provide several of the compared properties through different mechanisms.
“In particular, systems can provide several of them through different mechanisms.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThese observations demonstrate reuse, but do not by themselves establish its causal effect on performance.
“These observations demonstrate reuse, but do not by themselves establish its causal effect on performance.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenEach accepted commit in GitSwarm has a physical Git parent and must declare its semantic dependencies through informed_by=, including references to work on other branches.
“Each accepted commit has a physical Git parent and must declare its semantic dependencies through informed_by=, including references to work on other branches.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningGitSwarm shares the goal of building on earlier computation but uses a different memory model.
“GitSwarm shares the goal of building on earlier computation but uses a different memory model.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenGitSwarm separates each commit's physical Git parent from mandatory semantic informed_by= references and selects final candidates through worker nominations and deterministic plurality readout.
“It separates each commit's physical Git parent from mandatory semantic informed_by= references and selects final candidates through worker nominations and deterministic plurality readout.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationnone givenRather than repeatedly compressing drafts into a single workspace, GitSwarm preserves individual contributions in a shared, branchable repository.
“Rather than repeatedly compressing drafts into a single workspace, it preserves individual contributions in a shared, branchable repository.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenCAID delegates dependent software engineering subtasks; SwarmResearch chooses starting points and steers exploration.
“The former delegates dependent software engineering subtasks; the latter chooses starting points and steers exploration.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningGitSwarm studies a closely related design space to Agora with a different contribution and selection protocol.
“GitSwarm studies a closely related design space with a different contribution and selection protocol.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenThe evaluation examines fixed-answer benchmarks, controlled substrate properties, and sustained GPU-backed research, with separate measurements of declared dependencies, artifact consumption, and selected-solution ancestry.
“Our evaluation also examines fixed-answer benchmarks, controlled substrate properties, and sustained GPU-backed research, with separate measurements of declared dependencies, artifact consumption, and selected-solution ancestry.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMLGym additionally provides an environment for developing reinforcement learning algorithms for research agents.
“MLGym additionally provides an environment for developing reinforcement learning algorithms for research agents.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Nathani et al., 2025 source
Checked before — its earlier verdict is held back until a new check runs.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningAgora is closer in its control model to GitSwarm.
“Agora is closer in its control model.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe comparison is of the protocols described in the cited papers, rather than every configuration their underlying frameworks could support.
“We compare the protocols described in the cited papers, rather than every configuration their underlying frameworks could support.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAgentVerse studies configurable teams and collaboration patterns.
“AgentVerse studies configurable teams and collaboration patterns (Chen et al., 2024).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Chen et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningBranches and explicit dependencies let many asynchronous workers build on one another concurrently under version control, and beyond a small core layout, workers choose which artifacts to leave.
“Branches and explicit dependencies let many asynchronous workers build on one another concurrently under version control, and beyond a small core layout, workers choose which artifacts to leave.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenGitSwarm's repository preserves competing experiments, including unsuccessful ones, with explicit dependencies between contributions.
“Its repository preserves competing experiments, including unsuccessful ones, with explicit dependencies between contributions.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Premisenone givenMeta-Harness's search proceeds in sequential iterations, and every stored candidate is a complete harness, evaluated on a fixed search set and logged in a predefined per-candidate directory.
“Its search, however, proceeds in sequential iterations, and every stored candidate is a complete harness, evaluated on a fixed search set and logged in a predefined per-candidate directory.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenThe paper measures declared cross-branch dependencies, trace-visible artifact consumption, and the transitive ancestry of selected solutions across programming and proof benchmarks.
“We measure declared cross-branch dependencies, trace-visible artifact consumption, and the transitive ancestry of selected solutions across programming and proof benchmarks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenIndependent workers choose which previous results to inspect or extend, while each commit explicitly records its dependencies across branches.
“Independent workers choose which previous results to inspect or extend, while each commit explicitly records its dependencies across branches.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenGitSwarm instead preserves individual contributions and artifacts in a shared, branchable repository.
“GitSwarm instead preserves individual contributions and artifacts in a shared, branchable repository.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningThe GitSwarm workspace is durable and branchable, and dependencies between contributions are recorded explicitly, allowing competing lines of work to persist and later recombine.
“The workspace is durable and branchable, and dependencies between contributions are recorded explicitly, allowing competing lines of work to persist and later recombine.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningPDR and RSA allow inference to accumulate information without continually expanding the model's context.
“PDR and RSA allow inference to accumulate information without continually expanding the model's context.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenWorkers independently choose which contributions to extend, while each commit explicitly records its dependencies across branches.
“Workers independently choose which contributions to extend, while each commit explicitly records its dependencies across branches.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAgora, developed in parallel with this work, also investigates compounding research by self-directed agents over a persistent Git-backed contribution graph.
“Agora (Zhang et al., 2026), developed in parallel with our work, also investigates compounding research by self-directed agents over a persistent Git-backed contribution graph.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhang et al., 2026 source
Checked before — its earlier verdict is held back until a new check runs.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone givenLike GitSwarm, Meta-Harness exposes the full, uncompressed history for selective retrieval rather than summarizing it.
“Like GitSwarm, it exposes the full, uncompressed history for selective retrieval rather than summarizing it.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationAIRA 2 combines agentic experimentation, consistent evaluation, and asynchronous multi-GPU execution.
“AIRA 2 combines agentic experimentation, consistent evaluation, and asynchronous multi-GPU execution (Hambardzumyan et al., 2026).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hambardzumyan et al., 2026 source
Checked before — its earlier verdict is held back until a new check runs.
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenGitSwarm has no central scientist: independent workers, each starting with a fresh context, retrieve what they need from a shared version-controlled persistent repository that is never summarized or pruned, can combine separate branches into a single contribution, and select the final solution through decentralized nomination when no reliable validation score exists.
“GitSwarm has no central scientist: independent workers, each starting with a fresh context, retrieve what they need from a shared version-controlled persistent repository that is never summarized or pruned, can combine separate branches into a single contribution, and select the final solution through decentralized nomination when no reliable validation score exists.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningGitSwarm contributions can be bounded partial work that later workers extend, providing a higher branching factor and thus allowing more exploration.
“GitSwarm contributions can instead be bounded partial work that later workers extend, providing a higher branching factor and thus allowing more exploration.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenWorkers nominate candidate solutions, and the harness applies deterministic plurality readout.
“Workers nominate candidate solutions, and the harness applies deterministic plurality readout.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningGitSwarm inherits the shared-workspace principle but changes the temporal and structural role of the workspace.
“GitSwarm inherits the shared-workspace principle but changes the temporal and structural role of the workspace.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThe seven properties compared in Table 2 are not independent, nor are they a performance ranking.
“These properties are not independent, nor are they a performance ranking.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenSource-level checks confirm exact MLP/Block and attention-class transfers for the selected ports, rather than relying solely on matching prose descriptions.
“Source-level checks confirm exact MLP/Block and attention-class transfers for the selected ports, rather than relying solely on matching prose descriptions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThe seeded run was heavily rate-limited by a shared API quota, so its pace is not comparable to the original runs.
“The run was heavily rate-limited by a shared API quota, so its pace is not comparable to the original runs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThese comparisons retain initialization, optimizer-shape and fixed-time-throughput confounds, but go beyond scalar hyperparameter tuning.
“These comparisons retain initialization, optimizer-shape and fixed-time-throughput confounds, but go beyond scalar hyperparameter tuning.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown dataIn every condition, k > D, so an individual worker can reach the treasure by choosing the correct route; no treatment is made impossible by its step limit.
“In every condition, k > D, so an individual worker can reach the treasure by choosing the correct route; no treatment is made impossible by its step limit.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThe controlled experiments provide controlled illustrations of why persistence, branchability, and attribution may be useful for compounding inference.
“They provide controlled illustrations of why persistence, branchability, and attribution may be useful for compounding inference.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationIn Huginn, the prelude embeds the input, the core is iterated a variable number of times with the prelude output re-injected at every iteration, and the coda decodes the final latent state, so test-time compute can be scaled by unrolling more iterations instead of generating more tokens.
“The prelude embeds the input, the core is iterated a variable number of times with the prelude output re-injected at every iteration-an adapter mixes the current state with the encoded input-and the coda decodes the final latent state, so test-time compute can be scaled by unrolling more iterations instead of generating more tokens.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Geiping et al., 2025 source
Checked before — its earlier verdict is held back until a new check runs.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThis experiment compares practical memory interfaces rather than changing branchability alone: the branch-native treatment also makes checkpoint restoration explicit in the worker prompt.
“This experiment compares practical memory interfaces rather than changing branchability alone: the branch-native treatment also makes checkpoint restoration explicit in the worker prompt.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningGitSwarm's gains come entirely from architectural changes at the reference training budget, half of them specific to the RMT's matrix memory, which is the kind of change the task asks for.
“GitSwarm's gains, by contrast, come entirely from architectural changes at the reference training budget, half of them specific to the RMT's matrix memory, which is the kind of change the task asks for.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductreasoningThe aggregate budget of the two independent baselines is approximately 4Q/3 > Q, so their lower coverage cannot be explained by having too few total queries.
“Their aggregate budget is approximately 4Q/3 > Q, so their lower coverage cannot be explained by having too few total queries.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationown dataThis episode shows shared in-flight state changing whether an experiment occurred at all, but does not establish that coordination was universally efficient or quantify aggregate resource savings.
“This episode shows shared in-flight state changing whether an experiment occurred at all, but does not establish that coordination was universally efficient or quantify aggregate resource savings.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThus, the earlier evidence informed both a revised conclusion and a later implementation decision; it did not establish that the epsilon change was intrinsically harmful.
“Thus, the earlier evidence informed both a revised conclusion and a later implementation decision; it did not establish that the epsilon change was intrinsically harmful.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoningIt was not a randomized crossover study: training-token exposure, compiler state and measurement time remained uncontrolled, so it did not identify an order effect or establish expected architectural superiority.
“It was not a randomized crossover study: training-token exposure, compiler state and measurement time remained uncontrolled, so it did not identify an order effect or establish expected architectural superiority.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningBoth treatments have persistent observations: they differ in whether explicit attribution (dependency) metadata is available, not in whether their accumulated observations persist.
“Thus both treatments have persistent observations: they differ in whether explicit attribution (dependency) metadata is available, not in whether their accumulated observations persist.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationown dataThis documents measured consolidation rather than treating every inherited detail as indispensable; the synthesis does not isolate a coefficient-specific benefit.
“This documents measured consolidation rather than treating every inherited detail as indispensable; the synthesis does not isolate a coefficient-specific benefit.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThis is documented reconsideration of a failed design, not proof that trigram features alone caused the eventual gain: the surrounding model, capacity and optimizer also changed.
“This is documented reconsideration of a failed design, not proof that trigram features alone caused the eventual gain: the surrounding model, capacity and optimizer also changed.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningBranch-native memory makes both earlier endpoints and their relationships directly available for subsequent exploration.
“Branch-native memory thus makes both earlier endpoints and their relationships directly available for subsequent exploration.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThus this is substantive architecture and capacity search, but not an architecture-only, equal-parameter or equal-token comparison.
“Thus this is substantive architecture and capacity search, but not an architecture-only, equal-parameter or equal-token comparison.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningThe per-worker read budget is k = ⌈|A|/4⌉, where |A| is the generated archive size, so the combined budget of the four workers is sufficient to read the complete archive in principle.
“The per-worker read budget is k = ⌈|A|/4⌉, where |A| is the generated archive size, so the combined budget of the four workers is sufficient to read the complete archive in principle.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Study design and conductnone givenThe harness validates every restored checkpoint and commits the final state after the episode, so workers cannot fabricate states or modify previous work.
“The harness validates every restored checkpoint and commits the final state after the episode, so workers cannot fabricate states or modify previous work.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenEach synthetic task isolates a setting in which one property of the shared substrate (persistence, branchability, or attribution) should reduce a particular form of repeated work.
“Each synthetic task isolates a setting in which one property of the shared substrate (persistence, branchability, or attribution; Section 2.1) should reduce a particular form of repeated work.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone givenPrevious memory is read-only in all treatments.
“Previous memory is read-only in all treatments.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningThey do not establish that shared memory caused the performance gain or identify each reused idea's contribution; those claims require an appropriate comparison or ablation.
“They do not establish that shared memory caused the performance gain or identify each reused idea's contribution; those claims require an appropriate comparison or ablation.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe three studies extend the evaluation from benchmarks with well-defined target solutions to multi-hour research tasks requiring architectural decisions, experimental feedback, and shared GPU resources, and they provide complementary evidence: RMT includes completed held-out evaluations of improved candidates; the Looped Transformer shows improvements during the recorded development trajectories;
“The three studies extend the evaluation from benchmarks with well-defined target solutions to multi-hour research tasks requiring architectural decisions, experimental feedback, and shared GPU resources, and they provide complementary evidence: RMT includes completed held-out evaluations of improved candidates; the Looped Transformer shows improvements during the recorded development trajectories;”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThe repository provides a mechanism for preserving experimental artifacts and coordinating subsequent work, but attributing a specific architectural improvement to earlier contributions requires analysis of the corresponding execution traces and experimental dependencies.
“The repository provides a mechanism for preserving experimental artifacts and coordinating subsequent work, but attributing a specific architectural improvement to earlier contributions requires analysis of the corresponding execution traces and experimental dependencies (see also the discussion of causal attribution in Section 4).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningThe two panels of Figure 28 answer different questions: the ProgramBench review assesses whether the cited work covered the intended contribution, while the IMO audit checks only whether the named evidence existed.
“The two panels of Figure 28 answer different questions: the ProgramBench review assesses whether the cited work covered the intended contribution, while the IMO audit checks only whether the named evidence existed.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningAn improving research trajectory does not by itself establish how much of that improvement comes from access to previous workers' discoveries, nor does improving a research objective reveal which earlier experiments informed subsequent research decisions.
“An improving research trajectory does not by itself establish how much of that improvement comes from access to previous workers' discoveries, nor does improving a research objective reveal which earlier experiments informed subsequent research decisions.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenThese files act as persistent communication between episodes: a later worker can inspect a finding, rerun a probe, or adapt a test without reconstructing the producer's private tool transcript.
“These files act as persistent communication between episodes: a later worker can inspect a finding, rerun a probe, or adapt a test without reconstructing the producer's private tool transcript.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningThis explains how the final combination can emerge through measured branches without assuming that every isolated improvement transfers additively.
“This explains how the final combination can emerge through measured branches without assuming that every isolated improvement transfers additively.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe usual behavior is parallel work (editing, building, testing, and probing) followed by a late check of the durable result, after which the worker stops; abstention therefore prevents a duplicate commit but not the computation already spent.
“The usual behavior is parallel work (editing, building, testing, and probing) followed by a late check of the durable result, after which the worker stops; abstention therefore prevents a duplicate commit but not the computation already spent.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe high ProgramBench rate shows that abstention usually prevents duplicate commits, while the few unrecovered cases expose its main risk.
“The high ProgramBench rate shows that abstention usually prevents duplicate commits, while the few unrecovered cases expose its main risk.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisenone givenWorkers may nominate an existing candidate rather than publish another contribution.
“Workers may nominate an existing candidate rather than publish another contribution (Section 2.3).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisenone givenAs the repository grows, workers could incur increasing costs if they repeatedly inspected every previous contribution.
“As the repository grows, workers could incur increasing costs if they repeatedly inspected every previous contribution.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningThe resulting counts describe reusable units rather than the serialization format chosen by a worker.
“The resulting counts describe reusable units rather than the serialization format chosen by a worker.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisenone givenWorkers have no fixed solver, verifier, or integrator roles, whether from the scheduler or the prompt.
“Because workers have no fixed solver, verifier, or integrator roles, whether from the scheduler or the prompt (Section 2.3), we infer activity labels after the run to examine whether the mixture of observed work changes over its course.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenA nomination is a worker preference, not an independent verification of correctness: repeated nominations for one SHA are coordination signals, not independent proofs of correctness.
“A nomination is a worker preference, not an independent verification of correctness: repeated nominations for one SHA are coordination signals, not independent proofs of correctness.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.
InterpretationreasoningThis lower rate need not mean that the proof artifacts are less useful: a computation may rule out an approach after being read, while the metric requires it to be run or adapted and cited by a new contribution.
“This lower rate need not mean that the proof artifacts are less useful: a computation may rule out an approach after being read, while the metric requires it to be run or adapted and cited by a new contribution.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisenone givenWorkers may explicitly abstain when their intended contribution is already covered by published or in-flight work, or when no useful nonredundant change remains; voluntary abstainers are replaced without consuming the charged budget but still incur token costs for the inspection and reasoning they perform before abstaining.
“Workers may explicitly abstain when their intended contribution is already covered by published or in-flight work, or when no useful nonredundant change remains; voluntary abstainers are replaced without consuming the charged budget but still incur token costs for the inspection and reasoning they perform before abstaining (Section 2.3).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThis concentration reflects the task: proof workers often use symbolic checks or numerical experiments to test a proof idea, whereas programming workers more often preserve behavioral findings, fixtures, and regression tests.
“This concentration reflects the task: proof workers often use symbolic checks or numerical experiments to test a proof idea, whereas programming workers more often preserve behavioral findings, fixtures, and regression tests.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThese values do not include the larger set of no-action, invalid, and selection-related proof episodes.
“These values do not include the larger set of no-action, invalid, and selection-related proof episodes.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe traces document dependencies in experimental judgment, which need not be code-inheritance edges.
“The traces document dependencies in experimental judgment, which need not be code-inheritance edges.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe shared startup followed by different mature phases shows that one decentralized protocol can adapt its workflow to the task: program reconstruction keeps accumulating implementation changes, whereas these proof runs increasingly use the repository to evaluate work that already exists.
“The shared startup followed by different mature phases shows that one decentralized protocol can adapt its workflow to the task: program reconstruction keeps accumulating implementation changes, whereas these proof runs increasingly use the repository to evaluate work that already exists.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningWorkers are not losing access to old work; they are moving from broad orientation to selective use of a larger history.
“Workers are not losing access to old work; they are moving from broad orientation to selective use of a larger history.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenThis run explicitly assigned experimenter/synthesizer and critic/verifier roles at a configured 3:1 ratio: it supports adaptive work selection within those roles, not specialization without predefined roles.
“This run explicitly assigned experimenter/synthesizer and critic/verifier roles at a configured 3:1 ratio: it supports adaptive work selection within those roles, not specialization without predefined roles.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningWe therefore describe the extension's endpoint as a best observation rather than a verified final architecture.
“We therefore describe the extension's endpoint as a best observation rather than a verified final architecture.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningWith up to five concurrent workers per problem, several physical branches advance at once, so a single Git-parent chain cannot contain most contributions.
“with up to five concurrent workers per problem, several physical branches advance at once, so a single Git-parent chain cannot contain most contributions.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningAdditional compute extends implementation work on ProgramBench, whereas larger proof budgets extend comparison and ratification of a small frontier more than construction of new proofs.
“Additional compute therefore extends implementation work on ProgramBench, whereas larger proof budgets extend comparison and ratification of a small frontier more than construction of new proofs.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThis is late integration because the decisive candidate depends on several mature branches rather than one steadily refined line.
“This is late integration because the decisive candidate depends on several mature branches rather than one steadily refined line.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataThe proof runs differ in configuration, so we treat cross-budget differences in voting behavior and timing as descriptive rather than as a controlled change in budget alone.
“Moreover, the proof runs differ in configuration (Appendix B.4), so we treat cross-budget differences in voting behavior and timing as descriptive rather than as a controlled change in budget alone.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe contrast between frequent access and weaker winner linkage on the proof tasks shows why consumption and final synthesis are separate outcomes: a worker may read an artifact without using it in a subsequent contribution, and an artifact may support verification without appearing in the selected solution's ancestry.
“The contrast between frequent access and weaker winner linkage on the proof tasks shows why consumption and final synthesis are separate outcomes: a worker may read an artifact without using it in a subsequent contribution, and an artifact may support verification without appearing in the selected solution's ancestry.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductown dataWe calibrated the ProgramBench judgments with a score-blind review of all 422 committed-only collisions and five pre-registered in-flight-only cases per problem and budget, for 1,422 abstention traces in total: 289 at B100, 315 at B200, 346 at B300, and 472 at B500.
“We calibrated the ProgramBench judgments with a score-blind review of all 422 committed-only collisions and five pre-registered in-flight-only cases per problem and budget, for 1,422 abstention traces in total: 289 at B100, 315 at B200, 346 at B300, and 472 at B500.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Study design and conductreasoningThis is a trace-evidence check, not a mathematical equivalence judgment.
“This is a trace-evidence check, not a mathematical equivalence judgment.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningPersistent state supports both synthesis and selection, while winner ancestry measures only the former.
“Persistent state supports both synthesis and selection, while winner ancestry measures only the former.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningWe derive the activity labels bottom-up and independently for each benchmark, with separate taxonomies for programming and proof work.
“We derive the labels bottom-up and independently for each benchmark, with separate taxonomies for programming and proof work.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe artifact-flow measurements support a general claim of artifact-mediated coordination, but not a claim that every read is preserved in the final candidate; neither access nor winner linkage establishes the causal effect of artifact access on score.
“The artifact-flow measurements therefore support a general claim of artifact-mediated coordination, but not a claim that every read is preserved in the final candidate; neither access nor winner linkage establishes the causal effect of artifact access on score.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningFile evidence comes from the diff against the worker's physical Git parent: only files changed by that worker enter the classification, so inherited files are not attributed to the worker and do not make a descendant appear to repeat its parent's work.
“File evidence comes from the diff against the worker's physical Git parent: only files changed by that worker enter the classification, so inherited files are not attributed to the worker and do not make a descendant appear to repeat its parent's work.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenNo architecture-only attribution, calibrated confidence interval, or blanket semantic certification of all attempted programs is claimed.
“No architecture-only attribution, calibrated confidence interval, or blanket semantic certification of all attempted programs is claimed.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataIt establishes that the cited work existed or materialized, but not that two proofs are mathematically equivalent, that the cited proof is correct, or that abstaining workers correctly assessed the cited proofs or their own potential contributions.
“It establishes that the cited work existed or materialized, but not that two proofs are mathematically equivalent, that the cited proof is correct, or that abstaining workers correctly assessed the cited proofs or their own potential contributions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenA workflow archetype is not a role schedule imposed by the prompt.
“It is not a role schedule imposed by the prompt.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThe pivot from CLI/configuration work to renderer work in the ProgramBench run is a gradual change in the dominant search direction, not an abrupt replacement of all earlier work.
“The pivot is a gradual change in the dominant search direction, not an abrupt replacement of all earlier work.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningThey do not isolate the individual contributions of persistence, branchability, or semantic dependencies to benchmark performance; the controlled tests in Appendix C examine the operational benefits of these properties separately.
“They do not isolate the individual contributions of persistence, branchability, or semantic dependencies to benchmark performance; the controlled tests in Appendix C examine the operational benefits of these properties separately.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataIMOProofBench-Advanced has more selection outcomes and more no-action or invalid episodes, consistent with its smaller active proof frontier.
“IMOProofBench-Advanced has more selection outcomes and more no-action or invalid episodes, consistent with its smaller active proof frontier.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThese comparisons characterize execution and readout choices within GitSwarm.
“These comparisons characterize execution and readout choices within GitSwarm.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThis coordination includes synthesis into later solutions, but also comparison, redundancy avoidance, and selective navigation whose influence may not appear in the selected solution's ancestry.
“This coordination includes synthesis into later solutions, but also comparison, redundancy avoidance, and selective navigation whose influence may not appear in the selected solution's ancestry.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenFor the sequential shell interaction runs, the harness runs each shell block with set -e, skips the remaining blocks of a response once a block fails, and rejects any pre-branch command other than read-only inspection, coordination, a standalone gs_fork, or a standalone gs_abstain.
“For these runs, the harness also runs each shell block with set -e, skips the remaining blocks of a response once a block fails, and rejects any pre-branch command other than read-only inspection, coordination, a standalone gs_fork, or a standalone gs_abstain.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe exact contribution share is partly shaped by five-worker concurrency, but the persistence of repair, expansion, and synthesis matters more than the precise value.
“The exact contribution share is partly shaped by five-worker concurrency, but the persistence of repair, expansion, and synthesis (Figure 10) matters more than the precise value.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningThe sequential shell interaction extension asks workers to wait for each command's output before issuing dependent commands, to run gs_fork as a standalone step, and to use only read-only inspection and coordination commands before branching succeeds.
“The extension asks workers to wait for each command's output before issuing commands that depend on it, to run gs_fork as a standalone step, and to use only read-only inspection and coordination commands before branching succeeds.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premiseown dataThe default configuration in the ProgramBench architectural tradeoffs is asynchronous execution with decentralized nomination.
“The default configuration (asynchronous execution with decentralized nomination) is the control in both blocks.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe shared behavior across benchmarks is inspection and comparison, while the exact coordination tool depends on the worker interface.
“The shared behavior is inspection and comparison, while the exact coordination tool depends on the worker interface.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe central selector has the higher observed mean in this development-set comparison, but the limited number of seeds does not establish a general advantage of either readout or their statistical equivalence.
“The central selector has the higher observed mean in this development-set comparison, but the limited number of seeds does not establish a general advantage of either readout or their statistical equivalence.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe present budgets do not establish that this behavior will remain effective, or persist without degradation, at substantially larger graph sizes.
“The present budgets do not establish that this behavior will remain effective, or persist without degradation, at substantially larger graph sizes.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThese additions constrain how and when shell commands run (for example, scripts and tests can run only after branching), while the graph, scope, dependency, and nomination rules are unchanged.
“These additions constrain how and when shell commands run (for example, scripts and tests can run only after branching); the graph, scope, dependency, and nomination rules are unchanged.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataIn this study, forcing a final synthesis step was neither more accurate nor more efficient than decentralized nomination.
“In this study, forcing a final synthesis step was thus neither more accurate nor more efficient than decentralized nomination.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenThe two-node GPT-5.6 run in the Looped Transformer research is not compute-matched to the other runs.
“the two-node GPT-5.6 run is not compute-matched”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataAll configurations in the ProgramBench architectural tradeoffs used GPT-5.5-medium, B = 50 worker episodes per task, and maximum concurrency five.
“All configurations use GPT-5.5-medium,B = 50 worker episodes per task, and maximum concurrency five.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisenone givenThe synchronous variant executes barriers of five workers: it waits for every worker in a five-worker wave to finish before launching the next wave.
“The synchronous variant instead executes barriers of five workers: it waits for every worker in a five-worker wave to finish before launching the next wave.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisenone givenBy default, the submitted candidate is determined by decentralized worker nominations.
“By default, the submitted candidate is determined by decentralized worker nominations (Section 2.5).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisenone givenThe default scheduler launches a replacement worker whenever a worker slot becomes available.
“The default scheduler launches a replacement worker whenever a worker slot becomes available.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningUnder these conditions, asynchronous execution reduces latency with a small observed difference in mean score; the experiment does not establish statistical equivalence between the two schedules.
“Under these conditions, asynchronous execution therefore reduces latency with a small observed difference in mean score; the experiment does not establish statistical equivalence between the two schedules.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThese shared and task-specific patterns indicate selective navigation rather than exhaustive reinspection, and suggest that the repository is used as a searchable memory rather than a transcript that every worker rereads in full.
“These shared and task-specific patterns indicate selective navigation rather than exhaustive reinspection, and suggest that the repository is used as a searchable memory rather than a transcript that every worker rereads in full.”
Quote in the paperSubstantial claimMay combine two claims.
Premisenone givenUnsuccessful charged episodes are distinct from voluntary abstentions, which are replaced without consuming the charged budget.
“These unsuccessful charged episodes are distinct from voluntary abstentions, which are replaced without consuming the charged budget (Section 2.3).”
Quote in the paperNot a substantial claim
The paper relies on this without support.
Long-horizon problem solving and scientific research require computation to accumulate across successive attempts.
“Long-horizon problem solving and scientific research pose a complementary challenge: useful computation must also accumulate across attempts.”
The paper relies on this without support.
The value of intermediate work is often unknown when it is produced.
“The value of intermediate work is often unknown when it is produced.”
The paper relies on this without support.
If such work disappears when an episode terminates, later computation may need to rediscover it.
“If such work disappears when an episode terminates, later computation may need to rediscover it.”
The paper relies on this without support.
Preserving intermediate work allows subsequent agents to build on an accumulated body of prior work.
“Preserving it instead allows subsequent agents to build on an accumulated body of prior work.”
The paper relies on this without support.
The LLM analyzer's semantic judgments are reliable.
“Experiments combine semantic judgments by an LLM analyzer, and also to design synthetic tasks.”
The paper relies on this without support.
The fixed evaluators correctly score candidates.
“Candidates were scored by fixed evaluators on protected development and separate held-out sets.”
The paper relies on this without support.
The synthetic tasks isolate the intended substrate properties (persistence, branchability, attribution).
“Each synthetic task isolates a setting in which one property of the shared substrate (persistence, branchability, or attribution; Section 2.1) should reduce a particular form of repeated work.”
The paper relies on this without support.
The benchmarks (IMOProofBench-Advanced and ProgramBench) are valid measures of the capabilities studied.
“all experiments use public benchmarks (IMOProofBench-Advanced and ProgramBench) and research tasks based on publicly available research papers.”
The paper relies on this without support.
The fraction of unique archive artifacts read before the first exact submission measures the information-gathering cost of identifying valid conclusions.
“The primary metric is the fraction of unique archive artifacts read before the first exact submission. It measures the information-gathering cost of identifying valid conclusions, and distinguishes the ability to follow recorded dependencies from the cost of reconstructing those dependencies from content.”
The paper relies on this without support.
The architecture-improvement tasks require coordinating expensive experiments under limited computational resources.
“Unlike mathematical proofs or program reconstruction, these tasks require coordinating expensive experiments under limited computational resources.”
The paper relies on this without support.
The worker budget B is a controlled system-level knob.
“The worker budget B is the controlled system-level knob, while recorded input and output tokens provide model-level measures of the computation that was actually consumed.”
The paper relies on this without support.
The reference loss of 3.158 is a valid baseline against which RMT development loss is compared.
“the reference loss is 3.158”
The paper relies on this without support.
The benchmark score, defined as the fraction of hidden cases on which a candidate's behavior matches the reference, is a valid measure of behavioral reproduction.
“The benchmark score is the fraction of hidden cases on which the candidate's behavior matches the reference.”
The paper relies on this without support.
The 50 evaluation tasks are representative of the ProgramBench task distribution.
“The 50 evaluation tasks, grouped by implementation language.”
The paper relies on this without support.
The reference program's behavior is the correct target that a candidate solution must reproduce.
“The execute-only reference binary is at `./executable_orig`in the root of your current Git worktree.”
The paper relies on this without support.
The three judge passes give a reliable measure of proof correctness on IMOProofBench-Advanced.
“Gemini points: mean of three judge passes.”
The paper relies on this without support.
Worker budget is a valid proxy for inference-time compute.
“Each GitSwarm point represents one run at the indicated worker budget.”
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 1
- Bernard J. Baars. A Cognitive Theory of Consciousness. Cambridge University Press, Cambridge, 1988. ISBN 9780521301336.
- Authors
- Shah, V., Samanta, A., Dahal, P., Plekhanov, M., Wu, C.-J., Yih, S., Munos, R., Fergus, R., Foerster, J., Salakhutdinov, R., Arora, S., Weston, J., Courville, A., & Goyal, A.
- Affiliations
- Columbia University (US)
- Carnegie Mellon University (US)
- Princeton University (US)
- Mila - Quebec Artificial Intelligence Institute (CA)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2610.04862
- OpenAlex
- W7220715303
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Keywords
- decentralized agents, persistent memory, Git repositories, semantic dependencies, program synthesis, neural architecture search, GPT-5
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-28.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-28.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review