AgentRoom: Concurrent Multi-Agent Coding in a CRDT-Backed Shared Workspace

May combine two claims.

Implicit concurrent coordination via CRDTs has been explored but with mixed results (21% speedup / 39% slowdown across tasks).

“Implicit concurrent coordination via CRDTs has been explored [Pugachev, 2025] but with mixed results (21% speedup / 39% slowdown across tasks).”

• cites Pugachev, 2025 sourceSupported

May combine two claims.

CRDT convergence already provides the safety property advisory claims need; causality is established by the underlying CRDT which inherits Lamport's happens-before for ordering concurrent edits, in contrast to the operational-transformation lineage that requires a central transformation server.

“CRDT convergence already provides the safety property advisory claims need; causality is established by the underlying CRDT [Shapiro et al., 2011] which inherits Lamport's happens-before [Lamport, 1978] for ordering concurrent edits, in contrast to the operational-transformation lineage [Ellis and Gibbs, 1989, Sun et al., 1998] that requires a central transformation server.”

• cites Shapiro et al., 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Lamport, 1978 sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites Ellis and Gibbs, 1989 sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

CodeCRDT's empirical question is speedup over sequential baselines, reported bimodal at 21% speedup / 39% slowdown, semantic conflict rate 5-10%.

“the empirical question is speedup over sequential baselines (reported bimodal at 21% speedup / 39% slowdown, semantic conflict rate 5 -10% )”

• cites Pugachev, 2025 sourcePartly supported

  Section 6.2 (p. 7) compares parallel and sequential execution and reports task-dependent outcomes ranging from 21.1% speedup to 39.4% slowdown. The Introduction (p. 1) gives a preliminary estimate of 5–10% semantic conflicts. The paper does not characterize the outcomes as bimodal, and the claim omits the preliminary qualifier on the conflict estimate. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

May combine two claims.

CodeCRDT is implicit and observation-driven: agents infer teammate state by reading the merged workspace, roles are pre-assigned (outliner / implementer).

“CodeCRDT is implicit and observation-driven: agents infer teammate state by reading the merged workspace, roles are pre-assigned (outliner / implementer)”

• cites Pugachev, 2025 sourceSupported

May combine two claims.

Other recent multi-agent coding systems [Pan et al., 2026, Islam et al., 2025, Peng et al., 2026] share the LLM-team-coding goal but use sequential or staged-role paradigms.

“Other recent multi-agent coding systems [Pan et al., 2026, Islam et al., 2025, Peng et al., 2026] share the LLM-team-coding goal but use sequential or staged-role paradigms.”

• cites Pan et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Islam et al., 2025 sourceSupported
• cites Peng et al., 2026 sourceSupported

May combine two claims.

CodeCRDT's character-level concurrent edits produce the 5-10% semantic conflicts and the bimodal speed distribution.

“CodeCRDT's character-level concurrent edits produce the 5 -10% semantic conflicts and the bimodal speed distribution”

• cites Pugachev, 2025 sourcePartly supported

  Section 4.3 (p. 6) gives a preliminary 5–10% semantic-conflict estimate from inspection of 60 runs, but does not establish that character-level edits cause those conflicts. Section 6.2 (p. 7) describes task-dependent speedups and slowdowns, attributing apparent slowdowns chiefly to greater code volume; it does not report a bimodal speed distribution or attribute one to character-level edits. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

May combine two claims.

Across all seven per-model strata every Solo abandonment count is ≥ the AgentRoom count; two of four are individually significant on T4.

“Across all seven per-model strata every Solo abandonment count is ≥ the AgentRoom count; two of four are individually significant on T4.”

May combine two claims.

On T5 all three per-model strata are directionally consistent but individually underpowered.

“On T5 all three per-model strata are directionally consistent but individually underpowered (per-stratum counts in Figure 6).”

May combine two claims. The quote only partly states this claim.

A substrate-only bundle probe supports the mean ordering shared-only < prompt-only < AgentRoom, with the larger observed step at the MCP coordination layer.

“The bundle is co-designed, and a probe separates its components only partially: keeping the CRDT substrate and collaboration prompt but removing the MCP tool surface (Appendix B.13, n =7 ) lands at 0 . 588 , +0 . 013 above shared-only.”

May combine two claims.

The variance and abandonment results replicate across all three powered CLI-stable models (Sonnet 4.6, Haiku 4.5, Codex GPT-5.4) and both Tier I tasks (T4, T5).

“The result replicates across all three powered CLI-stable models (Sonnet 4.6, Haiku 4.5, Codex GPT-5.4) and both Tier I tasks (T4, T5).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The regex/AST composites preserve the same condition ordering as the LLM-judge.

“the regex/AST composites in Appendix E preserve the same condition ordering.”

May combine two claims.

The protocol is advisory at the prompt layer; agents typically deviate when a teammate stalls.

“The protocol is advisory at the prompt layer; agents typically deviate when a teammate stalls, an informative pattern documented in Appendix C.”

May combine two claims.

Sonnet tracks Haiku at r=0.87 (n=30), Sonnet tracks Codex at r=0.86 (n=26), and Haiku tracks Codex at r=0.78 (n=26).

“Sonnet tracks Haiku at r =0 . 87 ( n =30 ), Sonnet tracks Codex at r =0 . 86 ( n =26 ), and Haiku tracks Codex at r =0 . 78 ( n =26 );”

May combine two claims.

The main directional findings (CMH OR=13.7, AgentRoom > parallel-merge, AgentRoom > ChatDev) hold under the regex and AST scorers.

“The main directional findings (CMH OR = 13 . 7 , AgentRoom &gt; parallel-merge, AgentRoom &gt; ChatDev) hold under the regex and AST scorers (Appendix E).”

May combine two claims.

On T4 the reduction is individually significant for two models: Haiku falls from 12/35 to 1/17 (p=0.025) and Codex from 9/17 to 2/21 (p=0.005).

“On T4 the reduction is individually significant for two models: Haiku falls from 12 / 35 to 1 / 17 ( p = 0 . 025 ) and Codex from 9 / 17 to 2 / 21 ( p = 0 . 005 ).”

May combine two claims.

On T4 Haiku, the lone agent commits the 1-file abandonment on 12 of 35 attempts (34%) while AgentRoom ×2 commits it 1 of 17 times (6%; Fisher's exact one-sided p=0.025).

“On T4 Haiku, the lone agent commits the 1-file abandonment on 12 of 35 attempts (34%) while AgentRoom × 2 commits it 1 of 17 times (6%; Fisher's exact one-sided p =0 . 025 ).”

May combine two claims.

LLM-judge correlates with the regex scorer at Pearson r=0.67 over the T4 runs both scorers see (n=246); regex and AST correlate at Pearson r=0.79, Spearman ρ=0.62 on a 30-run cross-check.

“LLM-judge correlates with the regex scorer at Pearson r =0 . 67 over the T4 runs both scorers see ( n =246 , App. E); regex and AST correlate at Pearson r =0 . 79 , Spearman ρ =0 . 62 on a 30-run cross-check.”

May combine two claims.

The cross-vendor Codex judge is uniformly stricter, mean composite -0.151 vs Sonnet, but preserves the per-run ranking.

“the cross-vendor Codex judge is uniformly stricter, mean composite -0 . 151 vs Sonnet, but preserves the per-run ranking, so the composite's ranking is stable across judge vendors.”

May combine two claims.

Of the 13 budget-fair ChatDev runs, 7 are infrastructure failures-every phase agent exits with code 1 and zero output at ∼32 s because the orchestrator crashed before producing code.

“Of the 13 budget-fair ChatDev runs, 7 are infrastructure failures-every phase agent exits with code 1 and zero output at ∼ 32 s because the orchestrator crashed before producing code, and and we exclude them from the quality comparison, consistent with the infrastructure-failure exclusion in the Tier I codebook, and no run in any other condition meets this criterion.”

May combine two claims.

Per-run wall-clock stays near the task budget across N, so total compute grows roughly linearly with N, while broadcast traffic rises super-linearly past ×3.

“Per-run wall-clock stays near the task budget across N (Appendix B.6), so total compute grows roughly linearly with N , while broadcast traffic rises super-linearly past × 3 (Figure 5, right panel).”

May combine two claims.

Haiku replicates the pattern with AgentRoom 0.662 (n=17) above solo 0.530 (n=35); Codex posts the lowest solo mean of the four models at 0.384, consistent with its monolithic-architecture priors degrading multi-agent decomposition.

“Haiku replicates the pattern with AgentRoom 0 . 662 ( n =17 ) above solo 0 . 530 ( n =35 ); Codex posts the lowest solo mean of the four models at 0 . 384 , consistent with its monolithic-architecture priors degrading multi-agent decomposition.”

May combine two claims.

AgentRoom logs contain unprompted patterns: apologies for boundary violations, proactive gap-filling, 'do not' warnings, dependency documentation.

“AgentRoom logs contain unprompted patterns-apologies for boundary violations, proactive gap-filling, 'do not' warnings, dependency documentation (Appendix B, Appendix C).”

May combine two claims.

The Gemini failure root-caused to a non-conformance issue: the room MCP server emitted LSP-style Content-Length: framing while Gemini CLI 0.40+ strictly enforces the MCP-stdio NDJSON spec; switching the room server to NDJSON recovered Gemini fully.

“The Gemini failure root-caused to a non-conformance issue: the room MCP server emitted LSP-style Content-Length: framing while Gemini CLI 0.40+ strictly enforces the MCP-stdio NDJSON spec; switching the room server to NDJSON (a one-line server-side change) recovered Gemini fully.”

May combine two claims.

Across the remaining n=6 genuine ChatDev runs (Sonnet, T4, budget-fair) ChatDev averages ̄s=0.333 (σ=0.197) where concurrent AgentRoom ×2 on the same task at the same compute reaches 0.669 (n=14, σ=0.140). Welch's t=3.78, two-sided p=0.006.

“Across the remaining n =6 genuine ChatDev runs (Sonnet, T4, budget-fair) ChatDev averages ¯ s =0 . 333 ( σ =0 . 197 ) where concurrent AgentRoom × 2 on the same task at the same compute reaches 0 . 669 ( n =14 , σ =0 . 140 ). Welch's t =3 . 78 , two-sided p =0 . 006 .”

May combine two claims.

2× Haiku AgentRoom reaches T4 mean 0.662 (n=17) versus 1× Sonnet 0.544 (n=32) at roughly half the dollar cost-Welch t=2.10, p=0.036.

“2 × Haiku AgentRoom reaches T4 mean 0 . 662 ( n =17 ) versus 1 × Sonnet 0 . 544 ( n =32 ) at roughly half the dollar cost-Welch t =2 . 10 , p =0 . 036 .”

• AssumptionRests on an assumption: Haiku is roughly 4× cheaper per token than Sonnet.

May combine two claims.

On T4 Sonnet under the LLM-judge rubric, Solo averages 0.544 (n=32) where AgentRoom averages 0.669 (n=14), excluding catastrophic infrastructure-failure runs, so AgentRoom leads by +0.125; Welch t=2.30, p=0.022.

“On T4 Sonnet under the LLM-judge rubric, Solo averages 0 . 544 ( n =32 ) where AgentRoom averages 0 . 669 ( n =14 ), excluding catastrophic infrastructure-failure runs, so AgentRoom leads by +0 . 125 ; Welch t =2 . 30 , p =0 . 022 .”

May combine two claims.

The six conditions form a clean monotonic ordering-ChatDev-style sequential pipeline 0.333 (n=6) ≪ parallel-merge 0.456 (n=12) < solo 0.544 (n=32) < shared-only 0.575 (n=11) < shared+collab without MCP 0.588 (n=7) < AgentRoom 0.669 (n=14, σ=0.140).

“The six conditions form a clean monotonic ordering-ChatDev-style sequential pipeline 0 . 333 ( n =6 ) ≪ parallel-merge 0 . 456 ( n =12 ) &lt; solo 0 . 544 ( n =32 ) &lt; shared-only 0 . 575 ( n =11 ) &lt; shared+collab without MCP 0 . 588 ( n =7 ) &lt; AgentRoom 0 . 669 ( n =14 , σ =0 . 140 ).”

May combine two claims.

AgentRoom's file-level claim eliminates concurrent same-file edits (0% semantic conflicts in the headline pool).

“AgentRoom's file-level claim eliminates concurrent same-file edits ( 0% semantic conflicts in the headline pool)”

May combine two claims.

A post-fix Gemini T4 batch (Sonnet partner; n=5 Solo, n=5 AgentRoom ×2) reproduces the headline pattern-Solo abandonment 2/5=40% collapses to 0/5 under AgentRoom, with mean LLM-judge composite 0.602 (σ=0.326) → 0.870 (σ=0.046) and 5/5 vitest pass.

“A post-fix Gemini T4 batch (Sonnet partner; n =5 Solo, n =5 AgentRoom × 2 ) reproduces the headline pattern-Solo abandonment 2 / 5=40% (1-file stub-and-exit at 30-50 s) collapses to 0 / 5 under AgentRoom, with mean LLM-judge composite 0 . 602 ( σ =0 . 326 ) → 0 . 870 ( σ =0 . 046 ) and 5/5 vitest pass.”

May combine two claims.

Heterogeneous pairing on T4 shows ×2 Sonnet+Codex at 0.721 (n=3), the highest cell across the heterogeneous combinations tested, while ×2 Sonnet+Gemini reaches only 0.542 (n=3); the pattern suggests heterogeneity is not a free lunch: a stronger second agent contributes more than a weaker one even when the first agent is held fixed, and the coordination layer does not erase capability differences.

“Heterogeneous pairing on T4 (Sonnet primary partner) shows × 2 Sonnet+Codex at 0 . 721 ( n =3 ), the highest cell across the heterogeneous combinations we tested, while × 2 Sonnet+Gemini reaches only 0 . 542 ( n =3 ); the pattern suggests heterogeneity is not a free lunch: a stronger second agent contributes more than a weaker one even when the first agent is held fixed, and the coordination layer”

May combine two claims.

On 10 DevBench Python multi-file projects (Haiku 4.5), ×2 passes 8/10 against Solo's 7/10 with zero CRDT semantic conflicts.

“on 10 DevBench Python multi-file projects (Haiku 4.5) × 2 passes 8 / 10 against Solo's 7 / 10 with zero CRDT semantic conflicts (Appendix B.16)”

May combine two claims.

AgentRoom leads parallel-merge by +0.213 in mean quality, Welch's t=3.35, p=0.003.

“AgentRoom leads parallel-merge by +0 . 213 in mean quality, Welch's t =3 . 35 , p = 0 . 003 .”

May combine two claims.

Pooled per task, abandonment falls in every task-T4 32% → 5% (p=0.0001) and T5 23% → 0% (p=0.031) individually significant, T1 and T2 directionally consistent at small n.

“Pooled per task, abandonment falls in every task-T4 32% → 5% ( p =0 . 0001 ) and T5 23% → 0% ( p =0 . 031 ) individually significant, T1 and T2 directionally consistent at small n (Figure 6).”

May combine two claims.

Sequential pipelines amplify the lone-agent abandonment failure mode at every phase boundary: in the 6-run genuine subsample, 4/6 ChatDev runs produced a 1-file implementation-phase abandonment that nevertheless passes the boilerplate GET / test, while AgentRoom ×2 exhibited 0/10 such abandonments in the hand-inspected subsample.

“Sequential pipelines amplify the lone-agent abandonment failure mode at every phase boundary: in the 6-run genuine subsample, 4 / 6 ChatDev runs produced a 1-file implementation-phase abandonment that nevertheless passes the boilerplate GET / test, while AgentRoom × 2 exhibited 0 / 10 such abandonments in the hand-inspected subsample.”

May combine two claims.

The bundle is co-designed, and a probe separates its components only partially: keeping the CRDT substrate and collaboration prompt but removing the MCP tool surface (n=7) lands at 0.588, +0.013 above shared-only. The MCP-tool step accounts for the remaining +0.081 up to AgentRoom's 0.669.

“The bundle is co-designed, and a probe separates its components only partially: keeping the CRDT substrate and collaboration prompt but removing the MCP tool surface (Appendix B.13, n =7 ) lands at 0 . 588 , +0 . 013 above shared-only. The MCP-tool step accounts for the remaining +0 . 081 up to AgentRoom's 0 . 669 .”

May combine two claims.

Sonnet T4 quality peaks at ×2 then declines under the LLM-judge rubric-0.544/0.669/0.553/0.489 for ×1/×2/×3/×4 on the budget-fair pool-while mean tests passing peaks at ×3.

“Sonnet T4 quality peaks at × 2 then declines under the LLM-judge rubric0 . 544 / 0 . 669 / 0 . 553 / 0 . 489 for × 1 / × 2 / × 3 / × 4 on the budget-fair pool-while mean tests passing peaks at × 3 (Figure 5, middle panel).”

May combine two claims.

The full cross-model headroom matrix has 5/6 cells point-estimate-positive, and only Haiku T4 clears its bootstrap CI.

“the full cross-model headroom matrix (5/6 cells point-estimate-positive, only Haiku T4 clears its bootstrap CI) is what Appendix B reports.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Agents fix each other's bugs in only 2 of 6 runs.

“they fix each other's bugs in only 2 of 6”

May combine two claims.

At equal dollar cost on T4 under the LLM-judge rubric, 2× Haiku AgentRoom reaches mean 0.662 (n=17, σ=0.165) versus 1× Sonnet 0.544 (n=32, σ=0.222)-Welch's t=2.10, two-sided p=0.036.

“At equal dollar cost on T4 under the LLM-judge rubric, 2 × Haiku AgentRoom reaches mean 0 . 662 ( n =17 , σ =0 . 165 ) versus 1 × Sonnet 0 . 544 ( n =32 , σ =0 . 222 )-Welch's t =2 . 10 , two-sided p =0 . 036”

• AssumptionRests on an assumption: Haiku is roughly 4× cheaper per token than Sonnet.

May combine two claims.

The structured prompt alone is model-dependent: it hurts Sonnet and helps Haiku.

“The structured prompt alone is model-dependent: it hurts Sonnet (channel is the load-bearing variable) and helps Haiku”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Only Haiku T4 clears its 95% bootstrap CI under the AST scorer (and is point-estimate-positive under regex).

“only Haiku T4 clears its 95% bootstrap CI under the AST scorer (and is point-estimate-positive under regex)”

May combine two claims.

Across the 65 runs with at least one changed source file, the mean unclaimed-write rate is 0.11, and 39/65 runs have every changed file covered by a claim.

“Across the 65 runs with at least one changed source file, the mean unclaimed-write rate is 0 . 11 , and 39 / 65 runs have every changed file covered by a claim.”

May combine two claims.

Solo mean 0.714 (n=5, σ=0.104, individual 0.631/0.637/0.688/0.728/0.887) - AgentRoom ×2 mean 0.740 (n=4, σ=0.070, individual 0.683/0.704/0.731/0.841); ∆=+0.026.

“Solo mean 0 . 714 ( n =5 , σ =0 . 104 , individual 0 . 631 / 0 . 637 / 0 . 688 / 0 . 728 / 0 . 887 )-AgentRoom × 2 mean 0 . 740 ( n =4 , σ =0 . 070 , individual 0 . 683 / 0 . 704 / 0 . 731 / 0 . 841 ); ∆=+0 . 026”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

Probably not a substantial claim: may not pass S3 (load-bearing).

Code volume per AgentRoom ×2 run: 2008-2452 Rust lines across 3-6 .rs files.

“Code volume per AgentRoom × 2 run: 2008 -2452 Rust lines across 3 -6 .rs fi les.”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

May combine two claims.

×2 AgentRoom passes 8 of 10 tasks; solo passes 7 of 10.

“× 2 AgentRoom passes 8 of 10 tasks; solo passes 7 of 10.”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

Probably not a substantial claim: may not pass S3 (load-bearing).

The LLM-judge T4 cell (0.544 → 0.669, ES 1.23×, n=32/14, budget-fair) gives a directionally identical result.

“the LLM-judge T4 cell ( 0 . 544 → 0 . 669 , ES 1 . 23 × , n =32 / 14 , budget-fair) reported in the body (Table 1) gives a directionally identical result”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Heterogeneous and mixed T5 cells: Sonnet+GPT-5.4 0.857 (n=2), Sonnet+Haiku 0.900 (n=1), Sonnet+Gemini 0.626 (n=3), Mixed S+C+G ×3 0.673 (n=4), and Mixed S+H+C ×3 0.791 (n=2).

“Sonnet+GPT-5.4 0 . 857 ( n =2 ), Sonnet+Haiku 0 . 900 ( n =1 ), Sonnet+Gemini 0 . 626 ( n =3 ), Mixed S+C+G × 3 0 . 673 ( n =4 ), and Mixed S+H+C × 3 0 . 791 ( n =2 )”

May combine two claims.

Judged quality peaks at ×2 while mean tests passing peaks at ×3.

“judged quality peaks at × 2 while mean tests passing peaks at × 3”

May combine two claims.

Hard tasks (T4, T5) reach 1.16-1.20× under the regex scorer (1.23× on the LLM-judge T4 cell).

“hard tasks (T4, T5) reach 1 . 16 -1 . 20 × under the regex scorer ( 1 . 23 × on the LLM-judge T4 cell)”

May combine two claims.

The bundle component probe realises this substrate-coordination decomposition: removing only the MCP coordination tools while keeping the CRDT substrate and the collaboration prompt recovers +0.013 over shared-only, where the full AgentRoom recovers a further +0.081 above the prompt-only cell, and both steps carry wide intervals at these n.

“The bundle component probe (§5, Appendix B.13) realises this substrate-coordination decomposition: removing only the MCP coordination tools while keeping the CRDT substrate and the collaboration prompt recovers +0 . 013 over shared-only, where the full AgentRoom recovers a further +0 . 081 above the prompt-only cell, and both steps carry wide intervals at these n (ordering claim in §5).”

May combine two claims.

The +1 pass differential is driven by geotext, where solo timed out at 603 s while ×2 finished in 390 s.

“The +1 pass differential is driven by geotext , where solo timed out at 603 s while × 2 finished in 390 s.”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

Classified as probably not a substantial claim. May combine two claims.

Sonnet 4.6, T4, 1200 s budget, 30-700 s budget-fair envelope, n=7; LLM-judge mean 0.588 (σ=0.188).

“Sonnet 4.6, T4, 1200 s budget, 30 -700 s budget-fair envelope, n =7 ; LLM-judge mean 0 . 588 ( σ =0 . 188 )”

May combine two claims.

For Haiku, the same prompt scores above bare solo and statistically indistinguishable from AgentRoom ×2 at this n.

“For Haiku, the same prompt scores above bare solo and statistically indistinguishable from AgentRoom × 2 at this n .”

May combine two claims.

Only Haiku ×2 clears its CI direction, Sonnet ×2 is flat, and Codex ×2 regresses.

“only Haiku × 2 clears its CI direction, Sonnet × 2 is flat, and Codex × 2 regresses”

May combine two claims.

The duration cutoff 150-300 s does not shift any cell counts (every <0.3 abandonment in our pool either exited within 150 s or ran the full budget).

“The duration cutoff 150 -300 s does not shift any cell counts (every &lt; 0 . 3 abandonment in our pool either exited within 150 s or ran the full budget)”

May combine two claims.

Against the LLM-judge composite the correlation is positive-Pearson r=0.28 (bootstrap 95% CI [0.06, 0.46]), Spearman ρ=0.30 [0.06, 0.52]; restricted to ×2 runs, r=0.24 (n=54).

“Against the LLM-judge composite the correlation is positive-Pearson r =0 . 28 (bootstrap 95% CI [0 . 06 , 0 . 46] ), Spearman ρ =0 . 30 [0 . 06 , 0 . 52] ; restricted to × 2 runs, r =0 . 24 ( n =54 ).”

May combine two claims.

Both AgentRoom and the substrate baselines shift slightly under the wider pool, with the gap narrowing modestly because the long-running solo runs occasionally produce more code.

“both AgentRoom and the substrate baselines shift slightly under the wider pool, with the gap narrowing modestly because the long-running solo runs occasionally produce more code”

May combine two claims.

At low difficulty (T1 single auth route), ×2 saturates at the test-suite ceiling and ES = 0.99×.

“At low difficulty (T1 single auth route), × 2 saturates at the test-suite ceiling and ES =0 . 99 ×”

May combine two claims.

Across all 12 threshold combinations the T4-only CMH OR is at least 6.3 and CMH p <0.001.

“Across all 12 threshold combinations the T4-only CMH OR is at least 6 . 3 and CMH p &lt; 0 . 001”

May combine two claims.

The rate is uncorrelated with the number of changed files (r=0.02), so it is not a repository-size artifact.

“The rate is uncorrelated with the number of changed files ( r =0 . 02 ), so it is not a repository-size artifact.”

May combine two claims.

Nearly every run claims a module and adjusts the plan.

“Nearly every run claims a module and adjusts the plan”

May combine two claims.

The source-file cutoff matters more (more permissive thresholds reclassify Codex AgentRoom runs from infrastructure to abandonment, lowering OR but never below 6.3).

“the source-file cutoff matters more (more permissive thresholds reclassify Codex AgentRoom runs from infrastructure to abandonment, lowering OR but never below 6 . 3 )”

May combine two claims.

AgentRoom runs share the wall-clock budget concurrently, so wall stays close to solo while compute scales with N.

“AgentRoom runs share the wall-clock budget concurrently, so wall stays close to solo while compute scales with N”

Probably not a substantial claim: may not pass S3 (load-bearing).

The full-pool cells include some long-running attempts that exceeded the 700 s budget.

“The full-pool cells include some long-running attempts that exceeded the 700 s budget”

May combine two claims.

Mean capabilities passed (of 10): AgentRoom ×2 5.06 (n=17), shared-only 4.91 (n=11), shared+collab noMCP 4.70 (n=10), Solo 3.67 (n=33), ChatDev-style 1.46 (n=13), parallel-merge 1.00 (n=15).

“Mean capabilities passed (of 10 ): AgentRoom × 2 5 . 06 ( n =17 ), shared-only 4 . 91 ( n =11 ), shared+collab noMCP 4 . 70 ( n =10 ), Solo 3 . 67 ( n =33 ), ChatDev-style 1 . 46 ( n =13 ), parallel-merge 1 . 00 ( n =15 ).”

• AssumptionRests on an assumption: The ten binary capabilities probed by the execution oracle are sufficient to measure functional correctness of the ledger system.

May combine two claims.

Each point is one T4 run carrying both a regex composite and an LLM-judge composite (n=246, the runs scored by both); Pearson r=0.67.

“Each point is one T4 run carrying both a regex composite and an LLM-judge composite ( n =246 , the runs scored by both); Pearson r =0 . 67 .”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The LLM-judge spreads substrate-only conditions slightly more than the regex scorer and tightens the AgentRoom condition.

“the LLM-judge spreads substrate-only conditions slightly more than the regex scorer and tightens the AgentRoom condition”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

ChatDev-style edges parallel-merge under the oracle while the judge orders them the other way.

“ChatDev-style edges parallel-merge under the oracle while the judge orders them the other way.”

• AssumptionRests on an assumption: The ten binary capabilities probed by the execution oracle are sufficient to measure functional correctness of the ledger system.

May combine two claims.

The robustness conclusion does not depend on the subset choice: the condition ordering (AgentRoom > solo) is preserved under every single-dimension ablation, with ES ranging from 1.12 to 1.25 on this subset.

“The robustness conclusion does not depend on the subset choice: the condition ordering (AgentRoom &gt; solo) is preserved under every single-dimension ablation, with ES ranging from 1.12 to 1.25 on this subset.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

May combine two claims.

The AgentRoom > solo ordering is preserved under every single-dimension ablation and the ratio stays within 1.12-1.25 on this subset.

“The AgentRoom &gt; solo ordering is preserved under every single-dimension ablation and the ratio stays within 1.12-1.25 on this subset.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Probably not a substantial claim: may not pass S3 (load-bearing).

Table 13 reports a robustness check on a smaller solo subset (n=4, mean 0.786) than the canonical T4 Sonnet solo used in main Table 1 and Appendix B.

“Table 13 reports a robustness check on a smaller solo subset ( n =4 , mean 0.786) than the canonical T4 Sonnet solo used in main Table 1 and Appendix B.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

May combine two claims.

On a random sample of 30 runs spanning all tasks, models, and conditions, Pearson correlation between the two scorers is r=0.79 and Spearman rank correlation is ρ=0.62 (n=30).

“On a random sample of 30 runs spanning all tasks, models, and conditions, Pearson correlation between the two scorers is r =0 . 79 and Spearman rank correlation is ρ =0 . 62 ( n =30 ).”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

May combine two claims.

The two scorers disagree on adjacent pairs in between: shared-only and noMCP sit within 0.2 capabilities of each other but in the opposite order to the judge's.

“The two scorers disagree on adjacent pairs in between: shared-only and noMCP sit within 0 . 2 capabilities of each other but in the opposite order to the judge's”

• AssumptionRests on an assumption: The ten binary capabilities probed by the execution oracle are sufficient to measure functional correctness of the ledger system.

May combine two claims.

The agreement is far from perfect (in particular the AST scorer treats Codex's compact single-file implementations more harshly than our regex scorer).

“The agreement is far from perfect (in particular the AST scorer treats Codex's compact single-file implementations more harshly than our regex scorer)”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

May combine two claims.

Strong eventual consistency (SEC) eliminates data loss under concurrent edits but is a property over byte-level merge, not semantic compatibility.

“An op-based sequence CRDT[Shapiro et al., 2011, Roh et al., 2011, Kleppmann and Beresford, 2017] underlies the substrate, stamping edits with Lamport timestamps [Lamport, 1978] and satisfying strong eventual consistency (SEC); SEC eliminates data loss under concurrent edits but is a property over byte-level merge, not semantic compatibility: two agents inserting incompatible function signatures at”

• cites Shapiro et al., 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Roh et al., 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Kleppmann and Beresford, 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Two agents inserting incompatible function signatures at the same character offset both survive in the merged file and break compilation downstream.

“two agents inserting incompatible function signatures at the same character offset both survive in the merged file and break compilation downstream.”

May combine two claims.

The variance-reduction direction is consistent with self-consistency and mixture-of-agents ensembling, but AgentRoom achieves it through coordinated concurrent generation in a shared workspace rather than independent sampling and aggregation.

“The variance-reduction direction is consistent with self-consistency [Wang et al., 2023] and mixture-of-agents ensembling [Wang et al., 2024], but AgentRoom achieves it through coordinated concurrent generation in a shared workspace rather than independent sampling and aggregation.”

• cites Wang et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Wang et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

All headline cells share one Express.js/TypeScript runtime.

“All headline cells share one Express.js/TypeScript runtime”

Classified as probably not a substantial claim. May combine two claims.

Quality is assessed by the LLM-judge composite, not an execution oracle: tasks do not ship a held-out correctness suite (vitest suites are agent-authored).

“Quality is assessed by the LLM-judge composite, not an execution oracle: tasks do not ship a held-out correctness suite (vitest suites are agent-authored)”

May combine two claims.

Concurrent multi-agent coding has been treated as a problem of parallelism or of merge-correct substrate; our position is that it is primarily about explicit coordination.

“Concurrent multi-agent coding has been treated as a problem of parallelism or of merge-correct substrate; our position is that it is primarily about explicit coordination.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The Rust and Python checks are bounded mechanism-portability spot checks, not a claim of runtime-general magnitude.

“the Rust and Python checks (Appendices B.17, B.16) are bounded mechanism-portability spot checks, not a claim of runtime-general magnitude”

May combine two claims.

We read the ChatDev contrast as further evidence that AgentRoom's coordination layer collapses this specific mode independently of upstream task structure, not that ChatDev pipelines have higher catastrophic-failure rates overall.

“We read the ChatDev contrast as further evidence that AgentRoom's coordination layer collapses this specific mode independently of upstream task structure, not that ChatDev pipelines have higher catastrophic-failure rates overall.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Gemini 3 is underpowered at 5+5, and we exclude Codex-mini.

“Gemini 3 is underpowered at 5+5 , and we exclude Codex-mini (§4.4)”

May combine two claims.

A complementary prompt-vs-channel control on Haiku is less conclusive because the structured tool prompt also acts as workflow scaffolding for Haiku and is not separately identified from channel use at this n.

“A complementary prompt-vs-channel control on Haiku (Appendix B) is less conclusive because the structured tool prompt also acts as workflow scaffolding for Haiku and is not separately identified from channel use at this n .”

May combine two claims.

Within our setting, the substrate-coordination separation suggests that an explicit room-style primitive is the contributor we identify, and that matched-compute ablation is how we measure such contributions.

“Within our setting, the substrate-coordination separation suggests that an explicit room-style primitive is the contributor we identify, and that matched-compute ablation is how we measure such contributions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

We cross-validate the judge against the regex and AST scorers but do not claim execution-verified correctness.

“we cross-validate the judge against the regex and AST scorers (§2.5) but do not claim execution-verified correctness”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The headline results span four backend domains and the three powered CLI-stable models from two providers; per-cell n ranges 7-35.

“The headline results span four backend domains and the three powered CLI-stable models from two providers; per-cell n ranges 7 -35”

May combine two claims.

The N=2 cell is the operational sweet spot under our setup; ×3 and ×4 pay linearly more compute for declining mean quality.

“The N =2 cell is the operational sweet spot under our setup; × 3 and × 4 pay linearly more compute for declining mean quality.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

We claim as the delta here the first matched-compute quantification isolating the coordination layer from parallelism and from the CRDT substrate, on a failure mode that neither CodeCRDT nor the shipping systems measure.

“We claim as the delta here the first matched-compute quantification isolating the coordination layer from parallelism and from the CRDT substrate, on a failure mode that neither CodeCRDT nor the shipping systems measure.”

May combine two claims.

AgentRoom's claim protocol creates an advisory distributed lock by prompt-level cooperation; the advisory-plus-apology pattern matters because strict locking would block the cross-agent bug-fix case.

“AgentRoom's claim protocol creates an advisory distributed lock by prompt-level cooperation; the advisory-plus-apology pattern matters because strict locking would block the cross-agent bug-fix case (a teammate correcting an Express 5 routing error in another agent's file, Appendix C).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The bundle-component probe reuses the ablation cells; we read its substrate-vs-coordination split as indicative, not a precise variance decomposition.

“The bundle-component probe reuses the ablation cells (Appendix B.13); we read its substrate-vs-coordination split as indicative, not a precise variance decomposition.”

May combine two claims.

The design lineage is advisory locking in distributed systems; AgentRoom's file-level claim is a Chubby-style advisory lock surfaced as an MCP tool, with the agent rather than a kernel call as the cooperator.

“The design lineage is advisory locking in distributed systems [Burrows, 2006, Hunt et al., 2010, Gray and Cheriton, 1989]; AgentRoom's file-level claim is a Chubby-style advisory lock surfaced as an MCP tool, with the agent rather than a kernel call as the cooperator.”

• cites Burrows, 2006Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Hunt et al., 2010Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Gray and Cheriton, 1989 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The parallel-merge vs AgentRoom contrast holds the same Sonnet producer on both sides.

“The parallel-merge vs AgentRoom contrast holds the same Sonnet producer on both sides [Li et al., 2026, Wataoka et al., 2025]”

• cites Li et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Wataoka et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The ordering still argues against reading AgentRoom as 'CRDT plus prompts' integration: the largest observed step is the explicit coordination channel; the substrate is the merge primitive on which it stands.

“The ordering still argues against reading AgentRoom as 'CRDT plus prompts' integration: the largest observed step is the explicit coordination channel; the substrate is the merge primitive on which it stands.”

May combine two claims.

In our matched-compute setting, neither agent count nor the CRDT itself is the main contributor we identify; the difference comes from the runtime room's state-management operations-file-level claim, broadcast log, and per-agent status.

“In our matched-compute setting, neither agent count nor the CRDT itself is the main contributor we identify; the difference comes from the runtime room's state-management operations-file-level claim, broadcast log, and per-agent status.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The cost-equivalent cross-model comparison carries an exposed bias, and we frame it as suggestive.

“The cost-equivalent cross-model comparison (Appendix B.6) carries an exposed bias, and we frame it as suggestive.”

Classified as probably not a substantial claim.

We cross-validate the LLM-judge composite against independent regex and TypeScript-AST scorers.

“we cross-validate the LLM-judge composite against independent regex and TypeScript-AST scorers (§2.5)”

Classified as probably not a substantial claim.

The codebook is implemented as a deterministic Python script that reads each run's recorded duration, exit code, and snapshot source-file count, and emits a label without human judgment.

“We implement the codebook as a deterministic Python script that reads each run's recorded duration, exit code, and snapshot source-file count, and emits a label without human judgment.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

For a sequential patch sequence the chain rule gives I(P_seq; X) = sum_t I(P_t; X | P_<t), and a competent LLM avoids redundant edits, so the per-step conditional information shrinks fast as easy rubric bits are knocked out early.

“For a sequential patch sequence the chain rule gives I ( P seq ; X ) = ∑ t I ( P t ; X | P &lt;t ) , and a competent LLM avoids redundant edits, so the per-step conditional information I ( P t ; X | P &lt;t ) shrinks fast as easy rubric bits are knocked out early.”

May combine two claims.

Without coordination the N agents collide on the same files, H(P_i_t | P_j_t) collapses, and the rate of novel rubric-relevant information per wall-clock step is bottlenecked at the single-agent rate I(P; X), so the room wastes concurrent compute on redundant edits.

“Without coordination the N agents collide on the same files, H ( P i t | P j t ) collapses, and the rate of novel rubric-relevant information per wall-clock step is bottlenecked at the single-agent rate I ( P ; X ) , so the room wastes concurrent compute on redundant edits.”

May combine two claims.

The replication establishes that the LLM-judge composite extends to Rust without breaking and that Codex MCP-concurrent execution is stable on full GPT-5.4.

“The replication establishes that the LLM-judge composite extends to Rust without breaking and that Codex MCP-concurrent execution is stable on full GPT-5.4.”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

May combine two claims.

Eq. 1 explains the asymmetry-SEC alone leaves p_collision untouched-while Eq. 2 predicts that the gain should track rubric dimensionality D and per-agent novelty rate, consistent with the larger absolute lifts observed on higher-dimension T4 (≥15 files, multi-domain rubric) than on T1 (6 files, narrower rubric).

“Eq. 1 explains the asymmetry-SEC alone leaves p collision untouched-while Eq. 2 predicts that the gain should track rubric dimensionality D and per-agent novelty rate, consistent with the larger absolute lifts observed on higher-dimension T4 ( ≥ 15 fi les, multi-domain rubric) than on T1 (6 files, narrower rubric, Appendix B).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Welch underpowered at this n, t=0.44, p=0.68; we report descriptively.

“(Welch underpowered at this n , t =0 . 44 , p =0 . 68 ; we report descriptively)”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

May combine two claims.

Deviating from claim discipline does not predict lower quality in this pool; if anything, runs containing unclaimed writes score modestly higher.

“Deviating from claim discipline therefore does not predict lower quality in this pool; if anything, runs containing unclaimed writes score modestly higher.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The estimate is correlational, zero-inflated, and conditioned on rooms whose coordination went through the observable channel.

“The estimate is correlational, zero-inflated, and conditioned on rooms whose coordination went through the observable channel.”

Probably not a substantial claim: may not pass S3 (load-bearing).

At n=10 a one-task pass differential carries no inferential weight.

“At n =10 a one-task pass differential carries no inferential weight”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Zero-event strata sit at OR ≈ 1 and are marked †, where we apply a Haldane 0.5 correction so the interval is defined at all.

“Zero-event strata sit at OR ≈ 1 and are marked † , where we apply a Haldane 0 . 5 correction so the interval is defined at all.”

May combine two claims.

Codex ×2 regresses consistent with the model's monolithic-architecture default fighting a second-agent decomposition.

“Codex × 2 regresses (consistent with the model's monolithic-architecture default fighting a second-agent decomposition)”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The script was committed to the paper repository before the CMH analyses, and it ships with the release bundle.

“We committed the script to the paper repository before the CMH analyses, and it ships with the release bundle.”

May combine two claims.

Two tasks (stocktrends, textcnn) fail under both conditions, indicating task difficulty above what 600 s of Haiku affords regardless of agent count.

“Two tasks ( stocktrends , textcnn ) fail under both conditions, indicating task difficulty above what 600 s of Haiku affords regardless of agent count.”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

May combine two claims. The quote only partly states this claim.

In the concurrent setting with coordination, N agents sample at step t from posteriors anchored on the same W_t but pushed onto disjoint files by claim, so H(P_i_t | P_j_t, W_t) ≈ H(P_i_t | W_t) for i ≠ j, and the per-step joint information across agents scales near-linearly in N as long as claims keep the joint entropy from collapsing.

“In the concurrent setting with coordination, N agents sample at step t from posteriors anchored on the same W t but pushed onto disjoint files by claim , so H ( P i t | P j t , W t ) ≈ H ( P i t |W t ) for i = j , and the per-step joint information across agents”

May combine two claims.

Because merge is commutative and associative under the op-based sequence CRDT with Lamport timestamps, the workspace satisfies strong eventual consistency: for any two replicas i, j with the same observed operation set O, APPLY(O, W_i0) = APPLY(O, W_j0).

“Because merge is commutative and associative under the op-based sequence CRDT with Lamport timestamps, the workspace satisfies strong eventual consistency: for any two replicas i, j with the same observed operation set O , APPLY ( O, W i 0 ) = APPLY ( O, W j 0 ) .”

Probably not a substantial claim: may not pass S3 (load-bearing).

Residual classification risk lies in the threshold choice.

“residual classification risk lies in the threshold choice”

Classified as probably not a substantial claim.

The apologies, gap-filling, and explicit 'do not' warnings visible in the transcripts of Appendix C are not in the prompt.

“the apologies, gap-filling, and explicit 'do not' warnings visible in the transcripts of Appendix C are not in the prompt”

Probably not a substantial claim: may not pass S3 (load-bearing).

∆=+0.026 is direction-consistent with the TypeScript headline.

“direction-consistent with the TypeScript headline”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

GPT-5.4-mini produced a single valid solo run (0.077) before its CLI crashed, so we omit it.

“GPT-5.4-mini produced a single valid solo run ( 0 . 077 ) before its CLI crashed, so we omit it (§4.4)”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

One earlier AgentRoom ×2 run (score 0.580) is excluded from this cell for a coordination artifact-the same four enums duplicated across two files with slightly different shapes; it is the only run in the Rust pool with this pattern, and it remains in the run archive.

“One earlier AgentRoom × 2 run (score 0 . 580 ) is excluded from this cell for a coordination artifact-the same four enums duplicated across two files with slightly different shapes; it is the only run in the Rust pool with this pattern, and it remains in the run archive.”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Replication has a hosted-model ceiling: we pin CLI versions, model identifiers, and per-agent invocations, but hosted models can change server-side, so bit-for-bit reproduction is not claimed.

“Replication has a hosted-model ceiling: we pin CLI versions, model identifiers, and per-agent invocations (§3), but hosted models can change server-side, so bit-for-bit reproduction is not claimed”

Probably not a substantial claim: may not pass S3 (load-bearing).

Because classification is deterministic, blinded interrater agreement is trivially κ=1.

“Because classification is deterministic, blinded interrater agreement is trivially κ =1”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Contrasts are matched-CLI and matched-budget, so drift moves both arms together.

“contrasts are matched-CLI and matched-budget, so drift moves both arms together”

May combine two claims.

Because every token is a contiguous string of characters, E_tok embeds losslessly into the character-level Yrs alphabet, so the substrate accepts LLM-generated patches without re-encoding.

“Because every token is a contiguous string of characters, E tok embeds losslessly into the character-level Yrs alphabet [Jahns, 2024, Shapiro et al., 2011], so the substrate accepts LLM-generated patches without re-encoding.”

• cites Jahns, 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Shapiro et al., 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

We attribute this to Haiku's stronger benefit from workflow scaffolding (consistent with the lone-agent abandonment failure mode that the explicit tool list defuses).

“We attribute this to Haiku's stronger benefit from workflow scaffolding (consistent with the lone-agent abandonment failure mode that the explicit tool list defuses)”

May combine two claims.

The cross-domain evidence is consistent with the TypeScript results: ×2 does not break correctness on multi-file Python projects, and on at least one task the ×2 condition completes a task the solo condition cannot finish in budget.

“The cross-domain evidence is consistent with the TypeScript results: × 2 does not break correctness on multi-file Python projects, and on at least one task the × 2 condition completes a task the solo condition cannot finish in budget.”

• AssumptionRests on an assumption: The hidden test suites (vitest, pytest, cargo test) are valid and comprehensive measures of task correctness.
• AssumptionRests on an assumption: The tasks used (T1-T4, DevBench, Rust transposition) are representative of real-world software engineering tasks.

May combine two claims.

For Sonnet, the structured prompt alone scores below bare solo and far below AgentRoom ×2, so the AgentRoom gain is attributable to the channel rather than to prompt complexity.

“For Sonnet, the structured prompt alone scores below bare solo and far below AgentRoom × 2 , so the AgentRoom gain is attributable to the channel rather than to prompt complexity.”

May combine two claims.

Codex departs at the AgentRoom cell; multi-agent Codex pairings hurt regardless of whether the channel is on.

“Codex departs at the AgentRoom cell, multi-agent Codex pairings hurt regardless of whether the channel is on”

• cites §4.5Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

One Sonnet control run scored at the bottom of the range because the agent burned the wall-clock budget calling room tools that returned no useful state; without referent multi-agent context, the prompt becomes counterproductive.

“One Sonnet control run scored at the bottom of the range because the agent burned the wall-clock budget calling room tools that returned no useful state; without referent multi-agent context, the prompt becomes counterproductive.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For Haiku, only the AgentRoom-vs-baseline endpoints are read, not the fine-grained substrate-then-prompt step ordering, which is identified only on the larger Sonnet cells.

“for Haiku we read only the AgentRoom-vs-baseline endpoints, not the fine-grained substrate-then-prompt step ordering, which is identified only on the larger Sonnet cells”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

We replay a post-hoc capability probe over the archived T4 snapshots, and it corroborates the ordering endpoints.

“We replay a post-hoc capability probe over the archived T4 snapshots, and it corroborates the ordering endpoints (Appendix B.18).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The trade-off is that the scorer is lexical and can be fooled by appearance of defensive code that is never exercised.

“the trade-off is that the scorer is lexical and can be fooled by appearance of defensive code that is never exercised”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Classified as probably not a substantial claim.

Same-family preference [Li et al., 2026] can inflate Sonnet-vs-other comparisons.

“Same-family preference [Li et al., 2026] can inflate Sonnet-vs-other comparisons”

• cites Li et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

We scored the full 282 T4 budget-fair pool idempotently, with 14 transient claude -p CLI errors retried until convergence (final 282 / 282 ).

“We scored the full 282 T4 budget-fair pool idempotently, with 14 transient claude -p CLI errors retried until convergence (final 282 / 282 ).”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Classified as probably not a substantial claim.

The judge receives the task spec, the test command, the test pass/fail status, and the post-run repository snapshot (truncated to 4 KB per file and 30 KB total) and returns four 0-100 scores assessing spec coverage, correctness signals, code quality, and test rigor.

“The judge receives the task spec, the test command, the test pass/fail status, and the post-run repository snapshot (truncated to 4 KB per file and 30 KB total) and returns four 0-100 scores assessing (a) spec coverage (how many spec requirements have visible implementation), (b) correctness signals (proper types, error handling, no obvious bugs), (c) code quality (separation of concerns, naming,”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Classified as probably not a substantial claim.

The composite is a fixed-weight average (0.35 spec coverage, 0.30 correctness, 0.20 code quality, 0.15 test rigor).

“The composite is a fixed-weight average ( 0 . 35 spec coverage, 0 . 30 correctness, 0 . 20 code quality, 0 . 15 test rigor)”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Classified as probably not a substantial claim. May combine two claims.

The scorers of §2.5 judge code; they do not execute it.

“The scorers of §2.5 judge code; they do not execute it.”

Classified as probably not a substantial claim. May combine two claims.

Regex-based scoring was chosen over AST parsing for reproducibility across runtime variants (ESM, CJS, mixed) and because agents occasionally emit near-valid code that an AST parser would reject outright.

“Regex-based scoring was chosen over AST parsing for reproducibility across runtime variants (ESM, CJS, mixed) and because agents occasionally emit near-valid code that an AST parser would reject outright”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Classified as probably not a substantial claim. May combine two claims.

The LLM-judge here is Sonnet 4.6, evaluating outputs from a mix of distinct models including Haiku 4.5, Codex GPT-5.4, and Gemini 3 Flash, in addition to Sonnet itself.

“The LLM-judge here is Sonnet 4.6, evaluating outputs from a mix of distinct models including Haiku 4.5, Codex GPT-5.4, and Gemini 3 Flash, in addition to Sonnet itself.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The rubric and the judge code ship with the released repository.

“The rubric and the judge code ship with the released repository.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The probe pool covers all budget-fair runs, including runs without an LLM score, so n differs slightly from Table 1.

“The probe pool covers all budget-fair runs, including runs without an LLM score, so n differs slightly from Table 1.”

Classified as probably not a substantial claim.

All continuous-score statistics in the main text on the T4 budget-fair pool are produced by an LLM-judge: a single Sonnet 4.6 invocation per run with a fixed rubric.

“All continuous-score statistics in the main text on the T4 budget-fair pool are produced by an LLM-judge: a single Sonnet 4.6 invocation per run with a fixed rubric.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Classified as probably not a substantial claim. May combine two claims.

The sandbox template plus the snapshot's src/ overlay boots under tsx, and we probe ten binary capabilities against the live server: boot, accounts endpoint, account creation, account listing, balanced-transaction posting, unbalanced-transaction rejection, balance query, trial balance, audit trail, and reconciliation endpoint.

“The sandbox template plus the snapshot's src/ overlay boots under tsx , and we probe ten binary capabilities against the live server: boot, accounts endpoint, account creation, account listing, balanced-transaction posting, unbalanced-transaction rejection, balance query, trial balance, audit trail, and reconciliation endpoint.”

Classified as probably not a substantial claim. May combine two claims.

Weights were fixed before any multi-agent runs were evaluated, based on the relative importance of each dimension for production-grade TypeScript code as judged by the authors and not tuned to favour any condition.

“Weights were fixed before any multi-agent runs were evaluated, based on the relative importance of each dimension for production-grade TypeScript code as judged by the authors and not tuned to favour any condition.”

• AssumptionRests on an assumption: The regex scorer's dimension weights reflect the relative importance of each dimension for production-grade TypeScript code.

Classified as probably not a substantial claim.

We replayed every archived T4 budget-fair snapshot against a task-blind capability probe fixed before the replay.

“we replayed every archived T4 budget-fair snapshot against a task-blind capability probe fixed before the replay.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The regex scorer is retained as a secondary scorer and is the primary scorer for the T5 matrix and difficulty gradient; the cross-model effect-size table uses the AST scorer (Table 2).

“The regex scorer is retained below as a secondary scorer and is the primary scorer for the T5 matrix and difficulty gradient; the cross-model effect-size table uses the AST scorer (Table 2).”

• cites Table 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

May combine two claims.

The oracle agrees with the judge on the endpoints of the ordering: AgentRoom is the top condition, and concurrency without coordination is the floor.

“The oracle agrees with the judge on the endpoints of the ordering: AgentRoom is the top condition, and concurrency without coordination is the floor”

• AssumptionRests on an assumption: The ten binary capabilities probed by the execution oracle are sufficient to measure functional correctness of the ledger system.

Probably not a substantial claim: may not pass S3 (load-bearing).

Weights were fixed before any conditions were evaluated.

“weights were fixed before any conditions were evaluated”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Probably not a substantial claim: may not pass S3 (load-bearing).

We use the smaller subset here because it is the intersection of solo runs for which per-dimension intermediate scorer outputs were retained.

“We use the smaller subset here because it is the intersection of solo runs for which per-dimension intermediate scorer outputs were retained.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

Classified as probably not a substantial claim.

The spread off the diagonal is why the body uses one scorer throughout and reports the other as a cross-check rather than averaging them.

“The spread off the diagonal is why the body uses one scorer throughout and reports the other as a cross-check rather than averaging them.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.

May combine two claims.

The two scorers measure the same thing loosely, not interchangeably.

“The two scorers measure the same thing loosely, not interchangeably.”

• AssumptionRests on an assumption: The LLM-judge composite is a valid measure of code quality for comparing conditions.