PreprintCited by 0Topic ModelingMulti-Agent Systems and NegotiationAdversarial Robustness in Machine Learning

Measuring Collapse and Correction in Homogeneous-Panel LLM Debate

Open access
Li, Xin, Liu, Mengbing, Yuen, Chau·2026·arXiv

Abstract

Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but movement is not necessarily improvement: the same discussion can rescue an initially wrong majority or destroy an initially correct one. Standard final-accuracy evaluations conflate these opposing mechanisms. We introduce an auditable protocol for homogeneous debate on multiple-choice questions (MCQs) that records each run as a transition ledger over collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates, the protocol identifies 253 collapses and a parallel correction ledger that changes how interventions should be judged. Replay experiments reveal the central tradeoff: a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections under equal weights, so collapse prevention alone can recommend the wrong policy. A compact pre-debate 8-probe screen is a triage signal: its unadjusted family-level association with conditional-collapse risk is high (G=7, Spearman rho=0.893, exact two-sided p=0.0123), but initial-majority accuracy is a close comparator (rho=0.821; family partial rho=0.767, p=0.0877), so we do not treat it as calibrated or capability-adjusted prediction. Round-level traces localize many collapses to the first debate round, where early disagreement can precede both harmful cascades and useful recovery. We release replayable schemas, coders, audits, cost cards, and zero-API rebuild scripts so future model-scaffold rows can be compared under the same denominators and signed utility ledger.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper introduces an auditable transition-ledger protocol for homogeneous three-agent LLM debate on MCQs, separating final-accuracy changes into collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates it finds 253 collapses and shows that freeze-style interventions can prevent collapses while losing more corrections, so collapse prevention alone can select the wrong policy.

Key findings

  • Across 6,925 MMLU-Pro debates, the protocol identifies 253 collapses; onset is concentrated in Round 1: 149/253 (58.9%), versus Round 2 47 (18.6%) and Round 3 57 (22.5%). Adding Round 1 trajectory features raises pooled out-of-fold AUC from 0.669 to 0.768, and full trajectory features reach 0.846.
  • A pre-debate 8-probe screen has a high unadjusted family-level association with conditional-collapse risk: G=7 Spearman ρ=0.893, exact two-sided p=0.0123. Initial-majority accuracy is a close comparator (ρ=0.821), and the family partial after initial accuracy is ρ=0.767, p=0.0877, so the screen is treated as triage, not calibrated prediction.
  • Signed replay shows freeze-style policies are net-negative under equal collapse/correction weights: a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections (net -79), and a Round 1 majority-change rule prevents 56 collapses but loses 69 corrections (net -13; break-even ratio r*=1.23).
  • Cross-benchmark GPQA stress rows support transition accounting rather than screen transfer: Mistral Small 4 has 20/120 conditional collapses and 21/78 corrections (net +1), while DeepSeek V4 Flash has 10/111 collapses and 47/87 corrections (net +37).

Methods

  • Homogeneous, closed-book, three-agent LLM debate on multiple-choice questions: agents answer independently in Round 0, then discuss and update for Rounds 1–3 while seeing all current answers; the final majority is the group response. Runs are coded into a 2×2 ledger of preserved, collapse, correction, and unrepaired transitions, with collapse onset and signed utility.
  • The pre-debate probe is an 8-probe 4×2 battery crossing argument strength (weak, moderate, strong, very strong) with social pressure (solo counterargument vs. attributed to two other agents). It reports α_tot, α_adv, α_cor, and S/A; all labels are rule-based with no LLM judge or human coder.
  • The primary MMLU-Pro analysis covers N=14 model rows across G=7 family clusters after attrition from a pre-registered N=18 target. Analyses include family-level exact Spearman tests, model-row and leave-one-family sensitivity checks, out-of-fold AUC for Round 1 features, a Bayesian multilevel logistic model, and held-out signed replay of interventions under declared weights.

Limitations

  • The paper states that the primary association uses the realized matched cohort (N=14 rows, G=7 families), not the pre-registered N=18; it is small and uneven, exact family tests are primary, and held-out-family diagnostics test rank discipline rather than calibrated forecasting. Scope is narrow: homogeneous same-model, closed-book MCQ debate; MMLU-Pro is primary, and ARC, TruthfulQA, GPQA, and OpenRouter/Gemini rows do not establish benchmark transfer or cover heterogeneous judge-debater systems, tool use, retrieval, free-form generation, rubric- or judge-scored answers, or longer-context evidence-gathering workflows.
  • The paper states that the screen is protocol-bound triage, not a capability-free property, per-question oracle, or controller; initial-majority accuracy is a close comparator, and after Round 0 sampling, disagreement is the stronger runtime diagnostic. Round 1 trajectories localize risk without causal mediation, and controllers need correction-preserving uncertainty, prespecified weights, and held-out signed replay.
  • The paper states that the primary cohort does not isolate reasoning mode or effort, and a follow-up does not establish that debate outperforms a compute-matched solo reasoning baseline. The artifact is release-tiered: exact per-debate Round 1 and strict matched-τ DisagreementGate matrices remain gated, so those results are diagnostic rather than primary, and provider reruns may drift. The text also makes evident that Qwen contributes six of the 14 model rows and is a dependence cluster, and that parser/tie-rule choices affect some transition labels.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2609.35279

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 2 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Capability pressure ranks Qwen3-4B above Llama-3.1-8B in risk because their initial-majority accuracies are 45.3% and 50.7%, respectively.

    “Capability pressure ranks Qwen3-4B above Llama-3.1-8B in risk because their initial-majority accuracies are 45.3% and 50.7%, respectively.”
    Quote in the paperSubstantial claim
  • Findingown data

    Round-level traces localize many collapses to the first debate round.

    “Round-level traces localize many collapses to the first debate round”
    Quote in the paperNot a substantial claim
  • Findingown data

    Initial-majority accuracy is a close comparator to the screen (ρ = 0.821; family partial ρ = 0.767, p = 0.0877).

    “initial-majority accuracy is a close comparator (ρ = 0.821; family partial ρ = 0.767, p = 0.0877)”
    • Also stated in: § 4.3 Sensitivity: Capability and Per-Question Boundaries · plain
      “Capability pressure, measured as 1-initial-majority accuracy, is a close free comparator (ρ=+0.821at the family level)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The Round 1 trajectory features are available and meaningful for runtime diagnosis.
  • Findingown data

    In debate traces, 149 of 253 tracked collapses originate in Round 1.

    “In debate traces, 149/253 tracked collapses originate in Round 1”
    Quote in the paperSubstantial claim
  • Findingown data

    Llama-3.1-8B's conditional collapse rate is about four times higher than Sonnet 4.5's (8.46% vs. 2.15%).

    “yet Llama's conditional collapse rate is about four times higher (8.46% vs.”
    Quote in the paperSubstantial claim
  • May combine two claims. 157
  • Classified as probably not a substantial claim. 82
  • The quote only partly states this claim. 2
  • May combine two claims.

    Findingown data

    A post-hoc audit of all 91 model-row pairs found the screen reorders 26 pairs relative to initial accuracy, and observed conditional collapse follows the screen in 13 and initial accuracy in 12 (one tie).

    “A post-hoc audit of all 91 model-row pairs bounds this use: the screen reorders 26 pairs relative to initial accuracy, and observed conditional collapse follows the screen in 13 and initial accuracy in 12 (one tie; Section G.1).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Low-α_tot families occupy the low-collapse ranks, while high-α_tot open-model families carry most collapse risk; Meta is the visible upper-tail inversion.

    “Low-α tot families occupy the low-collapse ranks, while high-α tot open-model families carry most collapse risk; Meta is the visible upper-tail inversion.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Under equal collapse/correction weights, a leave-one-model-out probe-gated freeze prevents 29 collapses but discards 108 corrections, for a net utility of -79.

    “a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections under equal weights”
    • Also stated in: § 2.15%). · plain
      “under equal collapse/correction weights, a leave-one-model-out probe-gated freeze prevents 29 collapses but discards 108 corrections, for a net utility of -79.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Realized-cohort model-row partials remain positive after residualizing on initial accuracy (ρ = +0.658, p = 0.014) and initial accuracy plus raw revision (ρ = +0.667, p = 0.018).

    “Realized-cohort model-row partials remain positive after residualizing on initial accuracy (ρ=+0.658,p=0.014) and initial accuracy plus raw revision (ρ=+0.667,p=0.018)”
    Quote in the paperNot a substantial claim
  • The quote only partly states this claim.

    Findingown data

    Simple Round 1 replay gates show the same sign as the LOMO freeze.

    “Simple Round 1 replay gates show the same sign”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Capability pressure, measured as 1-initial-majority accuracy, is a close free comparator (ρ = +0.821 at the family level), leaving a modest marginal rank gain for α_tot of about Δρ = 0.07.

    “Capability pressure, measured as 1-initial-majority accuracy, is a close free comparator (ρ=+0.821at the family level), leaving a modest marginal rank gain for α tot of about ∆ρ=0.07”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The family-level rank partial after initial-majority accuracy remains positive but is no longer confirmatory (ρ = +0.767, exact two-sided p = 0.0877).

    “The family-level rank partial after initial-majority accuracy remains positive but is no longer confirmatory (ρ=+0.767,exact two-sided p=0.0877).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Probe ablations support the intended argument/social channel separation, but the effect is uneven, with Qwen-3 rows near zero.

    “Probe ablations support the intended argument/social channel separation, but the effect is uneven, with Qwen-3 rows near zero.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The probe reverses the capability-pressure order (α_tot = 0.774 vs. 0.607), matching the observed conditional-collapse ordering (8.46% vs. 5.11%).

    “The probe reverses this order (α tot =0.774 vs. 0.607), matching the observed conditional-collapse ordering (8.46% vs. 5.11%).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The combined collapse predictor changed by at most 0.004 AUC when very_strong was removed and by at most 0.005 AUC when only non-social probes were used.

    “The combined predictor used in the main text (mean probe flip rate + difficulty + initial disagreement) changed by at most 0.004 AUC when very_strong was removed and by at most 0.005 AUC when only non-social probes were used.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Removing the very_strong probe preserved the model-level S/A ordering exactly (Spearman ρ=1.00 across four models), with largest absolute shift in mean S/A only 0.024.

    “Removing very_strong preserved the model-level S/A ordering exactly (Spearman ρ = 1.00 across the four models), with the largest absolute shift in mean S/A only 0.024.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Alphabetic tie-breaking is used in 298/3,240 primary initial/final majority reductions (9.20%), with 272/1,620 rows having at least one primary initial/final tie.

    “Alphabetic tie-breaking is used in 298/3,240 primary initial/final majority reductions (9.20%), with 272/1,620 rows having at least one primary initial/final tie.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Recomputing S/A on a single shared 200-question pool leaves the model-level rank ordering of S/A unchanged (Spearman ρ=0.95 between per-model and shared-pool rankings on the four overlap models), with largest absolute shift in S/A 0.04.

    “The model-level rank ordering of S/A is unchanged (Spearman ρ = 0.95 between the per-model and shared-pool rankings on the four overlap models), and the largest absolute shift in S/A is 0.04.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Anti-argument and anti-social system prompts elicit measurably different flip distributions in all 16/16 tested models, with grand mean Cohen's d = 0.53 and 12/16 paired tests surviving Holm-Bonferroni correction.

    “Anti-argument and anti-social system prompts elicit measurably different flip distributions in all 16/16 tested models, with grand mean Cohen's d=0.53 and 12/16 paired tests surviving Holm-Bonferroni correction”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In eleven reasoning-mode rows, collapses and corrections both persist.

    “In eleven reasoning-mode rows, collapses and corrections both persist”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In the Mistral row, four questions lack a valid initial majority; excluding them changes the signed net from +1 to -1.

    “In the Mistral row, four questions lack a valid initial majority; excluding them changes the signed net from +1 to -1”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In the raw-log subset, removing the very_strong probe preserves the model-level S/A ordering exactly (Spearman ρ = 1.00 over four models) and changes matched collapse-prediction AUC by at most 0.004.

    “In the raw-log subset, removing that very_strong probe preserves the model-level S/A ordering exactly (Spearman ρ=1.00 over four models) and changes matched collapse-prediction AUC by at most 0.004”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Private revision has a negative point difference in all eleven reasoning-mode rows (intervals below zero in seven).

    “private revision has a negative point difference in all eleven (intervals below zero in seven; Table 8).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Private revision reduces wrong-answer collapses but leaves more ties.

    “it reduces wrong-answer collapses but leaves more ties”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Initial-majority accuracy is a close comparator; after Round 0 sampling, disagreement is the stronger runtime diagnostic.

    “Initial-majority accuracy is a close comparator; after Round 0 sampling, disagreement is the stronger runtime diagnostic.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The strict parser labels both initial and final majorities for 966/1,620 checked-in trace rows; among those strict-labeled rows, four-cell transition membership changes in 51 rows (5.28%), with 20 collapse-label and 29 correction-label changes.

    “The strict parser labels both initial and final majorities for 966/1,620 checked-in trace rows; among those strict-labeled rows, four-cell transition membership changes in 51 rows (5.28%), with 20 collapse-label and 29 correction-label changes.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Attrition and manifest-discipline exclusions reduced the matched post-A6 cohort to N = 14, and six of those rows are related Qwen variants.

    “attrition and manifest-discipline exclusions reduced the matched post-A6 cohort to N =14, and six of those rows are related Qwen variants.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Anti-argument prompts elicit more revision than anti-social prompts, with the ordering holding in 16/16 tested models.

    “Anti-argument prompts elicit more revision than anti-social prompts.The ordering holds in 16/16 tested models”
    • Also stated in: § B.6 Full 16-Model Channel Manipulation Results · plain
      “Anti-argument FR > Anti-social FR in 16/16 models”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The tie rule fires in 298/3,240 initial/final majority reductions (9.20%), and removing the last-valid-letter fallback changes 51/966 strict-labeled transition cells.

    “in the open trace subset, it fires in 298/3,240 initial/final majority reductions (9.20%), and removing the last-valid-letter fallback changes 51/966 strict-labeled transition cells.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    DeepSeek V4 Flash gains despite nonzero collapse because corrections dominate.

    “while DeepSeek V4 Flash gains despite nonzero collapse because corrections dominate”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Qwen3-4B (d = 0.092) and Qwen3-8B (d = 0.057) are near-zero separability cases.

    “Qwen3-4B (d=0.092) and Qwen3-8B (d=0.057) are near-zero separability cases.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The current model-row Spearman is ρ = +0.8295, and coding new-lane None post-answers as flips gives ρ = +0.8119.

    “the current model-row Spearman is ρ=+0.8295,and coding new-lane None post-answers as flips gives ρ=+0.8119.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    At the per-question level, after initial answers are observed, disagreement is the stronger runtime diagnostic, while probe FR and S/A do not act as row-level collapse detectors.

    “At the per-question level, the screen boundary is sharper: after initial answers are observed, disagreement is the stronger runtime diagnostic, while probe FR and S/A do not act as row-level collapse detectors”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    All three freeze-style gates prevent some collapses but lose more corrections; even the closest Round 1 gate is 56 prevented versus 69 lost.

    “All three freeze-style gates prevent some collapses but lose more corrections; even the closest Round 1 gate is 56 prevented versus 69 lost.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The freeze-style gates evaluated are representative of intervention policies for debate.
  • May combine two claims.

    Findingown data

    α tot remains associated with C cond under simple rank residualizations: controlling capability-pressure proxy gives ρ=+0.658 (p=0.014); controlling raw debate-revision proxy alone gives ρ=+0.780 (p=0.0017); controlling capability pressure plus raw debate-revision proxy gives ρ=+0.667 (p=0.018).

    “α tot remains associated with C cond under simple rank residualizations: controlling the capability-pressure proxy gives ρ=+0.658(p=0.014);controlling raw debate-revision proxy alone gives ρ=+0.780(p=0.0017);controlling capability pressure plus the raw debate-revision proxy gives ρ=+0.667(p=0.018).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Mistral Small 4 has 20/120 conditional collapses and 21/78 corrections, producing only +1 net debate under equal collapse/correction weights.

    “Mistral Small 4 has 20/120 conditional collapses and 21/78 corrections, producing only +1 net debate under equal collapse/correction weights.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The observed Round 1 lift is ∆R1=+0.099 with debate-index percentile 95% CI [+0.080, +0.118]; the residual full-trajectory lift is ∆full=+0.077 with 95% CI [+0.064, +0.092].

    “The observed Round 1 lift is ∆ R1 =+0.099 with debate-index percentile 95% CI [+0.080, +0.118]; the residual full-trajectory lift is ∆ full =+0.077with 95% CI [+0.064, +0.092].”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    If appended as a post-hoc N=15 model-row stress test, the Spearman association remains ρ=+0.822; averaging it into the Qwen family leaves the G=7 family association unchanged at +0.893, and treating it as a separate stress family gives G=8 ρ=+0.857.

    “If appended as a post-hoc N =15 model-row stress test, the Spearman association remains ρ=+0.822;averaging it into the Qwen family leaves the G=7 family association unchanged at +0.893, and treating it as a separate stress family gives G=8 ρ=+0.857.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Phi remains a high-collapse row (C cond=10.24%), while Anthropic/OpenAI/Google/DeepSeek occupy the lower-collapse ranks.

    “Phi remains a high-collapse row (C cond =10.24%), while Anthropic/OpenAI/Google/DeepSeek occupy the lower-collapse ranks.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Mistral Small 4 has the highest holdout probe rate (α tot=0.3772), 7/140 R4 collapses (5.00%), and 13/60 R4 corrections (21.67%).

    “Mistral Small 4 is the main new-family stress row: it has the highest holdout probe rate (α tot =0.3772), 7/140 R4 collapses (5.00%), and 13/60 R4 corrections (21.67%).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Dropping each Qwen variant in turn leaves the model-row Spearman in [+0.809, +0.875] over the remaining N=13 rows; worst cases are dropping Qwen3-4B or Qwen3-8B (ρ=+0.809).

    “Dropping each Qwen variant in turn leaves the model-row Spearman in [+0.809, +0.875] over the remaining N =13 rows; the worst cases are dropping Qwen3-4B or Qwen3-8B (ρ=+0.809).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The cascade is dominantly a Round 1 phenomenon (58.9% onset) but not exclusively.

    “The cascade is dominantly a Round 1 phenomenon (58.9% onset) but not exclusively”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Leave-one-model-out predictive R2 on a rank-linear fit is R2 LOMO=0.26 for total α and 0.12 for α adv.

    “(2) Leave-one-model-out predictive R 2 on a rank-linear fit: R 2 LOMO =0.26 for total α and 0.12 for α adv .”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The two 80% misses are Meta (observed below the interval) and Qwen (observed above it); Qwen is the only 95% miss, with the held-out fit underpredicting the upper tail.

    “The two 80% misses are Meta (observed below the interval) and Qwen (observed above it); Qwen is the only 95% miss, with the held-out fit underpredicting the upper tail.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Per-model α and α adv values drop uniformly by ∆α ∈ [-0.073, -0.022] but their rank order across models is preserved.

    “the per-model α and α adv values drop uniformly by ∆α ∈ [-0.073, -0.022] but their rank order across models is preserved.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The clipped EIV-propagated predictive intervals cover 5/7 families at 80% and 7/7 at 95%; Meta and Qwen remain the two 80% misses, while Qwen is now included only at the upper edge of the 95% interval.

    “The clipped EIV-propagated predictive intervals cover 5/7 families at 80% and 7/7 at 95%; Meta and Qwen remain the two 80% misses, while Qwen is now included only at the upper edge of the 95% interval.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    GPQA contributes the only collapse (1/31=3.2%) and all three corrections; TruthfulQA has 0/42 collapses and no corrections.

    “GPQA contributes the only collapse (1/31=3.2%)and all three corrections; TruthfulQA has 0/42 collapses and no corrections.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Pooled out-of-fold AUC is 0.669 for the pre-debate baseline, 0.768 for Round 1 trajectory, and 0.846 for the full trajectory.

    “Pooled out-of-fold AUC is 0.669 for the pre-debate baseline, 0.768 for Round 1 trajectory, and 0.846 for the full trajectory.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Llama 3.3 70B and DeepSeek V4 Flash both have measurable collapses, but standard debate is net positive because corrections are larger (+7 and +37 debates, respectively).

    “Llama 3.3 70B and DeepSeek V4 Flash add a useful contrast: both have measurable collapses, but standard debate is net positive because corrections are larger (+7 and +37 debates, respectively).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    α adv remains directionally positive after initial accuracy but is weaker and non-confirmatory (ρ=+0.52, raw p=0.148), while α cor points negative.

    “It does not: α adv remains directionally positive after initial accuracy but is weaker and non-confirmatory (ρ=+0.52,raw p=0.148), while α cor points negative.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    With model-constant predictors and N=9 models grouped into 6 families, the GEE coefficients on α adv and initial accuracy are individually statistically indistinguishable from zero (cluster-robust |z|≤0.5); the predictors are near-collinear (Spearman between α adv and initial accuracy: -0.58).

    “With model-constant predictors and N =9 models grouped into 6 families, the GEE coefficients on α adv and initial accuracy are individually statistically indistinguishable from zero (cluster-robust |z|≤0.5); the predictors are near-collinear (Spearman between α adv and initial accuracy: -0.58).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The family-level partial Spearman remains positive, ρ α|init acc=+0.767, with exact two-sided p=0.0877.

    “The family-level partial remains positive, ρ α|init acc =+0.767, but the small denominator is visible: enumerating all 7! assignments of family-level C cond while holding family α tot and initial accuracy fixed gives exact two-sided p=0.0877.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Expanding the same Gemini lane to the full 198-question GPQA Diamond split preserves the boundary interpretation: collapse remains sparse (2/147=1.4%), while corrections remain nonzero (9/51=17.6%).

    “Expanding the same Gemini lane to the full 198-question GPQA Diamond split preserves the boundary interpretation: collapse remains sparse (2/147=1.4%),while corrections remain nonzero (9/51=17.6%).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The permutation null for total α gives observed Spearman ρ=+0.78 with 2-sided permutation p<0.02; for α adv, ρ=+0.62 gives p≈0.09.

    “(1) Permutation null (10,000 random shuffles of C cond ): the observed Spearman ρ=+0.78 for total α has 2-sided permutation p<0.02; for α adv , ρ=+0.62 gives p≈0.09.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    GPT-5.4-mini has an S/A ratio of +0.048, near zero, consistent with argument-dominated revision.

    “GPT-5.4-mini was selected as a preregistered held-out sanity check because its S/A ratio (+0.048) is near zero, consistent with argument-dominated revision.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    On ARC-Challenge, both Haiku and GPT-4o-mini show dramatically lower flip rates (Haiku: 0.512→0.179, d=-1.13; GPT: 0.458→0.161, d=-1.07) and near-zero collapse (Haiku: 0.5%, GPT: 0.0%).

    “On ARC-Challenge [4] (an easier benchmark), both Haiku and GPT-4o-mini show dramatically lower flip rates (Haiku: 0.512 → 0.179, d = -1.13;GPT: 0.458 → 0.161, d = -1.07)and near-zero collapse (Haiku: 0.5%, GPT: 0.0%).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: ARC-Challenge is an easier benchmark.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Round 1 majority-change rate is even higher on this slice (ρ=+0.986).

    “Round 1 majority-change rate is even higher on this slice (ρ=+0.986),as expected for a runtime trajectory score.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The six near-complete additional OpenRouter alpha-channel lanes all preserve AA>AS.

    “The six near-complete additional lanes all preserve AA>AS.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Gemini 3-flash (S/A=-0.08) provides a second low-S/A 200-debate sanity row, with low collapse at 0.5% and a correction-to-collapse ratio of 2.0.

    “Gemini 3-flash (S/A = -0.08)provides a second low-S/A 200-debate sanity row, with low collapse at 0.5% and a correction-to-collapse ratio of 2.0.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The realized N=14 rerun gives point Spearman +0.8295 and median latent rank correlation +0.8022, with 95% interval [+0.7099, +0.8725] and Pr(r>0)=1.000 over 20,000 bootstrap draws.

    “The realized N =14 rerun gives point Spearman +0.8295 and median latent rank correlation +0.8022, with 95% interval [+0.7099, +0.8725] and Pr(r>0)=1.000over 20,000 bootstrap draws (seed 20260427).”
    • Also stated in: § C.7 Specification Curve and Errors-in-Variables · plain
      “The realized N =14 rerun gives point Spearman +0.8295 and median latent rank correlation +0.8022, 95% interval [+0.7099, +0.8725], Pr(r>0)=1.000.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Clipping the displayed intervals to the admissible [0,100]% range leaves both coverage counts unchanged.

    “Clipping the displayed intervals to the admissible [0, 100]% range leaves both coverage counts unchanged.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The held-out-family predictive check gives predicted-vs-observed Spearman +0.821 (p=0.023), 80% prediction intervals cover 5/7 families, 95% intervals cover 6/7, and mean absolute rank error 0.857.

    “The diagnostic supports a ranking interpretation but not a calibrated forecast: predictedvs.-observedSpearman is +0.821 (p=0.023),80% prediction intervals cover 5/7 families, 95% intervals cover 6/7, and the mean absolute rank error is 0.857.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Across all three day pairs, the 20 shared question-level mean-α values have Spearman 1.000, mean absolute delta 0.0000, max absolute delta 0.0000, and zero initial-answer majority changes.

    “Across all three day pairs, the 20 shared question-level mean-α values have Spearman 1.000, mean absolute delta 0.0000, max absolute delta 0.0000, and zero initial-answer majority changes.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Leave-one-family-out on the family-mean table has worst case ρ=+0.8286; dropping any of Anthropic, DeepSeek, Google, or OpenAI gives +0.829; dropping Meta gives +1.000; dropping Phi or Qwen gives +0.943.

    “Leave-one-familyout on the family-mean table has worst case ρ=+0.8286:dropping any of Anthropic, DeepSeek, Google, or OpenAI gives +0.829 because those rows occupy the low-collapse rank cluster; dropping Meta gives +1.000; dropping Phi or Qwen gives +0.943.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    OpenAI models show the strongest channel manipulation effects (mean d=0.82) with prominent channel substitution.

    “OpenAI models show the strongest channel manipulation effects (mean d = 0.82) with prominent channel substitution.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Dropping both Meta and Qwen leaves the remaining five families strictly monotone, ρ=+1.000 (exact two-sided p=0.0167 over 5! permutations).

    “Dropping both Meta (the visible upper-tail inversion) and Qwen (the multi-row upper-tail family) leaves the remaining five families strictly monotone, ρ=+1.000(exact two-sided p=0.0167 over 5! permutations).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Both mean and median family aggregation over the seven retained families give ρ=+0.8929; exact permutation over all 7! family-label assignments gives two-sided p=0.0123.

    “Both mean and median aggregation over the seven retained families give ρ=+0.8929;exact permutation over all 7! family-label assignments gives two-sided p=0.0123.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Within every day of the three-day Gemini Flash-Lite panel there are zero initial-answer instabilities, mean/max within-question α range 0.0000/0.0000, and 4.38% probe post-answer parse-None rate.

    “Within every day there are zero initial-answer instabilities, mean/max within-question α range 0.0000/0.0000,and 4.38% probe post-answer parse-None rate.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The high-accuracy Google/xAI lanes have very low observed R4 collapse (zero or one collapse among 145-167 initially-correct pluralities), while still producing nonzero corrections.

    “The highaccuracy Google/xAI lanes have very low observed R4 collapse (zero or one collapse among 145-167 initially-correct pluralities), while still producing nonzero corrections.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Across 100 debates in the Gemini 3.1 Flash-Lite 50+50 check, initial-majority accuracy is 73.0% and final-majority accuracy is 75.0%; there is only 1 collapse (1/73=1.4% conditional collapse) and 3 corrections.

    “The Gemini 3.1 Flash-Lite 50+50 check is not a powered transfer test: across 100 debates, initial-majority accuracy is 73.0% and final-majority accuracy is 75.0%;there is only 1 collapse (1/73=1.4% conditional collapse) and 3 corrections.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Qwen3-32B has probe α tot=0.6767 over 1,800 alpha rows, matched to 200 debated questions; in those debates initial-majority accuracy improves from 36.5% to 52.0%, with 8/73 at-risk initial-correct majorities collapsing (10.96%) and 39/127 initially-wrong majorities corrected (30.71%).

    “The paired slice is complete: probe α tot =0.6767 over 1,800 alpha rows (200 questions × 3 conditions × 3 agents), matched to the same 200 debated questions with no duplicate question rows.In those debates, initial-majority accuracy improves from 36.5% to 52.0%, with 8/73 at-risk initial-correct majorities collapsing (10.96%) and 39/127 initially-wrong majorities corrected (30.71%).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    All 27 specification-curve cells return ρ>0 on the N=9 cohort with median +0.62.

    “All 27 cells return ρ>0 on the N =9 cohort with median +0.62”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Alternative taxonomies preserve the positive association: splitting Qwen into 3/3.5/3.6 generations gives ρ=+0.883 over 9 groups; merging Phi and Gemma into a small-dense group gives ρ=+0.929 over 7 groups; applying both gives ρ=+0.933 over 9 groups.

    “Alternative taxonomies preserve the positive association: splitting Qwen into 3 / 3.5 / 3.6 generations gives ρ=+0.883 over 9 groups; merging Phi and Gemma into a small-dense group gives ρ=+0.929 over 7 groups; applying both choices gives ρ=+0.933 over 9 groups.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    On GPT-4o-mini, where collapse is already rare (2.0%), all four candidates (Standard, Shielded, ACC, DRS) sit within ±1.5pp of each other and none robustly beats Standard.

    “On GPT-4o-mini, where collapse is already rare (2.0%), all four candidates (Standard, Shielded, ACC, DRS) sit within ±1.5pp of each other and none robustly beats Standard.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    On Phi-4-mini, DisagreementGate is 55.7/2.6, DRS is 58.3/7.3, and ACC is 59.9/4.7.

    “On Phi-4-mini, DisagreementGate is 55.7/2.6,DRS is 58.3/7.3, and ACC is 59.9/4.7.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Private revision reduces wrong-answer collapses in every setting, with intervals below zero in eleven, but raises correct-to-tied transitions in all twelve, and corrections decrease in every setting.

    “It reduces wrong-answer collapses in every setting, with intervals below zero in eleven, but raises correct-to-tied transitions in all twelve, and corrections decrease in every setting.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Round 1 is not sufficient: 18.6% of collapses first appear in Round 2 and 22.5% in Round 3, while some debates later correct a Round 1 wrong majority.

    “Most importantly for policy, Round 1 is not sufficient: 18.6% of collapses first appear in Round 2 and 22.5% in Round 3, while some debates later correct a Round 1 wrong majority.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    F3 floor of -0.02, McNemar discordant counts 0/1 (p=1.0), Spearman perm.p=0.78 for Qwen3.5-9B.

    “F3 floor of -0.02, McNemar discordant counts 0/1 (p=1.0),Spearman perm.p=0.78.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    As a weaker same-trace sanity check on the available 1,255 replay cohort, DisagreementGate is also net-negative: 63 collapses prevented, 104 corrections lost, net -41 (-3.27pp).

    “As a weaker same-trace sanity check on the available 1,255 replay cohort, DisagreementGate is also net-negative: 63 collapses prevented, 104 corrections lost, net -41 (-3.27pp).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The 1,255-trace replay cohort is representative of the 6,525-debate LOMO cohort for the DisagreementGate sanity check.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The observed collapse rate is 7.98%.

    “Posterior predictive check: observed collapse rate 7.98%, replicate mean matches; PPC one-sided p=0.5426.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingreasoning

    Qwen3.6-27B-FP8 clears the F3-floor gate despite its full-ladder Spearman being underpowered (perm.p=0.247).

    “Notably Qwen3.6-27B-FP8clears the F3-floor gate despite its full-ladder Spearman being underpowered (perm.p=0.247)”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    On Haiku (200 development questions), DisagreementGate yields 73.0% accuracy / 7.0% collapse and DRS yields 74.5%/5.0%, both worse than ACC's 78.0%/1.5%.

    “On Haiku (200 development questions), DisagreementGate yields 73.0% accuracy / 7.0% collapse and DRS yields 74.5%/5.0%,both worse than ACC's 78.0%/1.5%.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    3,145 debates across 4 OSS models with per-round trace data: Phi-4-mini, Qwen3-4B, Llama-3.1-8B, Qwen3-8B.

    “3,145 debates across 4 OSS models with per-round trace data: Phi-4-mini, Qwen3-4B, Llama-3.1-8B,Qwen3-8B.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Under expected tie scoring the differences lie between -1.62 and +0.38 points and seven intervals remain below zero.

    “Under expected tie scoring the differences lie between -1.62 and +0.38 points and seven intervals remain below zero, so much of the binary loss comes from counting ties as wrong.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The F3-floor magnitudes are statistically separable from zero on both usable models: Qwen3.5-4B at +20 pp (McNemar p=0.003) and Qwen3.6-27B-FP8 at +10 pp (McNemar p=0.031).

    “The F3-floor magnitudes are statistically separable from zero on both usable models: Qwen3.5-4B at +20 pp (McNemar p=0.003) and Qwen3.6-27B-FP8 at +10 pp (McNemar p=0.031).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Collapse onset remains early: R1 accounts for 40/75 collapses (53.3%), followed by R2:10, R3:11, and R4:14.

    “Collapse onset remains early: R1 accounts for 40/75 collapses (53.3%), followed by R2: 10, R3: 11, and R4: 14.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    A prompt control on Qwen3-8B and Granite-4.1-8B (two runs per arm) loses 2.48 and 2.43 binary accuracy points relative to a contemporaneous peer arm, and both differences remain significant after Holm correction over four planned contrasts, but it does not reliably reduce total collapse.

    “A prompt control on Qwen3-8B and Granite-4.1-8B(two runs per arm) replaces the debate framing with a neutral self-review prompt.It still loses 2.48 and 2.43 binary accuracy points relative to a contemporaneous peer arm, and both differences remain significant after Holm correction over four planned contrasts, but it does not reliably reduce total collapse.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    αtot = 0.6767 for Qwen3-32B holdout.

    “αtot | 0.6767”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    On the seven-source primary cohort (1,255 usable debates), always freezing prevents 94 collapses but loses 124 corrections (-2.39pp).

    “On the seven-source primary cohort (1,255 usable debates), always freezing prevents 94 collapses but loses 124 corrections (-2.39pp).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Qwen3-32B local-vLLM holdout has 1,800 alpha rows over 200 questions with complete 3×3 design.

    “Alpha rows / questions | 1,800 / 200 | complete 3×3 design”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    On 1,200 debates in the post-A6 replication, 662 have an initially correct majority, 75 collapse (11.3% conditional), and 144 correct an initially wrong majority; final accuracy is 60.9%.

    “On 1,200 debates, 662 have an initially correct majority, 75 collapse (11.3% conditional), and 144 correct an initially wrong majority; final accuracy is 60.9%.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Qwen3.5-9B's p k trajectory across k=0,...,8 is {0.04, 0.06, 0.08, 0.02, 0.02, 0.04, 0.04, 0.06, 0.02}, flat at low magnitude with no graded response to ladder rung.

    “Q3.5-9B's p k trajectory across k=0, . . ., 8 is {0.04, 0.06, 0.08, 0.02, 0.02, 0.04, 0.04, 0.06, 0.02}, flat at low magnitude with no graded response to ladder rung.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In a smaller reviewer-requested zero-retune transfer check on Qwen3-4B (50 development questions, 300 held-out test questions), ACC is also null (p = 0.743 for accuracy, p = 1.000 for collapse).

    “In a smaller reviewer-requested zero-retune transfer check on Qwen3-4B (50 development questions, 300 held-out test questions), ACC is also null (p = 0.743 for accuracy, p = 1.000 for collapse).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Harness-detected anti-monotonicity in 1/3 pilot models, statistically indistinguishable from null (Spearman perm.p=0.78, McNemar discordant n=1).

    “harness-detected anti-monotonicity in 1/3 pilot models, statistically indistinguishable from null (Spearman perm.p=0.78,McNemar discordant n=1)”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Pooled with legacy/partial traces, R1 remains the largest localized onset bucket (40/77 collapses overall, with 15 R4 and one unlocalized).

    “Pooled with legacy/partial traces, R1 remains the largest localized onset bucket (40/77 collapses overall, with 15 R4 and one unlocalized).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In the 2×2 debate experiment (N = 100 questions per cell), the independence prompt does not reduce collapse for either Haiku (McNemar p = 0.593) or GPT-4o-mini (p = 1.000).

    “In the 2×2 debate experiment (N = 100 questions per cell), the independence prompt does not reduce collapse for either Haiku (McNemar p = 0.593) or GPT-4o-mini (p = 1.000).”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingown data

    In the sealed pre-A6 N=9 rank-correlation cross-check, probe α_tot has Spearman ρ = +0.78 (95% CI [+0.25, +0.95], p=0.013) with conditional collapse.

    “Probe αtot (8-probe total) ⋆ | +0.78 | [+0.25, +0.95] | 0.013 | n/a”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    On the original 800-question test, Phi-4-mini showed a modest positive trend (+1.25pp accuracy, -1.12pp collapse relative to Standard), but neither paired McNemar test was significant (p = 0.485 for accuracy, p = 0.336 for collapse).

    “On the original 800-question test, Phi-4-mini showed a modest positive trend (+1.25pp accuracy, -1.12pp collapse relative to Standard), but neither paired McNemar test was significant (p = 0.485 for accuracy, p = 0.336 for collapse).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The intercept is -4.254 with 95% interval [-5.46, -2.96].

    “Intercept -4.254 [-5.46, -2.96].”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    251 collapses (7.98%); 1,509 unique questions.

    “251 collapses (7.98%); 1,509 unique questions.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The intuitive R1-majority-changed rule gates 23.2% of debates and is still net-negative: 56 collapses prevented, 69 corrections lost, -1.04pp accuracy change.

    “The intuitive R1-majority-changed rule gates 23.2% of debates and is still net-negative: 56 collapses prevented, 69 corrections lost, -1.04pp accuracy change.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The positive trend did not replicate: ACC reduces accuracy by 1.93pp and increases collapse by 0.40pp (p = 0.106 for accuracy, p = 0.656 for collapse).

    “The positive trend did not replicate: ACC reduces accuracy by 1.93pp and increases collapse by 0.40pp (p = 0.106 for accuracy, p = 0.656 for collapse).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    They prevent only 2 collapses while losing 31 corrections in held-out replay.

    “preventing only 2 collapses while losing 31 corrections in held-out replay”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The aggregate Table 37 ledger is sufficient to evaluate the probe-gated freeze policy without the row-level decision matrix.
  • May combine two claims.

    Findingown data

    On Haiku, neither Shielded nor ACC beats Standard debate (p = 0.188 for accuracy, p = 1.000 for collapse).

    “On Haiku, neither Shielded nor ACC beats Standard debate (p = 0.188 for accuracy, p = 1.000 for collapse).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Observed collapse follows the screen in 13 pairs, initial accuracy in 12, and neither in one tie.

    “Observed collapse follows the screen in 13 pairs, initial accuracy in 12, and neither in one tie.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The temperature field is missing from the saved probe records of several later rows.

    “The temperature field is missing from the saved probe records of several later rows.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The missing temperature field in saved probe records does not invalidate the screen results.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The Gemini-3.1-Pro row has about 11% missing answers in both probe revisions and debate answers.

    “The Gemini-3.1-Prorow has about 11% missing answers in both probe revisions and debate answers.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Excluding the four leaves 20/120 collapses and 19/74 corrections, so the signed net changes from +1 to -1.

    “Excluding the four leaves 20/120 collapses and 19/74 corrections, so the signed net changes from +1 to -1.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Dropping each Qwen row in turn keeps the model-row correlation between 0.809 and 0.875 and leaves the family estimate at ρ = 0.893.

    “Dropping each Qwen row in turn keeps the model-row correlation between 0.809 and 0.875 and leaves the family estimate at ρ = 0.893.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Dropping the Tencent row gives ρ = 0.948 (exact p = 0.00013) and a partial ρ = 0.931 (p = 0.000265).

    “dropping it gives ρ = 0.948 (exact p = 0.00013) and a partial ρ = 0.931 (p = 0.000265)”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Three mixed-model panels answer the same 200 MMLU-Pro questions and produce both transition directions.

    “Three mixed-model panels answer the same 200 MMLU-Pro questions and produce both transition directions.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In 20 to 28 of 200 initial states no answer has a unique plurality, so the results depend on the aggregation rule: treating such states as undecided and wrong changes the three nets from +14, +6 and 0 to +22, +8 and +7.

    “In 20 to 28 of 200 initial states no answer has a unique plurality, so the results depend on the aggregation rule: treating such states as undecided and wrong changes the three nets from +14, +6 and 0 to +22, +8 and +7.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The alphabetic tie rule is an appropriate way to resolve initial states without a unique plurality.
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The two arms sampled their initial answers independently, so only the 162 questions whose initial-majority labels match are paired; on them, peer-only and self-only correct outcomes are 3 and 3 (exact McNemar p = 1.0).

    “The two arms sampled their initial answers independently, so only the 162 questions whose initial-majority labels match are paired; on them, peer-only and self-only correct outcomes are 3 and 3 (exact McNemar p = 1.0).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    A GLM-4.6 row with provider reasoning disabled shows hidden turnover: accuracy moves from 76.0% to 75.5% while the ledger records 6/152 collapses and 5/48 corrections.

    “A GLM-4.6 row with provider reasoning disabled shows hidden turnover: accuracy moves from 76.0% to 75.5% while the ledger records 6/152 collapses and 5/48 corrections.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Ten unadjusted tests have p < 0.05 (screen 4, initial accuracy 6); after Holm correction over the 26 tests, eight remain (screen 2, initial accuracy 6), mostly within the Qwen family.

    “Ten unadjusted tests have p < 0.05 (screen 4, initial accuracy 6); after Holm correction over the 26 tests, eight remain (screen 2, initial accuracy 6), mostly within the Qwen family.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In the Mistral Small 4 row of Table 30, four of the 198 questions have no valid initial majority and are counted as initially wrong; two of them end correct.

    “In the Mistral Small 4 row of Table 30, four of the 198 questions have no valid initial majority and are counted as initially wrong; two of them end correct.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The motivating Llama-3.1-8B/Qwen3-4B reordering has unadjusted p = 0.0033 but does not survive correction.

    “The motivating Llama-3.1-8B/Qwen3-4Breordering has unadjusted p = 0.0033 but does not survive correction.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    Early disagreement can precede both harmful cascades and useful recovery.

    “where early disagreement can precede both harmful cascades and useful recovery.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    In multi-agent LLM debate, the same discussion can rescue an initially wrong majority or destroy an initially correct one.

    “The same discussion that can correct an initially wrong majority can also collapse an initially correct majority into a wrong consensus, without an external adversary, tool output, or new evidence.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The screen is a distinct secondary signal for prioritizing trace collection, not a better selector.

    “The screen is therefore a distinct secondary signal for prioritizing trace collection, not a better selector.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    The study's scope is narrow: homogeneous same-model, closed-book MCQ debate.

    “Scope is narrow: homogeneous same-model, closed-book MCQ debate.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The April 19 pre-registration locked an N = 18 model-row Spearman extension as the confirmatory analysis.

    “The April 19 pre-registration locked an N =18 model-row Spearman extension as the confirmatory analysis.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductnone given

    All MCQ answer extraction, collapse/correction labels, and onset labels are rule-based, with no LLM-judge fallback.

    “There is no LLM-judge fallback: all MCQ answer extraction, collapse/correction labels, and onset labels are rule-based.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The rule-based parser and alphabetic tie-breaking produce transition labels accurate enough for the headline collapse/correction counts.
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The screen is protocol-bound triage, not a capability-free property, per-question oracle, or controller.

    “The screen is protocol-bound triage, not a capability-free property, per-question oracle, or controller.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The ledger keeps comparison auditable when user-stated utility weights change which controller is selected.

    “The ledger keeps that comparison auditable when user-stated utility weights change which controller is selected.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Round 3 being larger than Round 2 is not a contradiction because onset is the first correct-to-wrong majority switch.

    “Round 3 being larger than Round 2 is not a contradiction: onset is the first correct-to-wrong majority switch”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The family-level analysis avoids treating correlated variants as independent.

    “This family-level analysis avoids treating correlated variants as independent.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    For the same model, the selection procedure is deterministic given the random probe seed.

    “for the same model, the procedure is deterministic given the random probe seed.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Interpretationown data

    These results bound, rather than settle, the comparison with compute-matched solo reasoning.

    “These results bound, rather than settle, the comparison with compute-matched solo reasoning.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The S/A bootstrap intervals support the selection-sensitivity statement: full-pool S/A estimates stay within the bootstrap ranges, so the high-FR restriction sharpens measurement without changing the qualitative pool-check conclusion.

    “These intervals support the selection-sensitivity statement in Section B.1: full-pool S/A estimates stay within the bootstrap ranges, so the high-FR restriction sharpens measurement without changing the qualitative pool-check conclusion.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The primary cohort does not isolate reasoning mode or effort, and one of its rows (DeepSeek V4 Flash) ran with provider reasoning enabled.

    “The primary cohort does not isolate reasoning mode or effort, and one of its rows (DeepSeek V4 Flash) ran with provider reasoning enabled.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    MMLU-Pro is primary; ARC, TruthfulQA, GPQA, and OpenRouter/Gemini rows do not establish benchmark transfer or cover heterogeneous judge-debater systems, tool use, retrieval, free-form generation, rubric-or judge-scored answers, or agent workflows that gather evidence over longer contexts rather than static choices.

    “MMLU-Pro is primary; ARC, TruthfulQA, GPQA, and OpenRouter/Gemini rows do not establish benchmark transfer or cover heterogeneous judge-debater systems, tool use, retrieval, free-form generation, rubric-or judge-scored answers, or agent workflows that gather evidence over longer contexts rather than static choices.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    When every initial and final majority is defined, corrections minus collapses divided by n equals final-minus-initial majority accuracy, so the ledger decomposes accuracy rather than replacing it.

    “When every initial and final majority is defined, corrections minus collapses divided by n equals final-minus-initial majority accuracy, so the ledger decomposes accuracy rather than replacing it.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Mixed-model panels and GSM8K rows added after review are feasibility checks, not transfer evidence.

    “Mixed-model panels and GSM8K rows added after review are feasibility checks, not transfer evidence”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Because the cohort contains related variants, the family-aggregated Spearman exact test is primary, with the N = 14 model-row statistic as sensitivity.

    “Because the cohort contains related variants, the familyaggregated Spearman exact test is primary, with the N =14 model-row statistic as sensitivity.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The family-aggregated exact test is a design-aware non-independence correction for the realized cohort.

    “We therefore report the family-aggregated exact test as a design-aware non-independence correction for the realized cohort”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    Each S/A bootstrap interval resamples question-level S/A estimates within a model; it is a within-row measurement interval, not a cross-model hypothesis test.

    “Each interval resamples question-level S/A estimates within a model; it is therefore a within-row measurement interval, not a cross-model hypothesis test.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Interpretationreasoning

    This is a realized-cohort correction to the inferential unit, not a claim that the originally targeted N = 18 model-row primary was completed unchanged.

    “This is a realized-cohort correction to the inferential unit, not a claim that the originally targeted N =18 model-row primary was completed unchanged.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The adjusted evidence supports triage and cost allocation, not a capability-adjusted predictor.

    “The adjusted evidence supports triage and cost allocation, not a capability-adjusted predictor.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The work uses only public benchmark questions; no human-subjects data, personally identifying information, or crowd-worker labels were collected, so IRB review was not required.

    “The work uses only public benchmark questions; no human-subjects data, personally identifying information, or crowd-worker labels were collected, so IRB review was not required.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Qwen3-32B is valid in the sealed pre-A6 vintage; its exclusion applies only to the later post-A6 cohort, where the archived lane used the wrong question pool.

    “Qwen3-32B is valid in this sealed vintage; its exclusion applies only to the later post-A6 cohort, where the archived lane used the wrong question pool.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductnone given

    No item is selected, dropped, or reweighted using its debate transition label.

    “Selected items may later be debated as part of the model-row audit, but no item is selected, dropped, or reweighted using its debate transition label.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The paper does not claim a strict N = 14 one-agent validation without recovering per-agent raw rows for sealed lanes.

    “so we do not claim a strict N =14 one-agent validation without recovering per-agent raw rows for sealed lanes.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductnone given

    The selection is computed entirely on probe behavior, never on debate outcomes, so it is outcome-blind with respect to the targets reported in Table 16.

    “The selection is computed entirely on probe behavior, never on debate outcomes (collapse, correction, or any downstream label), so it is outcome-blind with respect to the targets reported in Table 16.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The α_tot is the pre-specified rank measurement.

    “Total α tot pools the two and is the pre-specified rank measurement (Table 23).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Exact per-debate Round 1 and strict matched-τ DisagreementGate matrices remain gated, so those results are diagnostic rather than primary and provider reruns may drift.

    “Exact per-debate Round 1 and strict matched-τ DisagreementGate matrices remain gated, so those results are diagnostic rather than primary and provider reruns may drift.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    The high unlabeled rate is concentrated in long Qwen traces, so the audit does not replace gated full-trace validation; it bounds fallback sensitivity where strict parsing still yields labels.

    “The high unlabeled rate is concentrated in long Qwen traces, so this audit does not replace gated full-trace validation; it bounds fallback sensitivity where strict parsing still yields labels”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Held-out-family diagnostics test rank discipline, not calibrated forecasting.

    “Held-out-family diagnostics test rank discipline, not calibrated forecasting.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The LOMO probe-gated freeze classifier, threshold τ, and freeze mode are tuned on held-in models and then evaluated frozen on the held-out model.

    “the classifier, τ , and the freeze mode are tuned on held-in models and then evaluated frozen on the held-out model.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    Channel separability varies with both model family and instruction-tuning approach.

    “suggesting that channel separability varies with both model family and instruction-tuning approach.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Five robustness checks were pre-registered before the cohort-extension data-collection round.

    “We pre-registered five robustness checks (see §6 of the supplementary pre-registration document) before the cohort-extension data-collection round”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    The held-out-family predictive check supports a ranking interpretation but not a calibrated forecast.

    “The diagnostic supports a ranking interpretation but not a calibrated forecast: predictedvs.-observedSpearman is +0.821 (p=0.023),80% prediction intervals cover 5/7 families, 95% intervals cover 6/7, and the mean absolute rank error is 0.857.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    These results confirm the preregistered prediction and illustrate healthy debate dynamics in this low-S/A held-out setting.

    “These results confirm the preregistered prediction and illustrate healthy debate dynamics in this low-S/A held-out setting: rapid early convergence, productive revisions concentrated in early rounds, and collapses confined to rare late-round cascades.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    The family aggregation is not a proof of benchmark-general robustness, but it bounds the realized-family imbalance directly.

    “The family aggregation is therefore not a proof of benchmark-general robustness, but it bounds the realized-family imbalance directly.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    High revisability can produce measurable collapses and many productive corrections in the same scaffold, so mitigation must preserve corrections rather than freezing revision wholesale.

    “Unlike the small Mistral slice above, this run is net accuracy-positive; it therefore reinforces the control lesson rather than weakening it: high revisability can produce measurable collapses and many productive corrections in the same scaffold, so mitigation must preserve corrections rather than freezing revision wholesale.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The social-pressure flip-rate proxy is not treated as an independent social mechanism score; the narrower social-over-solo lift is the cleaner conformity contrast, and it is not predictive.

    “We therefore do not treat it as an independent social mechanism score; the narrower social-over-solo lift is the cleaner conformity contrast, and it is not predictive.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    A collapse-only metric would penalize debate without accounting for corrections, while final accuracy alone would hide which harmful transitions occurred.

    “This reinforces the signed-utility lesson rather than the model-rank lesson: a collapse-only metric would penalize debate without accounting for corrections, while final accuracy alone would hide which harmful transitions occurred.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationown data

    Revision can create both harmful collapses and useful corrections, so interventions should be judged on both transitions.

    “This is not a powered independent validation of monotone ranking, but it reinforces the control lesson: revision can create both harmful collapses and useful corrections, so interventions should be judged on both transitions.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    These runtime diagnostics are not replacements for the selection-time screen: they are measured only after initial or Round 1 answers have been sampled, cover only three families in the current raw-trace slice, and leave the trace-only partial ρ α|init-disagree=+0.196 underpowered (p=0.752).

    “These are important runtime diagnostics, but not replacements for the selection-time screen: they are measured only after initial or Round 1 answers have been sampled, cover only three families in the current raw-trace slice, and leave the trace-only partial ρ α|init-disagree =+0.196 underpowered (p=0.752).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The clustered delta-AUC recomputation is unavailable in the open tier.

    “The audit record therefore marks the clustered delta-AUC recomputation as unavailable in the open tier rather than recomputing a different estimand from newer trace files.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The EIV-propagated calculation should be read as a Jeffreys conditional-collapse propagation, not a fully calibrated forecasting model.

    “This keeps α tot fixed and therefore should be read as a Jeffreys conditional-collapse propagation, not a fully calibrated forecasting model.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The family-level partial is used as a capability-pressure sensitivity, not as a separate confirmatory test.

    “We therefore use this as a capability-pressure sensitivity, not as a separate confirmatory test.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationown data

    α tot is kept as the pragmatic triage measurement and α adv/α cor are treated as a descriptive decomposition rather than a standalone leaderboard.

    “We therefore keep α tot as the pragmatic triage measurement and treat α adv /α cor as a descriptive decomposition rather than a standalone leaderboard.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The small panel supports same-snapshot stability for one closed API lane; it is not an input to the family-level association and does not bound future provider-side snapshot changes.

    “This small panel supports same-snapshot stability for one closed API lane; it is not an input to the family-level association and does not bound future provider-side snapshot changes.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    The reduction is consistent with a general anchoring effect rather than selectively targeting one channel.

    “consistent with a general anchoring effect rather than selectively targeting one channel”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The oracle detect-revert row is an upper bound.

    “the oracle detect-revert row is an upper bound”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Convergence: all R=1.00, no divergences after non-centered reparameterization, ESS bulk ≥1,013.

    “Convergence: all R=1.00, no divergences after non-centered reparameterization, ESS bulk ≥1,013.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductnone given

    Strict quality control requires at most 2% missing answers, truncations, and unclosed reasoning blocks; rows marked † pass only an amended criterion that separates explicit abstentions from unparsed complete responses.

    “Strict quality control requires at most 2% missing answers, truncations, and unclosed reasoning blocks; rows marked † pass only an amended criterion that separates explicit abstentions from unparsed complete responses.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The G=7 association is the primary analysis, and GPQA rows are post-hoc stress checks.

    “The G=7 association is the primary analysis, and GPQA rows are post-hoc stress checks.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Adding the observed Round 1 majority state to pre-debate features can only improve an ideal Bayes score if Round 1 carries information about the final collapse label.

    “Adding the observed Round 1 majority state to pre-debate features can only improve an ideal Bayes score if Round 1 carries information about the final collapse label.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The two Phi-4-mini tests use disjoint question sets.

    “the two Phi-4-mini tests use disjoint question sets”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationown data

    Converting that information into a deployable policy that wins under frozen evaluation remains the open problem.

    “converting that information into a deployable policy that wins under frozen evaluation remains the open problem”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    In a toy three-agent MCQ panel conditioned on the initial majority being correct, if each agent independently abandons the correct answer with probability α_adv, then the probability of a wrong Round 1 majority is bounded above by 3α_adv^2 - 2α_adv^3, monotone in α_adv on [0,1].

    “In a toy three-agent MCQ panel, condition on the initial majority being correct and suppose each agent independently abandons the correct answer with probability α adv .A wrong Round 1 majority then requires at least two agents to abandon the correct answer and coordinate on a wrong option, so a loose upper envelope is Pr(M (1) ̸ =C) ≤ 3α 2 adv -2α 3 adv , monotone in α adv on [0, 1].”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: In the toy three-agent MCQ panel, each agent independently abandons the correct answer with probability α_adv.
  • Classified as probably not a substantial claim.

    Interpretationown data

    We treat R1 as a diagnostic substrate, not a standalone deployable gate.

    “We therefore treat R1 as a diagnostic substrate, not a standalone deployable gate.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The aggregate Table 37 ledger is sufficient to evaluate the probe-gated freeze policy without the row-level decision matrix.
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    ACC is selected on pooled development data, then evaluated without retuning.

    “ACC is selected on pooled development data, then evaluated without retuning”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The floor metric isolates the refusal-rate shift on the affected rungs without staking the claim on rank-monotone behavior across all 9 rungs.

    “the floor metric isolates the refusal-rate shift on the affected rungs without staking the claim on rank-monotone behavior across all 9 rungs”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The wrong-answer coincidence factor is not uniform because a plausible but wrong argument can steer multiple agents to the same distractor.

    “The wrong-answer coincidence factor is not uniform because a plausible but wrong argument can steer multiple agents to the same distractor.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The posterior predictive check one-sided p-value is 0.5426.

    “Posterior predictive check: observed collapse rate 7.98%, replicate mean matches; PPC one-sided p=0.5426.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    The deployable intervention evidence is proof-of-concept on the development set only: the frozen policy does not transfer reliably to fresh held-out questions.

    “We therefore interpret the deployable intervention evidence as proof-of-concept on the development set only: the frozen policy does not transfer reliably to fresh held-out questions.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductown data

    The residual model-level α slope is weakly identified, so this is a localization check rather than causal mediation evidence.

    “the residual model-level α slope is weakly identified, so this is a localization check rather than causal mediation evidence”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Conditioning on initial_correct and dropping models with n pos ≤1 reduces from 6,925 raw debates.

    “Conditioning on initial_correct and dropping models with n pos ≤1 (Gemini 3-flash and GPT-5.4-mini,both with a single collapse, render σ model unidentifiable) reduces from 6,925 raw debates.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    A local sweep of the post-challenge reply temperature kept only a derived seven-row summary, not paired raw outputs, and reached 6.4% unparsed replies, so we do not use it.

    “A local sweep of the post-challenge reply temperature kept only a derived seven-row summary, not paired raw outputs, and reached 6.4% unparsed replies, so we do not use it.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The missing temperature field in saved probe records does not invalidate the screen results.
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    A Fisher-z approximation for a partial correlation needs about 12 families at the submitted partial ρ = 0.767 and about 21 families at the extension partial ρ = 0.597.

    “a Fisher-z approximation for a partial correlation needs about 12 families at the submitted partial ρ = 0.767 and about 21 families at the extension partial ρ = 0.597”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductreasoning

    The follow-up removes the main weakness of the quasi-paired check in Section G.4: every arm branches from the same saved Round 0 answers, so arms differ only in the revision rounds.

    “It removes the main weakness of the quasi-paired check in Section G.4: every arm branches from the same saved Round 0 answers, so arms differ only in the revision rounds.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The DeepSeek V4 Flash debate row of the primary cohort ran with the provider's default reasoning enabled: 2,985 of its 3,000 logged debate calls returned reasoning tokens.

    “the DeepSeek V4 Flash debate row of the primary cohort ran with the provider's default reasoning enabled: 2,985 of its 3,000 logged debate calls returned reasoning tokens.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The DeepSeek V4 Flash row's use of default reasoning does not materially affect the primary cohort results.
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    These are single-realization feasibility pilots, not a panel-composition result.

    “These are single-realization feasibility pilots, not a panel-composition result.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Tool use, retrieval, asymmetric judge/debater roles, unrestricted generation, and rubric-or judge-scored answers are outside the evidence in this paper.

    “Tool use, retrieval, asymmetric judge/debater roles, unrestricted generation, and rubric-or judge-scored answers are outside the evidence in this paper.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All pre-reg SHAs verified in the result JSON match the frozen pre-registered values.

    “All pre-reg SHAs verified in the result JSON match the frozen pre-registered values.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All follow-up results are post hoc relative to the submitted study.

    “All follow-up results are post hoc relative to the submitted study.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationown data

    This is a quasi-paired single-checkpoint null, not evidence of equivalence.

    “This is a quasi-paired single-checkpoint null, not evidence of equivalence.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    A GPT-5.4-mini reasoning-effort sweep on 200 MMLU-Pro questions resampled initial answers at each effort level and reached 7.6% parse missingness at high effort (0% and 2.5% at low and medium), so it is descriptive rather than directional.

    “A GPT-5.4-mini reasoning-effort sweep on 200 MMLU-Pro questions resampled initial answers at each effort level and reached 7.6% parse missingness at high effort (0% and 2.5% at low and medium), so it is descriptive rather than directional.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Robustness of the screen to the reply temperature remains unresolved.

    “Robustness of the screen to the reply temperature remains unresolved.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The missing temperature field in saved probe records does not invalidate the screen results.
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Both inputs are noisy observed effects, so these are neither exact small-sample calculations nor achieved power.

    “Both inputs are noisy observed effects, so these are neither exact small-sample calculations nor achieved power.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Three runtime/infrastructure-only fixes were applied to the harness during execution ... none touch the scientific surface.

    “Three runtime/infrastructure-only fixes were applied to the harness during execution (enforce_eager=True, max_model_len=8192, per-model partial-write checkpoint) via commits c3b40f8 + 69f98d0; none touch the scientific surface”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductown data

    The GSM8K rows use deterministic numeric grading with no parse failures, but with 1-2 collapses per roughly 289 initially correct majorities they are too sparse to test collapse.

    “The GSM8K rows use deterministic numeric grading with no parse failures, but with 1-2 collapses per roughly 289 initially correct majorities they are too sparse to test collapse.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    No pre-registration deviation is owed for this entry.

    “No pre-registration deviation is owed for this entry.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    These tests are exploratory.

    “so these tests are exploratory”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Pools contain 2,000 MMLU-Pro or SuperGPQA questions, 1,000 MMLU-Pro questions for most reasoning-mode rows, or all 1,324 level-5 MATH problems.

    “Pools contain 2,000 MMLU-Pro or SuperGPQA questions, 1,000 MMLU-Pro questions for most reasoning-mode rows, or all 1,324 level-5 MATH problems.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    A Qwen3-8B control on 200 MMLU-Pro questions compared peer debate with private self-revision at the same call count and per-call token cap, scored at Round 3.

    “A Qwen3-8B control on 200 MMLU-Pro questions compared peer debate with private self-revision at the same call count and per-call token cap, scored at Round 3.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The screen is a distinct secondary signal for prioritizing trace collection, not a better selector than initial accuracy.

    “The screen is therefore a distinct secondary signal for prioritizing trace collection, not a better selector than initial accuracy.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    Family aggregation prevents six related rows from counting as six families, but it does not remove the within-Qwen range of conditional collapse (5.11% to 43.90%).

    “Family aggregation prevents six related rows from counting as six families, but it does not remove the within-Qwen range of conditional collapse (5.11% to 43.90%).”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    Initial probe answers were in fact sampled at the debate temperatures (0.5, 0.7, 1.0); only the post-challenge reply used temperature 0 where the provider supported it.

    “Initial probe answers were in fact sampled at the debate temperatures (0.5, 0.7, 1.0); only the post-challenge reply used temperature 0 where the provider supported it.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The missing temperature field in saved probe records does not invalidate the screen results.
  • May combine two claims.

    Interpretationown data

    The row remains nearly accuracy-neutral, and it stays a stress row rather than screen-transfer evidence.

    “The row remains nearly accuracy-neutral, and it stays a stress row rather than screen-transfer evidence.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    None of the appendix analyses enters the primary G=7 family analysis.

    “None of them enters the primary G=7 family analysis.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Model identity is also confounded with agent slot and slot temperature, and parse missingness through Round 3 is 1.46%-1.87%.

    “Model identity is also confounded with agent slot and slot temperature, and parse missingness through Round 3 is 1.46%-1.87%.”
    Quote in the paperNot a substantial claim
200 of 200
  • The paper relies on this without support.

    Multi-agent LLM debate is often evaluated by whether final answers improve.

    “Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve”
  • The paper relies on this without support.

    The 8-probe argument/social battery measures a model-level pre-debate compliance/revisability property relevant to debate collapse.

    “Each MCQ item is re-asked under an 8-probe argument/social battery, and α tot records the total answer-change rate.”
  • The paper relies on this without support.

    The MMLU-Pro benchmark and the realized N = 14 / G = 7 cohort are adequate to support the paper's family-level triage conclusions.

    “Our matched MMLU-Pro evaluation covers N =14 model rows across G=7 family clusters”
  • The paper relies on this without support.

    The rule-based parser and alphabetic tie-breaking produce transition labels accurate enough for the headline collapse/correction counts.

    “All labels are rule-based; no LLM judge or human coder is used.”
  • The paper relies on this without support.

    The freeze-style gates evaluated are representative of intervention policies for debate.

    “All three freeze-style gates prevent some collapses but lose more corrections”
  • The paper relies on this without support.

    The initial-majority accuracy is a valid capability proxy for comparing against the pre-debate screen.

    “Capability pressure, measured as 1-initial-majority accuracy, is a close free comparator”
  • The paper relies on this without support.

    The Round 1 trajectory features are available and meaningful for runtime diagnosis.

    “Round 1 trajectory features raise pooled out-of-fold area under the curve (AUC) from 0.669 to 0.768 over pre-debate features.”
  • The paper relies on this without support.

    The reasoning-mode follow-up with shared Round-0 answers is comparable to compute-matched solo reasoning.

    “These results bound, rather than settle, the comparison with compute-matched solo reasoning.”
  • The paper relies on this without support.

    Gated release limits casual reuse of the convince-wrong probe templates as an attack corpus.

    “To limit casual reuse as an attack corpus, the convince-wrong probe templates and tuned phrasings are released under a gated artifact host”
  • The authors state this as an assumption.

    The calibration value of a transparent, pre-registered instrument outweighs the marginal uplift to attackers.

    “Our position is that the calibration value of a transparent, pre-registered instrument outweighs the marginal uplift to attackers, and we welcome critique of that judgement during review.”
  • The paper relies on this without support.

    Initial accuracy is a strong capability proxy.

    “Initial accuracy is a strong capability proxy”
  • The paper relies on this without support.

    ARC-Challenge is an easier benchmark.

    “On ARC-Challenge [4] (an easier benchmark)”
  • The authors state this as an assumption.

    In the toy three-agent MCQ panel, each agent independently abandons the correct answer with probability α_adv.

    “suppose each agent independently abandons the correct answer with probability α adv”
  • The paper relies on this without support.

    The alphabetic tie rule is an appropriate way to resolve initial states without a unique plurality.

    “under the paper's alphabetic tie rule”
  • The paper relies on this without support.

    The aggregate Table 37 ledger is sufficient to evaluate the probe-gated freeze policy without the row-level decision matrix.

    “the 6,525-debate probe-gated LOMO ledger (Table37) is released as a frozen aggregate, and its row-level decision matrix is not part of the release”
  • The paper relies on this without support.

    The 1,255-trace replay cohort is representative of the 6,525-debate LOMO cohort for the DisagreementGate sanity check.

    “As a weaker same-trace sanity check on the available 1,255 replay cohort”
  • The paper relies on this without support.

    The missing temperature field in saved probe records does not invalidate the screen results.

    “The temperature field is missing from the saved probe records of several later rows.”
  • The paper relies on this without support.

    The DeepSeek V4 Flash row's use of default reasoning does not materially affect the primary cohort results.

    “the DeepSeek V4 Flash debate row of the primary cohort ran with the provider's default reasoning enabled”
18 of 18

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • The stated very-strong probe phrase is absent from its quoted template

    Minor

    The robustness discussion says the very-strong probe contains an authority phrase, but the paper's quoted very-strong counterargument string does not contain it. This leaves the description of the prompt being ablated internally inconsistent.

1 of 1

Cites

Flagged first, then by citations

Authors
Li, X., Liu, M., & Yuen, C.
Affiliations
Nanyang Technological University (SG)
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
OpenAlex
W7214734896
Cited by
0
References
32
Type
posted content
Open access
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
collapse, corrections, multiple-choice questions, MMLU-Pro

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review