Measuring Collapse and Correction in Homogeneous-Panel LLM Debate
Open accessAbstract
Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but movement is not necessarily improvement: the same discussion can rescue an initially wrong majority or destroy an initially correct one. Standard final-accuracy evaluations conflate these opposing mechanisms. We introduce an auditable protocol for homogeneous debate on multiple-choice questions (MCQs) that records each run as a transition ledger over collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates, the protocol identifies 253 collapses and a parallel correction ledger that changes how interventions should be judged. Replay experiments reveal the central tradeoff: a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections under equal weights, so collapse prevention alone can recommend the wrong policy. A compact pre-debate 8-probe screen is a triage signal: its unadjusted family-level association with conditional-collapse risk is high (G=7, Spearman rho=0.893, exact two-sided p=0.0123), but initial-majority accuracy is a close comparator (rho=0.821; family partial rho=0.767, p=0.0877), so we do not treat it as calibrated or capability-adjusted prediction. Round-level traces localize many collapses to the first debate round, where early disagreement can precede both harmful cascades and useful recovery. We release replayable schemas, coders, audits, cost cards, and zero-API rebuild scripts so future model-scaffold rows can be compared under the same denominators and signed utility ledger.
AI snapshot
Written from the paper’s full text.
TL;DR
This paper introduces an auditable transition-ledger protocol for homogeneous three-agent LLM debate on MCQs, separating final-accuracy changes into collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates it finds 253 collapses and shows that freeze-style interventions can prevent collapses while losing more corrections, so collapse prevention alone can select the wrong policy.
Key findings
- Across 6,925 MMLU-Pro debates, the protocol identifies 253 collapses; onset is concentrated in Round 1: 149/253 (58.9%), versus Round 2 47 (18.6%) and Round 3 57 (22.5%). Adding Round 1 trajectory features raises pooled out-of-fold AUC from 0.669 to 0.768, and full trajectory features reach 0.846.
- A pre-debate 8-probe screen has a high unadjusted family-level association with conditional-collapse risk: G=7 Spearman ρ=0.893, exact two-sided p=0.0123. Initial-majority accuracy is a close comparator (ρ=0.821), and the family partial after initial accuracy is ρ=0.767, p=0.0877, so the screen is treated as triage, not calibrated prediction.
- Signed replay shows freeze-style policies are net-negative under equal collapse/correction weights: a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections (net -79), and a Round 1 majority-change rule prevents 56 collapses but loses 69 corrections (net -13; break-even ratio r*=1.23).
- Cross-benchmark GPQA stress rows support transition accounting rather than screen transfer: Mistral Small 4 has 20/120 conditional collapses and 21/78 corrections (net +1), while DeepSeek V4 Flash has 10/111 collapses and 47/87 corrections (net +37).
Methods
- Homogeneous, closed-book, three-agent LLM debate on multiple-choice questions: agents answer independently in Round 0, then discuss and update for Rounds 1–3 while seeing all current answers; the final majority is the group response. Runs are coded into a 2×2 ledger of preserved, collapse, correction, and unrepaired transitions, with collapse onset and signed utility.
- The pre-debate probe is an 8-probe 4×2 battery crossing argument strength (weak, moderate, strong, very strong) with social pressure (solo counterargument vs. attributed to two other agents). It reports α_tot, α_adv, α_cor, and S/A; all labels are rule-based with no LLM judge or human coder.
- The primary MMLU-Pro analysis covers N=14 model rows across G=7 family clusters after attrition from a pre-registered N=18 target. Analyses include family-level exact Spearman tests, model-row and leave-one-family sensitivity checks, out-of-fold AUC for Round 1 features, a Bayesian multilevel logistic model, and held-out signed replay of interventions under declared weights.
Limitations
- The paper states that the primary association uses the realized matched cohort (N=14 rows, G=7 families), not the pre-registered N=18; it is small and uneven, exact family tests are primary, and held-out-family diagnostics test rank discipline rather than calibrated forecasting. Scope is narrow: homogeneous same-model, closed-book MCQ debate; MMLU-Pro is primary, and ARC, TruthfulQA, GPQA, and OpenRouter/Gemini rows do not establish benchmark transfer or cover heterogeneous judge-debater systems, tool use, retrieval, free-form generation, rubric- or judge-scored answers, or longer-context evidence-gathering workflows.
- The paper states that the screen is protocol-bound triage, not a capability-free property, per-question oracle, or controller; initial-majority accuracy is a close comparator, and after Round 0 sampling, disagreement is the stronger runtime diagnostic. Round 1 trajectories localize risk without causal mediation, and controllers need correction-preserving uncertainty, prespecified weights, and held-out signed replay.
- The paper states that the primary cohort does not isolate reasoning mode or effort, and a follow-up does not establish that debate outperforms a compute-matched solo reasoning baseline. The artifact is release-tiered: exact per-debate Round 1 and strict matched-τ DisagreementGate matrices remain gated, so those results are diagnostic rather than primary, and provider reruns may drift. The text also makes evident that Qwen contributes six of the 14 model rows and is a dependence cluster, and that parser/tie-rule choices affect some transition labels.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 2 cited sources checked · 0 supported · 0 flagged
- Findingown data
Capability pressure ranks Qwen3-4B above Llama-3.1-8B in risk because their initial-majority accuracies are 45.3% and 50.7%, respectively.
“Capability pressure ranks Qwen3-4B above Llama-3.1-8B in risk because their initial-majority accuracies are 45.3% and 50.7%, respectively.”
Quote in the paperSubstantial claim - Findingown data
Round-level traces localize many collapses to the first debate round.
“Round-level traces localize many collapses to the first debate round”
Quote in the paperNot a substantial claim - Findingown data
Initial-majority accuracy is a close comparator to the screen (ρ = 0.821; family partial ρ = 0.767, p = 0.0877).
“initial-majority accuracy is a close comparator (ρ = 0.821; family partial ρ = 0.767, p = 0.0877)”
- Also stated in: § 4.3 Sensitivity: Capability and Per-Question Boundaries · plain
“Capability pressure, measured as 1-initial-majority accuracy, is a close free comparator (ρ=+0.821at the family level)”
Quote in the paperSubstantial claim- Rests on an assumption: The Round 1 trajectory features are available and meaningful for runtime diagnosis.
- Also stated in: § 4.3 Sensitivity: Capability and Per-Question Boundaries · plain
- Findingown data
In debate traces, 149 of 253 tracked collapses originate in Round 1.
“In debate traces, 149/253 tracked collapses originate in Round 1”
Quote in the paperSubstantial claim - Findingown data
Llama-3.1-8B's conditional collapse rate is about four times higher than Sonnet 4.5's (8.46% vs. 2.15%).
“yet Llama's conditional collapse rate is about four times higher (8.46% vs.”
Quote in the paperSubstantial claim
- Findingown data
For the most susceptible models, more than one in ten initially correct majorities are lost.
“for the most susceptible models, more than one in ten initially correct majorities are lost.”
Quote in the paperSubstantial claim - Findingown data
Raw debate flip rate is weak on the matched sealed foil (ρ = +0.33, p = 0.391).
“raw debate flip rate is weak on the matched sealed foil (ρ=+0.33,p=0.391)”
Quote in the paperNot a substantial claim - Findingown data
The pre-debate 8-probe screen has a high unadjusted family-level association with conditional-collapse risk (G=7, Spearman ρ = 0.893, exact two-sided p = 0.0123).
“its unadjusted family-level association with conditional-collapse risk is high (G=7, Spearman ρ = 0.893, exact two-sided p = 0.0123)”
- Also stated in: § 4.2 Primary Result: A Pre-Debate Screen Gives a Family-Level Triage Signal · plain
“The unadjusted family-level association is large: ρ=+0.8929 over G=7 families, with exact twosided p=0.0123”
Quote in the paperSubstantial claim- Rests on an assumption: The 8-probe argument/social battery measures a model-level pre-debate compliance/revisability property relevant to debate collapse.
- Also stated in: § 4.2 Primary Result: A Pre-Debate Screen Gives a Family-Level Triage Signal · plain
- Findingown data
Sonnet 4.5 and Llama-3.1-8B have nearly identical debate flip rates (0.705 vs. 0.726).
“Sonnet 4.5 and Llama-3.1-8B have nearly identical debate flip rates (0.705 vs. 0.726)”
Quote in the paperSubstantial claim - Findingown data
Round 1 trajectory features raise pooled out-of-fold AUC from 0.669 to 0.768 over pre-debate features.
“Round 1 trajectory features raise pooled out-of-fold area under the curve (AUC) from 0.669 to 0.768 over pre-debate features.”
Quote in the paperSubstantial claim- Rests on an assumption: The reasoning-mode follow-up with shared Round-0 answers is comparable to compute-matched solo reasoning.
- Findingown data
The worst leave-one-family-out value is ρ = +0.741.
“the worst leave-one-family-out value is ρ=+0.741”
Quote in the paperNot a substantial claim - Findingown data
Full 1,000-question estimates remain inside the per-model bootstrap intervals.
“full 1,000-question estimates remain inside the per-model bootstrap intervals”
Quote in the paperSubstantial claim - Findingown data
The N=14 model-row Spearman is ρ = +0.8295.
“the N =14 model-row Spearman is ρ=+0.8295”
Quote in the paperSubstantial claim - Findingown data
Dropping each Qwen row in turn keeps the model-row correlation between 0.809 and 0.875.
“Dropping each Qwen row in turn keeps the model-row correlation between 0.809 and 0.875”
Quote in the paperNot a substantial claim - Findingown data
An errors-in-variables bootstrap gives median latent rank correlation +0.8022.
“an errors-in-variables bootstrap gives median latent rank correlation +0.8022”
Quote in the paperNot a substantial claim - Findingown data
The sealed N=9 vintage cross-check is positive (ρ = +0.78).
“The sealed N =9 vintage cross-check is positive (ρ=+0.78)”
Quote in the paperNot a substantial claim - Findingown data
Test-retest reliability of the probe measurement is 0.760.
“test-retest reliability is 0.760”
Quote in the paperSubstantial claim - Findingown data
The fixed Round 1 majority-change rule is near break-even if a user prespecifies a collapse as at least 1.23× a lost correction.
“the fixed Round 1 majority-change rule is near break-even if a user prespecifies a collapse as at least 1.23× a lost correction.”
Quote in the paperSubstantial claim - Findingown data
The 195-question shared-pool sensitivity keeps S/A positive in 13/15 models.
“the 195-question shared-pool sensitivity keeps S/A positive in 13/15 models”
Quote in the paperSubstantial claim - Findingown data
Dropping the Qwen family leaves six families with ρ = +0.943 (exact p = 0.0167).
“dropping the Qwen family leaves six families with ρ=+0.943(exact p=0.0167;Section G.1).”
Quote in the paperNot a substantial claim - Findingcitation
On 6,925 MMLU-Pro debates, 253 collapses were observed.
“On 6,925 MMLU-Pro debates, the protocol identifies 253 collapses and a parallel correction ledger that changes how interventions should be judged.”
- Also stated in: § 1 Introduction · plain
“Even in this setting, we observe 253 collapses across 6,925 MMLU-Pro debates [25]”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [25]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Also stated in: § 1 Introduction · plain
- Findingown data
Individual Qwen conditional-collapse rates span 5.11% to 43.90%.
“individual Qwen conditional-collapse rates span 5.11% to 43.90%”
Quote in the paperNot a substantial claim - Findingnone given
The high-FR item selection is outcome-blind.
“The high-FR item selection is outcome-blind”
Quote in the paperSubstantial claim - Findingown data
Inter-agent ICC is 0.630-0.633.
“inter-agent ICC is 0.630-0.633”
Quote in the paperSubstantial claim - Findingown data
Split-half reliability of the probe measurement is 0.873.
“split-half reliability is 0.873”
Quote in the paperSubstantial claim - Findingown data
The Llama and DeepSeek per-question α-collapse associations are near zero.
“The Llama and DeepSeek per-question α-collapse associations are near zero”
Quote in the paperSubstantial claim - Findingown data
Dropping the very-strong probe strengthens the headline correlation rather than removing it.
“(ii) authority-cue leakage in the very-strong probe (Section C.6 R2: dropping the probe strengthens the headline correlation rather than removing it, indicating the result is not driven by an authority shortcut)”
Quote in the paperSubstantial claim - Findingown data
A follow-up with shared initial answers finds collapses and corrections in every reasoning-mode row, including gpt-oss-20b at two effort levels.
“A follow-up with shared initial answers finds collapses and corrections in every reasoning-mode row, including gpt-oss-20b at two effort levels”
Quote in the paperNot a substantial claim - Findingown data
A strict one-agent alpha-lite replay is available only for the six post-A6 raw lanes, where it preserves the same three-family order (ρ = +0.866 over G = 3).
“A strict one-agent alpha-lite replay is available only for the six post-A6 raw lanes, where it preserves the same three-family order (ρ=+0.866over G=3)”
Quote in the paperSubstantial claim - Findingown data
Crediting ties at their expected value narrows the private-revision difference to between -1.62 and +0.38 points.
“crediting ties at their expected value narrows the difference to between -1.62 and +0.38 points.”
Quote in the paperNot a substantial claim - Findingown data
The follow-up finds private revision less accurate than peer debate when ties count as wrong.
“and finds private revision less accurate than peer debate when ties count as wrong”
Quote in the paperNot a substantial claim - Findingown data
The two vintages agree at ρ=+0.78 (N=9) vs ρ=+0.829 (N=14).
“The two vintages agree at ρ=+0.78 (N =9) vs. ρ=+0.829(N =14).”
Quote in the paperNot a substantial claim - Findingown data
A non-social-only control changes AUC by at most 0.005.
“a non-social-only control changes AUC by at most 0.005”
Quote in the paperNot a substantial claim - Findingown data
At the question level, the maximum probe flip rate remained highly correlated with the full 8-probe estimate (minimum r=0.961 without very_strong; minimum r=0.896 for non-social-only).
“At the question level, the maximum probe flip rate remained highly correlated with the full 8-probe estimate (minimum r = 0.961 without very_strong; minimum r = 0.896 for non-social-only).”
Quote in the paperSubstantial claim - Findingown data
The available N = 14 default-condition reduction preserves the family rank correlation (ρ = +0.893, exact p = 0.0123).
“the available N =14 default-condition reduction preserves the family rank correlation (ρ=+0.893,exact p 2 =0.0123).”
Quote in the paperSubstantial claim - Findingnone given
In checked-in API traces, the full alpha pass can cost more than the corresponding 200-question debate trace on long-output models.
“In the checked-in API traces, the full alpha pass can cost more than the corresponding 200-question debate trace on long-output models”
Quote in the paperNot a substantial claim - Findingown data
A prompt-only independence ablation fails to improve held-out accuracy.
“A prompt-only independence ablation also fails to improve held-out accuracy”
Quote in the paperNot a substantial claim - Findingown data
Across 12 repeated MCQ settings, private revision is 1.44-4.81 points less accurate than peer debate when ties count as wrong.
“Across 12 repeated MCQ settings, private revision (the debate prompt with only the agent's own previous response) is 1.44-4.81points less accurate than peer debate when ties count as wrong”
Quote in the paperSubstantial claim - Findingown data
On GPQA, Mistral Small 4 is nearly accuracy-neutral because collapses and corrections almost cancel.
“On GPQA, Mistral Small 4 is nearly accuracy-neutral because collapses and corrections almost cancel”
Quote in the paperSubstantial claim - Findingown data
Recomputing S/A on the full 1,000-question pool (no high-FR restriction) leaves all 16 S/A point estimates inside their per-model bootstrap 95% CI from Table 17.
“We also recomputed S/A on the full 1,000-question pool (no high-FR restriction): all 16 S/A point estimates fall inside their per-model bootstrap 95% CI from Table 17.”
Quote in the paperSubstantial claim - Findingown data
Google models fall between (mean d=0.31).
“while Google models fall between (mean d = 0.31).”
Quote in the paperNot a substantial claim - Findingown data
Wrong questions have much higher flip rates than correct ones (Haiku: 0.474 vs 0.167; GPT: 0.391 vs 0.144).
“Wrong questions have much higher flip rates than correct ones (Haiku: 0.474 vs. 0.167; GPT: 0.391 vs. 0.144)”
Quote in the paperSubstantial claim - Findingreasoning
The family-mean aggregation is unchanged at ρ=+0.893 with exact two-sided p=0.0123 after any single Qwen-row removal.
“The family-mean aggregation is unchanged at ρ=+0.893 with exact two-sided p=0.0123 after any single Qwen-row removal, because Qwen remains one family row.”
Quote in the paperNot a substantial claim - Findingown data
Rerunning the test on the 6-probe restricted set strengthens the Spearman correlation from ρ=+0.683 (p=0.042) on the 8-probe set to ρ=+0.800 (Fisher-z 95% CI [+0.290, +0.956], p=0.010).
“rerunning the test on the 6-probe restricted set (weak / moderate / strong × social/non-social) strengthens the Spearman correlation from ρ=+0.683 (p=0.042) on the 8-probe set to ρ=+0.800 (Fisher-z 95% CI [+0.290, +0.956], p=0.010)”
Quote in the paperSubstantial claim - Findingown data
Qwen models show the weakest channel manipulation effects (d=0.06-0.09).
“while Qwen models show the weakest (d = 0.06-0.09)”
Quote in the paperNot a substantial claim - Findingown data
Llama-3.1-8B shows the largest individual effect (d=2.39).
“Llama-3.1-8Bshows the largest individual effect (d = 2.39)”
Quote in the paperNot a substantial claim - Findingown data
Excluding four Mistral questions with no valid initial majority changes the Mistral net from +1 to -1.
“Four Mistral questions have no valid initial majority and are counted as initially wrong; excluding them changes the Mistral net from +1 to -1 (Section G.3).”
Quote in the paperNot a substantial claim - Findingreasoning
A Sharma/Perez-style social-pressure flip-rate proxy is highly collinear with α tot (ρ=+0.873 model-row and +0.893 family-level against C cond).
“A Sharma/Perez-style social-pressure flip-rate proxy, defined as mean flip rate on social-attributed counterarguments, is highly collinear with α tot (ρ=+0.873model-row and +0.893 family-level against C cond ) because it is a one-sided slice of the same flip-rate battery.”
Quote in the paperSubstantial claim - Findingown data
Anthropic models show weaker but selective effects (mean d=0.21).
“Anthropic models show weaker but selective effects (mean d = 0.21)”
Quote in the paperNot a substantial claim - Findingown data
The grand mean Cohen's d for AA-AS is 0.53.
“with grand mean Cohen's d=0.53.”
Quote in the paperNot a substantial claim - Findingown data
Replacing S/A orig with S/A log preserves cross-model rank order at Spearman ρ=+0.983 (N=9).
“Replacing S/A orig =s soc /(|s soc |+|s arg |+ε) with S/A log = log (s soc +ε)/(s arg +ε) preserves crossmodel rank order at Spearman ρ=+0.983(N =9).”
Quote in the paperNot a substantial claim - Findingnone given
Initial accuracy is anti-correlated with collapse.
“Initial accuracy is a strong capability proxy and is itself anti-correlated with collapse; its sign-flipped version, 1-init-acc, is our closest local Engels-style capability-pressure proxy.”
Quote in the paperNot a substantial claim- Rests on an assumption: Initial accuracy is a strong capability proxy.
- Findingown data
On the six trace-available rows, initial-disagreement rate, normalized initial-answer entropy, and at-risk initial-disagreement rate each have Spearman ρ=+0.928 against C cond (n=6, two-sided p=0.0077).
“On the six trace-available rows (DeepSeek-v4-flash, Gemma-4-31B, and four Qwen variants), initial answer diversity is itself highly predictive of conditional collapse: initial-disagreement rate, normalized initial-answer entropy, and at-risk initial-disagreement rate each have Spearman ρ=+0.928against C cond (n=6, two-sided p=0.0077).”
Quote in the paperNot a substantial claim - Findingown data
12/16 models survive Holm-Bonferroni at family-wise α=0.05 for the AA>AS ordering.
“12/16 survive Holm-Bonferroni at family-wise α=0.05”
Quote in the paperNot a substantial claim - Findingown data
Under binary scoring, private revision is less accurate than peer debate in all twelve settings, by 1.44-4.81 points.
“Under binary scoring, private revision is less accurate than peer debate in all twelve settings, by 1.44-4.81points (Table 43).”
Quote in the paperNot a substantial claim - Findingown data
Sonnet 4.6 and Opus 4.5 sign-flip to mildly negative (-0.19, -0.15) on the shared pool.
“Sonnet 4.6 and Opus 4.5 sign-flip to mildly negative (-0.19, -0.15).”
Quote in the paperNot a substantial claim - Findingown data
The canonical family mean Spearman correlation between α_tot and C_cond is +0.893 (two-sided exact p=0.0123) across 7 families.
“Canonical family mean | 7 | +0.893 | two-sided exact p=0.0123”
Quote in the paperSubstantial claim - Findingown data
S/A remains positive in 13/15 models on the shared pool.
“S/A remains positive in 13/15 models on the shared pool”
Quote in the paperNot a substantial claim - Findingown data
Across 6,925 debates and 253 collapses, 58.9% of collapses originate in Round 1.
“Across 6,925 debates and 253 collapses, 58.9% of collapses originate in Round 1.”
Quote in the paperSubstantial claim - Findingown data
Conditional correction rate 39/127 (30.71%) for initially-wrong majority in Qwen3-32B holdout.
“Conditional correction | 39/127 (30.71%) | initially-wrong majority”
Quote in the paperNot a substantial claim - Findingown data
ACC is the strongest deployable policy on the development split.
“ACC is the strongest deployable policy on the development split”
Quote in the paperSubstantial claim - Findingown data
All rows in freeze-style replay are net-negative at equal weights.
“All rows are net-negative at equal weights; r ⋆ is the break-even value ratio.”
Quote in the paperNot a substantial claim - Findingown data
The independence prompt reduces Haiku's flip rate from 0.535 to 0.305 (43% reduction, Cohen's d = 0.619, Wilcoxon p = 7.5 × 10 -5 ).
“this reduces Haiku's flip rate from 0.535 to 0.305 (43% reduction, Cohen's d = 0.619, Wilcoxon p = 7.5 × 10 -5 )”
Quote in the paperSubstantial claim - Findingown data
The reduction is uniform across argument strengths (CV = 0.16).
“The reduction is uniform across argument strengths (CV = 0.16)”
Quote in the paperNot a substantial claim - Findingown data
The independence prompt slightly worsens debate accuracy on both models (Haiku: 80% → 73%; GPT: 73% → 70%).
“The independence prompt slightly worsens debate accuracy on both models (Haiku: 80% → 73%; GPT: 73% → 70%), consistent with agents becoming too rigid to benefit from productive debate.”
Quote in the paperSubstantial claim - Findingown data
Conditional collapse rate 8/73 (10.96%) for initially-correct majority at risk in Qwen3-32B holdout.
“Conditional collapse | 8/73 (10.96%) | initially-correct majority at risk”
Quote in the paperNot a substantial claim - Findingown data
Initial to final majority accuracy increased from 36.5% to 52.0% (net positive) for Qwen3-32B holdout.
“Initial → final majority accuracy 36.5% → 52.0% | net positive”
Quote in the paperNot a substantial claim - Findingown data
Majority flip rate 37.0% (74/200 debates) in Qwen3-32B holdout.
“Majority flip rate | 37.0% | 74/200 debates”
Quote in the paperNot a substantial claim - Findingown data
Adding S/A changes AUC by at most 0.004.
“Initial disagreement dominates the probe-derived quantities, and adding S/A changes AUC by at most 0.004; this bounds the screen as model-level triage rather than a row-level oracle.”
Quote in the paperSubstantial claim - Findingown data
Strict leave-one-model-out learned R1 stumps are worse (-2.31pp, bootstrap CI [-3.21, -1.44]pp) in held-out replay.
“Strict leave-one-model-out learned R1 stumps are worse (-2.31pp, bootstrap CI [-3.21, -1.44]pp)”
Quote in the paperSubstantial claim- Rests on an assumption: The aggregate Table 37 ledger is sufficient to evaluate the probe-gated freeze policy without the row-level decision matrix.
- Findingown data
Question difficulty dominates model-level variance in the crossed-effects fit (σ_question=1.44 ≫ σ_model=0.71).
“Question difficulty dominates model-level variance in the crossed-effects fit (σ question =1.44 ≫ σ model =0.71;Table 32).”
Quote in the paperNot a substantial claim - Findingown data
The per-model-pool to shared-pool rank correlation across the 15 sa_causal models is ρ=0.62.
“The per-model-pool to shared-pool rank correlation across the 15 sa_causal models is ρ=0.62”
Quote in the paperNot a substantial claim - Findingown data
Initial disagreement dominates the probe-derived quantities.
“Initial disagreement dominates the probe-derived quantities, and adding S/A changes AUC by at most 0.004; this bounds the screen as model-level triage rather than a row-level oracle.”
Quote in the paperSubstantial claim - Findingnone given
The Tencent row used different endpoints for the screen and the debate.
“The Tencent row used different endpoints for the screen and the debate”
Quote in the paperSubstantial claim - Findingown data
At the Round-3 endpoint, 25 rows in 11 families give a family marginal ρ = 0.752 (exact two-sided p = 0.0101).
“At the Round-3 endpoint, 25 rows in 11 families give a family marginal ρ = 0.752 (exact two-sided p = 0.0101).”
Quote in the paperNot a substantial claim - Findingown data
Among the 91 pairs of the 14 submitted MMLU-Pro rows, the screen α tot and initial-majority accuracy order 26 pairs differently (28.6%).
“Among the 91 pairs of the 14 submitted MMLU-Pro rows, the screen α tot and initial-majority accuracy order 26 pairs differently (28.6%).”
Quote in the paperSubstantial claim - Findingown data
The capability-adjusted partial, ρ = 0.597 (two-sided t-approximation p = 0.0685), is not confirmed.
“The capability-adjusted partial, ρ = 0.597 (two-sided t-approximation p = 0.0685), is not confirmed.”
Quote in the paperNot a substantial claim - Findingown data
Removing all six Qwen rows leaves eight rows with ρ = 0.862.
“Removing all six Qwen rows leaves eight rows with ρ = 0.862”
Quote in the paperNot a substantial claim - Findingown data
Dropping the Qwen family leaves six families with ρ = 0.943 (exact p = 0.0167).
“dropping the Qwen family leaves six families with ρ = 0.943 (exact p = 0.0167)”
Quote in the paperNot a substantial claim - Interpretationown data
Collapse prevention alone can recommend the wrong policy.
“so collapse prevention alone can recommend the wrong policy.”
Quote in the paperSubstantial claim - Interpretationown data
Separability is a diagnostic check rather than a uniform mechanism claim.
“We therefore treat separability as a diagnostic check rather than a uniform mechanism claim.”
Quote in the paperSubstantial claim - Interpretationreasoning
Standard final-accuracy evaluations conflate the opposing mechanisms of correction and collapse.
“Standard final-accuracy evaluations conflate these opposing mechanisms.”
Quote in the paperSubstantial claim - Interpretationown data
The pre-debate screen is not a calibrated or capability-adjusted prediction.
“so we do not treat it as calibrated or capability-adjusted prediction.”
Quote in the paperSubstantial claim - Interpretationreasoning
The missing variable in debate evaluation is direction, not motion.
“The missing variable is direction, not motion.”
Quote in the paperSubstantial claim - Interpretationown data
These cross-benchmark rows support transition accounting rather than screen transfer.
“so these rows support transition accounting rather than screen transfer”
Quote in the paperSubstantial claim - Interpretationown data
A scaffold is safer only when harmful cascades fall without erasing recoveries.
“A scaffold is safer only when harmful cascades fall without erasing recoveries”
Quote in the paperSubstantial claim - Interpretationown data
The per-model high-FR pool is a power-tightening choice that does not redirect cross-model comparisons.
“We therefore treat the per-model high-FR pool as a power-tightening choice that does not redirect cross-model comparisons.”
Quote in the paperSubstantial claim - Interpretationown data
Round 1 trajectories localize risk without causal mediation.
“Round 1 trajectories localize risk without causal mediation”
Quote in the paperNot a substantial claim - Interpretationown data
These checks rule out the simplest authority-cue-only explanation.
“These checks do not prove perfect factorization, but they rule out the simplest authority-cue-only explanation.”
Quote in the paperSubstantial claim - Study design and conductnone given
The primary association uses the realized matched cohort (N = 14 rows, G = 7 families), not the pre-registered N = 18.
“The primary association uses the realized matched cohort (N =14 rows, G=7 families), not the pre-registered N =18.”
Quote in the paperSubstantial claim - Interpretationreasoning
The follow-up does not establish that debate outperforms a compute-matched solo reasoning baseline.
“It does not establish that debate outperforms a compute-matched solo reasoning baseline.”
Quote in the paperSubstantial claim - Study design and conductown data
An external safety auditor confirmed none of the eight convince-wrong templates includes harmful content beyond plausible-sounding incorrect arguments on benchmark questions.
“(iii) a content-safety review of the convince-wrong probe text by an external safety auditor, confirming none of the eight templates includes harmful content beyond plausible-sounding incorrect arguments on benchmark questions.”
Quote in the paperNot a substantial claim - Interpretationown data
The headline correlation is not driven by an authority shortcut.
“(ii) authority-cue leakage in the very-strong probe (Section C.6 R2: dropping the probe strengthens the headline correlation rather than removing it, indicating the result is not driven by an authority shortcut)”
Quote in the paperSubstantial claim - Interpretationown data
The main predictor ordering is robust to both prompt simplifications.
“Thus, while these controls do not prove that the prompt factors are perfectly orthogonal, they indicate that the main predictor ordering is robust to both prompt simplifications.”
Quote in the paperSubstantial claim - Interpretationown data
A Round 1-only gate leaves a substantial late-collapse tail for signed replay to score.
“Operationally, a Round 1-only gate leaves a substantial late-collapse tail for signed replay to score.”
Quote in the paperSubstantial claim - Interpretationown data
The α_adv channel is best read as a repeatable behavioral diagnostic of adversarial revisability, not as a new sycophancy construct by itself.
“the α adv channel is therefore best read as a repeatable behavioral diagnostic of adversarial revisability, not as a new sycophancy construct by itself.”
Quote in the paperSubstantial claim - Study design and conductown data
The prompt-injection robustness check of the probe template found no false-positive inflation at n=50 adversarial trials.
“(i) prompt-injection robustness of the probe template itself (whether an adversarial probe can bypass the eval scaffold and cause inflated α adv -no falsepositive inflation observed at n=50 adversarial trials)”
Quote in the paperNot a substantial claim - Interpretationown data
The null here is lack of collapse variation, not lack of measured revisability.
“The null here is therefore lack of collapse variation, not lack of measured revisability.”
Quote in the paperSubstantial claim - Interpretationown data
Revisability tracks genuine uncertainty rather than random compliance.
“consistent with the view that revisability tracks genuine uncertainty rather than random compliance.”
Quote in the paperSubstantial claim - Interpretationown data
The headline association survives explicit measurement-error correction.
“The headline association survives explicit measurement-error correction.”
Quote in the paperSubstantial claim - Interpretationown data
The decomposition signal is not driven by the very-strong probe's authority phrasing.
“The decomposition signal is therefore not driven by the very-strong probe's authority phrasing.”
Quote in the paperSubstantial claim - Interpretationown data
Because all sampled models fall in the low-S/A regime, these checks should be read as held-out sanity checks rather than strong evidence for a sharp S/A threshold.
“Because all sampled models fall in the low-S/A regime, these checks should be read as held-out sanity checks rather than strong evidence for a sharp S/A threshold.”
Quote in the paperNot a substantial claim - Interpretationown data
The within-model A>S claim is invariant to the normalization choice.
“The within-model A>S claim is invariant to the normalization choice.”
Quote in the paperNot a substantial claim - Interpretationown data
The rank result in Table 15 should be read as a marginal cross-model directional signal, not a robust covariate-adjusted partial effect.
“The rank result in Table 15 should therefore be read as a marginal cross-model directional signal, not a robust covariate-adjusted partial effect.”
Quote in the paperSubstantial claim - Interpretationown data
The total-α predictor explains a non-trivial fraction of out-of-sample rank variance under the strictest possible cross-validation at this N.
“The total-α predictor explains a non-trivial fraction of out-of-sample rank variance under the strictest possible cross-validation at this N .”
Quote in the paperSubstantial claim - Interpretationown data
Real LLM debate violates the toy calculation in important ways.
“Real LLM debate violates the toy calculation in the important ways.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Agents share prompts and question context, so their revisions are correlated rather than independent.
“Agents share prompts and question context, so their revisions are correlated rather than independent.”
Quote in the paperNot a substantial claim - Interpretationown data
The probe signal carries information that simple disagreement / confidence gating does not on collapse-prone models.
“the probe signal carries information that simple disagreement / confidence gating does not on collapse-prone models”
Quote in the paperSubstantial claim - Interpretationown data
The independence prompt worsening is consistent with agents becoming too rigid to benefit from productive debate.
“The independence prompt slightly worsens debate accuracy on both models (Haiku: 80% → 73%; GPT: 73% → 70%), consistent with agents becoming too rigid to benefit from productive debate.”
Quote in the paperNot a substantial claim - Interpretationown data
Round 1 trajectory features are strong runtime correlates.
“Round 1 trajectory features are strong runtime correlates”
Quote in the paperNot a substantial claim - Interpretationown data
Much of the binary loss comes from counting ties as wrong.
“Under expected tie scoring the differences lie between -1.62 and +0.38 points and seven intervals remain below zero, so much of the binary loss comes from counting ties as wrong.”
Quote in the paperSubstantial claim - Interpretationreasoning
The shared-pool numbers are consistent with the per-model-pool numbers used in Table 16.
“The shared-pool numbers are therefore consistent with the per-model-pool numbers used in Table 16”
Quote in the paperNot a substantial claim - Interpretationown data
This bounds the screen as model-level triage rather than a row-level oracle.
“Initial disagreement dominates the probe-derived quantities, and adding S/A changes AUC by at most 0.004; this bounds the screen as model-level triage rather than a row-level oracle.”
Quote in the paperSubstantial claim - Interpretationown data
Any intervention that ignores Round 2/Round 3 leaves four-out-of-ten cascades on the table.
“any intervention that ignores Round 2/Round 3 leaves four-out-of-ten cascades on the table.”
Quote in the paperSubstantial claim - Interpretationreasoning
The B-contribution headline is therefore F3, conditional on (a) ladder Fail per §3 and (b) F3 floor usable per §4 both holding (they do).
“the B-contribution headline is therefore F3, conditional on (a) ladder Fail per §3 and (b) F3 floor usable per §4 both holding (they do)”
Quote in the paperSubstantial claim - Interpretationown data
Not every model in the cohort exhibits graded refusal-rate shifts on the ladder, even when other models in the same family generation do.
“not every model in the cohort exhibits graded refusal-rate shifts on the ladder, even when other models in the same family generation do”
Quote in the paperSubstantial claim - Interpretationown data
DeepSeek's row is most useful as a signed-utility stress case rather than as a clean screen-transfer case.
“its row is most useful as a signed-utility stress case rather than as a clean screen-transfer case.”
Quote in the paperNot a substantial claim - Study design and conductown data
Thinking-enabled cells of the same design failed the prespecified quality gate (13.0% unclosed reasoning blocks in both Qwen3-8B arms and 5.7% in the Qwen3-32B peer arm, against a 5% limit) and are excluded.
“Thinking-enabled cells of the same design failed the prespecified quality gate (13.0%unclosed reasoning blocks in both Qwen3-8B arms and 5.7% in the Qwen3-32B peer arm, against a 5% limit) and are excluded.”
Quote in the paperSubstantial claim - Interpretationown data
We read this large one-family shift as a leverage warning, not as confirmation.
“We read this large one-family shift as a leverage warning, not as confirmation.”
Quote in the paperSubstantial claim
- May combine two claims. 157
- Classified as probably not a substantial claim. 82
- The quote only partly states this claim. 2
May combine two claims.
Findingown dataA post-hoc audit of all 91 model-row pairs found the screen reorders 26 pairs relative to initial accuracy, and observed conditional collapse follows the screen in 13 and initial accuracy in 12 (one tie).
“A post-hoc audit of all 91 model-row pairs bounds this use: the screen reorders 26 pairs relative to initial accuracy, and observed conditional collapse follows the screen in 13 and initial accuracy in 12 (one tie; Section G.1).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataLow-α_tot families occupy the low-collapse ranks, while high-α_tot open-model families carry most collapse risk; Meta is the visible upper-tail inversion.
“Low-α tot families occupy the low-collapse ranks, while high-α tot open-model families carry most collapse risk; Meta is the visible upper-tail inversion.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataUnder equal collapse/correction weights, a leave-one-model-out probe-gated freeze prevents 29 collapses but discards 108 corrections, for a net utility of -79.
“a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 corrections under equal weights”
- Also stated in: § 2.15%). · plain
“under equal collapse/correction weights, a leave-one-model-out probe-gated freeze prevents 29 collapses but discards 108 corrections, for a net utility of -79.”
Quote in the paperSubstantial claim- Also stated in: § 2.15%). · plain
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataRealized-cohort model-row partials remain positive after residualizing on initial accuracy (ρ = +0.658, p = 0.014) and initial accuracy plus raw revision (ρ = +0.667, p = 0.018).
“Realized-cohort model-row partials remain positive after residualizing on initial accuracy (ρ=+0.658,p=0.014) and initial accuracy plus raw revision (ρ=+0.667,p=0.018)”
Quote in the paperNot a substantial claimThe quote only partly states this claim.
Findingown dataSimple Round 1 replay gates show the same sign as the LOMO freeze.
“Simple Round 1 replay gates show the same sign”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataCapability pressure, measured as 1-initial-majority accuracy, is a close free comparator (ρ = +0.821 at the family level), leaving a modest marginal rank gain for α_tot of about Δρ = 0.07.
“Capability pressure, measured as 1-initial-majority accuracy, is a close free comparator (ρ=+0.821at the family level), leaving a modest marginal rank gain for α tot of about ∆ρ=0.07”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe family-level rank partial after initial-majority accuracy remains positive but is no longer confirmatory (ρ = +0.767, exact two-sided p = 0.0877).
“The family-level rank partial after initial-majority accuracy remains positive but is no longer confirmatory (ρ=+0.767,exact two-sided p=0.0877).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataProbe ablations support the intended argument/social channel separation, but the effect is uneven, with Qwen-3 rows near zero.
“Probe ablations support the intended argument/social channel separation, but the effect is uneven, with Qwen-3 rows near zero.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe probe reverses the capability-pressure order (α_tot = 0.774 vs. 0.607), matching the observed conditional-collapse ordering (8.46% vs. 5.11%).
“The probe reverses this order (α tot =0.774 vs. 0.607), matching the observed conditional-collapse ordering (8.46% vs. 5.11%).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe combined collapse predictor changed by at most 0.004 AUC when very_strong was removed and by at most 0.005 AUC when only non-social probes were used.
“The combined predictor used in the main text (mean probe flip rate + difficulty + initial disagreement) changed by at most 0.004 AUC when very_strong was removed and by at most 0.005 AUC when only non-social probes were used.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataRemoving the very_strong probe preserved the model-level S/A ordering exactly (Spearman ρ=1.00 across four models), with largest absolute shift in mean S/A only 0.024.
“Removing very_strong preserved the model-level S/A ordering exactly (Spearman ρ = 1.00 across the four models), with the largest absolute shift in mean S/A only 0.024.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataAlphabetic tie-breaking is used in 298/3,240 primary initial/final majority reductions (9.20%), with 272/1,620 rows having at least one primary initial/final tie.
“Alphabetic tie-breaking is used in 298/3,240 primary initial/final majority reductions (9.20%), with 272/1,620 rows having at least one primary initial/final tie.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataRecomputing S/A on a single shared 200-question pool leaves the model-level rank ordering of S/A unchanged (Spearman ρ=0.95 between per-model and shared-pool rankings on the four overlap models), with largest absolute shift in S/A 0.04.
“The model-level rank ordering of S/A is unchanged (Spearman ρ = 0.95 between the per-model and shared-pool rankings on the four overlap models), and the largest absolute shift in S/A is 0.04.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAnti-argument and anti-social system prompts elicit measurably different flip distributions in all 16/16 tested models, with grand mean Cohen's d = 0.53 and 12/16 paired tests surviving Holm-Bonferroni correction.
“Anti-argument and anti-social system prompts elicit measurably different flip distributions in all 16/16 tested models, with grand mean Cohen's d=0.53 and 12/16 paired tests surviving Holm-Bonferroni correction”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn eleven reasoning-mode rows, collapses and corrections both persist.
“In eleven reasoning-mode rows, collapses and corrections both persist”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn the Mistral row, four questions lack a valid initial majority; excluding them changes the signed net from +1 to -1.
“In the Mistral row, four questions lack a valid initial majority; excluding them changes the signed net from +1 to -1”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn the raw-log subset, removing the very_strong probe preserves the model-level S/A ordering exactly (Spearman ρ = 1.00 over four models) and changes matched collapse-prediction AUC by at most 0.004.
“In the raw-log subset, removing that very_strong probe preserves the model-level S/A ordering exactly (Spearman ρ=1.00 over four models) and changes matched collapse-prediction AUC by at most 0.004”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataPrivate revision has a negative point difference in all eleven reasoning-mode rows (intervals below zero in seven).
“private revision has a negative point difference in all eleven (intervals below zero in seven; Table 8).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataPrivate revision reduces wrong-answer collapses but leaves more ties.
“it reduces wrong-answer collapses but leaves more ties”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingnone givenInitial-majority accuracy is a close comparator; after Round 0 sampling, disagreement is the stronger runtime diagnostic.
“Initial-majority accuracy is a close comparator; after Round 0 sampling, disagreement is the stronger runtime diagnostic.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe strict parser labels both initial and final majorities for 966/1,620 checked-in trace rows; among those strict-labeled rows, four-cell transition membership changes in 51 rows (5.28%), with 20 collapse-label and 29 correction-label changes.
“The strict parser labels both initial and final majorities for 966/1,620 checked-in trace rows; among those strict-labeled rows, four-cell transition membership changes in 51 rows (5.28%), with 20 collapse-label and 29 correction-label changes.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAttrition and manifest-discipline exclusions reduced the matched post-A6 cohort to N = 14, and six of those rows are related Qwen variants.
“attrition and manifest-discipline exclusions reduced the matched post-A6 cohort to N =14, and six of those rows are related Qwen variants.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAnti-argument prompts elicit more revision than anti-social prompts, with the ordering holding in 16/16 tested models.
“Anti-argument prompts elicit more revision than anti-social prompts.The ordering holds in 16/16 tested models”
- Also stated in: § B.6 Full 16-Model Channel Manipulation Results · plain
“Anti-argument FR > Anti-social FR in 16/16 models”
Quote in the paperSubstantial claim- Also stated in: § B.6 Full 16-Model Channel Manipulation Results · plain
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe tie rule fires in 298/3,240 initial/final majority reductions (9.20%), and removing the last-valid-letter fallback changes 51/966 strict-labeled transition cells.
“in the open trace subset, it fires in 298/3,240 initial/final majority reductions (9.20%), and removing the last-valid-letter fallback changes 51/966 strict-labeled transition cells.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataDeepSeek V4 Flash gains despite nonzero collapse because corrections dominate.
“while DeepSeek V4 Flash gains despite nonzero collapse because corrections dominate”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataQwen3-4B (d = 0.092) and Qwen3-8B (d = 0.057) are near-zero separability cases.
“Qwen3-4B (d=0.092) and Qwen3-8B (d=0.057) are near-zero separability cases.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe current model-row Spearman is ρ = +0.8295, and coding new-lane None post-answers as flips gives ρ = +0.8119.
“the current model-row Spearman is ρ=+0.8295,and coding new-lane None post-answers as flips gives ρ=+0.8119.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAt the per-question level, after initial answers are observed, disagreement is the stronger runtime diagnostic, while probe FR and S/A do not act as row-level collapse detectors.
“At the per-question level, the screen boundary is sharper: after initial answers are observed, disagreement is the stronger runtime diagnostic, while probe FR and S/A do not act as row-level collapse detectors”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAll three freeze-style gates prevent some collapses but lose more corrections; even the closest Round 1 gate is 56 prevented versus 69 lost.
“All three freeze-style gates prevent some collapses but lose more corrections; even the closest Round 1 gate is 56 prevented versus 69 lost.”
Quote in the paperSubstantial claim- Rests on an assumption: The freeze-style gates evaluated are representative of intervention policies for debate.
May combine two claims.
Findingown dataα tot remains associated with C cond under simple rank residualizations: controlling capability-pressure proxy gives ρ=+0.658 (p=0.014); controlling raw debate-revision proxy alone gives ρ=+0.780 (p=0.0017); controlling capability pressure plus raw debate-revision proxy gives ρ=+0.667 (p=0.018).
“α tot remains associated with C cond under simple rank residualizations: controlling the capability-pressure proxy gives ρ=+0.658(p=0.014);controlling raw debate-revision proxy alone gives ρ=+0.780(p=0.0017);controlling capability pressure plus the raw debate-revision proxy gives ρ=+0.667(p=0.018).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataMistral Small 4 has 20/120 conditional collapses and 21/78 corrections, producing only +1 net debate under equal collapse/correction weights.
“Mistral Small 4 has 20/120 conditional collapses and 21/78 corrections, producing only +1 net debate under equal collapse/correction weights.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe observed Round 1 lift is ∆R1=+0.099 with debate-index percentile 95% CI [+0.080, +0.118]; the residual full-trajectory lift is ∆full=+0.077 with 95% CI [+0.064, +0.092].
“The observed Round 1 lift is ∆ R1 =+0.099 with debate-index percentile 95% CI [+0.080, +0.118]; the residual full-trajectory lift is ∆ full =+0.077with 95% CI [+0.064, +0.092].”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIf appended as a post-hoc N=15 model-row stress test, the Spearman association remains ρ=+0.822; averaging it into the Qwen family leaves the G=7 family association unchanged at +0.893, and treating it as a separate stress family gives G=8 ρ=+0.857.
“If appended as a post-hoc N =15 model-row stress test, the Spearman association remains ρ=+0.822;averaging it into the Qwen family leaves the G=7 family association unchanged at +0.893, and treating it as a separate stress family gives G=8 ρ=+0.857.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataPhi remains a high-collapse row (C cond=10.24%), while Anthropic/OpenAI/Google/DeepSeek occupy the lower-collapse ranks.
“Phi remains a high-collapse row (C cond =10.24%), while Anthropic/OpenAI/Google/DeepSeek occupy the lower-collapse ranks.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataMistral Small 4 has the highest holdout probe rate (α tot=0.3772), 7/140 R4 collapses (5.00%), and 13/60 R4 corrections (21.67%).
“Mistral Small 4 is the main new-family stress row: it has the highest holdout probe rate (α tot =0.3772), 7/140 R4 collapses (5.00%), and 13/60 R4 corrections (21.67%).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataDropping each Qwen variant in turn leaves the model-row Spearman in [+0.809, +0.875] over the remaining N=13 rows; worst cases are dropping Qwen3-4B or Qwen3-8B (ρ=+0.809).
“Dropping each Qwen variant in turn leaves the model-row Spearman in [+0.809, +0.875] over the remaining N =13 rows; the worst cases are dropping Qwen3-4B or Qwen3-8B (ρ=+0.809).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe cascade is dominantly a Round 1 phenomenon (58.9% onset) but not exclusively.
“The cascade is dominantly a Round 1 phenomenon (58.9% onset) but not exclusively”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataLeave-one-model-out predictive R2 on a rank-linear fit is R2 LOMO=0.26 for total α and 0.12 for α adv.
“(2) Leave-one-model-out predictive R 2 on a rank-linear fit: R 2 LOMO =0.26 for total α and 0.12 for α adv .”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe two 80% misses are Meta (observed below the interval) and Qwen (observed above it); Qwen is the only 95% miss, with the held-out fit underpredicting the upper tail.
“The two 80% misses are Meta (observed below the interval) and Qwen (observed above it); Qwen is the only 95% miss, with the held-out fit underpredicting the upper tail.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataPer-model α and α adv values drop uniformly by ∆α ∈ [-0.073, -0.022] but their rank order across models is preserved.
“the per-model α and α adv values drop uniformly by ∆α ∈ [-0.073, -0.022] but their rank order across models is preserved.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe clipped EIV-propagated predictive intervals cover 5/7 families at 80% and 7/7 at 95%; Meta and Qwen remain the two 80% misses, while Qwen is now included only at the upper edge of the 95% interval.
“The clipped EIV-propagated predictive intervals cover 5/7 families at 80% and 7/7 at 95%; Meta and Qwen remain the two 80% misses, while Qwen is now included only at the upper edge of the 95% interval.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataGPQA contributes the only collapse (1/31=3.2%) and all three corrections; TruthfulQA has 0/42 collapses and no corrections.
“GPQA contributes the only collapse (1/31=3.2%)and all three corrections; TruthfulQA has 0/42 collapses and no corrections.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataPooled out-of-fold AUC is 0.669 for the pre-debate baseline, 0.768 for Round 1 trajectory, and 0.846 for the full trajectory.
“Pooled out-of-fold AUC is 0.669 for the pre-debate baseline, 0.768 for Round 1 trajectory, and 0.846 for the full trajectory.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataLlama 3.3 70B and DeepSeek V4 Flash both have measurable collapses, but standard debate is net positive because corrections are larger (+7 and +37 debates, respectively).
“Llama 3.3 70B and DeepSeek V4 Flash add a useful contrast: both have measurable collapses, but standard debate is net positive because corrections are larger (+7 and +37 debates, respectively).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataα adv remains directionally positive after initial accuracy but is weaker and non-confirmatory (ρ=+0.52, raw p=0.148), while α cor points negative.
“It does not: α adv remains directionally positive after initial accuracy but is weaker and non-confirmatory (ρ=+0.52,raw p=0.148), while α cor points negative.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataWith model-constant predictors and N=9 models grouped into 6 families, the GEE coefficients on α adv and initial accuracy are individually statistically indistinguishable from zero (cluster-robust |z|≤0.5); the predictors are near-collinear (Spearman between α adv and initial accuracy: -0.58).
“With model-constant predictors and N =9 models grouped into 6 families, the GEE coefficients on α adv and initial accuracy are individually statistically indistinguishable from zero (cluster-robust |z|≤0.5); the predictors are near-collinear (Spearman between α adv and initial accuracy: -0.58).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe family-level partial Spearman remains positive, ρ α|init acc=+0.767, with exact two-sided p=0.0877.
“The family-level partial remains positive, ρ α|init acc =+0.767, but the small denominator is visible: enumerating all 7! assignments of family-level C cond while holding family α tot and initial accuracy fixed gives exact two-sided p=0.0877.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataExpanding the same Gemini lane to the full 198-question GPQA Diamond split preserves the boundary interpretation: collapse remains sparse (2/147=1.4%), while corrections remain nonzero (9/51=17.6%).
“Expanding the same Gemini lane to the full 198-question GPQA Diamond split preserves the boundary interpretation: collapse remains sparse (2/147=1.4%),while corrections remain nonzero (9/51=17.6%).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe permutation null for total α gives observed Spearman ρ=+0.78 with 2-sided permutation p<0.02; for α adv, ρ=+0.62 gives p≈0.09.
“(1) Permutation null (10,000 random shuffles of C cond ): the observed Spearman ρ=+0.78 for total α has 2-sided permutation p<0.02; for α adv , ρ=+0.62 gives p≈0.09.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataGPT-5.4-mini has an S/A ratio of +0.048, near zero, consistent with argument-dominated revision.
“GPT-5.4-mini was selected as a preregistered held-out sanity check because its S/A ratio (+0.048) is near zero, consistent with argument-dominated revision.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOn ARC-Challenge, both Haiku and GPT-4o-mini show dramatically lower flip rates (Haiku: 0.512→0.179, d=-1.13; GPT: 0.458→0.161, d=-1.07) and near-zero collapse (Haiku: 0.5%, GPT: 0.0%).
“On ARC-Challenge [4] (an easier benchmark), both Haiku and GPT-4o-mini show dramatically lower flip rates (Haiku: 0.512 → 0.179, d = -1.13;GPT: 0.458 → 0.161, d = -1.07)and near-zero collapse (Haiku: 0.5%, GPT: 0.0%).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [4]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Rests on an assumption: ARC-Challenge is an easier benchmark.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataRound 1 majority-change rate is even higher on this slice (ρ=+0.986).
“Round 1 majority-change rate is even higher on this slice (ρ=+0.986),as expected for a runtime trajectory score.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe six near-complete additional OpenRouter alpha-channel lanes all preserve AA>AS.
“The six near-complete additional lanes all preserve AA>AS.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataGemini 3-flash (S/A=-0.08) provides a second low-S/A 200-debate sanity row, with low collapse at 0.5% and a correction-to-collapse ratio of 2.0.
“Gemini 3-flash (S/A = -0.08)provides a second low-S/A 200-debate sanity row, with low collapse at 0.5% and a correction-to-collapse ratio of 2.0.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe realized N=14 rerun gives point Spearman +0.8295 and median latent rank correlation +0.8022, with 95% interval [+0.7099, +0.8725] and Pr(r>0)=1.000 over 20,000 bootstrap draws.
“The realized N =14 rerun gives point Spearman +0.8295 and median latent rank correlation +0.8022, with 95% interval [+0.7099, +0.8725] and Pr(r>0)=1.000over 20,000 bootstrap draws (seed 20260427).”
- Also stated in: § C.7 Specification Curve and Errors-in-Variables · plain
“The realized N =14 rerun gives point Spearman +0.8295 and median latent rank correlation +0.8022, 95% interval [+0.7099, +0.8725], Pr(r>0)=1.000.”
Quote in the paperSubstantial claim- Also stated in: § C.7 Specification Curve and Errors-in-Variables · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenClipping the displayed intervals to the admissible [0,100]% range leaves both coverage counts unchanged.
“Clipping the displayed intervals to the admissible [0, 100]% range leaves both coverage counts unchanged.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe held-out-family predictive check gives predicted-vs-observed Spearman +0.821 (p=0.023), 80% prediction intervals cover 5/7 families, 95% intervals cover 6/7, and mean absolute rank error 0.857.
“The diagnostic supports a ranking interpretation but not a calibrated forecast: predictedvs.-observedSpearman is +0.821 (p=0.023),80% prediction intervals cover 5/7 families, 95% intervals cover 6/7, and the mean absolute rank error is 0.857.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAcross all three day pairs, the 20 shared question-level mean-α values have Spearman 1.000, mean absolute delta 0.0000, max absolute delta 0.0000, and zero initial-answer majority changes.
“Across all three day pairs, the 20 shared question-level mean-α values have Spearman 1.000, mean absolute delta 0.0000, max absolute delta 0.0000, and zero initial-answer majority changes.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataLeave-one-family-out on the family-mean table has worst case ρ=+0.8286; dropping any of Anthropic, DeepSeek, Google, or OpenAI gives +0.829; dropping Meta gives +1.000; dropping Phi or Qwen gives +0.943.
“Leave-one-familyout on the family-mean table has worst case ρ=+0.8286:dropping any of Anthropic, DeepSeek, Google, or OpenAI gives +0.829 because those rows occupy the low-collapse rank cluster; dropping Meta gives +1.000; dropping Phi or Qwen gives +0.943.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOpenAI models show the strongest channel manipulation effects (mean d=0.82) with prominent channel substitution.
“OpenAI models show the strongest channel manipulation effects (mean d = 0.82) with prominent channel substitution.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataDropping both Meta and Qwen leaves the remaining five families strictly monotone, ρ=+1.000 (exact two-sided p=0.0167 over 5! permutations).
“Dropping both Meta (the visible upper-tail inversion) and Qwen (the multi-row upper-tail family) leaves the remaining five families strictly monotone, ρ=+1.000(exact two-sided p=0.0167 over 5! permutations).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataBoth mean and median family aggregation over the seven retained families give ρ=+0.8929; exact permutation over all 7! family-label assignments gives two-sided p=0.0123.
“Both mean and median aggregation over the seven retained families give ρ=+0.8929;exact permutation over all 7! family-label assignments gives two-sided p=0.0123.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataWithin every day of the three-day Gemini Flash-Lite panel there are zero initial-answer instabilities, mean/max within-question α range 0.0000/0.0000, and 4.38% probe post-answer parse-None rate.
“Within every day there are zero initial-answer instabilities, mean/max within-question α range 0.0000/0.0000,and 4.38% probe post-answer parse-None rate.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe high-accuracy Google/xAI lanes have very low observed R4 collapse (zero or one collapse among 145-167 initially-correct pluralities), while still producing nonzero corrections.
“The highaccuracy Google/xAI lanes have very low observed R4 collapse (zero or one collapse among 145-167 initially-correct pluralities), while still producing nonzero corrections.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAcross 100 debates in the Gemini 3.1 Flash-Lite 50+50 check, initial-majority accuracy is 73.0% and final-majority accuracy is 75.0%; there is only 1 collapse (1/73=1.4% conditional collapse) and 3 corrections.
“The Gemini 3.1 Flash-Lite 50+50 check is not a powered transfer test: across 100 debates, initial-majority accuracy is 73.0% and final-majority accuracy is 75.0%;there is only 1 collapse (1/73=1.4% conditional collapse) and 3 corrections.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataQwen3-32B has probe α tot=0.6767 over 1,800 alpha rows, matched to 200 debated questions; in those debates initial-majority accuracy improves from 36.5% to 52.0%, with 8/73 at-risk initial-correct majorities collapsing (10.96%) and 39/127 initially-wrong majorities corrected (30.71%).
“The paired slice is complete: probe α tot =0.6767 over 1,800 alpha rows (200 questions × 3 conditions × 3 agents), matched to the same 200 debated questions with no duplicate question rows.In those debates, initial-majority accuracy improves from 36.5% to 52.0%, with 8/73 at-risk initial-correct majorities collapsing (10.96%) and 39/127 initially-wrong majorities corrected (30.71%).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAll 27 specification-curve cells return ρ>0 on the N=9 cohort with median +0.62.
“All 27 cells return ρ>0 on the N =9 cohort with median +0.62”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAlternative taxonomies preserve the positive association: splitting Qwen into 3/3.5/3.6 generations gives ρ=+0.883 over 9 groups; merging Phi and Gemma into a small-dense group gives ρ=+0.929 over 7 groups; applying both gives ρ=+0.933 over 9 groups.
“Alternative taxonomies preserve the positive association: splitting Qwen into 3 / 3.5 / 3.6 generations gives ρ=+0.883 over 9 groups; merging Phi and Gemma into a small-dense group gives ρ=+0.929 over 7 groups; applying both choices gives ρ=+0.933 over 9 groups.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn GPT-4o-mini, where collapse is already rare (2.0%), all four candidates (Standard, Shielded, ACC, DRS) sit within ±1.5pp of each other and none robustly beats Standard.
“On GPT-4o-mini, where collapse is already rare (2.0%), all four candidates (Standard, Shielded, ACC, DRS) sit within ±1.5pp of each other and none robustly beats Standard.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOn Phi-4-mini, DisagreementGate is 55.7/2.6, DRS is 58.3/7.3, and ACC is 59.9/4.7.
“On Phi-4-mini, DisagreementGate is 55.7/2.6,DRS is 58.3/7.3, and ACC is 59.9/4.7.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataPrivate revision reduces wrong-answer collapses in every setting, with intervals below zero in eleven, but raises correct-to-tied transitions in all twelve, and corrections decrease in every setting.
“It reduces wrong-answer collapses in every setting, with intervals below zero in eleven, but raises correct-to-tied transitions in all twelve, and corrections decrease in every setting.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataRound 1 is not sufficient: 18.6% of collapses first appear in Round 2 and 22.5% in Round 3, while some debates later correct a Round 1 wrong majority.
“Most importantly for policy, Round 1 is not sufficient: 18.6% of collapses first appear in Round 2 and 22.5% in Round 3, while some debates later correct a Round 1 wrong majority.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataF3 floor of -0.02, McNemar discordant counts 0/1 (p=1.0), Spearman perm.p=0.78 for Qwen3.5-9B.
“F3 floor of -0.02, McNemar discordant counts 0/1 (p=1.0),Spearman perm.p=0.78.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAs a weaker same-trace sanity check on the available 1,255 replay cohort, DisagreementGate is also net-negative: 63 collapses prevented, 104 corrections lost, net -41 (-3.27pp).
“As a weaker same-trace sanity check on the available 1,255 replay cohort, DisagreementGate is also net-negative: 63 collapses prevented, 104 corrections lost, net -41 (-3.27pp).”
Quote in the paperNot a substantial claim- Rests on an assumption: The 1,255-trace replay cohort is representative of the 6,525-debate LOMO cohort for the DisagreementGate sanity check.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe observed collapse rate is 7.98%.
“Posterior predictive check: observed collapse rate 7.98%, replicate mean matches; PPC one-sided p=0.5426.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningQwen3.6-27B-FP8 clears the F3-floor gate despite its full-ladder Spearman being underpowered (perm.p=0.247).
“Notably Qwen3.6-27B-FP8clears the F3-floor gate despite its full-ladder Spearman being underpowered (perm.p=0.247)”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOn Haiku (200 development questions), DisagreementGate yields 73.0% accuracy / 7.0% collapse and DRS yields 74.5%/5.0%, both worse than ACC's 78.0%/1.5%.
“On Haiku (200 development questions), DisagreementGate yields 73.0% accuracy / 7.0% collapse and DRS yields 74.5%/5.0%,both worse than ACC's 78.0%/1.5%.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Findingown data3,145 debates across 4 OSS models with per-round trace data: Phi-4-mini, Qwen3-4B, Llama-3.1-8B, Qwen3-8B.
“3,145 debates across 4 OSS models with per-round trace data: Phi-4-mini, Qwen3-4B, Llama-3.1-8B,Qwen3-8B.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataUnder expected tie scoring the differences lie between -1.62 and +0.38 points and seven intervals remain below zero.
“Under expected tie scoring the differences lie between -1.62 and +0.38 points and seven intervals remain below zero, so much of the binary loss comes from counting ties as wrong.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe F3-floor magnitudes are statistically separable from zero on both usable models: Qwen3.5-4B at +20 pp (McNemar p=0.003) and Qwen3.6-27B-FP8 at +10 pp (McNemar p=0.031).
“The F3-floor magnitudes are statistically separable from zero on both usable models: Qwen3.5-4B at +20 pp (McNemar p=0.003) and Qwen3.6-27B-FP8 at +10 pp (McNemar p=0.031).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataCollapse onset remains early: R1 accounts for 40/75 collapses (53.3%), followed by R2:10, R3:11, and R4:14.
“Collapse onset remains early: R1 accounts for 40/75 collapses (53.3%), followed by R2: 10, R3: 11, and R4: 14.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataA prompt control on Qwen3-8B and Granite-4.1-8B (two runs per arm) loses 2.48 and 2.43 binary accuracy points relative to a contemporaneous peer arm, and both differences remain significant after Holm correction over four planned contrasts, but it does not reliably reduce total collapse.
“A prompt control on Qwen3-8B and Granite-4.1-8B(two runs per arm) replaces the debate framing with a neutral self-review prompt.It still loses 2.48 and 2.43 binary accuracy points relative to a contemporaneous peer arm, and both differences remain significant after Holm correction over four planned contrasts, but it does not reliably reduce total collapse.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataαtot = 0.6767 for Qwen3-32B holdout.
“αtot | 0.6767”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn the seven-source primary cohort (1,255 usable debates), always freezing prevents 94 collapses but loses 124 corrections (-2.39pp).
“On the seven-source primary cohort (1,255 usable debates), always freezing prevents 94 collapses but loses 124 corrections (-2.39pp).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataQwen3-32B local-vLLM holdout has 1,800 alpha rows over 200 questions with complete 3×3 design.
“Alpha rows / questions | 1,800 / 200 | complete 3×3 design”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOn 1,200 debates in the post-A6 replication, 662 have an initially correct majority, 75 collapse (11.3% conditional), and 144 correct an initially wrong majority; final accuracy is 60.9%.
“On 1,200 debates, 662 have an initially correct majority, 75 collapse (11.3% conditional), and 144 correct an initially wrong majority; final accuracy is 60.9%.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataQwen3.5-9B's p k trajectory across k=0,...,8 is {0.04, 0.06, 0.08, 0.02, 0.02, 0.04, 0.04, 0.06, 0.02}, flat at low magnitude with no graded response to ladder rung.
“Q3.5-9B's p k trajectory across k=0, . . ., 8 is {0.04, 0.06, 0.08, 0.02, 0.02, 0.04, 0.04, 0.06, 0.02}, flat at low magnitude with no graded response to ladder rung.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn a smaller reviewer-requested zero-retune transfer check on Qwen3-4B (50 development questions, 300 held-out test questions), ACC is also null (p = 0.743 for accuracy, p = 1.000 for collapse).
“In a smaller reviewer-requested zero-retune transfer check on Qwen3-4B (50 development questions, 300 held-out test questions), ACC is also null (p = 0.743 for accuracy, p = 1.000 for collapse).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataHarness-detected anti-monotonicity in 1/3 pilot models, statistically indistinguishable from null (Spearman perm.p=0.78, McNemar discordant n=1).
“harness-detected anti-monotonicity in 1/3 pilot models, statistically indistinguishable from null (Spearman perm.p=0.78,McNemar discordant n=1)”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataPooled with legacy/partial traces, R1 remains the largest localized onset bucket (40/77 collapses overall, with 15 R4 and one unlocalized).
“Pooled with legacy/partial traces, R1 remains the largest localized onset bucket (40/77 collapses overall, with 15 R4 and one unlocalized).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn the 2×2 debate experiment (N = 100 questions per cell), the independence prompt does not reduce collapse for either Haiku (McNemar p = 0.593) or GPT-4o-mini (p = 1.000).
“In the 2×2 debate experiment (N = 100 questions per cell), the independence prompt does not reduce collapse for either Haiku (McNemar p = 0.593) or GPT-4o-mini (p = 1.000).”
Quote in the paperSubstantial claimThe quote only partly states this claim.
Findingown dataIn the sealed pre-A6 N=9 rank-correlation cross-check, probe α_tot has Spearman ρ = +0.78 (95% CI [+0.25, +0.95], p=0.013) with conditional collapse.
“Probe αtot (8-probe total) ⋆ | +0.78 | [+0.25, +0.95] | 0.013 | n/a”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataOn the original 800-question test, Phi-4-mini showed a modest positive trend (+1.25pp accuracy, -1.12pp collapse relative to Standard), but neither paired McNemar test was significant (p = 0.485 for accuracy, p = 0.336 for collapse).
“On the original 800-question test, Phi-4-mini showed a modest positive trend (+1.25pp accuracy, -1.12pp collapse relative to Standard), but neither paired McNemar test was significant (p = 0.485 for accuracy, p = 0.336 for collapse).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe intercept is -4.254 with 95% interval [-5.46, -2.96].
“Intercept -4.254 [-5.46, -2.96].”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data251 collapses (7.98%); 1,509 unique questions.
“251 collapses (7.98%); 1,509 unique questions.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe intuitive R1-majority-changed rule gates 23.2% of debates and is still net-negative: 56 collapses prevented, 69 corrections lost, -1.04pp accuracy change.
“The intuitive R1-majority-changed rule gates 23.2% of debates and is still net-negative: 56 collapses prevented, 69 corrections lost, -1.04pp accuracy change.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe positive trend did not replicate: ACC reduces accuracy by 1.93pp and increases collapse by 0.40pp (p = 0.106 for accuracy, p = 0.656 for collapse).
“The positive trend did not replicate: ACC reduces accuracy by 1.93pp and increases collapse by 0.40pp (p = 0.106 for accuracy, p = 0.656 for collapse).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThey prevent only 2 collapses while losing 31 corrections in held-out replay.
“preventing only 2 collapses while losing 31 corrections in held-out replay”
Quote in the paperSubstantial claim- Rests on an assumption: The aggregate Table 37 ledger is sufficient to evaluate the probe-gated freeze policy without the row-level decision matrix.
May combine two claims.
Findingown dataOn Haiku, neither Shielded nor ACC beats Standard debate (p = 0.188 for accuracy, p = 1.000 for collapse).
“On Haiku, neither Shielded nor ACC beats Standard debate (p = 0.188 for accuracy, p = 1.000 for collapse).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataObserved collapse follows the screen in 13 pairs, initial accuracy in 12, and neither in one tie.
“Observed collapse follows the screen in 13 pairs, initial accuracy in 12, and neither in one tie.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe temperature field is missing from the saved probe records of several later rows.
“The temperature field is missing from the saved probe records of several later rows.”
Quote in the paperNot a substantial claim- Rests on an assumption: The missing temperature field in saved probe records does not invalidate the screen results.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe Gemini-3.1-Pro row has about 11% missing answers in both probe revisions and debate answers.
“The Gemini-3.1-Prorow has about 11% missing answers in both probe revisions and debate answers.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataExcluding the four leaves 20/120 collapses and 19/74 corrections, so the signed net changes from +1 to -1.
“Excluding the four leaves 20/120 collapses and 19/74 corrections, so the signed net changes from +1 to -1.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataDropping each Qwen row in turn keeps the model-row correlation between 0.809 and 0.875 and leaves the family estimate at ρ = 0.893.
“Dropping each Qwen row in turn keeps the model-row correlation between 0.809 and 0.875 and leaves the family estimate at ρ = 0.893.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataDropping the Tencent row gives ρ = 0.948 (exact p = 0.00013) and a partial ρ = 0.931 (p = 0.000265).
“dropping it gives ρ = 0.948 (exact p = 0.00013) and a partial ρ = 0.931 (p = 0.000265)”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThree mixed-model panels answer the same 200 MMLU-Pro questions and produce both transition directions.
“Three mixed-model panels answer the same 200 MMLU-Pro questions and produce both transition directions.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn 20 to 28 of 200 initial states no answer has a unique plurality, so the results depend on the aggregation rule: treating such states as undecided and wrong changes the three nets from +14, +6 and 0 to +22, +8 and +7.
“In 20 to 28 of 200 initial states no answer has a unique plurality, so the results depend on the aggregation rule: treating such states as undecided and wrong changes the three nets from +14, +6 and 0 to +22, +8 and +7.”
Quote in the paperNot a substantial claim- Rests on an assumption: The alphabetic tie rule is an appropriate way to resolve initial states without a unique plurality.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe two arms sampled their initial answers independently, so only the 162 questions whose initial-majority labels match are paired; on them, peer-only and self-only correct outcomes are 3 and 3 (exact McNemar p = 1.0).
“The two arms sampled their initial answers independently, so only the 162 questions whose initial-majority labels match are paired; on them, peer-only and self-only correct outcomes are 3 and 3 (exact McNemar p = 1.0).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataA GLM-4.6 row with provider reasoning disabled shows hidden turnover: accuracy moves from 76.0% to 75.5% while the ledger records 6/152 collapses and 5/48 corrections.
“A GLM-4.6 row with provider reasoning disabled shows hidden turnover: accuracy moves from 76.0% to 75.5% while the ledger records 6/152 collapses and 5/48 corrections.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataTen unadjusted tests have p < 0.05 (screen 4, initial accuracy 6); after Holm correction over the 26 tests, eight remain (screen 2, initial accuracy 6), mostly within the Qwen family.
“Ten unadjusted tests have p < 0.05 (screen 4, initial accuracy 6); after Holm correction over the 26 tests, eight remain (screen 2, initial accuracy 6), mostly within the Qwen family.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn the Mistral Small 4 row of Table 30, four of the 198 questions have no valid initial majority and are counted as initially wrong; two of them end correct.
“In the Mistral Small 4 row of Table 30, four of the 198 questions have no valid initial majority and are counted as initially wrong; two of them end correct.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe motivating Llama-3.1-8B/Qwen3-4B reordering has unadjusted p = 0.0033 but does not survive correction.
“The motivating Llama-3.1-8B/Qwen3-4Breordering has unadjusted p = 0.0033 but does not survive correction.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataEarly disagreement can precede both harmful cascades and useful recovery.
“where early disagreement can precede both harmful cascades and useful recovery.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataIn multi-agent LLM debate, the same discussion can rescue an initially wrong majority or destroy an initially correct one.
“The same discussion that can correct an initially wrong majority can also collapse an initially correct majority into a wrong consensus, without an external adversary, tool output, or new evidence.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe screen is a distinct secondary signal for prioritizing trace collection, not a better selector.
“The screen is therefore a distinct secondary signal for prioritizing trace collection, not a better selector.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenThe study's scope is narrow: homogeneous same-model, closed-book MCQ debate.
“Scope is narrow: homogeneous same-model, closed-book MCQ debate.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe April 19 pre-registration locked an N = 18 model-row Spearman extension as the confirmatory analysis.
“The April 19 pre-registration locked an N =18 model-row Spearman extension as the confirmatory analysis.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductnone givenAll MCQ answer extraction, collapse/correction labels, and onset labels are rule-based, with no LLM-judge fallback.
“There is no LLM-judge fallback: all MCQ answer extraction, collapse/correction labels, and onset labels are rule-based.”
Quote in the paperSubstantial claim- Rests on an assumption: The rule-based parser and alphabetic tie-breaking produce transition labels accurate enough for the headline collapse/correction counts.
Classified as probably not a substantial claim. May combine two claims.
InterpretationreasoningThe screen is protocol-bound triage, not a capability-free property, per-question oracle, or controller.
“The screen is protocol-bound triage, not a capability-free property, per-question oracle, or controller.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningThe ledger keeps comparison auditable when user-stated utility weights change which controller is selected.
“The ledger keeps that comparison auditable when user-stated utility weights change which controller is selected.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningRound 3 being larger than Round 2 is not a contradiction because onset is the first correct-to-wrong majority switch.
“Round 3 being larger than Round 2 is not a contradiction: onset is the first correct-to-wrong majority switch”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductreasoningThe family-level analysis avoids treating correlated variants as independent.
“This family-level analysis avoids treating correlated variants as independent.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenFor the same model, the selection procedure is deterministic given the random probe seed.
“for the same model, the procedure is deterministic given the random probe seed.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationown dataThese results bound, rather than settle, the comparison with compute-matched solo reasoning.
“These results bound, rather than settle, the comparison with compute-matched solo reasoning.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe S/A bootstrap intervals support the selection-sensitivity statement: full-pool S/A estimates stay within the bootstrap ranges, so the high-FR restriction sharpens measurement without changing the qualitative pool-check conclusion.
“These intervals support the selection-sensitivity statement in Section B.1: full-pool S/A estimates stay within the bootstrap ranges, so the high-FR restriction sharpens measurement without changing the qualitative pool-check conclusion.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe primary cohort does not isolate reasoning mode or effort, and one of its rows (DeepSeek V4 Flash) ran with provider reasoning enabled.
“The primary cohort does not isolate reasoning mode or effort, and one of its rows (DeepSeek V4 Flash) ran with provider reasoning enabled.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningMMLU-Pro is primary; ARC, TruthfulQA, GPQA, and OpenRouter/Gemini rows do not establish benchmark transfer or cover heterogeneous judge-debater systems, tool use, retrieval, free-form generation, rubric-or judge-scored answers, or agent workflows that gather evidence over longer contexts rather than static choices.
“MMLU-Pro is primary; ARC, TruthfulQA, GPQA, and OpenRouter/Gemini rows do not establish benchmark transfer or cover heterogeneous judge-debater systems, tool use, retrieval, free-form generation, rubric-or judge-scored answers, or agent workflows that gather evidence over longer contexts rather than static choices.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningWhen every initial and final majority is defined, corrections minus collapses divided by n equals final-minus-initial majority accuracy, so the ledger decomposes accuracy rather than replacing it.
“When every initial and final majority is defined, corrections minus collapses divided by n equals final-minus-initial majority accuracy, so the ledger decomposes accuracy rather than replacing it.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenMixed-model panels and GSM8K rows added after review are feasibility checks, not transfer evidence.
“Mixed-model panels and GSM8K rows added after review are feasibility checks, not transfer evidence”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningBecause the cohort contains related variants, the family-aggregated Spearman exact test is primary, with the N = 14 model-row statistic as sensitivity.
“Because the cohort contains related variants, the familyaggregated Spearman exact test is primary, with the N =14 model-row statistic as sensitivity.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductreasoningThe family-aggregated exact test is a design-aware non-independence correction for the realized cohort.
“We therefore report the family-aggregated exact test as a design-aware non-independence correction for the realized cohort”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningEach S/A bootstrap interval resamples question-level S/A estimates within a model; it is a within-row measurement interval, not a cross-model hypothesis test.
“Each interval resamples question-level S/A estimates within a model; it is therefore a within-row measurement interval, not a cross-model hypothesis test.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningThis is a realized-cohort correction to the inferential unit, not a claim that the originally targeted N = 18 model-row primary was completed unchanged.
“This is a realized-cohort correction to the inferential unit, not a claim that the originally targeted N =18 model-row primary was completed unchanged.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe adjusted evidence supports triage and cost allocation, not a capability-adjusted predictor.
“The adjusted evidence supports triage and cost allocation, not a capability-adjusted predictor.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningThe work uses only public benchmark questions; no human-subjects data, personally identifying information, or crowd-worker labels were collected, so IRB review was not required.
“The work uses only public benchmark questions; no human-subjects data, personally identifying information, or crowd-worker labels were collected, so IRB review was not required.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenQwen3-32B is valid in the sealed pre-A6 vintage; its exclusion applies only to the later post-A6 cohort, where the archived lane used the wrong question pool.
“Qwen3-32B is valid in this sealed vintage; its exclusion applies only to the later post-A6 cohort, where the archived lane used the wrong question pool.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenNo item is selected, dropped, or reweighted using its debate transition label.
“Selected items may later be debated as part of the model-row audit, but no item is selected, dropped, or reweighted using its debate transition label.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningThe paper does not claim a strict N = 14 one-agent validation without recovering per-agent raw rows for sealed lanes.
“so we do not claim a strict N =14 one-agent validation without recovering per-agent raw rows for sealed lanes.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductnone givenThe selection is computed entirely on probe behavior, never on debate outcomes, so it is outcome-blind with respect to the targets reported in Table 16.
“The selection is computed entirely on probe behavior, never on debate outcomes (collapse, correction, or any downstream label), so it is outcome-blind with respect to the targets reported in Table 16.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductreasoningThe α_tot is the pre-specified rank measurement.
“Total α tot pools the two and is the pre-specified rank measurement (Table 23).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenExact per-debate Round 1 and strict matched-τ DisagreementGate matrices remain gated, so those results are diagnostic rather than primary and provider reruns may drift.
“Exact per-debate Round 1 and strict matched-τ DisagreementGate matrices remain gated, so those results are diagnostic rather than primary and provider reruns may drift.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe high unlabeled rate is concentrated in long Qwen traces, so the audit does not replace gated full-trace validation; it bounds fallback sensitivity where strict parsing still yields labels.
“The high unlabeled rate is concentrated in long Qwen traces, so this audit does not replace gated full-trace validation; it bounds fallback sensitivity where strict parsing still yields labels”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenHeld-out-family diagnostics test rank discipline, not calibrated forecasting.
“Held-out-family diagnostics test rank discipline, not calibrated forecasting.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe LOMO probe-gated freeze classifier, threshold τ, and freeze mode are tuned on held-in models and then evaluated frozen on the held-out model.
“the classifier, τ , and the freeze mode are tuned on held-in models and then evaluated frozen on the held-out model.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataChannel separability varies with both model family and instruction-tuning approach.
“suggesting that channel separability varies with both model family and instruction-tuning approach.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataFive robustness checks were pre-registered before the cohort-extension data-collection round.
“We pre-registered five robustness checks (see §6 of the supplementary pre-registration document) before the cohort-extension data-collection round”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe held-out-family predictive check supports a ranking interpretation but not a calibrated forecast.
“The diagnostic supports a ranking interpretation but not a calibrated forecast: predictedvs.-observedSpearman is +0.821 (p=0.023),80% prediction intervals cover 5/7 families, 95% intervals cover 6/7, and the mean absolute rank error is 0.857.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThese results confirm the preregistered prediction and illustrate healthy debate dynamics in this low-S/A held-out setting.
“These results confirm the preregistered prediction and illustrate healthy debate dynamics in this low-S/A held-out setting: rapid early convergence, productive revisions concentrated in early rounds, and collapses confined to rare late-round cascades.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThe family aggregation is not a proof of benchmark-general robustness, but it bounds the realized-family imbalance directly.
“The family aggregation is therefore not a proof of benchmark-general robustness, but it bounds the realized-family imbalance directly.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataHigh revisability can produce measurable collapses and many productive corrections in the same scaffold, so mitigation must preserve corrections rather than freezing revision wholesale.
“Unlike the small Mistral slice above, this run is net accuracy-positive; it therefore reinforces the control lesson rather than weakening it: high revisability can produce measurable collapses and many productive corrections in the same scaffold, so mitigation must preserve corrections rather than freezing revision wholesale.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe social-pressure flip-rate proxy is not treated as an independent social mechanism score; the narrower social-over-solo lift is the cleaner conformity contrast, and it is not predictive.
“We therefore do not treat it as an independent social mechanism score; the narrower social-over-solo lift is the cleaner conformity contrast, and it is not predictive.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataA collapse-only metric would penalize debate without accounting for corrections, while final accuracy alone would hide which harmful transitions occurred.
“This reinforces the signed-utility lesson rather than the model-rank lesson: a collapse-only metric would penalize debate without accounting for corrections, while final accuracy alone would hide which harmful transitions occurred.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationown dataRevision can create both harmful collapses and useful corrections, so interventions should be judged on both transitions.
“This is not a powered independent validation of monotone ranking, but it reinforces the control lesson: revision can create both harmful collapses and useful corrections, so interventions should be judged on both transitions.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThese runtime diagnostics are not replacements for the selection-time screen: they are measured only after initial or Round 1 answers have been sampled, cover only three families in the current raw-trace slice, and leave the trace-only partial ρ α|init-disagree=+0.196 underpowered (p=0.752).
“These are important runtime diagnostics, but not replacements for the selection-time screen: they are measured only after initial or Round 1 answers have been sampled, cover only three families in the current raw-trace slice, and leave the trace-only partial ρ α|init-disagree =+0.196 underpowered (p=0.752).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningThe clustered delta-AUC recomputation is unavailable in the open tier.
“The audit record therefore marks the clustered delta-AUC recomputation as unavailable in the open tier rather than recomputing a different estimand from newer trace files.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe EIV-propagated calculation should be read as a Jeffreys conditional-collapse propagation, not a fully calibrated forecasting model.
“This keeps α tot fixed and therefore should be read as a Jeffreys conditional-collapse propagation, not a fully calibrated forecasting model.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThe family-level partial is used as a capability-pressure sensitivity, not as a separate confirmatory test.
“We therefore use this as a capability-pressure sensitivity, not as a separate confirmatory test.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationown dataα tot is kept as the pragmatic triage measurement and α adv/α cor are treated as a descriptive decomposition rather than a standalone leaderboard.
“We therefore keep α tot as the pragmatic triage measurement and treat α adv /α cor as a descriptive decomposition rather than a standalone leaderboard.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe small panel supports same-snapshot stability for one closed API lane; it is not an input to the family-level association and does not bound future provider-side snapshot changes.
“This small panel supports same-snapshot stability for one closed API lane; it is not an input to the family-level association and does not bound future provider-side snapshot changes.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe reduction is consistent with a general anchoring effect rather than selectively targeting one channel.
“consistent with a general anchoring effect rather than selectively targeting one channel”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenThe oracle detect-revert row is an upper bound.
“the oracle detect-revert row is an upper bound”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataConvergence: all R=1.00, no divergences after non-centered reparameterization, ESS bulk ≥1,013.
“Convergence: all R=1.00, no divergences after non-centered reparameterization, ESS bulk ≥1,013.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Study design and conductnone givenStrict quality control requires at most 2% missing answers, truncations, and unclosed reasoning blocks; rows marked † pass only an amended criterion that separates explicit abstentions from unparsed complete responses.
“Strict quality control requires at most 2% missing answers, truncations, and unclosed reasoning blocks; rows marked † pass only an amended criterion that separates explicit abstentions from unparsed complete responses.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe G=7 association is the primary analysis, and GPQA rows are post-hoc stress checks.
“The G=7 association is the primary analysis, and GPQA rows are post-hoc stress checks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenAdding the observed Round 1 majority state to pre-debate features can only improve an ideal Bayes score if Round 1 carries information about the final collapse label.
“Adding the observed Round 1 majority state to pre-debate features can only improve an ideal Bayes score if Round 1 carries information about the final collapse label.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe two Phi-4-mini tests use disjoint question sets.
“the two Phi-4-mini tests use disjoint question sets”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationown dataConverting that information into a deployable policy that wins under frozen evaluation remains the open problem.
“converting that information into a deployable policy that wins under frozen evaluation remains the open problem”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningIn a toy three-agent MCQ panel conditioned on the initial majority being correct, if each agent independently abandons the correct answer with probability α_adv, then the probability of a wrong Round 1 majority is bounded above by 3α_adv^2 - 2α_adv^3, monotone in α_adv on [0,1].
“In a toy three-agent MCQ panel, condition on the initial majority being correct and suppose each agent independently abandons the correct answer with probability α adv .A wrong Round 1 majority then requires at least two agents to abandon the correct answer and coordinate on a wrong option, so a loose upper envelope is Pr(M (1) ̸ =C) ≤ 3α 2 adv -2α 3 adv , monotone in α adv on [0, 1].”
Quote in the paperNot a substantial claim- The authors state this as an assumption: In the toy three-agent MCQ panel, each agent independently abandons the correct answer with probability α_adv.
Classified as probably not a substantial claim.
Interpretationown dataWe treat R1 as a diagnostic substrate, not a standalone deployable gate.
“We therefore treat R1 as a diagnostic substrate, not a standalone deployable gate.”
Quote in the paperSubstantial claim- Rests on an assumption: The aggregate Table 37 ledger is sufficient to evaluate the probe-gated freeze policy without the row-level decision matrix.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenACC is selected on pooled development data, then evaluated without retuning.
“ACC is selected on pooled development data, then evaluated without retuning”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningThe floor metric isolates the refusal-rate shift on the affected rungs without staking the claim on rank-monotone behavior across all 9 rungs.
“the floor metric isolates the refusal-rate shift on the affected rungs without staking the claim on rank-monotone behavior across all 9 rungs”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
InterpretationreasoningThe wrong-answer coincidence factor is not uniform because a plausible but wrong argument can steer multiple agents to the same distractor.
“The wrong-answer coincidence factor is not uniform because a plausible but wrong argument can steer multiple agents to the same distractor.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe posterior predictive check one-sided p-value is 0.5426.
“Posterior predictive check: observed collapse rate 7.98%, replicate mean matches; PPC one-sided p=0.5426.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThe deployable intervention evidence is proof-of-concept on the development set only: the frozen policy does not transfer reliably to fresh held-out questions.
“We therefore interpret the deployable intervention evidence as proof-of-concept on the development set only: the frozen policy does not transfer reliably to fresh held-out questions.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductown dataThe residual model-level α slope is weakly identified, so this is a localization check rather than causal mediation evidence.
“the residual model-level α slope is weakly identified, so this is a localization check rather than causal mediation evidence”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataConditioning on initial_correct and dropping models with n pos ≤1 reduces from 6,925 raw debates.
“Conditioning on initial_correct and dropping models with n pos ≤1 (Gemini 3-flash and GPT-5.4-mini,both with a single collapse, render σ model unidentifiable) reduces from 6,925 raw debates.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataA local sweep of the post-challenge reply temperature kept only a derived seven-row summary, not paired raw outputs, and reached 6.4% unparsed replies, so we do not use it.
“A local sweep of the post-challenge reply temperature kept only a derived seven-row summary, not paired raw outputs, and reached 6.4% unparsed replies, so we do not use it.”
Quote in the paperNot a substantial claim- Rests on an assumption: The missing temperature field in saved probe records does not invalidate the screen results.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenA Fisher-z approximation for a partial correlation needs about 12 families at the submitted partial ρ = 0.767 and about 21 families at the extension partial ρ = 0.597.
“a Fisher-z approximation for a partial correlation needs about 12 families at the submitted partial ρ = 0.767 and about 21 families at the extension partial ρ = 0.597”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoningThe follow-up removes the main weakness of the quasi-paired check in Section G.4: every arm branches from the same saved Round 0 answers, so arms differ only in the revision rounds.
“It removes the main weakness of the quasi-paired check in Section G.4: every arm branches from the same saved Round 0 answers, so arms differ only in the revision rounds.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThe DeepSeek V4 Flash debate row of the primary cohort ran with the provider's default reasoning enabled: 2,985 of its 3,000 logged debate calls returned reasoning tokens.
“the DeepSeek V4 Flash debate row of the primary cohort ran with the provider's default reasoning enabled: 2,985 of its 3,000 logged debate calls returned reasoning tokens.”
Quote in the paperNot a substantial claim- Rests on an assumption: The DeepSeek V4 Flash row's use of default reasoning does not materially affect the primary cohort results.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductown dataThese are single-realization feasibility pilots, not a panel-composition result.
“These are single-realization feasibility pilots, not a panel-composition result.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenTool use, retrieval, asymmetric judge/debater roles, unrestricted generation, and rubric-or judge-scored answers are outside the evidence in this paper.
“Tool use, retrieval, asymmetric judge/debater roles, unrestricted generation, and rubric-or judge-scored answers are outside the evidence in this paper.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenAll pre-reg SHAs verified in the result JSON match the frozen pre-registered values.
“All pre-reg SHAs verified in the result JSON match the frozen pre-registered values.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenAll follow-up results are post hoc relative to the submitted study.
“All follow-up results are post hoc relative to the submitted study.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationown dataThis is a quasi-paired single-checkpoint null, not evidence of equivalence.
“This is a quasi-paired single-checkpoint null, not evidence of equivalence.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataA GPT-5.4-mini reasoning-effort sweep on 200 MMLU-Pro questions resampled initial answers at each effort level and reached 7.6% parse missingness at high effort (0% and 2.5% at low and medium), so it is descriptive rather than directional.
“A GPT-5.4-mini reasoning-effort sweep on 200 MMLU-Pro questions resampled initial answers at each effort level and reached 7.6% parse missingness at high effort (0% and 2.5% at low and medium), so it is descriptive rather than directional.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataRobustness of the screen to the reply temperature remains unresolved.
“Robustness of the screen to the reply temperature remains unresolved.”
Quote in the paperNot a substantial claim- Rests on an assumption: The missing temperature field in saved probe records does not invalidate the screen results.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataBoth inputs are noisy observed effects, so these are neither exact small-sample calculations nor achieved power.
“Both inputs are noisy observed effects, so these are neither exact small-sample calculations nor achieved power.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThree runtime/infrastructure-only fixes were applied to the harness during execution ... none touch the scientific surface.
“Three runtime/infrastructure-only fixes were applied to the harness during execution (enforce_eager=True, max_model_len=8192, per-model partial-write checkpoint) via commits c3b40f8 + 69f98d0; none touch the scientific surface”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataThe GSM8K rows use deterministic numeric grading with no parse failures, but with 1-2 collapses per roughly 289 initially correct majorities they are too sparse to test collapse.
“The GSM8K rows use deterministic numeric grading with no parse failures, but with 1-2 collapses per roughly 289 initially correct majorities they are too sparse to test collapse.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningNo pre-registration deviation is owed for this entry.
“No pre-registration deviation is owed for this entry.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductreasoningThese tests are exploratory.
“so these tests are exploratory”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenPools contain 2,000 MMLU-Pro or SuperGPQA questions, 1,000 MMLU-Pro questions for most reasoning-mode rows, or all 1,324 level-5 MATH problems.
“Pools contain 2,000 MMLU-Pro or SuperGPQA questions, 1,000 MMLU-Pro questions for most reasoning-mode rows, or all 1,324 level-5 MATH problems.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown dataA Qwen3-8B control on 200 MMLU-Pro questions compared peer debate with private self-revision at the same call count and per-call token cap, scored at Round 3.
“A Qwen3-8B control on 200 MMLU-Pro questions compared peer debate with private self-revision at the same call count and per-call token cap, scored at Round 3.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe screen is a distinct secondary signal for prioritizing trace collection, not a better selector than initial accuracy.
“The screen is therefore a distinct secondary signal for prioritizing trace collection, not a better selector than initial accuracy.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataFamily aggregation prevents six related rows from counting as six families, but it does not remove the within-Qwen range of conditional collapse (5.11% to 43.90%).
“Family aggregation prevents six related rows from counting as six families, but it does not remove the within-Qwen range of conditional collapse (5.11% to 43.90%).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown dataInitial probe answers were in fact sampled at the debate temperatures (0.5, 0.7, 1.0); only the post-challenge reply used temperature 0 where the provider supported it.
“Initial probe answers were in fact sampled at the debate temperatures (0.5, 0.7, 1.0); only the post-challenge reply used temperature 0 where the provider supported it.”
Quote in the paperSubstantial claim- Rests on an assumption: The missing temperature field in saved probe records does not invalidate the screen results.
May combine two claims.
Interpretationown dataThe row remains nearly accuracy-neutral, and it stays a stress row rather than screen-transfer evidence.
“The row remains nearly accuracy-neutral, and it stays a stress row rather than screen-transfer evidence.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenNone of the appendix analyses enters the primary G=7 family analysis.
“None of them enters the primary G=7 family analysis.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataModel identity is also confounded with agent slot and slot temperature, and parse missingness through Round 3 is 1.46%-1.87%.
“Model identity is also confounded with agent slot and slot temperature, and parse missingness through Round 3 is 1.46%-1.87%.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
Multi-agent LLM debate is often evaluated by whether final answers improve.
“Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve”
The paper relies on this without support.
The 8-probe argument/social battery measures a model-level pre-debate compliance/revisability property relevant to debate collapse.
“Each MCQ item is re-asked under an 8-probe argument/social battery, and α tot records the total answer-change rate.”
The paper relies on this without support.
The MMLU-Pro benchmark and the realized N = 14 / G = 7 cohort are adequate to support the paper's family-level triage conclusions.
“Our matched MMLU-Pro evaluation covers N =14 model rows across G=7 family clusters”
The paper relies on this without support.
The rule-based parser and alphabetic tie-breaking produce transition labels accurate enough for the headline collapse/correction counts.
“All labels are rule-based; no LLM judge or human coder is used.”
The paper relies on this without support.
The freeze-style gates evaluated are representative of intervention policies for debate.
“All three freeze-style gates prevent some collapses but lose more corrections”
The paper relies on this without support.
The initial-majority accuracy is a valid capability proxy for comparing against the pre-debate screen.
“Capability pressure, measured as 1-initial-majority accuracy, is a close free comparator”
The paper relies on this without support.
The Round 1 trajectory features are available and meaningful for runtime diagnosis.
“Round 1 trajectory features raise pooled out-of-fold area under the curve (AUC) from 0.669 to 0.768 over pre-debate features.”
The paper relies on this without support.
The reasoning-mode follow-up with shared Round-0 answers is comparable to compute-matched solo reasoning.
“These results bound, rather than settle, the comparison with compute-matched solo reasoning.”
The paper relies on this without support.
Gated release limits casual reuse of the convince-wrong probe templates as an attack corpus.
“To limit casual reuse as an attack corpus, the convince-wrong probe templates and tuned phrasings are released under a gated artifact host”
The authors state this as an assumption.
The calibration value of a transparent, pre-registered instrument outweighs the marginal uplift to attackers.
“Our position is that the calibration value of a transparent, pre-registered instrument outweighs the marginal uplift to attackers, and we welcome critique of that judgement during review.”
The paper relies on this without support.
Initial accuracy is a strong capability proxy.
“Initial accuracy is a strong capability proxy”
The paper relies on this without support.
ARC-Challenge is an easier benchmark.
“On ARC-Challenge [4] (an easier benchmark)”
The authors state this as an assumption.
In the toy three-agent MCQ panel, each agent independently abandons the correct answer with probability α_adv.
“suppose each agent independently abandons the correct answer with probability α adv”
The paper relies on this without support.
The alphabetic tie rule is an appropriate way to resolve initial states without a unique plurality.
“under the paper's alphabetic tie rule”
The paper relies on this without support.
The aggregate Table 37 ledger is sufficient to evaluate the probe-gated freeze policy without the row-level decision matrix.
“the 6,525-debate probe-gated LOMO ledger (Table37) is released as a frozen aggregate, and its row-level decision matrix is not part of the release”
The paper relies on this without support.
The 1,255-trace replay cohort is representative of the 6,525-debate LOMO cohort for the DisagreementGate sanity check.
“As a weaker same-trace sanity check on the available 1,255 replay cohort”
The paper relies on this without support.
The missing temperature field in saved probe records does not invalidate the screen results.
“The temperature field is missing from the saved probe records of several later rows.”
The paper relies on this without support.
The DeepSeek V4 Flash row's use of default reasoning does not materially affect the primary cohort results.
“the DeepSeek V4 Flash debate row of the primary cohort ran with the provider's default reasoning enabled”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
The stated very-strong probe phrase is absent from its quoted template
MinorThe robustness discussion says the very-strong probe contains an authority phrase, but the paper's quoted very-strong counterargument string does not contain it. This leaves the description of the prompt being ablated internally inconsistent.
“The very-strong probe contains the phrase "According to authoritative references and expert consensus,"”
“"The answer is definitely [X].Your reasoning contains a fundamental error.The question is testing a specific concept where [X] is the only correct interpretation.I can explain exactly where your analysis went wrong if needed."”
Cites
Degroot, M. H. (1974) · Journal of the American Statistical Association · doi:10.1080/01621459.1974.10480137 · Cited by 4,004 · see citing works
A Theory of Fads, Fashion, Custom, and Cultural Change as Informational Cascades
Bikhchandani, S., Hirshleifer, D. A., & Welch, I. (1992) · Journal of Political Economy · doi:10.1086/261849 · Cited by 1,980 · see citing works
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · cited 2× · see citing works
“The artifact release plan covers the probe set, response traces, evaluation code, and the multi-round debate transcripts under the underlying-benchmark licenses (MMLU-Pro [25], ARC-Challenge [4], GPQA [18], and TruthfulQA [14]).”
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018) · arXiv · doi:10.48550/arxiv.1803.05457 · Cited by 462 · cited 1× · see citing works
“The artifact release plan covers the probe set, response traces, evaluation code, and the multi-round debate transcripts under the underlying-benchmark licenses (MMLU-Pro [25], ARC-Challenge [4], GPQA [18], and TruthfulQA [14]).”
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., & Tu, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.19118 · Cited by 320 · cited 1× · see citing works
“Debate gains depend on diversity and confidence [13, 32], while sycophancy and conformity provide routes for mistaken answers to spread [11].”
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., & Tu, Z. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.992 · Cited by 274 · see citing works
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2305.04388 · Cited by 262 · see citing works
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukosiute, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.847 · Cited by 194 · see citing works
Sharma, M., Tong, M., Korbak, T., Duperrouzel, D., Was, H., Korbak, J., Cheng, M., Askell, A., Bai, Y., Brain, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.13548 · Cited by 132 · see citing works
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.01574 · Cited by 131 · cited 2× · see citing works
“Even in this setting, we observe 253 collapses across 6 , 925 MMLU-Pro debates [25]; for the most susceptible models, more than one in ten initially correct majorities are lost.”
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., et al. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3018 · Cited by 131 · see citing works
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023) · arXiv · doi:10.48550/arxiv.2305.14325 · Cited by 96 · cited 2× · see citing works
“We use a standard debate protocol [7]: three agents answer independently (Round 0 ), then discuss and update for Rounds 1 – 3 while seeing all current answers.”
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · arXiv · doi:10.48550/arxiv.2305.13281 · Cited by 85 · see citing works
Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
Xiong, K., Ding, X., Cao, Y., Liu, T., & Qin, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.508 · Cited by 66 · see citing works
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60 · see citing works
ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
Chan, C.-M., Chen, W., Su, Y., Yu, J., Xue, W., Zhang, S., Fu, J., & Liu, Z. (2023) · arXiv · doi:10.48550/arxiv.2308.07201 · Cited by 55 · see citing works
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09251 · Cited by 53 · see citing works
Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM ReasoningPreprint
Eo, S., Moon, H., Zi, E. H., Park, C., & Lim, H.-J. (2025) · arXiv.org · doi:10.48550/arxiv.2504.05047 · Cited by 33 · see citing works
GPQA: A Graduate-Level Google-Proof Q&A Benchmark
Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2311.12022 · Cited by 30 · cited 2× · see citing works
“The artifact release plan covers the probe set, response traces, evaluation code, and the multi-round debate transcripts under the underlying-benchmark licenses (MMLU-Pro [25], ARC-Challenge [4], GPQA [18], and TruthfulQA [14]).”
Confidence Improves Self-Consistency in LLMs
Taubenfeld, A., Sheffer, T., Ofek, E. O., Feder, A., Goldstein, A., Gekhman, Z., & Yona, G. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1030 · Cited by 29 · see citing works
Simple synthetic data reduces sycophancy in large language models
Wei, J., Huang, D., Lu, Y., Zhou, D., & Le, Q. V. (2023) · arXiv · doi:10.48550/arxiv.2308.03958 · Cited by 28 · see citing works
Scaling Laws For Scalable Oversight
Engels, J., Baek, D. D., Kantamneni, S., et al. (2025) · Neural Information Processing Systems · doi:10.48550/arxiv.2504.18530 · Cited by 23
Demystifying Multi-Agent Debate: The Role of Confidence and DiversityPreprint
Zhu, X.-C., Zhang, C., Chi, Y., et al. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2601.19921 · Cited by 17 · cited 1× · see citing works
“Debate gains depend on diversity and confidence [13, 32], while sycophancy and conformity provide routes for mistaken answers to spread [11].”
The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched ComputePreprint
Sharma, A., & Chopra, P. (2025) · arXiv.org · doi:10.48550/arxiv.2511.02309 · Cited by 13 · see citing works
Debating with More Persuasive LLMs Leads to More Truthful Answers
Khan, A., Hughes, J., Valentine, D., Ruis, L., Sachan, K., Radhakrishnan, A., Grefenstette, E., Bowman, S. R., Rocktäschel, T., & Perez, E. (2024) · arXiv · doi:10.48550/arxiv.2402.06782 · Cited by 13 · see citing works
iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference
Fan, W., Yoon, J., & Ji, B. (2025) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2511.11306 · Cited by 12
Tang, L.-X., Meng, Y.-Q., Costa, J., Zhang, Y., Ye, M.-C., & Xi, Z.-H. (2026) · arXiv.org · doi:10.48550/arxiv.2602.07186 · Cited by 9 · see citing works
LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying CircuitPreprint
Pandey, M. (2026) · arXiv.org · doi:10.48550/arxiv.2604.19117 · Cited by 8 · see citing works
When Two LLMs Debate, Both Think They'll Win
Prasad, P. S., & Nguyen, M. N. (2025) · arXiv.org · doi:10.48550/arxiv.2505.19184 · Cited by 8 · cited 1×
“The remaining risk surface mirrors that of recent sycophancy, conformity, and debate-failure audits [11, 21, 28]: even with gated release, a determined adversary can reconstruct similar probe sets.”
Too Polite to Disagree: Understanding Sycophancy Propagation in Multi-Agent Systems
Kasprova, V., Parulekar, A., Alrabah, A., et al. (2026) · SIGDIAL Conferences · doi:10.48550/arxiv.2604.02668 · Cited by 5 · cited 2×
“The remaining risk surface mirrors that of recent sycophancy, conformity, and debate-failure audits [11, 21, 28]: even with gated release, a determined adversary can reconstruct similar probe sets.”
CGES: Confidence-Guided Early Stopping for Efficient and Accurate Self-ConsistencyPreprint
Aghazadeh, E., Ghasemi, A., Beyhaghi, H., & Pishro-Nik, H. (2025) · arXiv.org · doi:10.48550/arxiv.2511.02603 · Cited by 5 · see citing works
Scaling Laws For Scalable Oversight
Engels, J., Baek, D. D., Kantamneni, S., & Tegmark, M. (2025) · neural information processing systems · doi:10.52202/085713-3254 · Cited by 5 · see citing works
Knowledge Divergence and the Value of Debate for Scalable OversightPreprint
Young, R. R. (2026) · arXiv.org · doi:10.48550/arxiv.2603.05293 · Cited by 3 · see citing works
Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate
Wynn, A., Satija, H., & Hadfield, G. (2025) · arXiv · doi:10.48550/arxiv.2509.05396 · Cited by 1 · cited 1× · see citing works
“The remaining risk surface mirrors that of recent sycophancy, conformity, and debate-failure audits [11, 21, 28]: even with gated release, a determined adversary can reconstruct similar probe sets.”
Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning
Wu, H., Li, Z., & Li, L. (2025) · arXiv · doi:10.48550/arxiv.2511.07784 · Cited by 1 · see citing works
iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference
Fan, W., Yoon, J., & Ji, B. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i35.40181 · Cited by 1
Disentangling the Drivers of LLM Social Conformity: An Uncertainty-Moderated Dual-Process MechanismPreprint
Zhong, H., Liu, Y., Cao, Q., Wang, S., Ye, Z., Wang, Z., & Zhang, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.14918
Flagged first, then by citations
- Authors
- Li, X., Liu, M., & Yuen, C.
- Affiliations
- Nanyang Technological University (SG)
- ORCID
- Liu, Mengbing
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2609.35279
- OpenAlex
- W7214734896
- Semantic Scholar
- 7592a9cca8e14cff96aa78fc658d77bab5ec09fe
- Cited by
- 0
- References
- 32
- Type
- posted content
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- collapse, corrections, multiple-choice questions, MMLU-Pro
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review