RoPoLL: Robust Panel of LLM Judges

POLL was sufficient, in Verga et al.'s experiments, to match or exceed a single large judge.

“The LLM Jury, instantiated by the Panel of LLM Evaluators (POLL) of Verga et al. (2024), ensembles smaller, diverse, cheaper backbones and reports the arithmetic mean of their scores as the consensus-sufficient, in their experiments, to match or exceed a single large judge.”

• cites Verga et al., 2024 sourcePartly supported

  The Introduction and Section 4.2 report that PoLL compared favorably with a single large judge in the experiments. However, the citing sentence also attributes arithmetic-mean aggregation to PoLL generally. Section 3.1 says the experiments used max voting for QA datasets and average pooling only for Chatbot Arena. (LLM, conf=0.90)

  Experimental — accuracy not yet measured

The statistical behaviour of the LLM Jury (POLL) remains poorly understood.

“The LLM Jury, a Panel of LLM Evaluators (POLL) (Verga et al., 2024) reporting consensus scores, has become a practical alternative to single judge LLM evaluation, yet its statistical behaviour remains poorly understood.”

• cites Verga et al., 2024 sourceSupported

Systematic biases of single judges (position, verbosity, self-enhancement, sycophancy, and refusal artefacts) are well documented.

“The systematic biases its backbone exhibits, e.g., position, verbosity, self-enhancement, sycophancy, and refusal artefacts, are by now well documented (Wang et al., 2023;Panickssery et al., 2024;Saito et al., 2023;Stureborg et al., 2024); they propagate uncorrected to every score, and the cost-quality profile of the resulting evaluation is fixed to that of the single model.”

• cites Wang et al., 2023 sourcePartly supported

  Section 2.2, “Revealing the Positional Bias” (page not provided), documents position-dependent judgments by GPT-4 and ChatGPT, supporting the position-bias example. The passages found do not document verbosity, self-enhancement, sycophancy, or refusal artefacts, so this source supports only part of the cited list. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites Panickssery et al., 2024 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

• cites Saito et al., 2023 sourcePartly supported

  Section 2.3, “Biases in Automated LLM Evaluation” (page not provided), describes position, self-enhancement, and verbosity biases; the paper also reports its own findings on verbosity bias. The passages do not document sycophancy or refusal artefacts, so this source supports only part of the cited list. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

The minimax-optimal estimator on the breakdown floor is the Tukey halfspace median.

“The minimax-optimal estimator on the breakdown floor is the Tukey halfspace median (Tukey, 1975;Donoho and Gasko, 1992)”

• cites Tukey, 1975Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Donoho and Gasko, 1992 sourcePartly supported

  The introduction (p. 1) identifies the halfspace-depth generalization of the median and gives its breakdown-point lower bound of 1/(d + 1). It does not establish that this estimator is minimax-optimal; the discussion of breakdown results (p. 9) instead says the outlyingness-weighted mean has better breakdown properties and is nearly best possible. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Exact computation of the Tukey halfspace median is NP-hard for d ≥ 3.

“whose exact computation is NP-hard for d ≥ 3 (Johnson and Preparata, 1978;Aloupis, 2006)”

• cites Johnson and Preparata, 1978 sourcePartly supported

  Section 2 establishes NP-completeness for the hemisphere problems when the dimension is not fixed in advance. The paper does not establish NP-hardness for computing the Tukey halfspace median or for any fixed dimension d ≥ 3; Sections 3–4 instead give polynomial-time algorithms for the hemisphere problems at fixed d. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites Aloupis, 2006 sourceNot checked — no full text of the source

The breakdown floor is matched up to a dimensional constant only by the (intractable) Tukey halfspace median.

“the breakdown floor, in turn, is matched up to a dimensional constant only by the (intractable) Tukey halfspace median.”

• cites Tukey, 1975Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Donoho and Gasko, 1992 sourcePartly supported

  The opening summary (p. 1) gives the halfspace-depth median a breakdown point of at least 1/(d + 1), and §5.3 (p. 13) says the paper’s estimators are computationally impractical in higher dimensions. But “only” is not supported: Proposition 3.5 (p. 9) establishes high breakdown for an outlyingness-weighted mean, and §5.2 (p. 12) describes another method with breakdown nearly 1/2. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

• AssumptionThe authors state this as an assumption: The joint distributions of judge estimates are consistent with Assumptions 2, 3, 4, and 5, including the i.i.d. baseline with identical σ_i.

If at least (1-α)k points lie within distance r of z, then any geometric median x* satisfies ||x*-z||_2 ≤ C_α r, where C_α = (1-α)/sqrt(1-2α).

“If |{j : ∥x j -z∥ 2 ≤ r}| ≥ (1 -α)k, then x * -z 2 ≤ C α r, C α ≜ 1 -α √ 1 -2α . (16”

• cites Minsker (2015) sourceSupported
• cites Lopuhaä and Rousseeuw (1991) sourcePartly supported

  Theorem 2.2 and its proof (p. 5) establish that the L1 estimator, which minimizes the sum of distances, remains bounded when fewer than half the points are replaced. The proof gives a different bound and does not state the claimed radius bound or the constant Cα = (1−α)/√(1−2α). (LLM, conf=0.82)

  Experimental — accuracy not yet measured

The smoothed-depth estimator of Chen et al. (2018) matches the σα floor in sub-exponential time.

“the smoothed-depth estimator of Chen et al. (2018) matches the σα floor in sub-exponential time.”

• cites Chen et al., 2018 sourcePartly supported

  The paper’s Abstract (page unavailable) says that optimizing its matrix-depth function achieves minimax rates under Huber contamination. It does not describe a smoothed-depth estimator or establish that one matches a σα floor in sub-exponential time; the same passage says its high-dimensional depth estimators are challenging to compute. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

The geometric median has no closed form for dimension d≥2.

“The geometric median has no closed form for d ≥ 2 (Bajaj, 1988); we compute it via the modified Weiszfeld iteration (Weiszfeld, 1937;Vardi and Zhang, 2000).”

• cites Bajaj, 1988 sourcePartly supported

  The Introduction (page unavailable) says the planar generalized Weber problem is, in general, not solvable by radicals for five or more points, while noting simple solutions for special configurations. Section 5 (page unavailable) says the unrestricted three-dimensional result is suspected, not proved; the paper proves a result for a line-restricted version. Thus the paper supports the absence of a general radical formula in the planar case, but not the unqualified claim of no closed form for every dimension d ≥ 2. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

The modified Weiszfeld iteration converges to the unique geometric median at a linear rate whenever the data are not collinear.

“Vardi and Zhang (2000) prove that the modified Weiszfeld iteration converges to the unique geometric median at a linear rate whenever the data are not collinear: ∥z (t) -ŷGM ∥ 2 ≤ ρ t ∥z (0) -ŷGM ∥ 2 for some ρ ∈ (0, 1).”

• Also stated in: § A.4 Weiszfeld Iteration: Full Derivation, Convergence, and Cost · plain

  “Vardi and Zhang (2000) prove that the modified Weiszfeld iteration converges to the unique geometric median at a linear rate whenever the data are not collinear”

• cites Vardi and Zhang, 2000 sourcePartly supported

  Section 2, “The L1-Median” (page unavailable), states that noncollinear data have a unique median. Section 5, “Proofs of 2.9–2.11” (page unavailable), establishes convergence of the modified iteration, but the available text does not establish a linear rate or the stated geometric error bound; it describes monotonic convergence instead. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites Vardi and Zhang (2000) sourcePartly supported

  Section 2, “The L1-Median,” says noncollinear data have a unique median. The Introduction and Section 5 say the modified algorithm converges to it from any starting point, but the readable passages do not establish a linear rate or the stated bound; the relevant formulas are not decoded in our copy. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

• AssumptionThe authors state this as an assumption: The data are not collinear.

Chen et al. (2018), Theorem 5.1, establish Θ(σ^2(d/N + α^2)) as the squared-error minimax for sub-Gaussian Huber, with no d in the contamination term.

“and indeed Chen et al. (2018), Theorem 5.1, establish Θ(σ 2 (d/N + α 2 )) as the squared-error minimax for sub-Gaussian Huber, with no d in the contamination term.”

• cites Chen et al. (2018), Theorem 5.1 sourcePartly supported

  Section 5 (Theorem 5.1) gives a general minimax lower bound and illustrates it with a Gaussian location model, obtaining (p/n) ∨ ε²—equivalent in rate to p/n + ε², with no dimension factor on the contamination term. Section 2 (Theorems 2.1 and 2.2) supplies the matching rate for that Gaussian model. The copy does not establish the stated σ²-scaled rate for the broader sub-Gaussian class, and Theorem 5.1 alone does not establish a matching upper bound. (LLM, conf=0.90)

  Experimental — accuracy not yet measured

Chen et al. (2018), Theorem 5.1, prove inf sup E||θ - θ||_2 ≍ σ^2(d/N + α^2), with no d on the squared-error contamination floor.

“Chen et al. (2018), Theorem 5.1, prove inf sup E∥ θ -θ∥ 2 ≍ σ 2 (d/N + α 2 ), with no d on the squared-error contamination floor.”

• cites Chen et al. (2018) sourcePartly supported

  Section 5 (Theorem 5.1; page unavailable) derives a minimax lower bound of order (p/n) ∨ ε² for the Gaussian location model with identity covariance. This supports a dimension-free squared-error contamination floor. It does not state the claimed two-sided equivalence for expected loss or the σ² scaling; the upper bound in Section 2 (Theorem 2.1; page unavailable) is a high-probability result under additional conditions. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

The geometric median has breakdown point exactly ⌈N/2⌉/N, which tends to 1/2 as N→∞.

“Hence the breakdown point is exactly ϵ ⋆ = ⌈N/2⌉/N , which tends to 1/2 as N → ∞.”

• cites Lopuhaä and Rousseeuw, 1991 sourceSupported

POLL admits unbounded bias under any positive contamination.

“showed that POLL (Verga et al., 2024) admits unbounded bias under any positive contamination (Proposition 2)”

• Also stated in: § Step 2: Linearity of the mean. By the linearity of expectation applied to · boosted

  “The bias is therefore unbounded over the corruption class for every fixed N , completing the proof.”

• cites Verga et al., 2024 sourceSupported

The geometric median objective has a unique minimizer when the data are not collinear.

“Suppose the data are not collinear: then for any line L ⊂ R d there exists at least one ŷi / ∈ L, so the corresponding summand is strictly convex along L. Hence F is strictly convex along every line, hence strictly convex on R d , and the minimizer is unique”

• cites Vardi and Zhang, 2000 sourceSupported

• AssumptionThe authors state this as an assumption: The data are not collinear.

If at least (1-α)k points are within distance r of z, then the minimizer x* satisfies ∥x* - z∥ ≤ C_α r with C_α = (1-α)/√(1-2α).

“If |{j : ∥x j -z∥ 2 ≤ r}| ≥ (1 -α)k, then ∥x * -z∥ 2 ≤ C α r with C α = (1 -α)/ √ 1 -2α.”

• cites Minsker (2015) sourceSupported

Real LLM juries trained on overlapping corpora violate Assumption 3: inter-judge correlation γ ∈ [0.3, 0.7] is typical.

“Real LLM juries trained on overlapping corpora violate this: inter-judge correlation γ ∈ [0.3, 0.7] is typical (Figure 14 and Section 6.9).”

ROPOLL outperforms POLL by orders of magnitude on heavy-tailed Byzantine adversaries.

“by orders of magnitude on heavy-tailed Byzantine adversaries (whose unbounded first moments make any breakdown-positive aggregator beat averaging unconditionally).”

Across 13 open-weight judges (4B-675B), three reward-model benchmarks, and four corruption regimes at rates up to 50%, ROPOLL dominates POLL on every biased corruption type.

“Across 13 open-weight judges (4 B-675 B), three reward-model benchmarks, and four corruption regimes at rates up to 50%, ROPOLL dominates POLL on every biased corruption type: by ≈ 19% on cross-dimensional attacks at matched compute, and by orders of magnitude on”

Mean parser-failure rates are 3.4% on HelpSteer 3 and 0.6% on HelpSteer 2 across the 13-judge panel.

“with mean rates of 3.4% on HelpSteer 3 and 0.6% on HelpSteer 2 across the 13-judge panel (Figure 2, §3.4).”

A judge that handles one attribute well may catastrophically mis-score another, producing a vector that is plausible per axis yet jointly anomalous.

“A judge that handles one attribute well may catastrophically mis-score another, producing a vector that is plausible per axis yet jointly anomalous.”

A judge that produces malformed JSON triggers a parser fallback to the all-zeros score, dropping a single observation onto the boundary of the score space.

“A judge that produces malformed JSON triggers a parser fallback to the all-zeros score, dropping a single observation onto the boundary of the score space.”

A judge whose parser hallucinates can emit values entirely outside the bounded score scale.

“A judge whose parser hallucinates can emit values entirely outside the bounded score scale.”

In the authors' corpus, parser-failure alone reaches 33% on the smallest judge (Gemma-4B) for HelpSteer 3 multilingual prompts.

“in our corpus, parser-failure alone reaches 33% on the smallest judge (Gemma-4B) for HelpSteer 3 multilingual prompts, with mean rates of 3.4% on HelpSteer 3 and 0.6% on HelpSteer 2 across the 13-judge panel (Figure 2, §3.4).”

A Noisy-GT control confirms the ROPOLL premium is paid against biased contamination, not benign Gaussian imprecision.

“A Noisy-GT control confirms the premium is paid against biased contamination, not benign Gaussian imprecision (where POLL is statistically optimal).”

The empirical indicator-correlation γW is in [0.45, 0.53] measured on the authors' judge panels.

“with empirical indicator-correlation γW ∈ [0.45, 0.53] measured on our judge panels, Section 6.10”

ROPOLL outperforms POLL by approximately 19% on cross-dimensional attacks at matched compute.

“by ≈ 19% on cross-dimensional attacks at matched compute”

A judge with sycophancy bias rates every response near the maximum, flattening genuine quality differences.

“A judge with sycophancy bias rates every response near the maximum, flattening genuine quality differences.”

A 3-judge ROPOLL committee at 38B beats Mistral-Large-3 (675B) by 1.31× on HelpSteer-2 under 30% bimodal-random corruption.

“A 3-judge ROPOLL committee at 38 B beats Mistral-Large-3 (675 B) by 1.31× on HelpSteer-2 under 30% bimodal-random corruption -an 18× parameter advantage with strictly better accuracy.”

The empirical indicator correlation γW of Lemma 3 on the 13-judge experimental panels is stable to within ±0.03 across cluster-radius calibrations.

“γW is stable to within ±0.03 across cluster-radius calibrations and lies in [0.45, 0.53] on both benchmarks, so N eff ∈ [1.45, 1.58] at the practical jury size N =3.”

Even in the favourable-to-the-mean regime of 30% zeros corruption, all four ROPOLL committees sit at or below the individual-at-30% scaling line on every benchmark.

“even in this favourable-to-the-mean regime, all four ROPOLL committees sit at or below the individual-at-30% scaling line on every benchmark”

The UltraFeedback correlation γUF = 0.71 is notably higher than the HelpSteer correlations.

“The UltraFeedback correlation γUF = 0.71 is notably higher than the HelpSteer correlations.”

At finite N, the discrete breakdown threshold of the geometric median is ⌈N/2⌉/N, which is strictly above 1/2 for odd N.

“The asymptotic 1/2 breakdown is attained as N → ∞; at finite N the discrete breakdown threshold is ⌈N/2⌉/N , strictly above 1/2 for odd N .”

The largest gap occurs on HelpSteer 2 at r = 40%, where POLL's RMSE reaches ≈ 4,951 while ROPOLL holds at ≈ 9.2, a ratio of ≈ 540×.

“The largest gap occurs on HelpSteer 2 at r = 40%, where POLL's RMSE reaches ≈ 4,951 while ROPOLL holds at ≈ 9.2-a ratio of ≈ 540×.”

In our benchmarks, d is small, ranging from 1 to 5.

“d is small (1-5 in our benchmarks)”

The empirical γW is on the same order as the score correlation γ ∈ [0.49, 0.71] reported in Figure 14.

“The empirical γW is on the same order as the score correlation γ ∈ [0.49, 0.71] reported in Figure 14”

At N = 3, the variance term σ d/N dominates the breakdown floor on every regime we test.

“at N = 3 the variance term σ d/N dominates the breakdown floor on every regime we test ( §6.6).”

On HelpSteer 3, the MEDIUM committee at 89 B matches DeepSeek-V3.1 (671 B) at RMSE = 1.85.

“On the harder HelpSteer 3 (signed-preference target), the MEDIUM committee at 89 B still matches DeepSeek-V3.1 (671 B) at RMSE = 1.85.”

For any set of N≥1 points in R^d, the geometric median objective F(z)=Σ_i ||z-ŷ_i||_2 has a minimizer.

“Existence.F is continuous, convex, and coercive, so a minimizer exists.”

The empirical indicator correlation γW lies in [0.45, 0.53] on both benchmarks.

“γW is stable to within ±0.03 across cluster-radius calibrations and lies in [0.45, 0.53] on both benchmarks, so N eff ∈ [1.45, 1.58] at the practical jury size N =3.”

Under Assumptions 2-5, for any slack β∈(0,1/2-α), with probability at least 1-exp(-Nβ2/2), ||ŷ_GM - y*||_2 ≤ C_{α+β} ρ, where ρ is the cluster radius of Lemma 2.

“Theorem 1 (ROPOLL Breakdown Bound under Huber Contamination).Under Assumptions 2-5, fix any slack β ∈ (0, 1/2 -α).With probability at least 1 -exp(-N β 2 /2), ŷGM -y ⋆ 2 ≤ 1 -α -β √ 1 -2α -2β C α+β • σ C 1 √ d + 1 c log 2(1 -α) β ρ . (19”

• AssumptionThe authors state this as an assumption: The Huber ε-contamination model holds for the corruption distribution.
• AssumptionThe authors state this as an assumption: Judges' errors are conditionally independent.
• AssumptionThe authors state this as an assumption: Competent judge noise is σ-sub-Gaussian.
• AssumptionThe authors state this as an assumption: The minority corruption rate α is less than 1/2.

If the points ŷ_1,...,ŷ_N are not collinear, the geometric median minimizer is unique.

“Uniqueness.If ŷ1 , . . ., ŷN are not collinear, then F is strictly convex and the minimizer is unique.”

The gap to POLL remains positive but small (≤ 0.3 RMSE for the MEDIUM jury across the full corruption sweep).

“the gap to POLL remains positive but small (≤ 0.3 RMSE for the MEDIUM jury across the full corruption sweep-see Figure 12).”

A controlled synthetic validation confirms that empirical MSE on an equicorrelated Gaussian jury matches the closed-form prediction (1+(N-1)γ)/N dσ^2 across the full range γ ∈ [0,0.95].

“A controlled synthetic validation (Figure 7a of Figure 7, §5.1) confirms that empirical MSE on an equicorrelated Gaussian jury matches the closed-form prediction 1+(N -1)γ N dσ 2 across the full range γ ∈ [0, 0.95].”

The inter-judge correlation γ measured in our experiments is in the range [0.3, 0.7].

“covering the inter-judge correlation γ ∈ [0.3, 0.7] measured in our experiments.”

Under the assumptions of Theorem 1, the minimax risk over all measurable estimators is at least cσ d/N + α/(1-α), where c > 0 is a universal constant.

“inf ŷ sup F ∈Fα,σ E F [∥ŷ -y ⋆ ∥ 2 ] ≥ cσ d/N + α 1 -α ,(21)”

• AssumptionThe authors state this as an assumption: The joint distributions of judge estimates are consistent with Assumptions 2, 3, 4, and 5, including the i.i.d. baseline with identical σ_i.

On the MEDIUM jury, POLL's RMSE exceeds ROPOLL's by one to three orders of magnitude at every r ≥ 10% on all three benchmarks.

“On the MEDIUM jury, POLL's RMSE exceeds ROPOLL's by one to three orders of magnitude at every r ≥ 10% on all three benchmarks.”

The empirical diminishing-returns knee at N = 3 in Figure 11 corroborates the prediction that N_eff saturates at N≈2-3.

“a prediction corroborated by the empirical diminishing-returns knee at N = 3 in Figure 11”

The coordinate-wise MEDIAN is competitive with ROPOLL under heavy-tailed Byzantine attacks.

“The coordinate-wise MEDIAN is competitive with ROPOLL here”

Naturally-occurring parser-failure rates span 0.59% on HelpSteer 2 to 33% on HelpSteer 3 multilingual.

“naturally-occurring rates span 0.59% on HelpSteer 2 to 33% on HelpSteer 3 multilingual”

Under cauchy-far corruption, the gap between ROPOLL and POLL reaches one to three orders of magnitude.

“under cauchy-far the gap reaches one to three orders of magnitude (see Figure 12)”

At the practical jury size N=3, the effective number of independent judges N_eff lies in [1.45, 1.58].

“γW is stable to within ±0.03 across cluster-radius calibrations and lies in [0.45, 0.53] on both benchmarks, so N eff ∈ [1.45, 1.58] at the practical jury size N =3.”

The total cost of the Weiszfeld iteration is O(N d log(1/ε)).

“With O(log(1/ϵ)) iterations the total cost is O(N d log(1/ϵ)).”

The minimax lower bound (Theorem 2) gives Ω(σ(√(d/N) + α/(1-α))).

“The minimax lower bound (Theorem 2) gives Ω(σ( d/N + α/(1 -α))).”

• cites Theorem 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Under the setup of Lemma 1, if at least (1-α)k of k points lie within distance r of z, then the geometric median lies within Cα r of z, where Cα=(1-α)/√(1-2α).

“Therefore the assumption ∥x * -z∥ 2 > C α r must fail, proving (16).■”

On the breakdown floor, the gap between the upper and lower bounds is of order √d/α.

“On the breakdown floor the upper bound scales as C α+β σ √ d while the lower bound scales as σα/(1-α), leaving a gap of order √ d/α.”

Under worst-case Huber contamination (alpha = 0.3, sigma = 0.3, d = 5, Dirac corruption at y* + 10*1), the geometric median converges to the predicted breakdown floor sigma alpha sqrt(d)/(1 - alpha) approximately 0.287 as N grows, matching Theorem 1.

“(b) Under worst-case Huber contamination (α=0.3,σ=0.3, d=5, Dirac corruption at y ⋆ + 10 1), the geometric median converges to the predicted breakdown floor σα √ d/(1-α) ≈ 0.287 (gray dashed) as N grows, matching Theorem 1”

• AssumptionThe authors state this as an assumption: Assumption 2: conditional on the truth y*, each judge's output follows a mixture (1 - alpha_i) P_i + alpha_i Q_i with selector Z_i ~ Bernoulli(alpha_i); the competent component is unbiased, E_Pi[y_i] = y*, and Q_i has finite first moments.
• AssumptionThe authors state this as an assumption: Assumption 4: each judge has a per-judge sub-Gaussian parameter sigma_i, and the analysis uses sigma = max_i sigma_i as the worst-case sub-Gaussian parameter.

A 3-judge ROPOLL committee at 38B beats Mistral-Large-3 (675B) by 1.31× on HelpSteer 2 under 30% bimodal-random corruption.

“a 3-judge ROPOLL committee at 38 B beats Mistral-Large-3 (675 B) by 1.31× on HelpSteer 2 under 30% bimodal-random corruption”

The bound with β=0.2 degenerates at small N_eff, including the practical N_eff≈1.5 of the N=3 panels.

“β = 0.2 gives a non-trivial bound) but degenerates at small N eff , including the practical N eff ≈ 1.5 of our N = 3 panels.”

Under the same worst-case Huber contamination, the arithmetic mean and under-trimmed mean (beta = alpha/2) plateau above the breakdown floor, confirming Proposition 2.

“the arithmetic mean and under-trimmed mean (β=α/2) plateau above the floor, confirming Proposition 2.”

• AssumptionThe authors state this as an assumption: Assumption 2: conditional on the truth y*, each judge's output follows a mixture (1 - alpha_i) P_i + alpha_i Q_i with selector Z_i ~ Bernoulli(alpha_i); the competent component is unbiased, E_Pi[y_i] = y*, and Q_i has finite first moments.

ROPOLL pays at most 6.4% in clean-baseline relative RMSE.

“paying ≤ 6.4% in clean-baseline relative RMSE”

ROPOLL reduces POLL's RMSE by orders of magnitude on heavy-tailed and cross-dimensional attacks.

“ROPOLL reduces POLL's RMSE by orders of magnitude on heavy-tailed and cross-dimensional attacks”

N_eff saturates at 1/γ.

“N eff saturates at 1/γ”

The geometric median is affine equivariant.

“The map z → Uz + b is a bijection on R d , so the two minimizers are related by exactly this transformation.”

Lemma 3 extends Theorem 1 to equicorrelated juries.

“Lemma 3 extends it to equicorrelated juries”

Under Assumptions 2-5, for any slack β ∈ (0, 1/2 - α), with probability at least 1 - exp(-N β^2/2), ∥ŷ_GM - y⋆∥2 ≤ C_{α+β} ρ, where ŷ_GM is any geometric median of the N judge outputs, C_{α+β} = (1-α-β)/√(1-2(α+β)), and ρ = σ(C1√d + (1/c) log(2(1-α)/β)).

“For convenience, we recall Theorem 1: under Assumptions 2-5, fix any slack β ∈ (0, 1/2 -α); with probability at least 1 -exp(-N β 2 /2), ∥ŷ GM -y ⋆ ∥ 2 ≤ C α+β ρ, where ŷGM is any geometric median of the N judge outputs”

• AssumptionThe authors state this as an assumption: Each judge output is generated by the noise decomposition ŷ_i = (1 - Z_i)(y⋆ + ϵ_i) + Z_i η_i, where Z_i ~ Bern(α) selects competent (Z_i=0) vs corrupted (Z_i=1).
• AssumptionThe authors state this as an assumption: The tuples {(Z_i, ϵ_i, η_i)} for i=1..N are mutually independent.
• AssumptionThe authors state this as an assumption: Conditional on Z_i=0, the competent noise ϵ_i ∈ R^d is σ-sub-Gaussian.

On the breakdown floor, the upper bound scales as C_{α+β} σ √d.

“On the breakdown floor the upper bound scales as C α+β σ √ d while the lower bound scales as σα/(1-α), leaving a gap of order √ d/α.”

On the breakdown floor, the lower bound scales as σα/(1-α).

“On the breakdown floor the upper bound scales as C α+β σ √ d while the lower bound scales as σα/(1-α), leaving a gap of order √ d/α.”

Holding N = 1000 fixed and sweeping the dimension, the geometric median tracks the predicted sqrt(d) scaling of the cluster radius (Lemma 2) to within an absolute constant.

“(c) Holding N =1000 fixed and sweeping the dimension, the geometric median tracks the predicted √ d scaling of the cluster radius (Lemma 2) to within an absolute constant.”

• AssumptionThe authors state this as an assumption: Assumption 4: each judge has a per-judge sub-Gaussian parameter sigma_i, and the analysis uses sigma = max_i sigma_i as the worst-case sub-Gaussian parameter.

Empirical MSE of the arithmetic mean for N = 10 equicorrelated clean Gaussian judges matches the closed-form prediction 1 + (N - 1) gamma / N * d sigma^2 (Corollary 1) across the full range gamma in [0, 0.95].

“(a) Empirical MSE of the arithmetic mean for N =10 equicorrelated clean Gaussian judges matches the closed-form prediction 1+(N -1)γ N dσ 2 (Corollary 1) across the full range γ ∈ [0, 0.95]”

Under Assumptions 2-5, for any slack β ∈ (0, 1/2 - α), with probability at least 1 - exp(-N β^2/2), at least (1 - α - β)N of the N judge outputs lie within distance ρ = σ(C1√d + (1/c) log(2(1-α)/β)) of y⋆.

“For convenience, we recall Lemma 2: under Assumptions 2-5, for any slack β ∈ (0, 1/2 -α), with probability at least 1 -exp(-N β 2 /2), at least (1 -α -β)N of the N judge outputs lie within distance ρ = σ C 1”

• cites Vershynin, 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Boucheron et al., 2013 sourceNot checked — no full text of the source

• AssumptionThe authors state this as an assumption: Each judge output is generated by the noise decomposition ŷ_i = (1 - Z_i)(y⋆ + ϵ_i) + Z_i η_i, where Z_i ~ Bern(α) selects competent (Z_i=0) vs corrupted (Z_i=1).
• AssumptionThe authors state this as an assumption: The tuples {(Z_i, ϵ_i, η_i)} for i=1..N are mutually independent.
• AssumptionThe authors state this as an assumption: Conditional on Z_i=0, the competent noise ϵ_i ∈ R^d is σ-sub-Gaussian.

ROPOLL's clean-case insurance premium is at most 6% relative RMSE.

“the clean-case insurance premium is small (≤ 6% relative RMSE, §6.5)”

In Figure 1, coordinate-wise MEDIAN is competitive with ROPOLL under heavy-tailed cauchy-far corruption for the MEDIUM jury (N=3, ≈89B).

“coordinate-wise MEDIAN is competitive with ROPOLL here and is omitted (full three-method comparison in Figure12).”

In random hypercube corners corruption, the per-coordinate marginal 1/2(δ_0+δ_K) is indistinguishable from plausible scoring.

“The per-coordinate marginal 1 2 (δ 0 + δ K ) is indistinguishable from plausible scoring”

• cites Figure 18Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The Claude family clusters near the bottom despite strong ranking ability.

“The Claude family clusters near the bottom despite strong ranking ability, with Instruction Following and Truthfulness showing the largest errors (> 1.0) due to systematic negative bias.”

In biased dimension corruption, the geometric median's joint-distance objective resists the off-axis pull that fools per-coordinate alternatives.

“the geometric median's joint-distance objective resists the off-axis pull that fools per-coordinate alternatives.”

• cites Figure 17Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Substituting (52) and (53) gives Var(Σ W_i) ≤ N/4 + N(N-1)γ_W/4 = N/4 [1+(N-1)γ_W] = N^2/(4 N_eff).

“Substituting (52) on the diagonal and (53) off-diagonal: Var i W i ≤ N • 1 4 + N (N -1) • γW 4 = N 4 1 + (N -1)γ W = N 2 4N eff”

Smaller open-weight models (Magistral Small, Gemma 4B, Nemotron 9B) show uniformly positive bias of comparable magnitude.

“Smaller open-weight models (Magistral Small, Gemma 4B, Nemotron 9B) show uniformly positive bias of comparable magnitude.”

When the corruption is benign Gaussian noise, the geometric median does not sacrifice accuracy.

“when the corruption is benign Gaussian noise, the geometric median does not sacrifice accuracy”

• cites §6.7Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In mode collapse corruption, as α grows, the arithmetic mean is pulled toward the origin.

“Figure 15 shows the mean pulled toward the origin as α grows”

• cites Figure 15Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In sycophantic corruption, the arithmetic mean drifts diagonally toward the corrupted cloud.

“the arithmetic mean drifts diagonally toward it”

• cites Figure 19Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Qwen3 32B achieves the lowest MAE across all four dimensions.

“Qwen3 32B achieves the lowest MAE across all four dimensions.”

The Claude family has strong ranking ability.

“despite strong ranking ability”

• cites Figure 21Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The Claude family shows uniformly negative bias across all dimensions.

“The Claude family shows uniformly negative bias across all dimensions”

In sycophantic corruption, the geometric median stays anchored to the competent majority near y*.

“while the geometric median stays anchored to the competent majority near y ⋆ .”

• cites Figure 19Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Under Assumptions 2, 4, 5 and the equicorrelated-indicator assumption replacing Assumption 3, the ROPOLL bound ∥ŷ_GM - y⋆∥_2 ≤ C_{α+β} ρ holds with probability at least 1 - 1/(β^2 N_eff), where N_eff = N/(1+(N-1)γ_W).

“the ROPOLL bound ∥ŷ GM -y ⋆ ∥ 2 ≤ C α+β ρ holds with probability at least 1 -1/(β 2 N eff ), where N eff = N/(1 + (N -1)γ W ).”

• cites Lemma 3Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionThe authors state this as an assumption: Assumption 2: Pr[Z_i=0]=1-α_i.
• AssumptionThe authors state this as an assumption: Assumption 5: α = N^{-1} Σ α_i.
• AssumptionThe authors state this as an assumption: Equicorrelated-indicator assumption: Cov(W_i,W_j) ≤ γ_W sqrt(Var(W_i)Var(W_j)) for i≠j, with γ_W∈[0,1].

The cluster radius ρ equals σ C_1 (√d + (1/c) log(2(1-α)/β)).

“ρ = σ C 1 √ d + 1 c log 2(1-α) β .”

• cites Lemma 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The advantage of the geometric median over the arithmetic mean is paid only against biased contamination.

“this advantage is paid only against biased contamination”

• cites §6.7Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In random hypercube corners corruption, the arithmetic mean drifts toward the centroid of the corrupted vertices.

“while the arithmetic mean drifts toward the centroid of the corrupted vertices.”

• cites Figure 18Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

At α = 0.30 the geometric median remains close to y⋆.

“At α = 0.30 the mean is already displaced past the midpoint, while the geometric median remains close to y ⋆ .”

In biased dimension corruption, each corrupted score is plausible per coordinate but jointly anomalous.

“each corrupted score is plausible per coordinate but jointly anomalous”

• cites Figure 17Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

By the tower rule, E W_i = Pr[Z_i=0] • Pr[∥ε_i∥_2 ≤ ρ_p | Z_i=0] ≥ (1-α_i)(1-p), using Pr[Z_i=0]=1-α_i from Assumption 2 and the per-sample tail Pr[∥ε_i∥_2 ≤ ρ_p | Z_i=0] ≥ 1-p from Step 1 of Lemma 2's proof.

“By the tower rule, EW i = Pr[Z i = 0] • Pr[∥ϵ i ∥ 2 ≤ ρ p | Z i = 0] ≥ (1 -α i )(1 -p), using Pr[Z i = 0] = 1 -α i from Assumption 2 and the per-sample tail Pr[∥ϵ i ∥ 2 ≤ ρ p | Z i = 0] ≥ 1 -p from Step 1 of Lemma 2's proof (which is correlation-free).”

• cites Assumption 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Lemma 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionThe authors state this as an assumption: Assumption 2: Pr[Z_i=0]=1-α_i.

The Claude family lies near the bottom of the calibration ranking.

“the Claude family lies near the bottom of the calibration ranking”

• cites Figure 20Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Figure 21Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Qwen3 32B and Qwen3 235B are closest to zero across all dimensions.

“Qwen3 32B and Qwen3 235B are closest to zero across all dimensions.”

The probability that the sum of weights W_i is less than (1-α-β)N is at most 1/(β^2 N_eff).

“Pr N i=1 W i < (1 -α -β)N ≤ 1 β 2 N eff ,(57)”

• cites [20]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In mode collapse corruption, the geometric median remains anchored to the competent cluster.

“while the geometric median remains anchored to the competent cluster.”

• cites Figure 15Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In random hypercube corners corruption, the geometric median resists the cross-dimensional pull and sits at y*.

“The geometric median resists the cross-dimensional pull (it sits at y ⋆ , beneath the gold star)”

• cites Figure 18Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

On the complementary event of (57), which has probability at least 1 - 1/(β^2 N_eff), the number of judges within distance ρ of y* is at least (1-α-β)N.

“i ∈ [N ] : ∥ŷ i -y ⋆ ∥ 2 ≤ ρ ≥ N i=1 W i ≥ (1 -α -β)N = (1 -(α + β))N.”

• cites [57]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Qwen3 32B and Mistral-Large-3 lead with sub-0.75 MAE across all four Ultra-Feedback rubric dimensions.

“Qwen3 32B and Mistral-Large-3 lead with sub-0.75MAE across all four dimensions”

• cites Figure 20Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Instruction Following and Truthfulness show the largest errors (> 1.0) for the Claude family.

“The Claude family clusters near the bottom despite strong ranking ability, with Instruction Following and Truthfulness showing the largest errors (> 1.0) due to systematic negative bias.”

At α = 0.40 the mean lies roughly 40% of the way from y⋆ to 0.

“The mean is pulled linearly toward the origin; at α = 0.40 it lies roughly 40% of the way from y ⋆ to 0.”

The largest errors for the Claude family are due to systematic negative bias.

“The Claude family clusters near the bottom despite strong ranking ability, with Instruction Following and Truthfulness showing the largest errors (> 1.0) due to systematic negative bias.”

In inverted corruption, at α = 0.30, the arithmetic mean has already crossed the midpoint.

“at α = 0.30 the mean has already crossed the midpoint”

• cites Figure 16Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The mean is pulled linearly toward the origin.

“The mean is pulled linearly toward the origin; at α = 0.40 it lies roughly 40% of the way from y ⋆ to 0.”

Models like Magistral Small and Gemma 4B exhibit strong positive bias.

“while models like Magistral Small and Gemma 4B exhibit strong positive bias.”

At α = 0.30 the mean is already displaced past the midpoint.

“At α = 0.30 the mean is already displaced past the midpoint, while the geometric median remains close to y ⋆ .”

Applying Chebyshev to X=Σ W_i with mean μ_N and using (54): Pr[Σ W_i ≤ μ_N - uN] ≤ Var(Σ W_i)/(uN)^2 ≤ 1/(4u^2 N_eff).

“Applying this to X = i W i with mean µ N and using (54): Pr i W i ≤ µ N -uN ≤ Var( i W i ) (uN ) 2 ≤ N 2 /(4N eff ) u 2 N 2 = 1 4u 2 N eff .(55)”

Summing over i and using α = N^{-1} Σ α_i (Assumption 5), μ_N ≜ Σ E W_i ≥ N(1-α)(1-p).

“Summing over i and using α = N -1 i α i (Assumption 5): µ N ≜ N i=1 EW i ≥ N i=1 (1 -α i )(1 -p) = N (1 -α)(1 -p).”

• cites Assumption 5Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionThe authors state this as an assumption: Assumption 5: α = N^{-1} Σ α_i.

The geometric median remains within the competent cluster because the majority of Euclidean distances still point toward y⋆.

“The geometric median remains within the competent cluster because the majority of Euclidean distances still point toward y ⋆ .”

In random hypercube corners corruption, jointly every corrupted vector sits at a corner far from y* in l_2.

“jointly, every corrupted vector sits at a corner far from y ⋆ in ℓ 2 .”

• cites Figure 18Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The Huber ε-contamination model admits all four failure modes as specific instantiations of the contamination distribution Q_i (zeros, inverted, bimodal-random, and cauchy-far).

“The Huber ϵ-contamination model (Assumption 2) admits all four failure modes as specific instantiations of the contamination distribution Q i (zeros, inverted, bimodal-random, and cauchy-far; mapped explicitly in §3.4 and evaluated in §6.2-6.4), and a direct calculation (Proposition 2, §3.6) shows that under any positive contamination rate POLL's conditional bias grows linearly with the corruption”

• AssumptionThe authors state this as an assumption: Judge failures follow the Huber ε-contamination model.

The deployment regime is dataset-dependent across one to two orders of magnitude in α.

“The deployment regime is therefore dataset-dependent across one to two orders of magnitude in α”

Systematic biases of single judges propagate uncorrected to every score.

“The systematic biases its backbone exhibits, e.g., position, verbosity, self-enhancement, sycophancy, and refusal artefacts, are by now well documented (Wang et al., 2023;Panickssery et al., 2024;Saito et al., 2023;Stureborg et al., 2024); they propagate uncorrected to every score, and the cost-quality profile of the resulting evaluation is fixed to that of the single model.”

POLL is statistically optimal under benign Gaussian imprecision.

“A Noisy-GT control confirms the premium is paid against biased contamination, not benign Gaussian imprecision (where POLL is statistically optimal).”

The cost-quality profile of an evaluation by a single judge is fixed to that of the single model.

“The systematic biases its backbone exhibits, e.g., position, verbosity, self-enhancement, sycophancy, and refusal artefacts, are by now well documented (Wang et al., 2023;Panickssery et al., 2024;Saito et al., 2023;Stureborg et al., 2024); they propagate uncorrected to every score, and the cost-quality profile of the resulting evaluation is fixed to that of the single model.”

The correlated-jury extension holds under the equicorrelated-indicator hypothesis (Lemma 3).

“a correlated-jury extension under the equicorrelated-indicator hypothesis (Lemma 3, with empirical indicator-correlation γW ∈ [0.45, 0.53] measured on our judge panels, Section 6.10)”

• AssumptionThe authors state this as an assumption: The equicorrelated-indicator hypothesis holds for the correlated-jury extension.

Among classical robust estimators (CoMed, trimmed mean, GM), only GM is simultaneously tuning-free, joint-distance preserving, and attains the optimal 1/2 breakdown point.

“Among classical candidates-the coordinate-wise median (CoMed), the trimmed mean, and the geometric median (GM)-only GM is simultaneously tuning-free (no contamination-rate hyperparameter unlike the trimmed mean), joint-distance preserving (operates on Euclidean distance over the full score vector unlike CoMed, which decouples coordinates and misses cross-attribute structure of Example 1), and atta”

The LLM-jury setting shares the mathematical structure of Byzantine-robust distributed learning.

“The LLM-jury setting shares the mathematical structure but differs operationally on three axes”

Real LLM judges fail in ways that are nothing like Gaussian noise.

“Real LLM judges fail in ways that are nothing like Gaussian noise.”

The boundedness of the response scale makes the sub-Gaussian assumption non-restrictive in our setting.

“The boundedness of the response scale therefore makes the sub-Gaussian assumption non-restrictive in our setting.”

The paper's setting has heterogeneous workers (per-judge σ_i, α_i vary across the panel, outside the i.i.d. regime that the classical robust-statistics literature targets).

“(iii) heterogeneous workers (per-judge σ i , α i vary across the panel, outside the i.i.d.regime that the classical robust-statistics literature targets)”

When judge errors are light-tailed and centered on the truth, averaging N judges contracts the variance at the parametric rate 1/N.

“POLL is the optimal aggregator precisely when judge errors are light-tailed and centered on the truth, in which case averaging N judges contracts the variance at the parametric rate 1/N (Proposition 1, §3.5); Figure 10 ( §6.5) shows the clean-baseline parameter-efficiency this delivers empirically.”

No prior work analyzes the aggregation step or its failure modes.

“no prior work analyzes the aggregation step or its failure modes.”

ROPOLL's finite-sample error bound and minimax lower bound differ on the breakdown floor by a factor of √d.

“We establish a finite-sample error bound and an information-theoretic minimax lower bound that match on the parametric rate σ d/N and differ on the breakdown floor by a factor of √ d -a statistical-computational gap that polynomial-time ROPOLL pays relative to the (intractable) Tukey halfspace median.”

In the LLM-jury setting, contamination is per-sample heterogeneous: the contamination indicator Z_i is conditional on the prompt-response pair x, not per round.

“(b) per-sample heterogeneity (the contamination indicator Z i is conditional on the prompt-response pair x, not per round);”

A single judge is a single point of statistical failure.

“A single judge, however, is a single point of statistical failure.”

The key gap across this literature is the absence of any analysis of catastrophic failure modes or formal robustness properties of the aggregation rule.

“The key gap across this literature is the absence of any analysis of catastrophic failure modes or formal robustness properties of the aggregation rule.”

The law of ŷf(x) may contain non-trivial point masses even when Tf has a diffuse generation law.

“the law of ŷf (x) may contain non-trivial point masses even when T f has a diffuse generation law.”

• AssumptionRests on an assumption: In practice, the parser ϕ maps malformed outputs, refusals, or missing fields to a fixed fallback vector such as 0.

The geometric median's tuning-free 1/2 breakdown, joint-distance objective, and O(N d log(1/ε)) cost make it the right default for the small-d, small-N, heterogeneous-worker, one-shot regime that LLM juries occupy.

“the geometric median's tuning-free 1/2 breakdown, joint-distance objective, and O(N d log(1/ϵ)) cost make it the right default for the small-d, small-N , heterogeneous-worker, one-shot regime that LLM juries occupy.”

The finite-sample upper bound for ROPOLL is ∥ŷGM - y⋆∥2 ≤ C α+β ρ with explicit absolute constants (Theorem 1).

“a finite-sample upper bound ∥ŷ GM -y ⋆ ∥ 2 ≤ C α+β ρ with explicit absolute constants (Theorem 1)”

Under any positive contamination rate, POLL's conditional bias grows linearly with the corruption shift and is unbounded over the corruption class, regardless of N.

“a direct calculation (Proposition 2, §3.6) shows that under any positive contamination rate POLL's conditional bias grows linearly with the corruption shift and is unbounded over the corruption class, regardless of N : the 1/N variance reduction that motivates juries cannot rescue an aggregator whose bias is itself unbounded.”

• AssumptionThe authors state this as an assumption: Judge failures follow the Huber ε-contamination model.

The 1/N variance reduction that motivates juries cannot rescue an aggregator whose bias is itself unbounded.

“the 1/N variance reduction that motivates juries cannot rescue an aggregator whose bias is itself unbounded.”

• AssumptionThe authors state this as an assumption: Judge failures follow the Huber ε-contamination model.

Under the Huber contamination model, POLL incurs unbounded bias under any positive contamination, regardless of jury size, whenever a single judge fails in a biased, LLM-typical way (mode collapse, sycophancy, safety refusal).

“We formalize the LLM Jury setup under the Huber contamination model, and show that POLL incurs unbounded bias under any positive contamination, regardless of jury size, whenever a single judge fails in a biased, LLM-typical way (mode collapse, sycophancy, safety refusal).”

• AssumptionThe authors state this as an assumption: Judge failures follow the Huber ε-contamination model.

Even under deterministic API settings (temperature τ = 0), the induced law f(•|x) need not be degenerate.

“Even under deterministic API settings (temperature τ = 0), the induced law f (• | x) need not be degenerate.”

ROPOLL's finite-sample error bound and information-theoretic minimax lower bound match on the parametric rate σ d/N.

“We establish a finite-sample error bound and an information-theoretic minimax lower bound that match on the parametric rate σ d/N and differ on the breakdown floor by a factor of √ d -a statistical-computational gap that polynomial-time ROPOLL pays relative to the (intractable) Tukey halfspace median.”

POLL is the optimal aggregator precisely when judge errors are light-tailed and centered on the truth.

“POLL is the optimal aggregator precisely when judge errors are light-tailed and centered on the truth, in which case averaging N judges contracts the variance at the parametric rate 1/N (Proposition 1, §3.5); Figure 10 ( §6.5) shows the clean-baseline parameter-efficiency this delivers empirically.”

The √d gap is a statistical-computational gap that polynomial-time ROPOLL pays relative to the (intractable) Tukey halfspace median.

“We establish a finite-sample error bound and an information-theoretic minimax lower bound that match on the parametric rate σ d/N and differ on the breakdown floor by a factor of √ d -a statistical-computational gap that polynomial-time ROPOLL pays relative to the (intractable) Tukey halfspace median.”

• AssumptionRests on an assumption: The Tukey halfspace median is intractable.

If Σ_i ⪯ σ^2 I_d uniformly, then E||ŷ_mean - y⋆||^2 ≤ dσ^2/N.

“If Σ i ⪯ σ 2 I d uniformly, then E ∥ŷ mean -y ⋆ ∥ 2 2 y ⋆ ≤ dσ 2 /N .”

Robust aggregation of a small, diverse committee is a parameter-efficient and statistically principled alternative to scaling a single large LLM-as-judge.

“Overall, we establish that robust aggregation of a small, diverse committee is a parameter-efficient and statistically principled alternative to scaling a single large LLM-as-judge.”

The geometric median pays a √d price for O(N d log(1/ε)) tractability via the Weiszfeld iteration.

“but pays a √ d price for O(N d log(1/ϵ)) tractability via the Weiszfeld iteration ( §4.3).”

On a bounded scale, POLL is mean-preserving under uniform-rate corruption when the corrupted point happens to lie at the scale midpoint.

“On a bounded scale POLL is mean-preserving under uniform-rate corruption when the corrupted point happens to lie at the scale midpoint”

The difference between the upper and lower bounds on the breakdown floor is not a slack in the analysis but a real statistical-computational gap.

“This is not a slack in the analysis but a real statisticalcomputational gap.”

• AssumptionThe authors state this as an assumption: The joint distributions of judge estimates are consistent with Assumptions 2, 3, 4, and 5, including the i.i.d. baseline with identical σ_i.

The mean-preservation of POLL under bounded corruption at the scale midpoint is an empirical accident, not a property of the mean as an estimator.

“an empirical accident, not a property of the mean as an estimator”

The geometric-breakdown structure (Cα+β and the cluster radius ρ) of Theorem 1 is preserved when the i.i.d. assumption is replaced by an equicorrelated-indicator hypothesis with γW ∈ [0, 1].

“it shows that the geometricbreakdown structure (C α+β and the cluster radius ρ) of Theorem 1 is preserved when the i.i.d.assumption is replaced by an equicorrelated-indicator hypothesis with γW ∈ [0, 1].”

• AssumptionThe authors state this as an assumption: Judge failures are equicorrelated indicators with γW ∈ [0, 1].

Under the equicorrelated jury model of Corollary 1, the effective jury size N_eff = N/(1+(N-1)γ) saturates at 1/γ as N→∞.

“under the equicorrelated jury model of Corollary 1 the effective jury size N eff = N/(1 + (N -1)γ) saturates at 1/γ as N → ∞”

• AssumptionThe authors state this as an assumption: The inter-judge correlations in the jury follow the equicorrelated model of Corollary 1.

Under heavy-tailed Byzantine attacks, any robust aggregator beats the mean.

“under heavy-tailed Byzantine attacks any robust aggregator beats the mean, exactly as the theory predicts.”

Robust aggregation, not the ensemble itself, is what delivers the win.

“Robust aggregation, not the ensemble itself, is what delivers the win.”

For any γ > 0 the effective jury size N_eff saturates at 1/γ, so adding more judges past N ≈ 1/γ buys essentially nothing.

“The corollary has a direct implication for jury design: for any γ > 0 the effective jury size N eff saturates at 1/γ, so adding more judges past N ≈ 1/γ buys essentially nothing.”

The ROPOLL bound is distribution-free over the corruption class {Q_i}.

“The bound is distribution-free over the corruption class {Q i }.”

The geometric median shares the optimal 1/2 breakdown point.

“it shares the optimal 1/2 breakdown point”

With γ in the moderate range [0.3,0.5] characteristic of diverse but non-orthogonal LLM backbones, N_eff saturates already at N ≈ 2-3, motivating the three-judge committees throughout §6.

“With γ in the moderate range [0.3, 0.5] characteristic of diverse but non-orthogonal LLM backbones, N eff saturates already at N ≈ 2-3, motivating the three-judge committees throughout §6.”

For any α > 0 the conditional bias E[ŷ_mean | y⋆] - y⋆ cannot be uniformly bounded under Assumption 2, regardless of N.

“and for any α > 0 the conditional bias E[ŷ mean | y ⋆ ] -y ⋆ cannot be uniformly bounded under Assumption 2, regardless of N .”

In the cauchy-far attack, each corrupted slot has an unbounded first moment.

“each corrupted slot has an unbounded first moment”

In the cauchy-far attack, a single contaminated judge can in principle drag POLL by an arbitrary amount.

“a single contaminated judge can in principle drag POLL by an arbitrary amount”

Theorems 1 and 2 match exactly on the parametric rate σ d/N.

“Theorems 1 and 2 match exactly on the parametric rate σ d/N .”

• AssumptionThe authors state this as an assumption: The joint distributions of judge estimates are consistent with Assumptions 2, 3, 4, and 5, including the i.i.d. baseline with identical σ_i.

The regimes where corruption is bounded and mean-preserving make ROPOLL and POLL hardest to separate.

“which makes these the regimes where ROPOLL and POLL should be hardest to separate.”

For moderate inter-judge correlation γ ∈ [0.3, 0.5] characteristic of diverse but non-orthogonal LLM backbones, N_eff saturates already at N ≈ 2-3.

“for the moderate inter-judge correlation γ ∈ [0.3, 0.5] characteristic of diverse but non-orthogonal LLM backbones, N eff saturates already at N ≈ 2-3”

• AssumptionRests on an assumption: The moderate inter-judge correlation γ ∈ [0.3, 0.5] is characteristic of diverse but non-orthogonal LLM backbones.
• AssumptionThe authors state this as an assumption: The inter-judge correlations in the jury follow the equicorrelated model of Corollary 1.

No estimator can do better on the parametric variance term.

“No estimator can do better on the parametric variance term”

• AssumptionThe authors state this as an assumption: The joint distributions of judge estimates are consistent with Assumptions 2, 3, 4, and 5, including the i.i.d. baseline with identical σ_i.

Under arbitrary Q in the Huber class, the asymptotic-N floor of the geometric median error is set by the cluster radius, not by sample averaging.

“This reflects the breakdown-point character of plain GM: under arbitrary Q in the Huber class, the asymptotic-N floor is set by the cluster radius, not by sample averaging.”

The well-centered distribution of HelpSteer 3 is consistent with the cancellation of per-attribute biases under the signed-difference reduction.

“consistent with the cancellation of per-attribute biases under the signed-difference reduction.”

The ROPOLL premium reported in §6.2-6.4 is paid against biased contamination, not imprecision.

“This rules out the most obvious confound: the ROPOLL premium reported in § §6.2-6.4 is paid against biased contamination, not imprecision.”

The probability event delivered by the Chebyshev step, Pr[•] ≥ 1 - 1/(β2 Neff), is informative in the large-Neff regime.

“The probability event delivered by the Chebyshev step, Pr[•] ≥ 1 -1/(β 2 N eff ), is informative in the large-N eff regime”

• AssumptionThe authors state this as an assumption: Judge failures are equicorrelated indicators with γW ∈ [0, 1].

The cost of using ROPOLL when corruption is absent is a small fraction of the gains it delivers when corruption is present.

“so the cost of using ROPOLL when corruption happens to be absent is a small fraction of the gains it delivers when corruption is present.”

Under Assumption 2, if each Q_i has finite first moment μ_{Q_i} ≜ E_{Q_i}[ŷ_i] ∈ R^d, then E[ŷ_mean | y⋆] = y⋆ + 1/N Σ_i α_i (μ_{Q_i} - y⋆).

“Under Assumption 2, suppose each Q i has finite first moment µ Q i ≜ E Qi [ŷ i ] ∈ R d . Then E[ŷ mean | y ⋆ ] = y ⋆ + 1 N N i=1 α i µ Q i -y ⋆ ,(12)”

If β > α, competent points are discarded, inflating variance.

“if β > α, competent points are discarded, inflating variance.”

The β-trimmed mean requires choosing β, which in turn requires knowledge of the contamination rate α.

“It requires choosing β, which in turn requires knowledge of the contamination rate α:”

Proposition 2 is the central impossibility motivating ROPOLL: variance reduction over N judges is irrelevant when the bias of the aggregator is unbounded over the corruption class.

“Proposition 2 is the central impossibility motivating ROPOLL: variance reduction over N judges is irrelevant when the bias of the aggregator is unbounded over the corruption class.”

If β < α, corrupted points survive trimming and the bias of Proposition 2 returns.

“if β < α, corrupted points survive trimming and the bias of Proposition 2 returns;”

Each Weiszfeld iteration is a reweighted mean in which points far from the current consensus receive small weights, automatically downweighting corrupted judges.

“Each iteration is a reweighted mean in which points far from the current consensus receive small weights, automatically downweighting corrupted judges.”

The empirical diminishing-returns knee at N = 3 in Figure 11 sits at or just past the saturation point predicted by the corpus's actual correlation structure.

“so the empirical diminishing-returns knee at N = 3 in Figure 11 sits at or just past the saturation point predicted by the corpus's actual correlation structure.”

The geometric median is the polynomial-time alternative.

“The geometric median is the polynomial-time alternative”

The picture in Example 1 extends to a class of corruptions that are bounded per-coordinate but jointly anomalous: random vertices of the score hypercube {0,K}^d, mixtures of extreme corner values, or any corruption whose marginals look plausible but whose joint structure is not.

“The picture in Example 1 extends to a class of corruptions that are bounded per-coordinate but jointly anomalous: random vertices of the score hypercube {0, K} d , mixtures of extreme corner values, or any corruption whose marginals look plausible but whose joint structure is not.”

The parameter-efficiency advantage holds because the geometric median's joint-distance objective extracts more signal from the same forward passes than POLL.

“the parameter-efficiency advantage holds because the geometric median's joint-distance objective extracts more signal from the same forward passes than POLL.”

The cross-dimensional structure (Example 1) is invisible to a per-coordinate median.

“where the cross-dimensional structure (Example 1) is invisible to a per-coordinate median.”

Each corrupted score under the bimodal-random attack is plausible per coordinate but jointly anomalous.

“each corrupted score is plausible per coordinate but jointly anomalous”

Any 3-judge committee (114 B-267 B forward-pass compute) beats a single 675 B judge at evaluation cost.

“any 3-judge committee (114 B-267 B forward-pass compute) beats a single 675 B judge at evaluation cost.”

The √d overhead is at most approximately 2.2×.

“so the √ d overhead is at most ∼ 2.2×”

The three-judge committees sit at the knee of the cost-accuracy frontier, confirming the Corollary 1 prediction.

“confirming the Corollary 1 prediction that the three-judge committees sit at the knee of the cost-accuracy frontier.”

• AssumptionThe authors state this as an assumption: The inter-judge correlation γ lies in [0.3, 0.5].

The cauchy-far attack is the empirical analogue of the adversarial choice in Proposition 2.

“The cauchy-far attack is the empirical analogue of the adversarial choice in Proposition 2”

The construction in the proof scales the corruption mean linearly with N, exactly cancelling the 1/N averaging-so increasing the jury size cannot fix the problem.

“The construction in the proof scales the corruption mean linearly with N , exactly cancelling the 1/N averaging-so increasing the jury size cannot fix the problem.”

The slack β trades off the geometric constant C_{α+β}, which grows with β, against the cluster radius ρ, which shrinks with β.

“The slack β trades two terms in (19): the geometric constant C α+β grows with β (since β erodes the safety margin to the breakdown point 1/2), while the cluster radius ρ shrinks with β (since a larger slack absorbs more competent samples, allowing a smaller per-sample tail).”

With only about 1.5 effective independent samples, no concentration argument can deliver a tight high-probability bound, regardless of the estimator.

“This is a fundamental limit of variance-only concentration at small N , not a slack in the analysis: with only ∼ 1.5 effective independent samples, no concentration argument can deliver a tight high-probability bound, regardless of the estimator.”

When the overlap mass exceeds alpha/(1 - alpha), the two Huber neighborhoods touch.

“The overlap is shaded; when the overlap mass exceeds α/(1-α), the two Huber neighborhoods touch.”

Under the threshold above, a single distribution F in F_alpha(y0) intersection F_alpha(y1) is consistent with both truths.

“under the threshold above, a single distribution F ∈ F α (y 0 ) ∩ F α (y 1 ) is consistent with both truths.”

The adversary's per-judge displacement grows linearly with the jury size, exactly cancelling the 1/N averaging.

“The construction (24) scales µ Q i0 with N : the adversary's per-judge displacement grows linearly with the jury size, exactly cancelling the 1/N averaging.”

Under arbitrary Q in the Huber class, the asymptotic-N floor of plain GM is set by the cluster radius, not by sample averaging.

“This reflects the breakdown-point character of plain GM: under arbitrary Q in the Huber class, the asymptotic-N floor is set by the cluster radius, not by sample averaging.”

Applying Lemma 1 with the stated substitutions gives ∥ŷ GM - y⋆∥2 ≤ C_{α+β} ρ with C_{α+β} and ρ unchanged from Theorem 1.

“The conclusion gives ∥ŷ GM -y ⋆ ∥ 2 ≤ C α+β ρ with C α+β and ρ unchanged from Theorem 1.”

∆⋆ ≥ √(2π) σ α/(1-α).

“Therefore ∆ ⋆ = 2σ Φ -1 1 2 + α 2(1-α) ≥ 2σ • π 2 α 1 -α = √ 2π σ α 1 -α .”

The geometric median error bound (19) holds with probability at least 1 - exp(-N β^2/2) under the stated model.

“Pr ∥ŷ GM -y ⋆ ∥ 2 ≤ 1 -α -β 1 -2(α + β) C α+β • σ C 1 √ d + 1 c log 2(1-α) β ρ ≥ 1 -exp(-N β 2 /2), which is exactly (19).■”

• AssumptionThe authors state this as an assumption: The observation model (8) holds for the judges.
• AssumptionThe authors state this as an assumption: There are N judges in R^d.
• AssumptionThe authors state this as an assumption: The contamination rate α is homogeneous and α < 1/2.
• AssumptionThe authors state this as an assumption: Competent noise is σ^2-sub-Gaussian.
• AssumptionThe authors state this as an assumption: The contamination distribution Q is arbitrary probability on R^d in the Huber class.

Lemma 1 forces the geometric median y_GM to lie inside the GM envelope B(y*, C_{alpha+beta} rho), regardless of where the remaining (alpha+beta)N corrupted points are placed; this is the distribution-free breakdown property of the geometric median.

“Lemma 1 then forces the geometric median ŷGM (blue triangle) to lie inside the GM envelope B(y ⋆ , C α+β ρ) (dashed disk), regardless of where the remaining (α+β)N corrupted points (red ×) are placed-this is the distribution-free breakdown property of the geometric median.”

• AssumptionThe authors state this as an assumption: Assumption 2: conditional on the truth y*, each judge's output follows a mixture (1 - alpha_i) P_i + alpha_i Q_i with selector Z_i ~ Bernoulli(alpha_i); the competent component is unbiased, E_Pi[y_i] = y*, and Q_i has finite first moments.
• AssumptionThe authors state this as an assumption: Assumption 4: each judge has a per-judge sub-Gaussian parameter sigma_i, and the analysis uses sigma = max_i sigma_i as the worst-case sub-Gaussian parameter.

Lemma 1 applies to any minimizer, so the conclusion is independent of any choice in the collinear case where the geometric median is non-unique.

“Lemma 1 as proved in §A.5 applies to any minimizer, so the conclusion is independent of any choice in the (collinear) case where the GM is non-unique.”

For each judge i, the conditional bias is E[y_i | y*] - y* = alpha_i (mu_Q_i - y*).

“Rearranging, E[ŷ i | y ⋆ ] -y ⋆ = α i (µ Q i -y ⋆ ).(23)”

• AssumptionThe authors state this as an assumption: Assumption 2: conditional on the truth y*, each judge's output follows a mixture (1 - alpha_i) P_i + alpha_i Q_i with selector Z_i ~ Bernoulli(alpha_i); the competent component is unbiased, E_Pi[y_i] = y*, and Q_i has finite first moments.

Fano's inequality gives, for any estimator ŷ, (1/M) Σ Pr_{H_m}[||ŷ - y_m||_2 ≥ ∆/2] ≥ 1 - (KL + log 2)/log M.

“Fano's inequality (Tsybakov, 2009, Cor.2.6) gives, for any estimator ŷ, 1 M M m=1 Pr Hm ∥ŷ -y m ∥ 2 ≥ ∆/2 ≥ 1 - KL + log 2 log M ,(61)”

• cites Tsybakov, 2009Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Theorem 2: Under observation model (8) with N judges in R^d, homogeneous contamination rate α < 1/2, and σ^2-sub-Gaussian competent noise, there exists a universal constant c > 0 such that inf_ŷ sup_{F∈F_{α,σ}} E_F ||ŷ - y⋆||_2 ≥ c σ (√(d/N) + α/(1-α)).

“under the observation model (8) with N judges in R d , homogeneous contamination rate α < 1/2, and σ 2 -sub-Gaussian competent noise (Assumptions 2, 3, 4, 5), there exists a universal constant c > 0 such that inf ŷ sup F ∈Fα,σ E F ŷ -y ⋆ 2 ≥ c σ d/N + α 1 -α .(58)”

• AssumptionThe authors state this as an assumption: The observation model (8) holds for the judges.
• AssumptionThe authors state this as an assumption: There are N judges in R^d.
• AssumptionThe authors state this as an assumption: The contamination rate α is homogeneous and α < 1/2.
• AssumptionThe authors state this as an assumption: Competent noise is σ^2-sub-Gaussian.
• AssumptionThe authors state this as an assumption: The contamination distribution Q is arbitrary probability on R^d in the Huber class.

The √d in the upper bound comes from the geometric median's cluster radius (Lemma 2), reflecting the price plain GM pays for O(N d log(1/ε)) tractability relative to the (intractable) Tukey halfspace median or the (sub-exponential) smoothed-depth estimator.

“The √ d in the upper bound comes from the geometric median's cluster radius (Lemma 2), reflecting the price plain GM pays for O(N d log(1/ϵ)) tractability relative to the (intractable) Tukey halfspace median or the (sub-exponential) smoothed-depth estimator.”

• cites Lemma 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Total variation between two equal-covariance Gaussians at separation Delta* is 2 Phi(Delta*/(2 sigma)) - 1, dimension-free and depending only on the line through the two centers.

“Top: total variation between two equal-covariance Gaussians at separation ∆ ⋆ is 2Φ(∆ ⋆ /2σ) -1 (dimension-free; depends only on the line through the two centers).”

The gap between upper and lower bounds on the breakdown floor is not slack in the analysis but a real statistical-computational gap.

“This is not slack in the analysis but a real statistical-computational gap.”

By the Gilbert-Varshamov bound, for radius ∆ > 0 there exists a packing {y_1,...,y_M} ⊂ R^d with pairwise distance ≥ ∆ and M ≥ 2^{d/8}.

“by the Gilbert-Varshamov bound (Massart, 2007, Lem. 4.7) there exists a packing {y 1 , . . ., y M } ⊂ R d with ∥y m -y m ′ ∥ 2 ≥ ∆ for all m ̸ = m ′ , M ≥ 2 d/8 .(60)”

• cites Massart, 2007 sourceNot checked — no full text of the source

If competent scores are bounded in [0,K]^d, the cluster radius ρ in (19) can be replaced by the deterministic worst-case K√d, removing the (1/c) log(2(1-α)/β) term and the high-probability event for the cluster.

“If competent scores are bounded in [0, K] d (Remark 15), the cluster radius ρ in (19) can be replaced by the deterministic worst-case K √ d, removing the (1/c) log(2(1 -α)/β) term and the high-probability event for the cluster.”

• cites Remark 15Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionThe authors state this as an assumption: Competent scores are bounded in [0,K]^d.

Robust aggregation, not the ensemble itself, delivers the win.

“robust aggregation, not the ensemble itself, delivers the win.”

Proposition 2 shows that the arithmetic mean's bias is unbounded over the corruption class (Assumption 2) regardless of N, so any variance reduction the jury affords is dominated by an adversarial choice of {Q_i}.

“Proposition 2 shows why this benefit is insufficient under contamination: the arithmetic mean's bias is unbounded over the corruption class (Assumption 2) regardless of N , so any variance reduction the jury affords is dominated by an adversarial choice of {Q i }.”

• AssumptionThe authors state this as an assumption: Assumption 2: conditional on the truth y*, each judge's output follows a mixture (1 - alpha_i) P_i + alpha_i Q_i with selector Z_i ~ Bernoulli(alpha_i); the competent component is unbiased, E_Pi[y_i] = y*, and Q_i has finite first moments.

Φ^{-1}(1/2 + y) ≥ y√(2π) for all y ∈ [0, 1/2).

“Hence Φ -1 (1/2 + y) ≥ y √ 2π for all y ∈ [0, 1/2).”

Under the substitutions of Lemma 1, the geometric median satisfies ||y_GM - y*||_2 <= C_{alpha+beta} rho.

“The conclusion of Lemma 1 under the substitutions (50) is ∥x * -z∥ 2 ≤ C α+β r i.e. ∥ŷ GM -y ⋆ ∥ 2 ≤ C α+β ρ”

• AssumptionThe authors state this as an assumption: Assumption 2: conditional on the truth y*, each judge's output follows a mixture (1 - alpha_i) P_i + alpha_i Q_i with selector Z_i ~ Bernoulli(alpha_i); the competent component is unbiased, E_Pi[y_i] = y*, and Q_i has finite first moments.
• AssumptionThe authors state this as an assumption: Assumption 4: each judge has a per-judge sub-Gaussian parameter sigma_i, and the analysis uses sigma = max_i sigma_i as the worst-case sub-Gaussian parameter.

No m-budget corruption can drive the geometric median to infinity.

“Therefore ∥z ′ ∥ 2 remains bounded, i.e. no m-budget corruption can drive the GM to infinity.”

The cost-accuracy knee sits at whatever N reaches the ceiling for the inter-judge correlation γ of the judge pool at hand.

“the cost-accuracy knee sits at whatever N reaches that ceiling for the inter-judge correlation γ of the judge pool at hand.”

The Noisy-GT control rules out the confound that ROPOLL's premium is paid against benign imprecision rather than biased contamination.

“the Noisy-GT control ( §6.7) rules out the obvious confound that ROPOLL's premium is paid against benign imprecision rather than biased contamination”

F_α(y0) ∩ F_α(y1) is nonempty whenever ∆ ≤ ∆⋆ = 2σ Φ^{-1}(1/2 + α/(2(1-α))).

“Combining ( 62) and ( 64), F α (y 0 )∩F α (y 1 ) ̸ = ∅ whenever 2Φ(∆/(2σ)) -1 ≤ α/(1 -α), i.e. ∆ ≤ ∆ ⋆ ≜ 2σ Φ -1 1 2 + α 2(1-α) .”

The clean-rate term d/N in the minimax lower bound matches the upper bound exactly.

“The clean-rate term d/N matches the upper bound exactly.”

The total-variation distance between N(y0, σ^2 I_d) and N(y1, σ^2 I_d) depends only on ∆ = ||y0 - y1||_2 and equals 2Φ(∆/(2σ)) - 1.

“N (y 0 , σ 2 I d ) -N (y 1 , σ 2 I d ) TV = 2Φ ∆ 2σ -1,(64)”

Lemma 2 guarantees that at least (1 - alpha - beta)N judge outputs lie inside the cluster ball B(y*, rho) of sub-Gaussian radius rho.

“Lemma 2 guarantees that at least (1-α-β)N judge outputs (blue dots) lie inside the cluster ball B(y ⋆ , ρ) of sub-Gaussian radius ρ (solid disk).”

• AssumptionThe authors state this as an assumption: Assumption 2: conditional on the truth y*, each judge's output follows a mixture (1 - alpha_i) P_i + alpha_i Q_i with selector Z_i ~ Bernoulli(alpha_i); the competent component is unbiased, E_Pi[y_i] = y*, and Q_i has finite first moments.
• AssumptionThe authors state this as an assumption: Assumption 4: each judge has a per-judge sub-Gaussian parameter sigma_i, and the analysis uses sigma = max_i sigma_i as the worst-case sub-Gaussian parameter.

Two Huber neighborhoods F_α(y0) and F_α(y1) overlap if the total variation between their Gaussian centers is ≤ α/(1-α).

“N (y 0 , σ 2 I d ) -N (y 1 , σ 2 I d ) TV ≤ α 1 -α =⇒ F α (y 0 ) ∩ F α (y 1 ) ̸ = ∅.(62)”

In the parametric regime, the rate of Theorem 1 cannot be improved by any estimator.

“The following result shows that, in the parametric regime, it cannot.”

The threat to LLM juries is biased contamination rather than imprecision.

“the threat to LLM juries is biased contamination rather than imprecision ( §6.7)”

Partial competence in corrupted judges is challenging for coordinate-wise methods because the corruption is invisible on one axis.

“This partial competence is challenging for coordinate-wise methods because the corruption is invisible on one axis.”

The conditional expectation of the mean judge output given the truth is the truth.

“Linearity of conditional expectation gives E[ŷ mean | y ⋆ ] = y ⋆”

The inverted corruption scenario is the sharpest demonstration of the breakdown-point advantage.

“This is the sharpest demonstration of the breakdown-point advantage.”

Substituting the equicorrelated structure Cov(ŷ_i, ŷ_j | y⋆) = γΣ for i≠j and Cov(ŷ_i | y⋆) = Σ into (10) gives Cov(ŷ_mean | y⋆) = (1+(N-1)γ)/N Σ.

“Substituting the equicorrelated structure Cov(ŷ i , ŷj | y ⋆ ) = γ Σ for i ̸ = j and Cov(ŷ i | y ⋆ ) = Σ into (10) gives Cov(ŷ mean | y ⋆ ) = 1+(N -1)γ N Σ;taking traces yields the MSE expression.”

Sycophantic corruption complements mode collapse at the opposite extreme of the score scale.

“This complements mode collapse (corruption at the lower-left extremum) at the opposite extreme of the score scale.”

• cites Figure 19Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Since the conditional error is centered, the expected squared error of the mean given the truth equals the trace of its conditional covariance.

“Since the conditional error is centered,E[∥ŷ mean -y ⋆ ∥ 2 2 | y ⋆ ] = tr(Cov(ŷ mean | y ⋆ )) , which is the second equation in (11).”

Under α_i = 0, the conditional expectation of each judge's output given the truth is the truth.

“Under α i = 0, Assumption 2 gives E[ŷ i | y ⋆ ] = y ⋆ and Cov(ŷ i | y ⋆ ) = Σ i .”

The geometric median is robust to mixed-direction biases because the joint subgradient balance does not weight per-coordinate sign.

“the geometric median is robust to such mixed-direction biases because the joint subgradient balance does not weight per-coordinate sign.”

• cites Proposition 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Across all three failure modes, the arithmetic mean acquires a bias proportional to α and aligned with the corruption locus.

“Across all three failure modes, the arithmetic mean acquires a bias proportional to α and aligned with the corruption locus”

• cites Theorem 1Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Independence of judges zeroes the cross-covariances, yielding the off-diagonal vanishing in (11).

“Independence (Assumption 3) zeroes the cross-covariances, yielding the off-diagonal vanishing in (11).”

• AssumptionThe authors state this as an assumption: The judges' outputs are independent conditional on the truth.

Across all three failure modes, the geometric median remains close to y* as long as α < 1/2.

“the geometric median remains close to y ⋆ as long as α < 1/2”

• cites Theorem 1Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Mixing systematically over-scoring and under-scoring judges leaves the arithmetic mean's bias bounded only by the worst per-judge displacement.

“mixing systematically over-scoring and under-scoring judges leaves the arithmetic mean's bias bounded only by the worst per-judge displacement”

• cites Proposition 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The observed bias direction is precisely the contamination structure formalised by Proposition 2.

“The bias direction is precisely the contamination structure Proposition 2 formalises”

• cites Proposition 2Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.