Demystifying Multi-Agent Debate: The Role of Confidence and Diversity

Vanilla multi-agent debate often underperforms simple majority vote despite higher computational cost.

“yet recent work shows that vanilla MAD often underperforms simple majority vote despite higher computational cost.”

• Also stated in: § 1 Introduction · plain

  “In practice, vanilla MAD often underperforms a simple majority vote despite incurring substantially higher computational cost.”

• Also stated in: § 7 Results & Analysis · hedged

  “We observe that vanilla MAD performs similarly or worse than simple majority vote, echoing findings reported in prior work (Choi et al., 2025).”

• cites Choi et al., 2025 sourceSupported
• cites Wu et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

A confidence-modulated debate protocol in which agents express calibrated confidence and condition their updates on others' confidence enables debate to systematically drift to the correct hypothesis.

“Second, a confidence-modulated debate protocol in which agents express calibrated confidence and condition their updates on others' confidence.”

• Also stated in: § Abstract · plain

  “while confidence-modulated updates enable debate to systematically drift to the correct hypothesis.”

• Also stated in: § 2. Confidence communication and alignment. · plain

  “confidence-weighted updates break the martingale limitation and allow beliefs to drift toward correctness.”

• cites Bahrami et al., 2010 sourcePartly supported

  The Main Text (page unavailable) supports confidence sharing: participants communicated trial-by-trial estimates of their reliability and used one another’s confidence in joint perceptual decisions. It does not establish a debate protocol involving successive updates or systematic drift toward a correct hypothesis; the reported outcome is improved perceptual sensitivity in dyads of similarly sensitive observers. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Fusaroli et al., 2012Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionThe authors state this as an assumption: Agents are homogeneous and fully connected.
• AssumptionThe authors state this as an assumption: Confidence is positively correlated with correctness (higher-confidence answers are on average more likely to be correct).

Initial answer diversity in vanilla MAD is significantly correlated with final answer accuracy (performance gain).

“By analysing debate dialogues, we show that initial answer diversity correlates with performance gains with statistical significance”

• Also stated in: § 7 Results & Analysis · plain

  “As shown in Table 2, even without any intervention, initial answer diversity in vanilla MAD is significantly correlated with final answer accuracy.”

MAD tends to yield larger gains on more difficult datasets.

“MAD tends to yield larger gains on more difficult datasets.”

MAD tends to be more beneficial on more challenging tasks.

“as illustrated in Figure 3b, MAD tends to be more beneficial on more challenging tasks.”

Confidence-modulated debate consistently outperforms simple aggregation across all evaluated datasets, including OOD benchmarks.

“Even without diversity-aware initialisation, the confidence-modulated debate consistently outperforms simple aggregation across all evaluated datasets, including the two OOD benchmarks.”

Empirically, across six reasoning-oriented QA benchmarks, our methods consistently outperform vanilla MAD and majority vote.

“Empirically, across six reasoning-oriented QA benchmarks, our methods consistently outperform vanilla MAD and majority vote.”

• Also stated in: § 2. Confidence communication and alignment. · plain

  “Empirically, our methods consistently outperform vanilla MAD and majority vote across six reasoning-oriented question-answering (QA) benchmarks.”

• Also stated in: § 8 Conclusion · plain

  “Empirically, our methods consistently outperform vanilla MAD and majority vote across six benchmarks.”

The model struggled to converge when initially subsampling 12k data points uniformly from the training datasets.

“In practice, the model struggled to converge under this setting.”

During GRPO training, agents exhibited reward hacking by stopping engagement with others' reasoning or confidence signals and outputting only a final answer.

“During training, we observed reward hacking behavior, where agents stopped engaging with others' reasoning or confidence signals and instead output only a final answer.”

Raising the sampling temperature from 1.0 to 1.2 did not improve MAD performance for Qwen-2.5-7B-Instruct.

“raising the temperature from 1.0 to 1.2 did not yield more diverse answers or improved MAD performance.”

Raising the sampling temperature from 1.0 to 1.2 did not increase answer diversity for Qwen-2.5-7B-Instruct.

“Increasing temperature does not increase answer diversity.”

• Also stated in: § C What We Tried but Did Not Work · plain

  “raising the temperature from 1.0 to 1.2 did not yield more diverse answers or improved MAD performance.”

MAD naturally produces a more diverse initial answer pool on harder datasets (low accuracy).

“Figure 3a shows that MAD naturally produces a more diverse initial answer pool on harder datasets (i.e., low accuracy).”

Trained models exhibit well-calibrated verbalised confidence.

“Table 6 demonstrate that trained models exhibits wellcalibrated verbalised confidence.”

Theoretically, diversity-aware initialisation improves the prior probability of MAD success without changing the underlying update dynamics.

“We show theoretically that diversity-aware initialisation improves the prior probability of MAD success without changing the underlying update dynamics”

• Also stated in: § 2. Confidence communication and alignment. · boosted

  “We provide theoretical guarantees showing that diversity increases the prior probability of the debate success, while confidence-weighted updates break the martingale limitation and allow beliefs to drift toward correctness.”

• Also stated in: § 5.1 Diversity Improves What Is Debated · plain

  “Proposition 1 (Diversity-aware initialization improves prior success).”

• AssumptionThe authors state this as an assumption: Conditional success probability P(A_T | S=s) is nondecreasing in s.
• AssumptionThe authors state this as an assumption: The distribution of S under the diversity-aware initialiser first-order stochastically dominates that under random i.i.d. sampling.

The model's already high accuracy on the uniformly subsampled datasets caused many GRPO trajectories to result in correct answers with maximal confidence (10), leading to a collapse in reward diversity and poor sampling efficiency.

“We attribute this to the model's already high accuracy on these datasets: during GRPO training, many trajectories resulted in correct answers with maximal confidence (10), leading to a collapse in reward diversity and poor sampling efficiency.”

Under the confidence-weighted update, the belief of agent i in the correct answer at round t can be written as a convex combination of the current belief p_i,t-1 and the realised confidence-weighted fraction q_i,t-1.

“Lemma 1 (Convex combination form).Assume ∆ i,t > 0. Define the realised fraction of confidence mass that lands on the correct answer as qi,t-1 := ∆ (1) i,t ∆ i,t .(38) Then, for each agent i and round t, p i,t = λi,t p i,t-1 + 1 -λi,t qi,t-1 , where λi,t := S i,t-1 S i,t-1 + ∆ i,t ∈ (0, 1).”

Confidence-weighted debate yields a strict submartingale over agents' beliefs on the correct answer under stated assumptions.

“Then the belief process {p i,t } t≥0 becomes a strict submartingale:”

• Also stated in: § 5.2 Confidence Improves How Debate Aggregates · plain

  “Conceptually speaking, confidence-modulated debate breaks the martingale symmetry of vanilla MAD and, in expectation, strictly improves correctness over rounds.”

• AssumptionThe authors state this as an assumption: Agents are homogeneous and fully connected.
• AssumptionThe authors state this as an assumption: Confidence is positively correlated with correctness (higher-confidence answers are on average more likely to be correct).

Confidence-modulated debate, by incorporating a confidence score positively correlated with correctness, breaks the martingale process into a submartingale.

“For confidence-modulated debate, the incorporation of a confidence score that is positively correlated with correctness breaks martingale process into submartingale.”

• AssumptionThe authors state this as an assumption: Confidence is positively correlated with correctness: for each agent j and round t-1, ρ_j,t-1 ≥ p_j,t-1 for all j,t, with strict inequality on a set of positive probability.

In the signal-coverage model, conditional on observing S=s distinct informative hypotheses, Pr(A_T | S=s) = 1-(1-p)^s, which is strictly increasing in s.

“Pr(A T | S = s) = 1 -(1 -p) s ,(29) which is strictly increasing in s, verifying the monotonicity assumption ( 22).”

• AssumptionThe authors state this as an assumption: Each distinct informative hypothesis in the initial pool independently has correctness probability p > 0.

If diversity-aware initialization increases the effective spread of hypotheses so that S_div stochastically dominates S_rand, then the probability of eventual correctness is higher under diversity-aware initialization than under random initialization.

“If diversity-aware initialization increases the effective spread of hypotheses so that S div stochastically dominates S rand , then Pr(A T | diverse) = E 1 -(1 -p) S div ≥ E 1 -(1 -p) S rand = Pr(A T | random),(30) in agreement with Theorem 2.”

• AssumptionThe authors state this as an assumption: The success probability Pr(A_T | S=s) is weakly increasing in S (monotonicity in coverage).
• AssumptionThe authors state this as an assumption: Diversity-aware initialization increases coverage: S_div first-order stochastically dominates S_rand, with strict inequality for some s.

Under the confidence-weighted update, q_i,t-1 decomposes as a weighted sum of neighbour-level ρ_j,t-1.

“Lemma 2 (Decomposition of confidence-weighted correctness).For each agent i and round t, q i,t-1 = j∈N (i) ω j,t-1 ρ j,t-1 ,(47)”

Confidence-weighted debate can, in expectation, do strictly better than simple unweighted majority vote.

“In other words, confidence-weighted debate can, in expectation, do strictly better than simple unweighted majority vote.”

Naive temperature scaling is insufficient for inducing answer-level diversity.

“This observation suggests that naive temperature scaling is insufficient for inducing answer-level diversity and motivates the need for more targeted diversity-aware initialization strategies.”

Higher temperature primarily induces diversity in reasoning paths rather than in final answers.

“We attribute this to the fact that higher temperature primarily induces diversity in reasoning paths rather than in final answers.”

Under Assumption 1, if the debate graph is fully connected and all agents are homogeneous, then q_i,t-1 ≥ p_i,t-1 with strict inequality on a set of positive probability.

“Corollary 1.Under Assumption 1, if the debate graph is fully connected and all agents are homogeneous so that p j,t-1 = p i,t-1 for all j ∈ N (i), then q i,t-1 ≥ p i,t-1 ,(51) with strict inequality on a set of positive probability.”

• AssumptionThe authors state this as an assumption: Confidence is positively correlated with correctness: for each agent j and round t-1, ρ_j,t-1 ≥ p_j,t-1 for all j,t, with strict inequality on a set of positive probability.