CoChat
Sign Up
Cited by 3Multi-Agent Systems and NegotiationTopic ModelingAdversarial Robustness in Machine Learning

Free-MAD: Consensus-Free Multi-Agent Debate

Open access
Cui, Yu, Fu, Hang, Zhang, Haibin, Wang, Licheng, Zuo, Cong·2026·Findings of the Association for Computational Linguistics: ACL 2026
DOI 10.18653/v1/2026.findings-acl.1600
Open this paper in CoChat — free accountOpen PDF

Abstract

Multi-agent debate (MAD) is an emerging approach to improving the reasoning capabilities of large language models (LLMs).Existing MAD methods rely on multiple rounds of interaction among agents to reach consensus, and the final output is decided by majority voting in the last round.However, this consensus-based design faces several limitations.First, multiple rounds of communication increases token overhead and limits scalability.Second, due to the inherent conformity of LLMs, agents that initially produce correct responses may be influenced by incorrect ones during the debate process, causing error propagation.Third, majority voting introduces randomness and unfairness in the decision-making phase, and can degrade the reasoning performance.To address these issues, we propose FREE-MAD, an alternative and novel MAD framework that eliminates the need for consensus among agents.FREE-MAD introduces a novel score-based decision mechanism that evaluates the entire debate trajectory rather than relying on the last round only.This mechanism tracks how each agent's reasoning evolves, enabling more accurate and fair outcomes.In addition, FREE-MAD reconstructs the debate phase by introducing anticonformity, a mechanism that enables agents to mitigate excessive influence from the majority.Experiments on eight benchmark datasets demonstrate that FREE-MAD significantly improves reasoning performance while requiring only a single-round debate and thus reducing token costs.We also show that compared to existing MAD approaches, FREE-MAD exhibits improved robustness in real-world attack scenarios.

AI snapshot

Written from the paper’s full text.

TL;DR

FREE-MAD is a consensus-free multi-agent debate framework that scores the entire debate trajectory instead of relying on final-round majority voting, and it adds anti-conformity prompting to reduce error propagation. Across eight benchmarks it reports higher reasoning accuracy with only a single debate round, plus improved robustness under communication attacks.

Key findings

  • FREE-MAD outperformed baselines on eight benchmarks, with average improvements of 13.0% at R = 1 and 16.5% at R = 2. At R = 1, Table 3 reports FREE-MAD-N at 64.43% (↑16%/19%) and FREE-MAD-C at 61.41% (↑10%/14%), versus Baseline 1 at 55.73% and Baseline 2 at 54.06%.
  • A single debate round achieved accuracy comparable to or higher than the two-round Baseline 2 setting “while incurring almost no additional token consumption,” and the method eliminates the need for multiple rounds to reach consensus.
  • Under communication attacks with 50% compromised agents, baselines dropped by up to 20%, while FREE-MAD “consistently maintains very high accuracy” and in some cases slightly outperformed its original accuracy.
  • For logical and knowledge-based reasoning, FREE-MAD still significantly outperformed baselines, but FREE-MAD-C consistently achieved better results than FREE-MAD-N in those settings.

Methods

  • The paper formalizes MAD as a two-stage Debate and Decision protocol, then proposes FREE-MAD, which integrates decision-making into the debate phase and uses a score-based mechanism over the full answer matrix \(A \in \mathbb{R}^{N \times (R+1)}\), tracking whether agents change answers across rounds.
  • The debate stage uses either anti-conformity prompting (FREE-MAD-N) or conformity prompting (FREE-MAD-C); the anti-conformity prompt instructs agents to identify flaws in peers’ reasoning and not rely on majority opinion.
  • Experiments cover eight benchmarks: GSM-Ranges levels 4 and 6, AIME2024, AIME2025, MATH500, StrategyQA, MMLU Logical Fallacies, and AICrypto. AIME datasets use \(N=3\) agents with Qwen1.5-7B-Chat and DeepSeek-V3; other datasets use \(N=4\) with Qwen1.5-7B-Chat and Qwen2.5-72B-Instruct. Most datasets use 80 samples, while AIME2024 and AIME2025 use complete datasets; the baseline is SoM, and metrics are accuracy and token consumption.

Limitations

  • Stated in the paper: token overhead still affects practical deployment compared with single-model inference, and further optimization is necessary; evaluating optimization gains across more schemes is also necessary, but resource constraints prevented such evaluations.
  • Stated in the results: a fixed set of agents cannot be universally optimal across all task categories, and weaker models under anti-conformity showed rigidity on MATH500.
  • Evident from the setup: most benchmarks were evaluated on 80 samples, and the security evaluation used a 50% compromised-agent setting on four datasets, which may limit generality beyond those configurations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18653/v1/2026.findings-acl.1600

Cites

0 citation errors · 18 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

11 of 23 cited sources checked · 8 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    In existing MAD frameworks, conformity-driven consensus is employed during the debate process to obtain the correct answer as the final decision.

    “In existing MAD frameworks, conformity-driven consensus (Sun et al., 2024;Zeng et al., 2025;Li et al., 2024b) is employed during the debate process to obtain the correct answer as the final decision (Chan et al., 2024).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Sun et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Zeng et al., 2025 source

      The Introduction and “Reaching Consensus” section describe agents updating answers through debate, then reaching consensus or selecting an answer by majority vote. They do not characterize that process as conformity-driven; the claim’s “correct answer” wording is attached to a different citation. Page numbers are not available in our copy. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites Li et al., 2024b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Interpretationcitation

    Due to inherent conformity of LLMs, agents that initially produce correct responses may be influenced by incorrect ones during debate, causing error propagation.

    “Second, due to the inherent conformity of LLMs, agents that initially produce correct responses may be influenced by incorrect ones during the debate process, causing error propagation.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Weng et al., 2025 source
  • Premisecitation

    LLM agents can exhibit conformity, meaning that during multi-agent interactions, agents tend to favor answers endorsed by the majority.

    “Meanwhile, recent studies reveal that LLM agents can exhibit conformity (Weng et al., 2025;Zhu et al., 2025;Cho et al., 2025), meaning that during multi-agent interactions, agents tend to favor answers endorsed by the majority.”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites Weng et al., 2025 source
    • cites Zhu et al., 2025 source
    • cites Cho et al., 2025 source
  • Premisecitation

    Most existing multi-agent systems lack robustness.

    “Most existing multi-agent systems lack robustness (Chen et al., 2024a;Zhang et al., 2024;He et al., 2025).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Chen et al., 2024a source
    • cites Zhang et al., 2024 source

      Section 5.5 (page unavailable) reports effective attacks in two multi-agent scenarios, supporting a robustness concern for the systems tested. But it says the evaluation used only two implemented agents; it does not establish that most existing multi-agent systems lack robustness. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites He et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingnone given

    Under anti-conformity, the reasoning process of LLMs appears to be more rational.

    “Notably, under anti-conformity, the reasoning process of LLMs appears to be more rational.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 16
  • May combine two claims. 12
  • The quote only partly states this claim. 2

  • Classified as probably not a substantial claim.

    Premisecitation

    Multi-agent debate (MAD) has emerged as a prominent solution, showing that multiple agents engaged in multi-round interactions can achieve substantially better reasoning performance than a single agent.

    “In contrast, multi-agent debate (MAD) (Zeng et al., 2025;Du et al., 2024;Chan et al., 2024;Liu et al., 2025) has emerged as a prominent solution, showing that multiple agents engaged in multi-round interactions can achieve substantially better reasoning performance than a single agent.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Zeng et al., 2025 source
    • cites Du et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Chan et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Token consumption is adopted as the metric for scalability following Zeng et al. (2025).

    “Following (Zeng et al., 2025), we adopt token consumption as the metric for scalability.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Zeng et al., 2025 source

  • Classified as probably not a substantial claim.

    Premisecitation

    Prior work provides a general complexity analysis of the token cost (TC) in MAD as O(N R^2 V + N^2 R V + N R |q|).

    “Prior work (Zeng et al., 2025) provides a general complexity analysis of the token cost (TC) in MAD as: O N R 2 V + N 2 RV + N R|q| , where V is the maximum token cost for each agent.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Zeng et al., 2025 source

  • Classified as probably not a substantial claim.

    Premisecitation

    The consensus-building process generally takes 2 to 3 rounds to be effective.

    “The consensus-building process generally takes 2 to 3 rounds (Chen et al., 2024b;Du et al., 2024;Yang et al., 2025;Xiong et al., 2023) to be effective.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Chen et al., 2024b source
    • cites Du et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Yang et al., 2025 source

      The Models and Number of generations sections describe debate configurations ranging from 2 to 8 rounds and say that mathematical-reasoning debates may stop early upon consensus. They do not report that consensus generally takes 2–3 rounds or that this is the number needed for effectiveness (searched for round counts, early stopping, and consensus). No page numbers are available in our copy. (LLM, conf=0.72)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    FREE-MAD achieves accuracy comparable to or higher than the two-round baseline 2 setting with only a single debate round, while incurring almost no additional token consumption.

    “Notably, FREE-MAD achieves accuracy comparable to or even higher than the two-round baseline 2 setting with only a single debate round, while incurring almost no additional token consumption, demonstrating promising scalability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    FREE-MAD-N achieves stronger reasoning accuracy compared to baselines, whereas FREE-MAD-C offers better scalability.

    “Specifically, FREE-MAD-N achieves stronger reasoning accuracy compared to the baselines, whereas FREE-MAD-C offers better scalability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    FREE-MAD outperforms baselines in reasoning accuracy, achieving average improvements of 13.0% and 16.5% over baselines for R=1 and R=2 respectively.

    “Overall, our proposed FREE-MAD substantially outperforms the baselines, achieving average improvements of 13.0% and 16.5% over baselines, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    FREE-MAD significantly improves reasoning performance while requiring only a single-round debate and thus reducing token costs.

    “Experiments on eight benchmark datasets demonstrate that FREE-MAD significantly improves reasoning performance while requiring only a single-round debate and thus reducing token costs.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    FREE-MAD-C uses score-based decision.

    “| FREE-MAD-C | score | ✗ | ✓ | ✓ |”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Existing MAD approaches may obtain final answers that are even less accurate than the initial ones.

    “Existing MAD approaches may obtain final answers that are even less accurate than the initial ones.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    FREE-MAD-N uses score-based decision.

    “| FREE-MAD-N | score | ✓ | ✓ | ✓ |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Answers that agents abandon are considered more likely to be incorrect, whereas newly adopted answers are treated as more likely to be correct.

    “Specifically, answers that agents abandon are considered more likely to be incorrect, whereas newly adopted answers are treated as more likely to be correct.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Multiple rounds of communication in consensus-based MAD increase token overhead and limit scalability.

    “First, multiple rounds of communication increases token overhead and limits scalability.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationreasoning

    FREE-MAD agents do not require predefined roles and participate equally, improving fairness.

    “In contrast, agents in FREE-MAD do not require any predefined roles, and they participate equally in the debate process.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    FREE-MAD theoretically requires only a single round of debate without any consensus constraint, substantially reducing token consumption.

    “In contrast, our method theoretically requires only a single round of debate without any consensus constraint, which substantially reduces token consumption.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Conformity-oriented and anti-conformity-oriented debate modes should be adapted and switched according to the task.

    “Therefore, we argue that conformity-oriented and anti-conformity-oriented debate modes should be adapted and switched according to the task.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Two configurations of MAD are designed to ensure the framework possesses basic capability to handle the datasets.

    “To ensure that the MAD framework possesses the basic capability to handle our datasets, we design two configurations of MAD.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    Majority voting introduces randomness and unfairness in the decision-making phase, and can degrade reasoning performance.

    “Third, majority voting introduces randomness and unfairness in the decision-making phase, and can degrade the reasoning performance.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. The quote only partly states this claim.

    Interpretationreasoning

    The structured critical reasoning prompt balances conformity.

    “Second, it balances conformity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The score strategy is immune to LLM hallucination because it is executed outside LLM reasoning and follows a deterministic protocol.

    “In addition, the score strategy is executed entirely outside the LLM reasoning and follows a deterministic protocol, rendering it immune to LLM hallucination.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The SoM framework is selected as the baseline to ensure a fair comparison.

    “we select the SoM framework (Du et al., 2024) as the baseline for our experiments to ensure a fair comparison”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Du et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    A change in an agent's response is interpreted as an indicator that a more accurate answer has been identified.

    “Therefore, a change in an agent's response is interpreted as an indicator that a more accurate answer has been identified.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    Consensus naturally demands more debate rounds, increasing token consumption and limiting scalability.

    “More importantly, consensus naturally demands more debate rounds, increasing token consumption and limiting scalability.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    FREE-MAD is implemented on top of SoM to minimize the influence of confounding factors.

    “Moreover, our proposed FREE-MAD is implemented on top of SoM to minimize the influence of confounding factors.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Existing MAD methods rely on multiple rounds of interaction among agents to reach consensus, and the final output is decided by majority voting in the last round.

    “Existing MAD methods rely on multiple rounds of interaction among agents to reach consensus, and the final output is decided by majority voting in the last round.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Setting the proportion of compromised agents to 50% better reflects the adversarial capability in real-world deployments.

    “We evaluate the accuracy of MAD across multiple benchmark datasets by setting the proportion of compromised agents to |V|/|N | = 50% (see Algorithm 2), which better reflects the adversarial capability in real-world deployments.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    To mitigate the influence of randomness in the evaluation, the authors reported outcomes that tend toward the middle of repeated runs.

    “To mitigate the influence of randomness in the evaluation, we reported outcomes that tend toward the middle of repeated runs.”
    Quote in the paperNot a substantial claim
27 of 27

  • The authors state this as an assumption.

    If p encourages critical reasoning, β(p) can be negative, acting as a regularizer that reduces alignment and pushes responses toward P_in.

    “If p encourages critical reasoning, β(p) can be negative, acting as a regularizer that reduces alignment and pushes responses toward P in .”

  • The authors state this as an assumption.

    If p encourages agreement with majority opinions, β(p) assumes a larger positive value, which speeds up convergence towards consensus.

    “Conversely, if p encourages agreement with majority opinions, β(p) assumes a larger positive value, which speeds up convergence towards consensus.”

  • The authors state this as an assumption.

    The overall probability distribution P_ai can be modeled as separating independent reasoning and conformity via an exponential transformation.

    “To study the interplay between independent reasoning and conformity, we model the overall probability distribution P a i with a formulation that separates the contributions of the two factors:”

  • The authors state this as an assumption.

    Agents' responses are sampled from a conditional probability distribution defined over the preceding context and p.

    “In round k, agent a i produces its response r k i by sampling from a conditional probability distribution defined over the preceding context and p:”
4 of 4

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 5

  1. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st Inter- national Conference on Machine Learning, ICML'24. JMLR.org.
  2. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Con- ference on Learning Representations (ICLR).
  3. Yexiang Liu, Jie Cao, Zekun Li, Ran He, and Tieniu Tan. 2025. Breaking mental set to improve reasoning through diverse multi-agent debate. In The Thir- teenth International Conference on Learning Repre- sentations.
  4. Ishwara Vasista, Imran Mirza, Cole Huang, Rohan Ra- jasekhara Patil, Aslihan Akalin, Kevin Zhu, and Sean O'Brien. 2025. MALIBU benchmark: Multi-agent LLM implicit bias uncovered. In ICLR 2025 Work- shop on Building Trust in Language Models and Ap- plications.
  5. Haibin Zhang, Sisi Duan, Boxin Zhao, and Liehuang Zhu. 2023. Waterbear: practical asynchronous bft matching security guarantees of partially syn- chronous bft. In Proceedings of the 32nd USENIX Conference on Security Symposium, SEC '23, USA. USENIX Association.

Cited by

Sorted by citations

Authors
Cui, Y., Fu, H., Zhang, H., Wang, L., & Zuo, C.
Affiliations
  • Beijing Institute of Technology (CN)
  • Tsinghua University (CN)
ORCID
Venue
Findings of the Association for Computational Linguistics: ACL 2026
Publisher
Association for Computational Linguistics
Published
2026
Pages
31977-31997
DOI
10.18653/v1/2026.findings-acl.1600
OpenAlex
W4415087697
Cited by
3
References
0
Citations by year
2026: 3
FWCI
31.42
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
multi-agent debate, large language models, anticonformity, majority voting, error propagation, token efficiency, reasoning robustness, adversarial attacks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review