CoChat
Sign Up
PreprintCited by 3Multi-Agent Systems and NegotiationAI-based Problem Solving and PlanningArtificial Intelligence Applications

Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges

Open access
Motger, Quim, Oriol, Marc, Marco, Jordi, Franch, Xavier·2026·arXiv
DOI 10.48550/arxiv.2607.26212arXiv 2607.26212
Open this paper in CoChat — free accountOpen PDF

Abstract

Multi-Agent Debate (MAD) is a promising paradigm for improving the accuracy and robustness of Large Language Model (LLM)-based agentic systems. It enables multiple agents to exchange arguments, critique each other's outputs, and iteratively converge towards a solution. However, research remains fragmented, with inconsistent terminology and no rigorous synthesis of MAD design dimensions. We present a systematic literature review characterizing 141 primary studies on MAD. We derive a three-dimensional taxonomy covering debate participants, the interaction mechanisms structuring the exchange, and the agreement protocols governing debate resolution, supported by formal notations to render MAD configurations. Our analysis reveals that the field has implicitly converged on a narrow design pattern - static, fully connected topologies, verbatim exchange, short-term memory and voting resolution strategies - adopted by convention rather than systematic comparison, while promising alternatives remain marginal. Because any MAD setting reflects roughly a dozen interacting design decisions, cross-study comparison is unreliable when these are left implicit. We position the taxonomy as a descriptive map of the research landscape, a framework for controlled benchmarking, and potentially as a schema for machine-readable MAD specifications. As future work, we propose formalizing it into an executable specification, enabling cost-aware benchmarking and automated tuning of debate configurations.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic literature review characterizes 141 primary studies on Multi-Agent Debate (MAD) and derives a three-dimensional taxonomy covering participants, interaction, and agreement. It finds the field has converged on a narrow default pattern—static, fully connected topologies, verbatim exchange, short-term memory, and voting resolution—while alternatives remain marginal and cross-study comparison is hindered by implicit design choices.

Key findings

  • Across 141 papers yielding 151 distinct MAD approaches, the field is young and growing: 14 studies appeared in 2023, 57 in 2024, and 70 in 2025; 85 were conference papers, 10 journal articles, 1 book chapter, and 45 arXiv preprints.
  • The dominant interaction pattern is static (88.7%), fully connected (72.2%), natural-language (99.3%), verbatim (86.1%), and short-term memory (94.7%); dynamic topologies (9.9%), structured networks (23.2%), summarized content (10.6%), long-term memory (2.7%), retrieval memory (6.0%), and parametric memory (2.0%) remain marginal.
  • Participants are mostly homogeneous in base model (81.5% vs. 27.8% heterogeneous); closed-weight-only configurations are 43.7%, open-weight-only 21.2%, and mixed 35.1%, with GPT dominating at 78.1%, followed by Llama (40.4%) and Qwen (18.5%). Judge roles appear in 38.4% of approaches, summarizers in 21.9%, leaders in 17.2%, verifiers in 6.0%, and editors in 3.3%.
  • Agreement is most often collective authority (52.3%) or judge-based (39.1%); among the 97 approaches with a formalized resolution process, majority vote is dominant (46.4%), followed by consensus (20.6%), argument evaluation (13.4%), scoring (10.3%), weighted vote (5.2%), and average (4.1%).

Methods

  • The authors conducted a systematic literature review following Kitchenham and Charters, complemented by Wohlin-style backward and forward snowballing, with five stages: MAD definition/scoping, seed search-string definition, independent screening, snowballing, and structured data extraction with inductive coding.
  • The Scopus search used `(LLM OR "Large Language Model" OR LLMS OR "Large Language Models" OR "AI Agent" OR "AI Agents") AND (MAD OR "Multi-Agent Debate")` over title, abstract, and keywords, with no domain or date restriction; the initial search yielded 29 results, 18 were excluded, leaving 11 seed papers. Backward snowballing identified 527 papers (357 after duplicate removal) and added 12, bringing the total to 23; forward snowballing yielded 929 papers (889 after duplicate removal), removed 367 in initial filtering, left 522 for review, and added 118, producing a final set of 141 papers. Average inter-rater agreement across screening rounds was Cohen’s kappa = 0.70.
  • Data extraction produced a three-dimensional taxonomy—participants, interaction, and agreement—with formal notations; quantitative analysis covers 151 distinct MAD approaches derived from the 141 papers, while resolution percentages are computed over n = 97 approaches where a resolution process is formalized and implemented.

Limitations

  • The paper states internal-validity threats: selection and annotation are subjective, mitigated by independent dual review, but data extraction was performed by a single author per paper due to corpus size; the taxonomy was derived inductively and may be shaped by the included studies.
  • The paper states construct-validity threats: there is no universally accepted definition of MAD, and boundaries with adjacent paradigms such as multi-agent collaboration or iterative self-refinement are fuzzy; external validity is limited because the study is scoped to natural-language tasks and excludes multimodal debate; conclusion validity is limited because the primary search used only Scopus with a deliberately narrow string, and snowballing was a single iteration rather than until saturation.
  • Evident from the text: percentages may not sum to 100 because some properties are unspecified or because a single approach may combine or compare several strategies and be counted under multiple values; resolution percentages cover only 97 of 151 approaches, and many participant prompts, personas, role combinations, and model assignments are described only partially, limiting reproducibility and cross-study comparison.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2607.26212

Cites

185 of 185 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 117 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    In software engineering and cybersecurity, MAD applications target technical reliability of software artifacts, including code generation, summarization, translation, phishing detection, and vulnerability assessment.

    “Software Engineering & Cybersecurity: Applications in this domain focus on the technical reliability of software artifacts.This includes code generation [8,63,76,110,128,149,156], code summarization [9,21], and code translation [21,34].In the security sector, frameworks are applied to detect phishing attempts in emails or websites [65,87] and perform software vulnerability assessments [132,134].”
    Quote in the paperNot a substantial claim
  • Findingcitation

    General knowledge and reasoning is the dominant application domain for MAD.

    “General Knowledge & Reasoning: This remains the dominant domain, where MAD is used to benchmark the ability of agents to resolve open-ended questions or general-purpose tasks [9,15,70,101,137,156].”
    Quote in the paperNot a substantial claim
  • Findingown data

    Reasoning and problem solving is the most frequent task category addressed by MAD.

    “Reasoning & Problem Solving: This is the most frequent category, focusing on enhancing the logical and computational capabilities of models.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The selected studies comprised 85 conference papers, 10 journal articles, 1 book chapter, and 45 arXiv preprints.

    “Among the selected studies, 85 were published in conferences, 10 in journals, 1 as a book chapter, and 45 were available as arXiv preprints.”
    Quote in the paperNot a substantial claim
  • Findingreasoning

    To the authors' knowledge, no existing study offers a large-scale, methodologically rigorous, cross-domain synthesis that systematically consolidates MAD design dimensions and architectural patterns.

    “To the best of our knowledge, no study provides a large-scale, methodologically rigorous, and cross-domain synthesis that systematically consolidates MAD design dimensions and architectural patterns.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 52
  • May combine two claims. 71
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    In practice, personas are implemented via the agent's system prompt, which can specify background, personality traits, and stance on the debate topic.

    “In practice, personas are instantiated through the agent's system prompt, which may specify different kinds of persona-related information, such as the agent's background, personality traits, and stance towards the debate topic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    In social sciences, politics, and ethics, MAD research addresses societal issues and value alignment, including fake news and rumor detection, political coalition negotiations, and mitigation of bias or harmful content.

    “Social Sciences, Politics & Ethics: This domain focuses on societal issues and value alignment.Key research areas include fake news and rumor detection [38,48,50,153], political coalition negotiations [83], and the mitigation of bias or harmful content in model outputs [3,10,18,73,84,94].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    MAD is increasingly applied in clinical settings, with research exploring mitigation of cognitive biases in diagnostic reasoning, improvement of medical question answering, and modeling of electronic health records for mortality prediction.

    “Healthcare & Medicine: MAD is increasingly applied to clinical settings.Research explores how multi-agent conversations can mitigate cognitive biases in diagnostic reasoning [10,16,52], improve medical Q&A performance [10,16,52,109,119,120], and model electronic health records (EHR) for mortality prediction [129].”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingreasoning

    There is currently no widely accepted definition of MAD and no clear consensus on its essential characteristics, boundaries, and distinguishing features.

    “As a result, there is currently no widely accepted definition of MAD, nor a clear consensus regarding its essential characteristics, boundaries, and distinguishing features.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Background attributes range from specialized functional roles to richer social or cultural identities.

    “Backgrounds range from specialized functional roles to richer social or cultural identities.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Evaluation maturity varies by task type: objective tasks allow direct benchmark-oriented evaluation, whereas open-ended tasks depend on weaker proxies like LLM-as-a-judge and qualitative synthesis.

    “Evaluation maturity varies strongly by task type.Objective tasks such as mathematics, logic, and code generation support direct benchmark-oriented evaluation, while open-ended tasks such as text evaluation, legal reasoning, negotiation, and creative generation rely on weaker proxies, including LLM-as-a-judge and qualitative synthesis [9,34,55,57,69,70,83,90,152,156].”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    The analysed corpus shows a diverse set of MAD objectives, grouped into categories including reasoning and problem solving, fact-checking, classification and security detection, evaluation and meta-agent judgment, technical and creative generation, strategic interaction and negotiation, and prediction and data analysis.

    “The analysed corpus reveals a diverse range of objectives, which have been classified and grouped into the following categories:”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Of the 141 selected publications, 14 appeared in 2023, 57 in 2024, and 70 in 2025, showing a clear increasing trend in MAD research activity.

    “Among the 141 selected publications, the earliest papers appeared in 2023 (14 papers), followed by 57 publications in 2024 and 70 in 2025, indicating a clear and increasing trend in research activity on this topic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The literature on MAD is fragmented, lacking consistent terminology and a rigorous synthesis of its design dimensions.

    “However, research remains fragmented, with inconsistent terminology and no rigorous synthesis of MAD design dimensions.”
    • Also stated in: § 2026. Manuscript submitted to ACM · plain
      “Despite this growing body of work, the field remains fragmented, with limited consolidation of terminology, design dimensions, and architectural patterns across application domains.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    In linguistics and natural language, MAD is applied to event extraction, machine translation evaluation, summary source alignment, and specialized generation tasks such as Chinese couplet generation.

    “Linguistics & Natural Language: Beyond general reasoning, specific linguistic tasks are addressed through MAD.These include event extraction [36,124], machine translation evaluation [34], summary source alignment [57], and specialized generation tasks like Chinese couplet generation and evaluation [90].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Summarization can substantially reduce communication overhead while remaining competitive in performance.

    “Empirically, summarization can substantially cut communication overhead while remaining competitive [120,134,139]”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [134]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [139]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    MAD research is strongly centred on GPT-family and other closed-weight models, and participant prompts, personas, role combinations, and model assignments are often only partially described.

    “MAD research remains strongly centred on GPT-family and other closed-weight models, while participant prompts, personas, role combinations, and model assignments are often described only partially”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • May combine two claims.

    Findingcitation

    The identified roles are not mutually exclusive, and a single agent can combine several of them, such as judge and summarizer.

    “These roles are not mutually exclusive: a single agent may combine judge and summarizer”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Debate-to-Detect reports that mixed-model configurations outperform homogeneous setups in misinformation detection.

    “Debate-to-Detect [38] reports mixed-model configurations outperforming homogeneous setups in misinformation detection.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [38]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    Sequential protocols are common, but agent order is rarely controlled as an experimental variable.

    “Sequential protocols are common, yet agent order is rarely controlled as an experimental variable.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Pham et al. report consistent gains of 0.5-5.0% for embeddings-based debate over natural-language debate, particularly for smaller models.

    “Pham et al. report consistent gains over natural-language debate (0.5-5.0%), particularly for smaller models [96].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [96]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A knowledge retrieval strategy, augmenting accessible memory with external sources queried during the debate, accounts for 6.0% of the surveyed approaches.

    “A knowledge retrieval strategy (6.0%) augments M (𝑡 ) with external sources”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • May combine two claims.

    Findingown data

    Among the surveyed approaches, 66 use only closed-weight models (43.7%), 32 use only open-weight models (21.2%), and 53 combine both (35.1%).

    “Among the surveyed approaches, 66 rely exclusively on closed-weight models (43.7%), 32 exclusively on open-weight models (21.2%), and 53 combine both (35.1%).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • May combine two claims.

    Findingown data

    Where diversity is used, it usually comes from manually assigned roles, stances, or expertise profiles rather than being treated as an independent design variable, which makes it hard to tell which forms of diversity improve debate quality.

    “When diversity is used, it is typically introduced through manually assigned roles, stances, or expertise profiles rather than evaluated as an independent design variable, making it hard to determine which forms of diversity actually improve debate quality”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • May combine two claims.

    Findingcitation

    Huang et al. and Wen et al. show that simultaneous interaction reduces inter-agent dependency while preserving diversity.

    “Huang et al. [46] and Wen et al. [134] show that simultaneous interaction reduces inter-agent dependency while preserving diversity”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [134]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    The field has largely standardized on static structures, simultaneous or sequential natural-language exchange, verbatim message sharing, and short-term memory.

    “The field has largely standardized around static structures, simultaneous or sequential natural-language exchange, verbatim message sharing, and short-term memory.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • May combine two claims.

    Findingcitation

    Li et al. show that reducing connectivity in sparse communication can retain or even improve performance while substantially lowering inference cost.

    “Li et al. [67] systematically study sparse communication (e.g., neighbour-connected graphs) and show that reducing connectivity can retain or even improve performance while significantly lowering inference cost”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [67]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A long-term memory strategy, extending accessible memory with information persisting across sessions, accounts for 2.7% of the surveyed approaches.

    “A long-term strategy (2.7%) extends M (𝑡 ) with information persisting across sessions”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • May combine two claims.

    Findingown data

    By model family, GPT is the most used (118 approaches, 78.1%), followed by Llama (61, 40.4%) and Qwen (28, 18.5%).

    “By family, GPT dominates (118 approaches, 78.1%), followed by Llama (61, 40.4%),Qwen (28, 18.5%)”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • May combine two claims.

    Findingcitation

    Only a few studies compare the standardized configuration against alternatives, leaving trade-offs in accuracy, cost, and communication efficiency unresolved.

    “This configuration is easy to implement, but only a few studies compare it against alternatives, leaving trade-offs in accuracy, cost, and communication efficiency unresolved [46,120,134].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [134]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Hybrid resolution systems combine collective agreement first and invoke a judge only if a tie occurs or consensus is not reached within a specific number of rounds.

    “Hybrid (4.0%):These systems combine both methods, usually attempting a collective agreement first and invoking a judge only if a tie occurs or consensus is not reached within a specific number of rounds [6,30,57,68,123].”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [6]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [30]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [57]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [68]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [123]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingcitation

    Majority vote is the dominant resolution mechanism in collective systems, selecting the answer chosen by the highest number of agents.

    “Majority vote (46.4%):The dominant mechanism in collective systems, where the answer selected by the highest number of agents is chosen as the final outcome [9,25,28,29,39,48,67,69,74,80,91,96,101,104,111,114,120,122,130,138,141,145,146,151].”
    Quote in the paperNot a substantial claim0 of 10 sources checked
    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [25]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [28]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [29]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [39]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [48]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [67]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [69]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [74]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [80]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Consensus processes are usually capped at a maximum number of rounds to prevent infinite loops.

    “To prevent infinite loops, these processes are usually capped at a maximum number of rounds.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    The 'Other' resolution category includes frameworks where decision authority is not explicitly defined, is user-dependent, or follows non-standard protocols.

    “Other (4.6%):This category encompasses frameworks where the decision authority is not explicitly defined, is user-dependent, or follows non-standard protocols.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [92]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [4]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [148]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingcitation

    Inter-agent communication is almost exclusively natural language, despite early evidence that embedding-based communication can improve performance while reducing overhead.

    “Inter-agent communication is almost exclusively natural language, despite early evidence that embedding-based communication can improve performance while reducing overhead.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Scoring resolution has agents assign numerical values or ranks to responses, and the system selects the option with the best aggregate performance.

    “Scoring (10.3%):In this mechanism, agents assign numerical values or ranks to different responses, and the system selects the option with the best aggregate performance.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [76]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [160]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [161]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    MAD systems almost universally rely on short-term memory restricted to the current debate, while long-term, retrieval-based, and parametric memory remain marginal.

    “MAD systems almost universally rely on short-term memory restricted to the current debate, while long-term, retrieval-based, and parametric memory remain marginal.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Average resolution is used when agent outputs are numerical, computing the mathematical mean of all individual values to derive a unified final answer.

    “Average (4.1%):This method is specifically employed when the agents' outputs are numerical.Instead of choosing one agent's response, the system computes the mathematical mean of all individual values to derive a unified final answer.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [140]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Argument Evaluation is a resolution mechanism specific to judge-based systems, where a judge or meta-agent evaluates the quality of debate arguments to synthesize a final judgment or summary.

    “Argument Evaluation (13.4%foot_4 ): Specific to judge-based systems, where the designated judge or meta-agent evaluates the quality of the debate arguments to synthesize a final judgment or summary [3,8,15,43,51,58,59,70,127,132,149].”
    Quote in the paperNot a substantial claim0 of 10 sources checked
    • cites [3]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [8]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [15]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [43]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [51]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [58]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [59]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [70]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [127]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [132]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Weighted vote resolution applies specific weights to agent responses rather than treating them equally.

    “Weighted vote (5.2%):In this mechanism, final decisions are reached by applying specific weights to agent responses rather than treating them equally [10,13,97,105,148].”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [10]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [13]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [97]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [105]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [148]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Consensus resolution requires agents to deliberate until they all converge on a single, unanimous answer.

    “Consensus (20.6%):Agents deliberate until they all converge on a single, unanimous answer [12,14,16,20,44,45,73,118,134,139,158].”
    Quote in the paperNot a substantial claim0 of 10 sources checked
    • cites [12]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [14]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [16]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [20]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [44]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [45]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [73]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [118]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [134]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [139]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    Alternatives to the recurring pattern are not absent but marginal: dynamic topologies, structured and sparse networks, summarized or embedding-based content, and long-term, retrieval-augmented, or parametric memory each appear in a small minority of studies.

    “Crucially, the alternatives to this pattern are not absent but marginal: dynamic topologies, structured and sparse networks, summarized or embedding-based content, and long-term, retrieval-augmented, or parametric memory each appear in a small minority of studies”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Full connectivity is adopted by default, yet sparse and structured alternatives match or exceed it at lower cost.

    “Topology as a deliberate design lever.Full connectivity is adopted by default, yet sparse and structured alternatives match or exceed it at lower cost.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The bibliometric trend is unambiguous: from 14 studies in 2023 to 70 in 2025, with 45 of 141 studies available only as preprints.

    “The bibliometric trend is unambiguous: from 14 studies in 2023 to 70 in 2025, with 45 of 141 studies available only as preprints”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Agent ordering in sequential protocols is seldom controlled or reported, leaving an underappreciated source of anchoring and conformity bias.

    “Interaction order.Agent ordering in sequential protocols is seldom controlled or reported, leaving an underappreciated source of anchoring and conformity bias.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Studies typically vary one dimension in isolation; how participant, interaction, and agreement choices interact is unstudied.

    “Cross-dimension interactions.Studies typically vary one dimension in isolation; how participant, interaction, and agreement choices interact -e.g., topology effects under different agreement mechanisms -is unstudied.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    MAD research remains strongly centred on GPT, although open-weight alternatives are increasingly used.

    “MAD research thus remains strongly centred on GPT, although open-weight alternatives are increasingly used.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Adaptive topologies are rare despite their potential to prune redundant exchanges.

    “Dynamic interaction.Adaptive topologies are rare despite their potential to prune redundant exchanges.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    A fully connected structure (72.2%) links every pair of agents, and most surveyed approaches adopt this structure.

    “A fully connected structure (72.2%) links every pair of agents (∀𝑢, 𝑣 ∈ 𝑉 , (𝑢, 𝑣) ∈ 𝐸), so every message reaches all other agents.The bilateral case (|𝑉 | = 2) is its trivial instance.Most surveyed approaches adopt this structure[38,44,46,91,96,99,120].”
    Quote in the paperSubstantial claim0 of 7 sources checked
    • cites [38]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [44]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [91]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [96]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [99]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A divided into groups structure (3.3%) partitions agents into disjoint subsets with communication restricted mostly within each group.

    “A divided into groups structure (3.3%) partitions agents into disjoint subsets 𝑉 1 , 𝑉 2 , . . ., 𝑉 𝑘 with communication restricted mostly within each group”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The search string executed in Scopus yielded 29 results.

    “The search string executed in Scopus yielded 29 results.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    The conditions under which a given topology is preferable remain uncharacterized.

    “The conditions under which a given topology is preferable remain uncharacterized.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Latent-space message exchange is virtually unexplored despite early evidence of joint effectiveness and efficiency gains.

    “Embedding-based communication.Latent-space message exchange is virtually unexplored despite early evidence of joint effectiveness and efficiency gains.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Application is concentrated on general reasoning and mathematics, while specialized professional domains remain comparatively untested.

    “Underexplored domains.Application is concentrated on general reasoning and mathematics, while specialized professional domains remain comparatively untested.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Chen et al. found that stubborn and suggestible traits influence consensus dynamics, with stubborn agents having stronger influence on the final value.

    “Chen et al. [12] show that stubborn and suggestible traits affect consensus dynamics, with stubborn agents exerting stronger influence on the final value; related work explores easy-going and overconfident profiles [57], and DEBATE varies a critic's degree of criticality from weakly to strictly negative [55].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation

    Knowledge-intensive and subjective tasks benefit from heterogeneous personas that broaden perspective coverage and help reduce individual cognitive biases, especially in medical contexts.

    “In contrast, knowledge-intensive and subjective tasks benefit from the design of heterogeneous personas [44,51,127] that ensure a broad coverage of perspectives and help mitigate individual cognitive biases, particularly in medical contexts [10,16,52].”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationown data

    The variety of tasks addressed by MAD requires a strategic mapping between task requirements and multi-agent architectures.

    “From a design perspective, the diversity of addressed tasks necessitates a strategic mapping between task requirements and multi-agent architectures.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Across all screening rounds, the average inter-rater agreement was a Cohen's kappa of 0.70.

    “Inter-rater agreement across all screening rounds (initial search, backward and forward snowballing) yielded an average Cohen's kappa of 0.70.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Retrieved papers were randomly assigned to two authors as reviewers, the selection process was conducted independently, and disagreements were resolved through consensus.

    “All retrieved papers were randomly assigned to two authors of the paper acting as reviewers.The entire selection process was conducted independently, with each reviewer assessing their assigned studies against the predefined inclusion and exclusion criteria.Any disagreements were discussed and resolved through consensus.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Arithmetic and logical tasks permit objective measurement via accuracy on established benchmarks, whereas open-ended and creative tasks increasingly depend on meta-evaluation frameworks with agents as judges.

    “While arithmetic and logical tasks allow for objective measurement through accuracy scores on established benchmarks like GSM8K [97], open-ended and creative tasks increasingly rely on meta-evaluation frameworks where agents act as judges to score coherence, faithfulness, and alignment [9,55,57,69,70,98,156].”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Existing definitions of MAD share the notion of multiple AI agents in a deliberative process but differ on the role of consensus, the structure of interactions, and the presence of coordination or judging mechanisms.

    “While they share the common idea of multiple AI agents engaging in a deliberative process, they differ with respect to key aspects such as the role of consensus, the structure of agent interactions, and the presence of dedicated coordination or judging mechanisms.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [28]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [71]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Smit et al. provided an early overview of MAD research, covering eight studies and extracting six design features.

    “Smit et al. [108] offered an early overview of MAD research, identifying eight studies and extracting six design features, such as the number of debate rounds and the inclusion of specialized roles like judge or summarizer.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [108]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    Evaluation clarity varies by domain: mathematics has binary correctness, whereas social and linguistic domains depend on subjective, multi-dimensional metrics.

    “From an evaluation standpoint, these domains offer varying degrees of ground-truth clarity: while mathematics provides binary correctness, social and linguistic domains rely on subjective, multi-dimensional metrics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Wang et al. proposed a flat categorization of MAD strategies based on three communication patterns.

    “Wang et al. proposed a flat categorization of MAD strategies based on three communication patterns [126].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [126]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Candidate papers were independently evaluated by two authors, with disagreements resolved through discussion and consensus.

    “All candidate papers were independently evaluated by two distinct authors, and disagreements were resolved through discussion and consensus.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    The move toward multi-agent evaluation brings challenges such as inter-agent sycophancy and reinforcement of existing biases, requiring monitoring with diverse reasoning datasets and multi-dimensional scoring.

    “This shift towards multi-agent evaluation introduces new challenges, such as the risk of inter-agent sycophancy [97] or the reinforcement of existing biases, which must be carefully monitored through diverse reasoning datasets and multi-dimensional scoring systems.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [97]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The application domain shapes agent design, with specialized fields such as medicine or law often requiring agents that have specific personas or domain-specific knowledge.

    “From a design perspective, the choice of domain necessitates tailored agent profiles; specialized fields like medicine or law often require agents with specific personas or access to domain-specific knowledge.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Tillmann et al. also reviewed MAD approaches, but their review lacks a clear systematic protocol, relies solely on an AI-assisted tool, and does not report the exact number of included studies.

    “Tillmann et al. [116] also reviewed MAD approaches, although their review lacks a clear systematic protocol, relies exclusively on an AI-assisted tool, and does not report the exact number of included studies.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [116]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    Persona diversity is acknowledged in the literature but seldom put into practice in a systematic way.

    “Persona diversity is recognized but rarely operationalized systematically.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Liu et al. found that distinct personas alone may not produce genuinely divergent reasoning, so they paired them with distinct reasoning strategies to break the agents' shared mental set.

    “Notably, Liu et al. [75] find that distinct personas alone may not elicit genuinely divergent reasoning, and pair them with distinct reasoning strategies to break the agents' shared”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [75]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation

    Personality traits are defined less consistently and are harder to isolate experimentally, though early evidence indicates that traits such as stubbornness, suggestibility, confidence, or criticality can substantially affect convergence.

    “Personality traits are less consistently defined and harder to isolate experimentally, though early evidence suggests that stubbornness, suggestibility, confidence, or criticality can substantially affect convergence”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Automatic persona assignment increases adaptability and lowers manual effort, but ties debate quality to the persona-generation step.

    “Automatic assignment improves adaptability and reduces manual effort, but makes debate quality dependent on the persona-generation step.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Reproducibility in MAD is limited by reliance on closed-weight models and by underspecified participant design.

    “Reproducibility is constrained by closed-weight dependence and underspecified participant design.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Partial reporting limits reproducibility and cross-study comparison, so explicit participant specifications are needed for future benchmarking.

    “This limits reproducibility and cross-study comparison, making explicit participant specifications essential for future benchmarking.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Model-level diversity may improve robustness, but it makes attribution harder because gains could come from model capability rather than the debate protocol.

    “Such diversity may increase robustness, but it complicates attribution, since gains may stem from model capability rather than the debate protocol alone.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Using a shared persona simplifies implementation and isolates the interaction mechanism, but it limits reasoning diversity and makes agents prone to premature convergence and shared errors.

    “A shared persona simplifies implementation and isolates the interaction mechanism, but constrains reasoning diversity, leaving agents prone to premature convergence and shared errors.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Topology structures embody a trade-off: more connectivity improves information sharing and robustness, while less connectivity improves efficiency and scalability.

    “These structures expose a central trade-off: greater connectivity improves information sharing and robustness, while restricting it improves efficiency and scalability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Manual persona assignment affords precise control over panel composition but presupposes knowledge of which perspectives are relevant to the debate.

    “Manual assignment gives precise control over panel composition but requires prior assumptions about which perspectives are relevant to the debate.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Structured alternatives such as sparse, ring, or group-divided topologies can match performance while reducing token cost, so topology remains an underexploited design lever.

    “Yet structured alternatives such as sparse, ring, or group-divided topologies can achieve competitive performance while reducing token cost, leaving topology an underexploited design lever [67,71,74,123].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [67]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [71]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [74]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [123]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationcitation

    Because early contributions may anchor later reasoning and amplify conformity, randomized turn-taking and centralized speaker selection remain important but underused ways to reduce ordering bias.

    “Since early contributions may anchor subsequent reasoning and amplify conformity, randomized turn-taking and centralized speaker selection remain important but underused mechanisms for reducing ordering bias [19,80,110,120].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [19]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [80]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [110]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [120]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Early contributions can anchor later reasoning and amplify conformity, making interaction order a key factor in the diversity and robustness of deliberation.

    “early contributions can anchor later reasoning and amplify conformity, making interaction order a key factor in the diversity and robustness of deliberation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Heterogeneous personas are the main way MAD operationalizes perspective diversity and reduces conformity, but they add variability because outcomes depend on which personas are assigned and how they are specified.

    “Heterogeneous personas are the primary mechanism through which MAD operationalizes perspective diversity and mitigates conformity, at the cost of additional variability, since outcomes become sensitive to which personas are assigned and how they are specified.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Chen et al. assessed the impact of doubling debate rounds on performance and token efficiency.

    “For example, Chen et al. assessed the impact of doubling debate rounds (e.g., from 2 to 4, from 13 to 25) observing performance and token efficiency [16].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductreasoning

    The taxonomy was derived inductively from the literature rather than from a pre-established framework, meaning its categories may be shaped by the particular set of studies included.

    “Second, the taxonomy was derived inductively from the literature rather than from a pre-established framework, meaning its categories may be shaped by the particular set of studies included.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    These attributes offer a finer-grained lever over debate dynamics, diversity, and convergence, but their effects are harder to isolate than stance.

    “These attributes offer a finer-grained lever over debate dynamics, diversity, and convergence, but their effects are harder to isolate than stance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    Data extraction was performed by a single author per paper due to corpus size, but the taxonomy vocabulary was iteratively consolidated across all authors throughout the process.

    “Data extraction was performed by a single author per paper due to corpus size, but the taxonomy vocabulary was iteratively consolidated across all authors throughout the process.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    All percentages are computed over the set of distinct MAD approaches (n = 151).

    “All percentages are computed over the set of distinct MAD approaches (𝑛 = 151).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    For a given property, totals may be below 100% when some approaches leave that property unspecified and it cannot be inferred from the primary study without assumptions.

    “For a given property, totals may be below 100% when some approaches leave that property unspecified and it cannot be inferred from the primary study without assumptions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    All research materials are publicly available in a replication package.

    “All research materials are publicly available in a replication package.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    MAD is a young, fast-moving field whose body of knowledge remains largely uncurated and fragmented across domains and terminologies.

    “MAD is a young, fast-moving field whose body of knowledge remains largely uncurated and fragmented across domains and terminologies.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    We organize threats to validity following the taxonomy proposed by Zhou et al. [159].

    “We organize threats to validity following the taxonomy proposed by Zhou et al. [159].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [159]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Interpretationown data

    A MAD setting is defined by a large number of design decisions - on the order of a dozen attributes, each admitting several values - which jointly induce an enormous configuration space.

    “As the results show, a MAD setting is defined by a large number of design decisions -on the order of a dozen attributes, each admitting several values -which jointly induce an enormous configuration space.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Paper selection and annotation are susceptible to subjectivity.

    “First, paper selection and annotation are susceptible to subjectivity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Exclusively for resolution, percentages are computed over n = 97 MAD approaches, representing the set of studies where a resolution agreement process is actually formalized and implemented.

    “Exclusively, for resolution, percentages are computed over 𝑛 = 97 MAD approaches, representing the set of studies where a resolution agreement process is actually formalized and implemented.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Study design and conductreasoning

    The boundary with adjacent paradigms such as multi-agent collaboration or iterative self-refinement is inherently fuzzy, and borderline cases required judgment calls that the dual-review protocol only partially mitigates.

    “Nevertheless, the boundary with adjacent paradigms such as multi-agent collaboration or iterative self-refinement is inherently fuzzy, and borderline cases required judgment calls that the dual-review protocol only partially mitigates.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Any MAD setting is the product of roughly a dozen interacting design decisions, which makes cross-study comparison unreliable whenever those decisions are left implicit.

    “Second, any MAD setting is the product of roughly a dozen interacting design decisions, which makes cross-study comparison unreliable whenever those decisions are left implicit.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Generative AI tools were used exclusively to support language refinement and polishing of the manuscript, and were not used for literature analysis, data extraction, interpretation of results, or generation of scientific claims.

    “Generative AI tools were used exclusively to support language refinement and polishing of the manuscript.These tools were not used for literature analysis, data extraction, interpretation of results, or generation of scientific claims.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Snowballing was executed in a single iteration rather than until full saturation; given that 70 of the 141 included studies were published in 2025 alone, additional rounds would yield diminishing returns, as later-included papers would largely cite works already in the corpus.

    “Snowballing was executed in a single iteration rather than until full saturation; given that 70 of the 141 included studies were published in 2025 alone, additional rounds would yield diminishing returns, as later-included papers would largely cite works already in the corpus.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The authors iteratively reviewed and expanded the vocabulary included in the taxonomy, revisiting studies as new paradigms or design alternatives emerged.

    “To mitigate this threat, we iteratively reviewed and expanded the vocabulary included in our taxonomy, revisiting studies as new paradigms or design alternatives emerged.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    The primary search was conducted exclusively on Scopus using a deliberately narrow search string, prioritizing precision in the seed set at the cost of potentially missing relevant works.

    “The primary search was conducted exclusively on Scopus using a deliberately narrow search string, prioritizing precision in the seed set at the cost of potentially missing relevant works.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The same domains shape agreement (RQ4): objective tasks gravitate towards collective authority resolved by majority vote, while subjective tasks delegate to judge-based authority and argument evaluation, where a meta-agent performs qualitative synthesis rather than aggregation.

    “The same domains shape agreement (RQ4): objective tasks gravitate towards collective authority resolved by majority vote, while subjective tasks delegate to judge-based authority and argument evaluation, where a meta-agent performs qualitative synthesis rather than aggregation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    This combinatorial variance is the root cause of a recurring methodological weakness: comparisons across studies are difficult, and rigorous benchmarking is harder still, because reported results conflate the effect of the debate mechanism with the effect of unstated design choices.

    “This combinatorial variance is the root cause of a recurring methodological weakness: comparisons across studies are difficult, and rigorous benchmarking is harder still, because reported results conflate the effect of the debate mechanism with the effect of unstated design choices.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The rapid evolution of the field means that findings and taxonomic categories may require revision as new architectural patterns emerge.

    “Finally, the rapid evolution of the field means that findings and taxonomic categories may require revision as new architectural patterns emerge.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The main threat to construct validity is the absence of a universally accepted definition of MAD.

    “Concerning construct validity, the main threat is the absence of a universally accepted definition of MAD.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The authors make their full replication package available.

    “To support external auditability and incremental updates, we make our full replication package available [86].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [86]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductreasoning

    Every candidate paper was independently assessed by at least two authors, with disagreements resolved by consensus.

    “To mitigate this threat, every candidate paper was independently assessed by at least two authors, with disagreements resolved by consensus.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Interaction (RQ3) shows the strongest internal coupling: across topology, protocol, format, and memory, the field has implicitly converged on a recurring pattern - static, fully connected topologies, verbatim natural-language content, and short-term memory - adopted by convention rather than through systematic comparison.

    “Interaction (RQ3) shows the strongest internal coupling: across topology, protocol, format, and memory, the field has implicitly converged on a recurring pattern -static, fully connected topologies, verbatim natural-language content, and short-term memory -adopted by convention rather than through systematic comparison.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The taxonomy generalizes to text-based MAD and may need to be extended to capture modality-specific dimensions.

    “Consequently, our taxonomy generalizes to text-based MAD and may need to be extended to capture modality-specific dimensions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Predefined stances make disagreement explicit and provide a controllable adversarial structure, though they may impose conflict even when the task would not naturally require it.

    “Predefined stances make disagreement explicit and provide a controllable adversarial structure, though they may impose conflict even when the task would not naturally require it.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    Domain choice (RQ1) propagates into participant design (RQ2): objective-ground-truth domains such as mathematics and coding favour homogeneous debaters converging on a single answer, whereas less objective domains such as healthcare or social reasoning motivate heterogeneous personas and explicit roles to broaden perspective coverage.

    “Domain choice (RQ1) propagates into participant design (RQ2): objective-ground-truth domains such as mathematics and coding favour homogeneous debaters converging on a single answer, whereas less objective domains such as healthcare or social reasoning motivate heterogeneous personas and explicit roles to broaden perspective coverage.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The authors addressed this by deriving explicit inclusion and exclusion criteria (Table 2) from a synthesis of existing definitions.

    “We addressed this by deriving explicit inclusion and exclusion criteria (Table 2) from a synthesis of existing definitions.”
    Quote in the paperSubstantial claim
106 of 106

  • The paper relies on this without support.

    The 141 selected studies are representative of the broader MAD literature.

    “It is important to clarify that the goal of this initial search was not to retrieve all existing publications on MAD but rather to identify a relevant set of high-quality papers to be used as the seed for the subsequent snowballing process.”

  • The paper relies on this without support.

    The three-dimensional taxonomy (participants, interaction, agreement) captures the essential design dimensions of MAD.

    “We derive a three-dimensional taxonomy covering debate participants, the interaction mechanisms structuring the exchange, and the agreement protocols governing debate resolution, supported by formal notations to render MAD configurations.”

  • The paper relies on this without support.

    The set of papers collected and coded for this survey is representative of the multi-agent debate literature, so prevalence figures and generalizations drawn from it hold for the field as a whole.

    “Among the surveyed approaches, 66 rely exclusively on closed-weight models (43.7%), 32 exclusively on open-weight models (21.2%), and 53 combine both (35.1%).”

  • The paper relies on this without support.

    The set of 141 included studies is sufficiently representative of the MAD research landscape for the derived taxonomy and trends to generalize.

    “This paper surveyed MAD research through a systematic review of 141 primary studies published between 2023 and 2025.”
    “Concerning conclusion validity, the main threat concerns the completeness and repeatability of the search.”

  • The paper relies on this without support.

    The design dimensions in the taxonomy are orthogonal.

    “Decomposing MAD into participants, interaction, and agreement, and each dimension into orthogonal sub-dimensions with explicit attribute values”

  • The paper relies on this without support.

    The resolution percentages computed over n = 97 are representative of all MAD approaches.

    “Exclusively, for resolution, percentages are computed over 𝑛 = 97 MAD approaches, representing the set of studies where a resolution agreement process is actually formalized and implemented.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Single-author filtering contradicts the claimed dual review of every candidate

    Minor

    The validity discussion says every candidate paper was independently assessed by at least two authors, but the selection account says one author alone performed an initial filtering step that excluded 367 papers.

  • ChatEval summarizer sends messages over absent graph edges

    Minor

    The ChatEval topology includes edges from each debater to the summarizer, but no edges from the summarizer back to debaters. The same specification nevertheless defines summarized messages sent from the summarizer to debaters, contrary to its directed-graph definition of permitted communication.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. ACM SIGSOFT Empirical Standards. n.d.. Systematic Reviews Standard. https://www2.sigsoft.org/EmpiricalStandards/docs/standards?standard= SystematicReviews#. Accessed: 2026-02-27.
  2. Yong Guan, Hao Peng, Lei Hou, and Juanzi Li. 2025. Mmd-ere: multi-agent multi-sided debate for event relation extraction. In Procs. of the 31st International Conference on Computational Linguistics.
  3. Yidong He, Yongbin Liu, Chunping Ouyang, et al. 2024. Evaluating Human-Large Language Model Alignment in Group Process. In CCF International Conference on Natural Language Processing and Chinese Computing. Springer.
  4. Renhao Li, Minghuan Tan, Derek F. Wong, and Min Yang. 2024. CoEvol: Constructing Better Responses for Instruction Finetuning through Multi-Agent Cooperation. In Procs. of the 2024 Conference on Empirical Methods in Natural Language Processing. 4703-4721.
  5. Yexiang Liu, Jie Cao, Zekun Li, Ran He, and Tieniu Tan. 2025. Breaking mental set to improve reasoning through diverse multi-agent debate. In Procs. of the 13th International Conference on Learning Representations.
  6. Long Ouyang, Jeff Wu, Xu Jiang, et al. 2022. Training language models to follow instructions with human feedback. In Procs. of the 36th International Conference on Neural Information Processing Systems. Curran Associates Inc., Article 2011, 15 pages.
  7. Andries Petrus Smit, Paul Duckworth, Nathan Grinsztajn, Kale ab Tessera, Thomas D Barrett, and Arnu Pretorius. 2023. Are we going mad? benchmarking multi-agent debate between language models for medical q&a. In Deep Generative Models for Health Workshop NeurIPS 2023.

Cited by

Sorted by citations

Authors
Motger, Q., Oriol, M., Marco, J., & Franch, X.
Affiliations
Universitat Politècnica de Catalunya (ES)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2607.26212
arXiv
2607.26212
OpenAlex
W7171878187
Cited by
0
References
147
Type
posted content
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
multi-agent debate, large language models, agentic systems, debate strategy, taxonomy (general classification scheme), consensus algorithms, benchmarking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review