The DOI matches its Crossref record; 42 of the 49 references it cites were found in a registry or an index.
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
Open accessAbstract
Large Language Models (LLMs) still struggle with natural language reasoning tasks.Motivated by the society of minds (Minsky, 1988), we propose RECONCILE, a multi-model multiagent framework designed as a round table conference among diverse LLM agents.RECON-CILE enhances collaborative reasoning between LLM agents via multiple rounds of discussion, learning to convince other agents to improve their answers, and employing a confidenceweighted voting mechanism that leads to a better consensus.In each round, RECONCILE initiates discussion between agents via a 'discussion prompt' that consists of (a) grouped answers and explanations generated by each agent in the previous round, (b) their confidence scores, and (c) demonstrations of answerrectifying human explanations, used for convincing other agents.Experiments on seven benchmarks demonstrate that RECONCILE significantly improves LLMs' reasoning -both individually and as a team -surpassing prior single-agent and multi-agent baselines by up to 11.4% and even outperforming GPT-4 on three datasets.RECONCILE also flexibly incorporates different combinations of agents, including API-based, open-source, and domainspecific models, leading to an 8% improvement on MATH.Finally, we analyze the individual components of RECONCILE, demonstrating that the diversity originating from different models is critical to its superior performance.1 Self-Refine MAD+Judge Multi-Agent Debate (MAD) ReConcile (Group-Discuss-and-Convince) Yes, with 95% confidence No, with 50% confidence No, with 40% confidence yes no no yes no no yes no no Question (Q): Is an ammonia fighting cleaner good for pet owners?Human Explanation (Exp): Ammonia is a component in pet urine.It has an unpleasant odor.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Other studies have demonstrated that structured contrast and deliberation among agents can enhance argumentative coherence and mitigate bias (Chen et al., 2024;Liu et al., 2026).”
From Multi-Agent debate system based on large language models: structured deliberation and validation in satellite communications (2026) · cites this paper as Chen et al., 2024
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Some studies show that discussion, often run as multi-agent debate, outperforms standalone reasoning or voting (Du et al., 2024; Liang et al., 2024; Chen et al., 2024), whereas others report no effect (Smit et al., 2024; Wang et al., 2024b; Kaesberg et al., 2025).”
From Multi-agent discussion gains less when dissent is withheld (2026) · cites this paper as Chen et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“In contrast, other studies report that discussion can outperform individual reasoning or voting (Du et al., 2024; Liang et al., 2024; Chen et al., 2024).”
From Multi-agent discussion gains less when dissent is withheld (2026) · cites this paper as Chen et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“The consensus-building process generally takes 2 to 3 rounds (Chen et al., 2024b;Du et al., 2024;Yang et al., 2025;Xiong et al., 2023) to be effective.”
From Free-MAD: Consensus-Free Multi-Agent Debate (2026) · cites this paper as Chen et al., 2024b
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This framework explains conflicting findings in prior work (Du et al., 2024; Liang et al., 2024; Chen et al., 2024; Smit et al., 2024; Wang et al., 2024b; Kaesberg et al., 2025) as arising from tasks operating in different regimes.”
From Multi-agent discussion gains less when dissent is withheld (2026) · cites this paper as Chen et al., 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 12
- BIG-bench collaboration. 2023. Beyond the imitation game: Quantifying and extrapolating the capabili- ties of language models. Transactions on Machine Learning Research.
- Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. 2022. Glam: Efficient scaling of language models with mixture-of-experts. In International Conference on Machine Learning, pages 5547-5569. PMLR.
- Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. 2023. Specializing smaller language models towards multi-step reasoning. In ICML.
- Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Wein- berger. 2017. On calibration of modern neural net- works. In International conference on machine learn- ing, pages 1321-1330. PMLR.
- Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. NeurIPS.
- Shuang Li, Yilun Du, Joshua B Tenenbaum, Antonio Torralba, and Igor Mordatch. 2022a. Composing en- sembles of pre-trained models via iterative consensus. International Conference on Learning Representa- tions (ICLR).
- Yanhong Li, Gang Kou, Guangxu Li, and Yi Peng. 2022b. Consensus reaching process in large-scale group decision making based on bounded confidence and social network. European Journal of Opera- tional Research, 303(2):790-802. · DOI
- John Platt et al. 1999. Probabilistic outputs for support vector machines and comparisons to regularized like- lihood methods. Advances in large margin classifiers, 10(3):61-74.
- Swarnadeep Saha, Peter Hase, and Mohit Bansal. 2023. Can language models teach weaker agents? teacher explanations improve students via theory of mind. In NeurIPS.
- Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural net- works: The sparsely-gated mixture-of-experts layer. In International Conference on Learning Representa- tions.
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023b. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
- Andy Zeng, Maria Attarian, brian ichter, Krzysztof Marcin Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael S Ryoo, Vikas Sindhwani, Johnny Lee, Vin- cent Vanhoucke, and Pete Florence. 2023. Socratic models: Composing zero-shot multimodal reasoning with language. In The Eleventh International Conference on Learning Representations.
Cited by
Sorted by citations
Large Language Models Cannot Self-Correct Reasoning Yet
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2023) · arXiv · doi:10.48550/arxiv.2310.01798 · Cited by 712
Mixture-of-Agents Enhances Large Language Model Capabilities
Wang, J., Wang, J., Athiwaratkun, B., Zhang, C., & Zou, J. (2024) · arXiv · doi:10.48550/arxiv.2406.04692 · Cited by 386
Empowering biomedical discovery with AI agents
Gao, S., Fang, A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., & Zitnik, M. (2024) · Cell · doi:10.1016/j.cell.2024.09.022 · Cited by 315
Pan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., & Wang, W. Y. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00660 · Cited by 86
When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs
Kamoi, R., Zhang, Y., Zhang, N., Han, J., & Zhang, R. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00713 · Cited by 200
Metadata
- Authors
- Chen, J., Saha, S., & Bansal, M.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Venue
- Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 7066-7085
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4402671543
- Cited by
- 64
- References
- 0
- Citations by year
- 2026: 37 · 2025: 20 · 2024: 6
- FWCI
- 18.59
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1846185, 2112635, CAREER)
- Defense Advanced Research Projects Agency (N66001-19-2-4031, N66001-19)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- multi-agent LLM systems, collaborative reasoning, Society of Mind, discussion prompts, model diversity, natural language reasoning, multi-agent debate, MATH benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
