Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
Open accessAbstract
Multi-agent debate (MAD) is a promising strategy for improving LLM reasoning, but when agents converge on a shared answer, it is unclear whether that convergence reflects genuine deliberation or social compliance. We show that the conventional answer flip rate conflates three distinct mechanisms: spontaneous instability, stance-induced conformity, and reasoning-induced persuasion. Our three-source decomposition framework isolates each through controlled counterfactual conditions. In the primary MMLU-Pro setting, 37% of agent-question observations change under self-reflection alone, while robustness tests show substantial model-dependent instability across GPQA-Diamond and three model families; strict conformity is 29% in the primary setting and remains predominantly harmful across model replications (57-77% correct-to-wrong). A controlled information-gradient experiment reveals that even vacuous reasoning is associated with 20-39% error adoption among resistant agents, with reasoning-like presentation carrying substantial persuasive weight. Harmful conformity can be predicted from Round 0 features (AUC = 0.79), and risk-targeted intervention reduces it by 13.6 percentage points (p < 0.001). However, without correctness labels or self-reflection controls, reducing peer adoption does not improve accuracy, because harmful and beneficial influence cannot be distinguished.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An agent that withholds its belief and conforms to the surroundings internalizes what it stated as its own belief with probability a.
“An agent that withholds its belief and conforms to the surroundings internalizes what it stated as its own belief with probability a (Weng et al., 2025; Hao et al., 2026).”
From Multi-agent discussion gains less when dissent is withheld (2026) · cites this paper as Hao et al., 2026
Section 3.1 describes agents changing to a peer’s answer under stance exposure, supporting the conformity portion of the claim. It does not say that agents withhold a private belief or later internalize their stated answer with probability a; it explicitly treats its labels as behavioral attributions, not claims about latent cognitive mechanisms. No page number is available in our copy. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 17
- Solomon E Asch. 1956. Studies of independence and conformity: I. a minority of one against a unanimous majority. Psychological monographs: General and applied, 70(9):1.
- Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2024. Chateval: Towards better llm-based evaluators through multi-agent debate. In International conference on learning representations, volume 2024, pages 9079-9093.
- Hyeong Kyu Choi, Jerry Zhu, and Sharon Li. 2026. Debate or vote: Which yields better decisions in multi- agent large language models? Advances in Neural Information Processing Systems, 38:101732-101764.
- Hyeong Kyu Choi, Xiaojin Zhu, and Sharon Li. 2025a. When identity skews debate: Anonymization for bias-reduced multi-agent reasoning. arXiv preprint arXiv:2510.07517. · arXiv
- Morton Deutsch and Harold B Gerard. 1955. A study of normative and informational social influences upon individual judgment. The journal of abnormal and social psychology, 51(3):629.
- Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Forty-first international conference on machine learning.
- Andrew Estornell and Yang Liu. 2024. Multi-llm debate: Framework, principals, and interventions. Advances in Neural Information Processing Systems, 37:28938-28964.
- Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Yu, Xinying Song, and Denny Zhou. 2024. Large language models cannot self-correct reasoning yet. In International conference on learning representations, volume 2024, pages 32808-32824.
- Hsien-Tsung Lin, Pei-Cing Huang, Chan-Tung Ku, Chan Hsu, Pei-Xuan Shieh, and Yihuang Kang. 2025. Towards simulating social influence dynamics with llm-based multi-agents. In 2025 IEEE International Conference on Information Reuse and Integration and Data Science (IRI), pages 307-312. IEEE.
- Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in neural information processing systems, 36:46534-46594.
- Richard E Petty and John T Cacioppo. 1986. Communi- cation and persuasion: Central and peripheral routes to attitude change. Springer Science & Business Media.
- David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
- Mrinank Sharma, Meg Tong, Tomek Korbak, David Duvenaud, Amanda Askell, Sam Bowman, Esin Durmus, Zac Hatfield-Dodds, Scott Johnston, Shauna Kravec, and 1 others. 2024. Towards understanding sycophancy in language models. In International Conference on Learning Representations, volume 2024, pages 110-144.
- Andries Petrus Smit, Nathan Grinsztajn, Paul Duck- worth, Thomas D Barrett, and Arnu Pretorius. 2024. Should we be going mad? a look at multi-agent debate strategies for llms. In International Conference on Machine Learning, pages 45883-45905.
- PMLR.
- Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2023. Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36:74952-74965.
- Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024b. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266-95290.
Cited by
Sorted by citations
Multi-agent discussion gains less when dissent is withheld
Mansuri, C. S., Wang, X., Li, M., Acton, B., Eckardt, R., Patel, D., & Kojaku, S. (2026) · arXiv · doi:10.48550/arxiv.2609.38324
“The second is internalization, in which agents come to adopt the answer they stated as their own private belief (Weng et al., 2025; Hao et al., 2026).”
- Authors
- Hao, X., Wu, Z., Qiu, Y.-X., Xiao, C., Xu, R., Zheng, S., & Qin, J.
- Affiliations
- Zhuhai Institute of Advanced Technology (CN)
- The University of Osaka (JP)
- Shenzhen University (CN)
- ORCID
- Qin, Jianbin
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2606.00820
- OpenAlex
- W7163272606
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Natural Science Foundation of China (62532007, 62472289, 2017B030314073)
- Key Laboratory of Popular Type of High-performance Computer of Guangdong Province (2017B030314073)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Opinion Dynamics and Social Influence
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- multi-agent debate, LLM reasoning, persuasion, self-reflection, peer influence, MMLU-Pro, counterfactual decomposition, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
