Multi-agent discussion gains less when dissent is withheld
Open accessAbstract
Multi-agent systems of LLMs add discussion to majority voting and are therefore expected to be more capable. However, empirical reports conflict on whether discussion improves accuracy or leads to an incorrect consensus. Here, we introduce a parsimonious model that explains when discussion improves accuracy and when it ends in an incorrect consensus, built from four behaviors repeatedly observed in LLM agents: (1) withholding dissent, (2) internalizing a stated answer, (3) reconsidering after seeing dissent, and (4) correcting toward the correct answer. The model shows that discussion can overturn an incorrect initial majority only when the withholding rate $c$ is below a critical rate $c^* = γ/(γ+ a)$, set by the net correction rate $γ$ and the internalization rate $a$. We estimate these rates from conversation logs with a Bayesian method and place LLM teams relative to $c^*$. As the model predicts, the gain from discussion shrinks as withholding rises, across LLMs and on a hidden profile benchmark, HiddenBench, and MedEInst. Instructing agents not to withhold dissent increases this gain. Turning reasoning off also increases the gain, because reasoning raises the internalization rate $a$ and keeps agents from reconsidering a minority answer. These findings reconcile the conflicting reports and identify when discussion outperforms majority voting.
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces a parsimonious model of LLM multi-agent discussion in which teams recover from an incorrect initial majority only when the withholding rate \(c\) is below a critical value \(c^* = \gamma/(\gamma+a)\), set by net correction \(\gamma\) and internalization \(a\). Across LLMs and benchmarks, discussion gains shrink as the estimated margin \(\hat c - c^*\) grows, and instructions or reasoning settings that raise withholding or internalization reduce the benefit of discussion over majority voting.
Key findings
- Discussion can overturn an incorrect initial majority only when \(c < c^* = \gamma/(\gamma+a)\); above \(c^*\), teams can end in an incorrect consensus. The model predicts diminishing gains as withholding rises toward and above the threshold.
- In the hidden profile task, stronger conformity instructions raised the withholding rate \(\hat c\), and reasoning models remained above threshold for all instructions despite lower withholding because their internalization rate \(\hat a\) was high and lowered \(c^*\). The gain from discussion fell as the margin \(\hat c - c^*\) grew.
- Turning reasoning off produced no consistent shift in withholding but lowered internalization, raised net correction \(\hat\gamma\), and increased the gain from discussion. In a follow-up causal experiment across four LLMs, forcing \(c=1\) nearly removed the gain, while forcing \(c=0\) raised it only for gpt-4o-mini, the model whose public and private answers had diverged.
- The relationship held on HiddenBench and MedEInst, where discussion gains decreased as \(\hat c - c^*\) increased; on MuSiQue the relation was unclear, and net correction was always negative so \(c^*\) stayed at zero. Reasoning models shared evidence supporting their own choices and withheld evidence pointing to the correct answer, leaving the team short of decisive evidence.
Methods
- The authors build a mean-field mathematical model of binary opinion dynamics with four behaviors: withholding dissent, internalizing a stated answer, reconsidering after visible dissent, and correcting toward the correct answer. Agents observe \(q\) neighbors, follow local majority rule, and update public statements and private beliefs over rounds.
- They estimate withholding \(c\), internalization \(a\), and net correction \(\gamma\) from LLM conversation logs using Bayesian inference: a hierarchical Beta-Binomial model for \(c\) and \(a\), and a Bernoulli likelihood for directed belief flows \(u=\rho r\) and \(v=\rho(1-r)\) to obtain \(\gamma\). They compute \(c^*\) and the margin \(\hat c - c^*\) from the joint posterior.
- Experiments use nine LLMs, temperature 0.7, \(T=3\) discussion rounds after round 0, and a complete observation graph. The hidden profile benchmark has \(N=5\), 120 tasks with \(n_{\text{inf}}\in\{0,1,2,3\}\) informed agents fixing \(p\in\{0,0.2,0.4,0.6\}\); HiddenBench has \(N=4\) and 65 tasks; MuSiQue and MedEInst are adapted so clues are split across agents. Three instructions of increasing conformity pressure are used: A (honest), B (none), C (value cohesion).
Limitations
- The paper states that the study is limited to abstracted tasks, short-round discussions, homogeneous agents, fixed communication, and single sessions. It also states two model premises: agents reconsider with probability proportional to visible dissent, which logs cannot identify, and LLMs can state answers differing from what they actually believe; private answers are operational measurements, not guaranteed internal beliefs.
- The paper reports that the model orders settings by margin correctly but underestimates how often a team whose round-0 majority was wrong recovers; those recoveries occur when hidden clues reach the shared evidence, a path the model does not include. On MuSiQue the relation between margin and gain is unclear.
- The paper notes that synthetic validation tasks are statistically identical to one another, so coverage on real data may be lower; settings whose interval for \(c-c^*\) includes zero are treated as undecided.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
26 of 47 cited sources checked · 11 supported · 1 flagged · accuracy not yet measured
- Premisecitation§ 5 RELATED WORK · p. 8
LLMs similarly withhold unshared information.
“LLMs similarly withhold unshared information (Li et al., 2026; Zhang et al., 2026; Yadav et al., 2026; Cemri et al., 2025).”
Quote in the paperSubstantial claim2 of 3 sources checked- cites Li et al., 2026
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Zhang et al., 2026 source
The Conclusion (p. 9) says agents exchange information actively but fail to integrate what they gather. Section 5.2 (p. 8) locates the bottleneck in reasoning about and synthesizing distributed information, not in an unwillingness to communicate. That is contrary to the claim that LLMs withhold information. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured - cites Yadav et al., 2026 source
- Premisecitation§ ABSTRACT · p. 1
Empirical reports conflict on whether discussion improves accuracy or leads to an incorrect consensus.
“However, empirical reports conflict on whether discussion improves accuracy or leads to an incorrect consensus.”
- Also stated in: § 1 INTRODUCTION · p. 1 · plain
“If conformity were the only effect, discussion would make the group less accurate, but the empirical reports are inconsistent (Choi et al., 2025; Estornell & Liu, 2024; Huang et al., 2024; Wang et al., 2024a; Zhang et al., 2025).”
Quote in the paperSubstantial claim1 of 3 sources checked - Also stated in: § 1 INTRODUCTION · p. 1 · plain
- Interpretationcitation§ 1 INTRODUCTION · p. 2
As internalization turns conformed statements into private beliefs, even teams starting with a correct majority can end in an incorrect consensus.
“As internalization turns conformed statements into private beliefs, even teams starting with a correct majority can end in an incorrect consensus (Wang et al., 2024b; Wu et al., 2025; Okawa, 2026).”
Quote in the paperSubstantial claim1 of 3 sources checked- cites Wang et al., 2024b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Wu et al., 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Okawa, 2026 source
The Introduction and Discussion report that conformity can amplify modest initial biases into biased consensus. But the paper does not describe conformed statements becoming private beliefs or show a team with an initially correct majority ending in an incorrect consensus. Section 6 says its controlled tasks have no correct answers. (LLM, conf=0.86)
Experimental — accuracy not yet measured
- Premisecitation§ 1 INTRODUCTION · p. 1
Galton's argument hinges on the premise that votes are independent of one another.
“However, Galton's argument hinges on an important premise, namely that the votes are independent of one another.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Galton, 1907 source
In “Vox Populi” (page unavailable), Galton says the judgments were “unbiassed by passion and uninfluenced by oratory,” which supports a limited claim about freedom from influence. He does not state that the estimates were independent of one another or that his argument depends on that premise; he also allows that some competitors used information they picked up. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Interpretationreasoning§ 6 DISCUSSION · p. 9
The framework explains conflicting findings in prior work as arising from tasks operating in different regimes.
“This framework explains conflicting findings in prior work (Du et al., 2024; Liang et al., 2024; Chen et al., 2024; Smit et al., 2024; Wang et al., 2024b; Kaesberg et al., 2025) as arising from tasks operating in different regimes.”
Quote in the paperSubstantial claim1 of 3 sources checked- cites Du et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Liang et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Chen et al., 2024 source
The Introduction (page unavailable) discusses diversity and consensus among LLM agents, and the Results (page unavailable) compare their performance across reasoning tasks. Our copy does not explain conflicting findings in prior work as arising from tasks operating in different regimes; searches for that explanation and for task regimes found no such passage. (LLM, conf=0.70)
Experimental — accuracy not yet measured
- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Premisecitation§ 2.2 CRITICAL WITHHOLDING RATE · p. 4
Local majority rule buries minority statements.
“Local majority rule buries minority statements (Mehdizadeh & Hilbert, 2025).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Mehdizadeh & Hilbert, 2025 source
Section 2.1 (p. 3) describes a majority-vote baseline in which individuals tend to align with their local majority. Section 3.5 (p. 17) finds that a weak minority can be overwhelmed, but also that some network structures preserve a strong minority opinion. The unqualified claim that local majority rule buries minority statements overstates the paper's findings. (LLM, conf=0.86)
Experimental — accuracy not yet measured
- Findingnone given§ ABSTRACT · p. 1
Reasoning raises the internalization rate a.
“Turning reasoning off also increases the gain, because reasoning raises the internalization rate a and keeps agents from reconsidering a minority answer.”
Quote in the paperNot a substantial claim - Findingreasoning§ 1 INTRODUCTION · p. 2
As teams move further above the critical threshold, the benefit they gain from discussion compared to majority voting diminishes.
“As teams move further above the critical threshold, the benefit they gain from discussion compared to majority voting diminishes.”
Quote in the paperSubstantial claim - Findingreasoning§ ABSTRACT · p. 1
The gain from discussion shrinks as withholding rises, across LLMs and on a hidden profile benchmark, HiddenBench, and MedEInst.
“As the model predicts, the gain from discussion shrinks as withholding rises, across LLMs and on a hidden profile benchmark, HiddenBench, and MedEInst.”
Quote in the paperSubstantial claim - Findingreasoning§ ABSTRACT · p. 1
Instructing agents not to withhold dissent increases the gain from discussion.
“Instructing agents not to withhold dissent increases this gain.”
Quote in the paperSubstantial claim - Findingreasoning§ ABSTRACT · p. 1
Reasoning keeps agents from reconsidering a minority answer.
“Turning reasoning off also increases the gain, because reasoning raises the internalization rate a and keeps agents from reconsidering a minority answer.”
Quote in the paperSubstantial claim - Findingown data§ 1 INTRODUCTION · p. 2
Instructions systematically shift how often agents withhold their opinions.
“Across settings, instructions systematically shift how often agents withhold their opinions.”
Quote in the paperSubstantial claim - Findingreasoning§ ABSTRACT · p. 1
Turning reasoning off increases the gain from discussion.
“Turning reasoning off also increases the gain, because reasoning raises the internalization rate a and keeps agents from reconsidering a minority answer.”
Quote in the paperSubstantial claim - Findingown data§ 1 INTRODUCTION · p. 2
The relationship between distance from the threshold and discussion gain holds in different benchmarks, HiddenBench, MedEInst, and MuSiQue.
“This relationship between the distance from the threshold and discussion gain holds in different benchmarks, HiddenBench, MedEInst, and MuSiQue.”
Quote in the paperSubstantial claim - Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
Across the four benchmarks, withholding above a team's own critical value predicts a smaller gain from discussion.
“Across the four benchmarks, withholding above a team's own critical value predicts a smaller gain from discussion, and the decomposition and the evidence measures indicate which of the two rates a setting moves.”
Quote in the paperSubstantial claim - Findingown data§ 6 DISCUSSION · p. 9
The effect of the margin on discussion effectiveness decomposes into the LLM, the benchmark, and the instruction.
“decomposing the effect into the LLM, the benchmark, and the instruction.”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
On MuSiQue the relation between margin and discussion gain is unclear.
“On MuSiQue the relation is unclear (Fig. 3b).”
Quote in the paperNot a substantial claim - Findingreasoning§ 4.1 HIDDEN PROFILE TASK · p. 6
The gain from discussion falls as the margin ˆc - c* grows.
“The mathematical model predicts that the gain from discussion falls as withholding rises above the setting's own threshold, that is, as the margin ˆ c -c ∗ grows, and the results follow this prediction (Fig. 2d).”
Quote in the paperSubstantial claim - Findingreasoning§ 4.1 HIDDEN PROFILE TASK · p. 6
Reasoning models remain above the threshold for all instructions, despite lower withholding, because their internalization rate ˆa is high and lowers c*.
“Reasoning models remain above the threshold for all instructions, despite lower withholding, because their internalization rate ˆ a , the rate at which the private answer moves to the majority along with the statement, is high and lowers c ∗ .”
Quote in the paperSubstantial claim - Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
Reasoning models tend to share evidence supporting their own choices and to withhold evidence pointing the team to the correct answer.
“Reasoning models tend to share evidence supporting their own choices and to withhold evidence pointing the team to the correct answer (Fig. 3e).”
Quote in the paperSubstantial claim - Findingown data§ 4.1 HIDDEN PROFILE TASK · p. 6
In a follow-up experiment, fixing c externally, the observed gain moved in the direction the theory predicts across four LLMs.
“In a follow-up experiment, we fixed c externally to test whether the decline in gain with increased withholding is causal, and across four LLMs the observed gain moved in the direction the theory predicts.”
Quote in the paperSubstantial claim - Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
In HiddenBench and MedEInst, discussion gains decrease as the margin ˆc - c* increases.
“In HiddenBench and MedEInst, discussion gains decrease as the margin ˆ c -c ∗ increases (Fig. 3a,c).”
Quote in the paperSubstantial claim - Findingreasoning§ 6 DISCUSSION · p. 9
A team's ability to reach correct consensus depends on how far the withholding rate c is from the critical threshold c*.
“showing that a team's ability to reach correct consensus depends on how far the withholding rate c is from the critical threshold c ∗ .”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingown data§ 6 DISCUSSION · p. 9
The effectiveness of discussion falls as the margin ĉ - c* grows.
“finding that the effectiveness of discussion falls as the margin ˆ c -c ∗ grows”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
The ordering of settings by margin survives when the rates and the outcome are estimated on separate tasks and on separate rounds.
“The ordering of settings by margin also survives when the rates and the outcome are estimated on separate tasks and on separate rounds (Fig. 7, Appendix C.1).”
Quote in the paperNot a substantial claim - Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
The benchmark and the instruction move withholding by comparable amounts.
“The decomposition reveals that the benchmark and the instruction move withholding by comparable amounts (Fig. 3d).”
Quote in the paperNot a substantial claim - Findingown data§ A.2 PHASE BOUNDARIES · p. 16
When reconsideration is no better than guessing (r ≤ 1/2), the system does not reach the recovery regime even at zero withholding.
“when reconsideration is no better than guessing ( r ≤ 1 / 2 ) the system does not reach the recovery regime even at zero withholding.”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingown data§ C.1 ADDITIONAL ANALYSES · p. 25
Setting c = 1 (forced-majority) nearly eliminated the gain from discussion.
“Setting c = 1 (forced-majority) nearly eliminated the gain from discussion, as expected.”
Quote in the paperSubstantial claim - Findingreasoning§ 6 DISCUSSION · p. 9
Suppressing reasoning can improve group outcomes.
“Suppressing reasoning, in contrast, can improve group outcomes”
Quote in the paperSubstantial claim - Findingown data§ C.1 ADDITIONAL ANALYSES · p. 25
In experiment E1, the gain from discussion moved in the predicted direction.
“The gain from discussion moved in the predicted direction (Fig. 6).”
Quote in the paperNot a substantial claim - Findingnone given§ B.2 ESTIMATORS · p. 23
Weaker LLMs change their private answer more easily, even in rounds with no dissent and no newly shared evidence.
“Weaker LLMs change their private answer more easily, even in rounds with no dissent and no newly shared evidence.”
Quote in the paperSubstantial claim - Findingnone given§ B.2 ESTIMATORS · p. 23
The gain from discussion and the disclosure rate are similar under either elicitation (joint vs separate private answer).
“The gain from discussion and the disclosure rate are similar under either elicitation.”
Quote in the paperSubstantial claim - Findingown data§ B.2 ESTIMATORS · p. 22
When the same transcripts are replayed with the dissent shuffled, reconsideration grows in proportion to dissent in only a minority of settings.
“When we replay the same transcripts with the dissent shuffled, reconsideration grows in proportion to dissent in only a minority of settings.”
Quote in the paperSubstantial claim - Findingown data§ B.2 ESTIMATORS · p. 23
The reconsideration rate rho differs widely between rounds with a tie and rounds with a majority.
“The reconsideration rate ˆ ρ also differs widely between rounds with a tie and rounds with a majority.”
Quote in the paperSubstantial claim - Findingreasoning§ B.2 ESTIMATORS · p. 23
The difference between the two elicitations is larger for weaker LLMs.
“The difference between the two elicitations is larger for weaker LLMs.”
Quote in the paperNot a substantial claim - Findingnone given§ A.2 PHASE BOUNDARIES · p. 16
When γ < 0, stability of the correct consensus requires c > |γ|/(|γ|+a).
“(when γ < 0 , stability requires c > | γ | / ( | γ | + a ) )”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingreasoning§ A.1 DERIVATION · p. 15
A correct agent in a sea of incorrect statements always sees dissent (d = 1).
“A correct agent in a sea of incorrect statements therefore always sees dissent ( d = 1 )”
Quote in the paperSubstantial claim - Findingown data§ A.2 PHASE BOUNDARIES · p. 16
The closed-form critical withholding rate agrees with the numerical solution in every setting obtained by varying a, ρ, r, and q.
“The closed-form critical withholding rate agrees with the numerical solution in every setting obtained by varying a , ρ , r , and q ;”
Quote in the paperNot a substantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingreasoning§ 6 DISCUSSION · p. 9
Individual reasoning can undermine collective team accuracy.
“it can undermine collective team accuracy”
Quote in the paperSubstantial claim - Findingown data§ C ADDITIONAL RESULTS · p. 24
On HiddenBench, the gain from discussion falls as the margin c - c* grows.
“On HiddenBench, the gain from discussion also falls as the margin ˆ c -c ∗ grows (Section 4.2, Fig. 3a), so the central claims do not depend on our task construction or aggregation procedure.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Li et al., 2026
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingown data§ A.2 PHASE BOUNDARIES · p. 16
The withholding rate at which the incorrect consensus becomes absorbing matches the closed-form c*.
“The first is the withholding rate at which the incorrect consensus becomes absorbing, and it matches the closed-form c ∗ .”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingnone given§ 6 DISCUSSION · p. 9
The structure of the task-how information is distributed and what remains private-directly impacts both c and c*.
“Third, the structure of the task-specifically, how information is distributed and what remains private-directly impacts both c and c ∗ .”
Quote in the paperNot a substantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingnone given§ A.2 PHASE BOUNDARIES · p. 16
Internalization lowers the jump point to realistic withholding rates.
“internalization lowers the jump point to realistic withholding rates”
Quote in the paperNot a substantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
- Findingreasoning§ B.2 ESTIMATORS · p. 23
What changes between joint and separate elicitation is how rounds are attributed.
“What changes is how rounds are attributed.”
Quote in the paperSubstantial claim - Premisenone given§ 1 INTRODUCTION · p. 1
Discussion can rescue the group from an incorrect majority: members who are right can convince the rest even when they are in the minority.
“At the same time, discussion can rescue the group from an incorrect majority: members who are right can convince the rest even when they are in the minority.”
Quote in the paperNot a substantial claim - Interpretationreasoning§ 1 INTRODUCTION · p. 2
Stronger reasoning alone does not prevent an incorrect consensus unless agents remain open to revising their answers and share contrary evidence.
“Stronger reasoning alone therefore does not prevent an incorrect consensus unless agents remain open to revising their answers and share contrary evidence.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 1 INTRODUCTION · p. 1
Whether discussion outperforms majority voting depends on whether members voice their dissent or withhold it.
“Whether discussion outperforms majority voting therefore depends on whether members voice their dissent or withhold it.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 1 INTRODUCTION · p. 2
Conflicting empirical reports on discussion efficacy correspond to different regimes on one phase diagram over single-agent accuracy p and withholding rate c.
“Conflicting empirical reports on discussion efficacy then correspond to different regimes on one phase diagram over the single-agent accuracy p and the withholding rate c , on which each region is labelled by where the team ends up.”
Quote in the paperSubstantial claim - Interpretationown data§ ABSTRACT · p. 1
A parsimonious model built from four behaviors (withholding dissent, internalizing a stated answer, reconsidering after seeing dissent, and correcting toward the correct answer) explains when discussion improves accuracy and when it ends in an incorrect consensus.
“Here, we introduce a parsimonious model that explains when discussion improves accuracy and when it ends in an incorrect consensus, built from four behaviors repeatedly observed in LLMagents: (1) withholding dissent, (2) internalizing a stated answer, (3) reconsidering after seeing dissent, and (4) correcting toward the correct answer.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 1 INTRODUCTION · p. 2
Below c*, discussion can outperform majority voting even when starting from an incorrect majority.
“Below c ∗ , discussion can outperform majority voting even when starting from an incorrect majority.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 1 INTRODUCTION · p. 2
From the minimal specification of the four behaviors, the paper derives a critical withholding rate c*.
“From the minimal specification of these four behaviors, we derive a critical withholding rate c ∗ .”
Quote in the paperSubstantial claim - Interpretationreasoning§ 1 INTRODUCTION · p. 2
Together, the withholding rate and single-agent accuracy separate cases where discussion outperforms majority voting from cases where the team ends in an incorrect consensus.
“Together, the two quantities separate cases where discussion outperforms majority voting from cases where the team ends in an incorrect consensus.”
Quote in the paperSubstantial claim - Interpretationnone given§ 1 INTRODUCTION · p. 2
A simple mathematical model explains the success and failure of conforming teams from two quantities: the withholding rate and the single-agent accuracy.
“We propose a simple mathematical model that explains the success and failure of conforming teams from two quantities, the withholding rate and the single-agent accuracy, the probability that an agent answers correctly without discussion.”
Quote in the paperSubstantial claim - Interpretationreasoning§ ABSTRACT · p. 1
The model shows that discussion can overturn an incorrect initial majority only when the withholding rate c is below a critical rate c* = γ/(γ+a), set by the net correction rate γ and the internalization rate a.
“The model shows that discussion can overturn an incorrect initial majority only when the withholding rate c is below a critical rate c ∗ = γ/ ( γ + a ) , set by the net correction rate γ and the internalization rate a .”
Quote in the paperSubstantial claim- The authors state this as an assumption: The mean-field limit, where each agent effectively sees a random sample of the others, holds for the modeled teams.
- The authors state this as an assumption: Agents are abstract units that follow probabilistic rules rather than LLMs themselves.
- Premisereasoning§ 1 INTRODUCTION · p. 1
The independence premise does not hold when people or LLM agents reach a consensus, because they can discuss and influence one another's answers.
“This premise does not hold when people or large language model (LLM) agents reach a consensus, because they can discuss and influence one another's answers.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 2.2 CRITICAL WITHHOLDING RATE · p. 4
The all-correct state is stable for any c if γ > 0 (r > 1/2).
“The all-correct state is stable for any c if γ > 0 ( r > 1 / 2 ), with details in Appendix A.2.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
Their selective sharing nevertheless leaves the team short of decisive evidence.
“Their selective sharing nevertheless leaves the team short of decisive evidence.”
Quote in the paperSubstantial claim - Interpretationown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
The ordering of benchmark effects tracks how strongly a single private clue can challenge the majority.
“The ordering of the benchmark effects tracks how strongly a single private clue can challenge the majority.”
Quote in the paperNot a substantial claim - Interpretationown data§ 5 RELATED WORK · p. 8
The paper uses public and private responses to estimate key rates, such as withholding, internalization, and net correction, in a dynamical model, enabling prediction of when discussion creates gains over voting.
“We use public and private responses to estimate key rates, such as withholding, internalization, and net correction, in a dynamical model, enabling us to predict when discussion creates gains over voting.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 2.2 CRITICAL WITHHOLDING RATE · p. 4
Equation (6) sets the tipping point between collective success and failure by the balance between correction (γ), which raises c*, and internalization (a), which lowers it.
“Equation (6) sets the tipping point between collective success and failure by the balance between correction ( γ ), which raises c ∗ , and internalization ( a ), which lowers it (Appendix A.2).”
Quote in the paperSubstantial claim - Interpretationreasoning§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7
A team almost never answers correctly without sufficient shared evidence, so its accuracy is approximately the product of the evidence pooling rate and the evidence utilization rate.
“Because a team almost never answers correctly without sufficient shared evidence, its accuracy is approximately the product of the two rates.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 2.2 CRITICAL WITHHOLDING RATE · p. 4
The system can recover from an incorrect consensus only if a minority of correct agents grows at each round.
“The system can recover from an incorrect consensus only if a minority of correct agents grows at each round.”
Quote in the paperSubstantial claim - Interpretationown data§ A.2 PHASE BOUNDARIES · p. 16
Whether an absorbing state exists is decided by the fixed points, not by a round budget.
“whether an absorbing state exists is decided by the fixed points, not by a round budget.”
Quote in the paperSubstantial claim - Interpretationreasoning§ B.2 ESTIMATORS · p. 22
In settings where no agent ever enters a round with the correct answer, the data say nothing about v.
“In settings where no agent ever enters a round with the correct answer, for example when withholding is nearly complete, the data say nothing about v .”
Quote in the paperSubstantial claim - Premisereasoning§ A.1 DERIVATION · p. 15
Once the list of constraints F is fixed, the solution is unique, because maximum caliber maximizes a concave function over a convex set.
“Once the list of constraints F is fixed, the solution is unique, because maximum caliber maximizes a concave function over a convex set.”
Quote in the paperNot a substantial claim - Premisenone given§ A.1 DERIVATION · p. 15
The four rules (withholding, internalization, reconsideration after visible dissent, and correction) run in one direction in time.
“The four rules (withholding, internalization, reconsideration after visible dissent, and correction; Section 2.1) run in one direction in time.”
Quote in the paperSubstantial claim - Interpretationreasoning§ A.1 DERIVATION · p. 15
The collective behavior cannot be written as an energy over instantaneous states.
“The collective behavior therefore cannot be written as an energy over instantaneous states.”
Quote in the paperSubstantial claim - Interpretationown data§ B.2 ESTIMATORS · p. 23
The share of opposing statements alone does not determine how often an agent reconsiders.
“The share of opposing statements alone therefore does not determine how often an agent reconsiders.”
Quote in the paperSubstantial claim - Interpretationreasoning§ A.2 PHASE BOUNDARIES · p. 16
This stability explains why conformity undermines only the recovery from an incorrect majority.
“this stability explains why conformity undermines only the recovery from an incorrect majority.”
Quote in the paperSubstantial claim - Premisenone given§ A.1 DERIVATION · p. 15
At the all-incorrect state, the fraction Y of agents stating the correct answer is negligibly small (Y ≃ 0).
“At the all-incorrect state, the fraction Y of agents stating the correct answer is negligibly small ( Y ≃ 0 ).”
Quote in the paperSubstantial claim - Interpretationreasoning§ A.1 DERIVATION · p. 15
The four rules break detailed balance.
“Detailed balance requires a transition and its reverse to occur equally often, and the four rules break detailed balance.”
Quote in the paperSubstantial claim - Interpretationown data§ B.2 ESTIMATORS · p. 22
gamma should be read as the observed net flow of beliefs and not as the causal effect of dissent.
“We therefore read γ as the observed net flow of beliefs and not as the causal effect of dissent.”
Quote in the paperSubstantial claim - Study design and conductreasoning§ B.2 ESTIMATORS · p. 21
The agent-rounds from which c, a and gamma are estimated are not independent, because rounds of the same task are correlated.
“The agent-rounds from which c , a and γ are estimated (Section 3) are not independent, because rounds of the same task are correlated.”
Quote in the paperNot a substantial claim - Interpretationreasoning§ A.3 K ALTERNATIVES · p. 16
Recovery then persists at higher withholding rates.
“Recovery then persists at higher withholding rates.”
Quote in the paperSubstantial claim - Study design and conductnone given§ A.3 K ALTERNATIVES · p. 16
The threshold formula is unchanged.
“The threshold formula is unchanged.”
Quote in the paperSubstantial claim - Interpretationreasoning§ A.3 K ALTERNATIVES · p. 16
When three agents give three different incorrect answers, no answer constitutes a majority for reconsideration.
“When three agents give three different incorrect answers, no answer constitutes a majority for reconsideration.”
Quote in the paperSubstantial claim - Study design and conductreasoning§ A.3 K ALTERNATIVES · p. 16
The rules remain the same for K alternatives once the majority is the plurality of visible statements.
“The rules remain the same for K alternatives once the majority is the plurality of visible statements.”
Quote in the paperSubstantial claim - Interpretationreasoning§ A.3 K ALTERNATIVES · p. 16
When incorrect answers are divided, the correct answer is more likely to hold the relative majority.
“When incorrect answers are divided, the correct answer is more likely to hold the relative majority.”
Quote in the paperSubstantial claim - Interpretationreasoning§ A.3 K ALTERNATIVES · p. 16
Binary choice is the worst case, because all errors gather behind a single wrong answer.
“Binary choice is the worst case, because all errors gather behind a single wrong answer.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 49
- May combine two claims. 74
- The quote only partly states this claim. 2
Classified as probably not a substantial claim.
Premisecitation§ 1 INTRODUCTION · p. 1LLM multi-agent systems can fall short.
“LLM multi-agent systems can fall short (Weng et al., 2025; Zhu et al., 2025; Zhang et al., 2024; Sharma et al., 2024).”
Quote in the paperSubstantial claim2 of 3 sources checkedMay combine two claims.
Premisecitation§ 1 INTRODUCTION · p. 1As members see one another's views, the majority exerts pressure to conform, and the group converges on a shared answer without becoming more accurate.
“As members see one another's views, the majority exerts pressure to conform (Asch, 1956), and the group converges on a shared answer without becoming more accurate (Lorenz et al., 2011).”
Quote in the paperSubstantial claim1 of 2 sources checkedClassified as probably not a substantial claim. May combine two claims.
Premisecitation§ 1 INTRODUCTION · p. 1Some studies show that discussion, often run as multi-agent debate, outperforms standalone reasoning or voting, whereas others report no effect.
“Some studies show that discussion, often run as multi-agent debate, outperforms standalone reasoning or voting (Du et al., 2024; Liang et al., 2024; Chen et al., 2024), whereas others report no effect (Smit et al., 2024; Wang et al., 2024b; Kaesberg et al., 2025).”
Quote in the paperSubstantial claim1 of 3 sources checked- cites Du et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Liang et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Chen et al., 2024 source
Classified as probably not a substantial claim.
Premisecitation§ 2.1 AGENTS AND UPDATE RULES · p. 3Only public statements are visible to other agents.
“Only public statements are visible to other agents (Kuran, 1995; Garcia-Millan, 2020).”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Kuran, 1995
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Garcia-Millan, 2020 source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisenone given§ 5 RELATED WORK · p. 8Our model extends truth-seeking models based only on four repeatedly observed behaviors of LLM agents.
“our model does so based only on four repeatedly observed behaviors of LLM agents.”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites Fukushima, 2026 source
The Introduction (page unavailable) says the theory combines a crowding-generated network with a measured single-agent rule, supporting the general idea of a model grounded in agent behavior. The Conclusion (page unavailable) identifies two single-agent measurements, not four repeatedly observed behaviors; our copy does not substantiate that number or characterization. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites Pavlova & Tanaka, 2026 source
Section 4 reports observed differences in how LLM agents use visual and social evidence, and Section 5.1 reports further differences in their responses to social memory. These passages support the general reference to observed agent behaviors, but the paper does not identify four repeatedly observed behaviors as the basis for a truth-seeking model. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
Premisecitation§ 2.1 AGENTS AND UPDATE RULES · p. 3An agent that withholds its belief and conforms to the surroundings internalizes what it stated as its own belief with probability a.
“An agent that withholds its belief and conforms to the surroundings internalizes what it stated as its own belief with probability a (Weng et al., 2025; Hao et al., 2026).”
Quote in the paperSubstantial claim2 of 2 sources checked- cites Weng et al., 2025 source
Section 4.3 (page unavailable) reports that an LLM may conform to others even when it knows the correct answer. That supports the conformity part of the sentence, but the paper does not establish that the agent subsequently internalizes its stated answer as a belief, or give that event a probability of a. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites Hao et al., 2026 source
Section 3.1 describes agents changing to a peer’s answer under stance exposure, supporting the conformity portion of the claim. It does not say that agents withhold a private belief or later internalize their stated answer with probability a; it explicitly treats its labels as behavioral attributions, not claims about latent cognitive mechanisms. No page number is available in our copy. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationcitation§ 6 DISCUSSION · p. 9Private answers are operational measurements and are not guaranteed to reflect a belief held inside the LLM.
“we define the private answers as operational measurements, which are not guaranteed to reflect a belief held inside the LLM (Song et al., 2025; Rupprecht et al., 2026; Röttger et al., 2024).”
Quote in the paperNot a substantial claim3 of 3 sources checked- cites Song et al., 2025 source
The Discussion (p. 10) says prompted responses should not be conflated with a model’s direct knowledge of language, supporting the caution that an answer need not reflect an internal state. But the paper tests linguistic knowledge and word prediction, not whether private answers generally reflect beliefs held inside an LLM. The definition of private answers as operational measurements is the citing authors’ own framing. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites Rupprecht et al., 2026 source
- cites Röttger et al., 2024 source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 5 RELATED WORK · p. 7Other studies report that discussion can outperform individual reasoning or voting.
“In contrast, other studies report that discussion can outperform individual reasoning or voting (Du et al., 2024; Liang et al., 2024; Chen et al., 2024).”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites Du et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Liang et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Chen et al., 2024 source
May combine two claims.
Interpretationcitation§ 2.2 CRITICAL WITHHOLDING RATE · p. 4Above c*, two stable states coexist, and the single-agent accuracy p selects the destination.
“Above c ∗ , two stable states coexist, and the single-agent accuracy p selects the destination (Abramiuk-Szurlej et al., 2021; Ashery et al., 2025).”
Quote in the paperSubstantial claim2 of 2 sources checked- cites Abramiuk-Szurlej et al., 2021 source
The Monte Carlo results describe a range between two p spinodals in which ordered and disordered states can persist, depending on the initial condition. They do not describe coexistence as occurring above a c* threshold. The Model section defines p as the probability of anticonformity, not single-agent accuracy, and the reported destination depends on both p and the initial condition. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites Ashery et al., 2025 source
The Results section, “Tipping points and critical mass” (page unavailable), describes consensus on either of two conventions as a steady state and reports a critical mass that can flip one convention to the other. It does not establish that two stable states coexist above a threshold c*, or that single-agent accuracy p selects the destination; its threshold concerns the size of a committed minority. (LLM, conf=0.84)
Experimental — accuracy not yet measured
May combine two claims.
Premisenone given§ 5 RELATED WORK · p. 8Most prior models of collective opinion either lack a ground truth or assume full information sharing.
“However, most either lack a ground truth or assume full information sharing.”
Quote in the paperSubstantial claim2 of 3 sources checked- cites De Marzo et al., 2026b;a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Okawa, 2026 source
Section 6 (Experiments; page unavailable) says the paper’s own synthetic tasks have no correct answers. Section 4 (Background: Social Dynamics Models; page unavailable) describes prior models, but neither it nor the searched passages says that most prior models lack ground truth or assume full information sharing. Searched for those terms and for prior opinion-dynamics models. (LLM, conf=0.73)
Experimental — accuracy not yet measured - cites De Nobili, 2026 source
The Introduction and Conclusion discuss shared bias, cooperation, and whether multi-agent consensus is informative, but do not characterize most prior collective-opinion models as lacking a ground truth or assuming full information sharing. Searches for “ground truth” and “information sharing” found no passage in our copy addressing that claim. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim. May combine two claims.
Premisecitation§ 2.1 AGENTS AND UPDATE RULES · p. 3An agent that directly states its belief reconsiders its view with probability ρd_i proportional to the fraction of visible statements that dissent from its own, and reaches the correct answer through this reconsideration with probability r.
“Meanwhile, an agent that directly states its belief reconsiders its view with probability ρd i proportional to the fraction d i ∈ [0 , 1] of visible statements that dissent from its own, and reaches the correct answer through this reconsideration with probability r (Stechly et al., 2025; Jiang et al., 2025) (Fig. 1b).”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Stechly et al., 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Jiang et al., 2025 source
Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
Premisecitation§ 6 DISCUSSION · p. 9Post-training with reinforcement learning is known to narrow what a model produces.
“since post-training with reinforcement learning is known to narrow what a model produces (Kirk et al., 2024; Yue et al., 2025).”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Kirk et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Yue et al., 2025 source
May combine two claims.
Findingreasoning§ 1 INTRODUCTION · p. 2Disabling reasoning lowers a and raises γ, which raises c*, allowing the team to recover even when more agents withhold.
“We find that disabling reasoning lowers a and raises γ , which raises c ∗ , allowing the team to recover even when more agents withhold.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone given§ 1 INTRODUCTION · p. 2Reasoning models often state their dissent but also internalize the majority answers that other agents present to them.
“Reasoning models exemplify this situation: they often state their dissent but also internalize the majority answers that other agents present to them.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Findingnone given§ 1 INTRODUCTION · p. 2Reasoning models share evidence selectively, disclosing what supports their own answer and keeping contrary evidence private, so the clues needed for correction stay hidden from the team.
“Reasoning models also share evidence selectively, disclosing what supports their own answer and keeping contrary evidence private, so the clues needed for correction stay hidden from the team.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 1 INTRODUCTION · p. 2Instructions encouraging honesty reduce withholding, whereas instructions valuing cohesion push teams farther above the threshold.
“Specifically, instructions encouraging honesty reduce withholding, whereas instructions valuing cohesion push teams farther above the threshold.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7GPT-4o-mini achieves a higher evidence pooling rate but a lower utilization rate, while reasoning models show the opposite pattern.
“We find that GPT-4o-mini achieves a higher evidence pooling rate but a lower utilization rate, while reasoning models show the opposite pattern (Fig. 3f; Fig. 4 for each instruction), a divergence explained by this selective evidence sharing.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.1 HIDDEN PROFILE TASK · p. 6As conformity instructions strengthen, the withholding rate ˆc rises, and LLM agents more often align with the majority.
“As conformity instructions strengthen, the withholding rate ˆ c , the share of opportunities to dissent in which an agent states the majority, rises, and LLM agents more often align with the majority (Fig. 2b).”
Quote in the paperSubstantial claimMay combine two claims.
Findingreasoning§ 4.1 HIDDEN PROFILE TASK · p. 6As a result, agents that keep their own answer in private correct an incorrect consensus more often, and the net correction rate and the gain from discussion rise.
“As a result, agents that keep their own answer in private correct an incorrect consensus more often, and the net correction rate and the gain from discussion rise (Fig. 2eg).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.1 HIDDEN PROFILE TASK · p. 6Forcing c = 1 nearly removed the gain, while forcing c = 0 raised it only for gpt-4o-mini, and the other three models already agreed in private and in public and thus had no sizeable gain to begin with.
“Forcing c = 1 nearly removed the gain, while forcing c = 0 raised it only for gpt-4o-mini, the model whose public statements and private answers had diverged, and the other three models, gemini-3.8-flash among them, already agreed in private and in public and thus had no sizeable gain to begin with (Appendix C.1).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.1 HIDDEN PROFILE TASK · p. 6Switching reasoning off in the same LLM produces no consistent shift in withholding but lowers internalization.
“Switching reasoning off in the same LLM produces no consistent shift in withholding (Fig. 5a) but lowers internalization.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone given§ 6 DISCUSSION · p. 9Discussion yields gains below c*, with diminishing returns as c approaches c* and zero above c*.
“Discussion yields gains below c ∗ , with diminishing returns as c approaches c ∗ and zero above c ∗ .”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
May combine two claims.
Findingreasoning§ 4.1 HIDDEN PROFILE TASK · p. 6Settings under instruction C concentrate at the high-withholding end, so the team fails more often when instructed to conform to the majority.
“Settings under instruction C concentrate at this high-withholding end, so the team fails more often when instructed to conform to the majority, which is in line with what the mathematical model predicts.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7The model orders settings by margin correctly but underestimates how often a team whose round-0 majority was wrong recovers, and those recoveries occur when hidden clues reach the shared evidence, a path the model does not have.
“The model orders settings by margin correctly but underestimates how often a team whose round-0 majority was wrong recovers, and those recoveries occur when hidden clues reach the shared evidence, a path the model does not have (Fig. 8, Appendix C.1).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7Reasoning models withhold less overall and use shared evidence more effectively.
“Reasoning models withhold less overall and use shared evidence more effectively.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7Discussion gains are large on HiddenBench and small on MuSiQue and MedEInst; on MuSiQue the net correction is always negative, so c* stays at zero.
“Discussion gains are large on HiddenBench and small on MuSiQue and MedEInst (Fig. 3a-c); on MuSiQue the net correction is always negative, so c ∗ stays at zero.”
Quote in the paperNot a substantial claimMay combine two claims. The quote only partly states this claim.
Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7Withholding is lowest on HiddenBench, higher on MuSiQue, higher again on the hidden profile task, and highest on MedEInst.
“Withholding is lowest on HiddenBench, where each private clue rules out one option and gives the agent holding it a clear reason to dissent. It is higher on MuSiQue, where a private paragraph covers only one step of a multi-step answer and gives weaker grounds to disagree. It is higher again on our hidden profile task, where the answer depends on counting findings and one private clue changes the”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.2 EVALUATION ON OTHER BENCHMARKS · p. 7GPT-4o-mini pools more decisive evidence and therefore gains more from discussion despite being the weaker model.
“GPT-4o-mini pools more of this decisive evidence and therefore gains more from discussion despite being the weaker model.”
Quote in the paperSubstantial claimThe quote only partly states this claim.
Findingown data§ C.1 ADDITIONAL ANALYSES · p. 25The ordering of settings by the margin c - c* remained consistent across split-half analyses.
“The ordering of settings remained consistent across these splits.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ A.1 DERIVATION · p. 15For q ≥ 3, a correct majority requires at least two correct statements among the q observed, so M_q(Y) = O(Y^2) ≃ 0.
“For q ≥ 3 , a correct majority requires at least two correct statements among the q observed, so M q ( Y ) = O ( Y 2 ) ≃ 0 .”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ A.2 PHASE BOUNDARIES · p. 16The jump point, at which a stable state of low accuracy appears, lies very close to but distinct from c*, and the jump occurs slightly before c*.
“The second is the jump point, at which a stable state of low accuracy appears. The two points lie very close to each other but are distinct, and the jump occurs slightly before c ∗ .”
Quote in the paperNot a substantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
May combine two claims.
Findingown data§ B.2 ESTIMATORS · p. 23Team size, number of rounds, and initial accuracy introduce no bias in the estimates.
“Team size, number of rounds, and initial accuracy introduce no bias.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ B.2 ESTIMATORS · p. 21The ordering reported in the main text is the same either way (with or without accounting for correlation).
“The ordering reported in the main text is the same either way.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ B.2 ESTIMATORS · p. 23When the true withholding rate is far from c*, the sign of c - c* is estimated correctly; at c* itself, the sign is correct only about half the time.
“When the true withholding rate is far from c ∗ , the sign of c -c ∗ is estimated correctly. At c ∗ itself, the sign is correct only about half the time.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ A.1 DERIVATION · p. 15Withholding and internalization enter only where the majority contradicts the belief, and reconsideration enters wherever the agent states its belief.
“Withholding and internalization therefore enter only where the majority contradicts the belief, and reconsideration enters wherever the agent states its belief.”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
May combine two claims.
Findingown data§ C.1 ADDITIONAL ANALYSES · p. 25Setting c = 0 (transparent) increased the gain only for gpt-4o-mini, the one model where public and private answers differed in the control condition.
“Setting c = 0 (transparent) increased the gain only for gpt-4o-mini, the one model where public and private answers differed in the control condition.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ A.2 PHASE BOUNDARIES · p. 16Increasing q lowers the jump point but leaves c* unchanged.
“increasing q lowers the jump point but leaves c ∗ unchanged”
Quote in the paperNot a substantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
May combine two claims.
Findingown data§ B.2 ESTIMATORS · p. 22In the logs, the beliefs of gpt-4o-mini change whether or not dissent is visible, and reasoning models change their beliefs by yielding and then internalizing.
“In the logs, the beliefs of gpt-4o-mini change whether or not dissent is visible, and reasoning models change their beliefs by yielding and then internalizing.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Findingnone given§ 6 DISCUSSION · p. 9Reasoning agents tend to internalize exposed answers and to share only the evidence that supports their own answer.
“due to the tendency of reasoning agents to internalize exposed answers and to share only the evidence that supports their own answer.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ A.1 DERIVATION · p. 15M3 vanishes as O(Y^2) for small Y, because a single correct statement among three never forms a majority.
“M 3 vanishes as O ( Y 2 ) for small Y , because a single correct statement among three never forms a majority.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ C.1 ADDITIONAL ANALYSES · p. 25On MuSiQue, the margin equals c, because the net correction is negative in every setting and c* is therefore zero.
“On MuSiQue, the margin equals ˆ c , because the net correction is negative in every setting and c ∗ is therefore zero (Fig. 7).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ C.1 ADDITIONAL ANALYSES · p. 26In the reasoning models the visible shared evidence and the private answer sit on the majority side of most rounds, and divergence between the two appears mainly for gpt-4o-mini and under instruction C.
“In the reasoning models the visible shared evidence and the private answer sit on the majority side of most rounds, and divergence between the two appears mainly for gpt-4o-mini and under instruction C (Fig. 9).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ C.1 ADDITIONAL ANALYSES · p. 26The model predictions track the observations when the round-0 majority was already correct, and fall short when a team whose round-0 majority was incorrect recovers.
“The model predictions track the observations when the round-0 majority was already correct, and fall short when a team whose round-0 majority was incorrect recovers (Fig. 8).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone given§ B.2 ESTIMATORS · p. 22In the hidden profile task, the agent facing the most dissent is usually the one holding the correct answer.
“In the hidden profile task, the agent facing the most dissent is usually the one holding the correct answer.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ A.1 DERIVATION · p. 15Near the all-incorrect state a tie needs at least two correct statements for q ≥ 4, so the tie terms are O(Y^2) and do not change c*.
“Near the all-incorrect state a tie needs at least two correct statements for q ≥ 4 , so the tie terms are O ( Y 2 ) and do not change c ∗ .”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
May combine two claims.
Findingreasoning§ A.2 PHASE BOUNDARIES · p. 16The correct consensus (1,1) stays stable for every c and a when γ > 0, because when every agent holds and states the correct answer there is no visible disagreement to prompt reconsideration and nothing to withhold.
“The correct consensus (1 , 1) stays stable for every c and a when γ > 0 , because when every agent holds and states the correct answer there is no visible disagreement to prompt reconsideration and nothing to withhold”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
May combine two claims.
Findingnone given§ B.2 ESTIMATORS · p. 22A pooled Beta-Binomial on agent-rounds leaves interval width nearly unchanged, and coverage of the pooled model falls below the nominal level.
“A pooled Beta-Binomial on agent-rounds leaves interval width nearly unchanged, and coverage of the pooled model falls below the nominal level.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ B.2 ESTIMATORS · p. 21Treating agent-rounds as independent makes the intervals too narrow, and a task-level cluster bootstrap gives much wider intervals.
“Treating them as independent makes the intervals too narrow, and a task-level cluster bootstrap gives much wider intervals.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ A.1 DERIVATION · p. 15Having the correct answer in the majority requires k > q/2, and for q ≥ 3 this implies k ≥ 2, so the probability that the majority is correct vanishes at first order, M_q(Y) = O(Y^2).
“Having the correct answer in the majority requires k > q/ 2 , and because for q ≥ 3 this implies k ≥ 2 , the probability that the majority is correct vanishes at first order, M q ( Y ) = ∑ k>q/ 2 w k ( Y ) = O ( Y 2 ) .”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ C.1 ADDITIONAL ANALYSES · p. 25For the other three models, public and private answers already matched, so the transparent setting had little effect.
“For the other three models, public and private answers already matched, so the transparent setting had little effect, as predicted.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Findingnone given§ 6 DISCUSSION · p. 9Individual reasoning tends to benefit solo agent performance.
“while individual reasoning tends to benefit solo agent performance”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ B.2 ESTIMATORS · p. 23As a result, a, rho and r depend on the elicitation, while c is more stable.
“As a result, ˆ a , ˆ ρ and ˆ r depend on the elicitation, while ˆ c is more stable.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Findingnone given§ A.1 DERIVATION · p. 16T(1;Y) stays at the constant T(1;0) = c(1-a) + (1-c)(1-ρ(1-r)).
“Meanwhile T (1; Y ) stays at the constant T (1; 0) = c (1 -a ) + (1 -c ) ( 1 -ρ (1 -r ) ) .”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ B.2 ESTIMATORS · p. 23In the estimates for each LLM and instruction, r is close to one, while rho varies across settings.
“In the estimates for each LLM and instruction, ˆ r is close to one, while ˆ ρ varies across settings.”
Quote in the paperSubstantial claimMay combine two claims.
Findingreasoning§ A.2 PHASE BOUNDARIES · p. 16In the voter limit q = 1, no absorbing state exists and the recovery time diverges as c → 1.
“In the voter limit q = 1 no absorbing state exists and the recovery time diverges as c → 1 ;”
Quote in the paperNot a substantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingreasoning§ B.1 TASKS AND PROMPTS · p. 17The pooled clues favor the stronger candidate for every n_inf.
“The pooled clues favor the stronger candidate for every n inf .”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ Round 0 asks for an independent answer. · p. 21On the hidden profile benchmark, the round-0 accuracy p = n_inf/N is below one half except at n_inf = 3.
“On the hidden profile benchmark, the round-0 accuracy p = n inf /N is below one half except at n inf = 3 .”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ ABSTRACT · p. 1These findings reconcile the conflicting reports and identify when discussion outperforms majority voting.
“These findings reconcile the conflicting reports and identify when discussion outperforms majority voting.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationreasoning§ 1 INTRODUCTION · p. 2Instructions remedy the first pathway by reducing the withholding rate c.
“Instructions remedy the first pathway by reducing the withholding rate c .”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 1 INTRODUCTION · p. 2If the second pathway is open, voicing dissent is insufficient, because agents with strong internalization do not change their answers when a minority disagrees with them.
“However, if the second pathway is open, voicing dissent is insufficient, because agents with strong internalization do not change their answers when a minority disagrees with them.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 1 INTRODUCTION · p. 2Withholding suppresses visible dissent, removing the triggers needed for correction.
“Withholding suppresses visible dissent, removing the triggers needed for correction.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 1 INTRODUCTION · p. 2The model explains two pathways by which an LLM team can arrive at an incorrect consensus: high withholding (agents remain silent) and high internalization with low net correction (dissent is stated but fails to shift the team).
“Our model explains two pathways by which an LLM team can arrive at an incorrect consensus: one is that agents who disagree remain silent (a high withholding rate c ), so the team never hears the dissent it needs to correct itself; the other is that dissent is stated but fails to shift the team, because agents adhere to what they have already stated (a high internalization rate a ) and rarely reach”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisecitation§ 1 INTRODUCTION · p. 1The same conformity phenomenon is shown to benefit or harm performance depending on the LLM and task setting.
“The same conformity phenomenon is shown to benefit or harm performance depending on the LLM and task setting (Zhang et al., 2024).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Zhang et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone given§ 6 DISCUSSION · p. 9The study remains limited to abstracted tasks, short-round discussions, homogeneous agents, fixed communication, and single sessions.
“our study remains limited to abstracted tasks, short-round discussions, homogeneous agents, fixed communication, and single sessions.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ 3 PARAMETER ESTIMATION · p. 5The paper treats the elicited response as the measured counterpart of the private belief and does not claim that it is what an LLM believes.
“We treat this elicited response as the measured counterpart of the private belief and do not claim that it is what an LLM believes (Section 6).”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 2.2 CRITICAL WITHHOLDING RATE · p. 4If Λ(c) > 1, the correct minority (in terms of private belief Z_t) grows and otherwise vanishes.
“If Λ( c ) > 1 , the correct minority (in terms of private belief Z t ) grows and otherwise vanishes.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown data§ 5 RELATED WORK · p. 8The paper quantifies qualitative patterns of information withholding from conversation logs of LLM teams and identifies a condition under which discussion outperforms voting.
“We quantify these qualitative patterns of information withholding from conversation logs of LLM teams and identify a condition under which discussion outperforms voting.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 2.2 CRITICAL WITHHOLDING RATE · p. 4The condition Λ(c) > 1 directly compares the rate (1-c)γ at which voiced dissent corrects beliefs against the rate c·a at which agents adopt the majority in both statement and private belief, and it is equivalent to (1-c)γ > c·a.
“The condition Λ( c ) > 1 directly compares the rate (1 -c ) γ at which voiced dissent corrects beliefs against the rate c · a at which agents adopt the majority in both statement and private belief, and it is equivalent to (1 -c ) γ > c · a .”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown data§ 2.2 CRITICAL WITHHOLDING RATE · p. 4If an agent observes only one counterpart, a single correct statement is the majority for whichever agent observes that statement, and no threshold appears.
“If an agent observes only one counterpart, a single correct statement is the majority for whichever agent observes that statement, and no threshold appears (Appendix A.2).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductreasoning§ 3 PARAMETER ESTIMATION · p. 5Internalization measures the agreement of the two answers within a round rather than a sequence of stating an answer and later adopting it.
“Because y i,t and z i,t come from two calls on the same input and the private query does not show the agent its upcoming statement, internalization measures the agreement of the two answers within a round rather than a sequence of stating an answer and later adopting it.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 2.2 CRITICAL WITHHOLDING RATE · p. 4This threshold requires local majority rule with q ≥ 3.
“This threshold requires local majority rule with q ≥ 3 .”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 4.1 HIDDEN PROFILE TASK · p. 6A team collapses into an incorrect consensus when its withholding rate lies above its own critical value, and the quantity that decides the outcome is the margin ˆc - c*.
“A team collapses into an incorrect consensus when its withholding rate lies above its own critical value, and the quantity that decides the outcome is therefore the margin ˆ c -c ∗ .”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationnone given§ 5 RELATED WORK · p. 7The model reconciles these findings by focusing on the withholding rate: if it remains below c*, teams can recover from incorrect initial majorities; if it exceeds this threshold, recovery is unlikely.
“Our model reconciles these findings by focusing on the withholding rate: if it remains below a critical threshold ( c ∗ ), teams can recover from incorrect initial majorities; if it exceeds this threshold, recovery is unlikely.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationreasoning§ 2.2 CRITICAL WITHHOLDING RATE · p. 4The curve p*(c) that separates the two destinations is the phase boundary.
“The curve p ∗ ( c ) that separates the two destinations is the phase boundary (Fig. 1c).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone given§ 3 PARAMETER ESTIMATION · p. 5The margin ˆc - c* between the estimated withholding rate and the critical value decides whether an LLM-based multi-agent system is on the incorrect-consensus side of the threshold.
“We seek the margin ˆ c -c ∗ between the estimated withholding rate and the critical value, which decides whether an LLM-based multi-agent system is on the incorrect-consensus side of the threshold.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ 2.2 CRITICAL WITHHOLDING RATE · p. 4If a → 0, c* → 1, showing that withholding alone without internalization does not lock the team into an incorrect consensus.
“If a → 0 , c ∗ → 1 , showing that withholding alone without internalization does not lock the team into an incorrect consensus.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Premisenone given§ A.1 DERIVATION · p. 16u_0(0) = 0 reflects that incorrect agents do not reconsider when they observe no dissent.
“Here u 0 (0) = 0 reflects that incorrect agents do not reconsider when they observe no dissent”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ C.1 ADDITIONAL ANALYSES · p. 25MuSiQue still shows a positive gain, because its free-text answers spread the incorrect answers over many strings instead of a single competitor and because disclosed evidence stays visible.
“MuSiQue still shows a positive gain, because its free-text answers spread the incorrect answers over many strings instead of a single competitor (Appendix A.3) and because disclosed evidence stays visible.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Study design and conductnone given§ AI USE STATEMENT · p. 9We verified every result against the source data and reviewed all AI-assisted work.
“We verified every result against the source data, reviewed all AI-assisted work, and take responsibility for the final content of this paper, including text, claims, and artifacts produced with the aid of generative AI.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationreasoning§ B.2 ESTIMATORS · p. 23The synthetic tasks are statistically identical to one another, and the coverage on real data may therefore be lower.
“The synthetic tasks are statistically identical to one another, and the coverage on real data may therefore be lower.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisereasoning§ A.1 DERIVATION · p. 15A tie (k = q/2, possible only for even q) offers no majority to concede to, so the agent states its belief and only the reconsideration factor u_k(z) applies.
“A tie ( k = q/ 2 , possible only for even q ) offers no majority to concede to, so the agent states its belief and only the reconsideration factor u k ( z ) applies.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationnone given§ 6 DISCUSSION · p. 9The withholding rate c and its critical threshold c* fundamentally shape discussion effectiveness and can be modulated via task design and agent instruction.
“First, the withholding rate c and its critical threshold c ∗ fundamentally shape discussion effectiveness and can be modulated via task design and agent instruction.”
Quote in the paperSubstantial claim- The authors state this as an assumption: An agent reconsiders with a probability proportional to the dissent it sees.
- The authors state this as an assumption: LLMs can make statements that differ from what they actually believe.
May combine two claims.
Study design and conductown data§ B.2 ESTIMATORS · p. 23The estimates of all four rates and of c* are validated on synthetic logs with known true values; errors are small and intervals cover the true values.
“We validate the estimates of all four rates and of c ∗ on synthetic logs with known true values. The errors are small, and the intervals cover the true values.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone given§ Round 0 asks for an independent answer. · p. 21In the manipulation checks, we verify that accuracy at round 0 reproduces the designed value p = n_inf/N, that LLMs select the dominant candidate within their field of view, and that they can count observations.
“In the manipulation checks, we verify that accuracy at round 0 reproduces the designed value p = n inf /N , that LLMs select the dominant candidate within their field of view, and that they can count observations.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone given§ AI USE STATEMENT · p. 9We checked every reference returned by literature search tools.
“We searched the literature with Google Scholar and generative AI tools and checked every reference returned by these search tools.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationreasoning§ B.2 ESTIMATORS · p. 23In a joint reply the private answer has not yet moved to the majority, so the same round is counted as withholding instead of internalization.
“In a joint reply the private answer has not yet moved to the majority, so the same round is counted as withholding instead of internalization.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationreasoning§ A.1 DERIVATION · p. 15Whether the mathematical model is parsimonious is a question about the list of four constraints alone.
“Whether the mathematical model is parsimonious is therefore a question about the list of four constraints alone.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone given§ REPRODUCIBILITY STATEMENT · p. 9Every experiment is organized as a Snakemake workflow and the code, configurations, and prompts are provided as an anonymized archive in the supplementary material.
“We organize every experiment as a Snakemake workflow and provide the code, configurations, and prompts as an anonymized archive in the supplementary material.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone given§ B.1 TASKS AND PROMPTS · p. 17During initial manipulation checks, two candidate models were excluded for failing to reproduce designed accuracy p = n_inf/N, failing to count findings reliably, or routing through endpoints with inconsistent backend architectures.
“During initial manipulation checks, two candidate models were excluded for failing to reproduce designed accuracy p = n inf /N , failing to count findings reliably, or routing through endpoints with inconsistent backend architectures.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ AI USE STATEMENT · p. 9We tested and verified the code written with generative AI tools.
“We used generative AI tools to write the code for the experiments and the analysis, and we tested and verified that code.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationreasoning§ B.2 ESTIMATORS · p. 22The reconsideration rate measures how often a belief changes in the presence of dissent, not how often dissent causes the change.
“The reconsideration rate thus measures how often a belief changes in the presence of dissent, not how often dissent causes the change.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ C.1 ADDITIONAL ANALYSES · p. 26Such recoveries occur when evidence held privately reaches the team, and the current model includes no such recovery path.
“Such recoveries occur when evidence held privately reaches the team, and the current model includes no such recovery path.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ REPRODUCIBILITY STATEMENT · p. 9The hosted models are versioned proprietary endpoints, so the same identifier may later resolve to a different model version.
“The hosted models are versioned proprietary endpoints, so the same identifier may later resolve to a different model version.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoning§ A.3 K ALTERNATIVES · p. 16The estimator first identifies the majority by relative majority and then projects the outcomes to binary values; reversing this projection order fails when errors are dispersed and shifts the critical withholding rate.
“The estimator first identifies the majority by relative majority and then projects the outcomes to binary values; reversing this projection order fails when errors are dispersed and shifts the critical withholding rate.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Study design and conductnone given§ Round 0 asks for an independent answer. · p. 21A public answer that matches neither the agent's own private answer nor the visible majority is off-rule, and we exclude LLMs that give such answers often.
“A public answer that matches neither the agent's own private answer nor the visible majority is off-rule, and we exclude LLMs that give such answers often.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone given§ Round 0 asks for an independent answer. · p. 21The tasks are designed so that most agents cannot reach the correct answer alone.
“The tasks are designed so that most agents cannot reach the correct answer alone.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ B.1 TASKS AND PROMPTS · p. 17Settings whose LLM solved no item before or after discussion (mixtral on MuSiQue) are not drawn in Fig. 3.
“Settings whose LLM solved no item before or after discussion (mixtral on MuSiQue) are not drawn in Fig. 3.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Premisenone given§ B.1 TASKS AND PROMPTS · p. 17MedEInst is a differential diagnosis benchmark; each task pairs a typical case with a counterfactual case; in the counterfactual case, one altered finding flips the diagnosis to the correct one.
“MedEInst is a differential diagnosis benchmark. Each task pairs a typical case with a counterfactual case. In the counterfactual case, one altered finding flips the diagnosis to the correct one.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ B.2 ESTIMATORS · p. 23We treat settings whose interval for c - c* includes zero as undecided and leave them out of any decision based on the sign.
“We therefore treat settings whose interval for c -c ∗ includes zero as undecided and leave them out of any decision based on the sign.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown data§ B.1 TASKS AND PROMPTS · p. 17We exclude the MedEInst run of qwen3.8 because 3.3% of its discussion replies carry no public answer, more than the 2% we allow.
“We exclude the MedEInst run of qwen3.8: 3.3% of its discussion replies carry no public answer, more than the 2% we allow.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoning§ B.1 TASKS AND PROMPTS · p. 17Only informed agents see more findings for the stronger candidate, and the round-0 accuracy is p = n_inf/N for n_inf informed agents.
“Only informed agents therefore see more findings for the stronger candidate, and the round-0 accuracy is p = n inf /N for n inf informed agents.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Premisecitation§ B.1 TASKS AND PROMPTS · p. 17In its original form, HiddenBench is a multi-agent hidden profile benchmark; its shared briefing favors an incorrect option; each private clue rules out one incorrect option; only the correct option remains once the team pools all private clues.
“In its original form, HiddenBench is a multi-agent hidden profile benchmark. Its shared briefing favors an incorrect option. Each private clue rules out one incorrect option. Only the correct option remains once the team pools all private clues.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Li et al., 2026
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone given§ B.1 TASKS AND PROMPTS · p. 17External tasks use the same harness, so differences across tasks do not arise from the harness.
“External tasks use the same harness, so differences across tasks do not arise from the harness.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ B.2 ESTIMATORS · p. 23Asking for the private and the public answer in one reply would halve the calls.
“Asking for the private and the public answer in one reply would halve the calls.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductcitation§ B.1 TASKS AND PROMPTS · p. 17The three-line format is distinct from unstructured multi-agent debate and isolates opinion conformity and information pooling while tracking all communicated evidence.
“The three-line format is distinct from unstructured multi-agent debate (Du et al., 2024), and it isolates opinion conformity and information pooling while tracking all communicated evidence.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Du et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based).
Premisereasoning§ B.1 TASKS AND PROMPTS · p. 17The independent vote is the natural comparison on hidden profile tasks, where partitioned private clues cannot be assembled by repeatedly querying the same agent.
“The independent vote is the natural comparison on hidden profile tasks, where partitioned private clues cannot be assembled by repeatedly querying the same agent.”
Quote in the paperNot a substantial claim
The authors state this as an assumption.
The mean-field limit, where each agent effectively sees a random sample of the others, holds for the modeled teams.
“We focus on the mean-field limit, where each agent effectively sees a random sample of the others.”
The authors state this as an assumption.
Agents are abstract units that follow probabilistic rules rather than LLMs themselves.
“Within this setting, the agents are abstract units that follow probabilistic rules rather than LLMs themselves.”
The authors state this as an assumption.
Each agent starts with the correct belief independently with probability p.
“Each agent starts with the correct belief independently with probability p , the single-agent accuracy.”
The paper relies on this without support.
The four behaviors (withholding, internalization, reconsideration after dissent, and correction toward the correct answer) are the intersection of prior observations and are sufficient to model multi-agent discussion.
“While prior studies of LLM teams diverge on which mechanisms drive the outcome of discussion, we focus on the intersection of their observations, not their union, to provide a parsimonious model that applies across multi-agent systems.”
The authors state this as an assumption.
The elicited private response is a valid measured counterpart of the private belief.
“We treat this elicited response as the measured counterpart of the private belief and do not claim that it is what an LLM believes (Section 6).”
The authors state this as an assumption.
An agent reconsiders with a probability proportional to the dissent it sees.
“First premise is that an agent reconsiders with a probability proportional to the dissent it sees, which the logs cannot identify.”
The authors state this as an assumption.
LLMs can make statements that differ from what they actually believe.
“Second premise is that LLMs can make statements that differ from what they actually believe.”
The authors state this as an assumption.
The effects add on the logit scale, which means that they multiply the odds of withholding; an LLM changes the odds by the same factor under every instruction, and the residual absorbs any departure from this.
“The effects add on the logit scale, which means that they multiply the odds of withholding. An LLM changes the odds by the same factor under every instruction, and the residual absorbs any departure from this.”
The authors state this as an assumption.
The task-to-task variation nu_l is normal on the logit scale, with one variance per benchmark.
“The task-to-task variation ν ℓ is normal on the logit scale, with one variance per benchmark.”
The authors state this as an assumption.
The LLM, benchmark, interaction, and residual effects each come from their own zero-mean normal distribution, whose variance is estimated from the data.
“The LLM, benchmark, interaction, and residual effects each come from their own zero-mean normal distribution, whose variance is estimated from the data.”
The authors state this as an assumption.
The instructions are conditions we designed, not a sample from a population; their effects are therefore fixed and measured relative to B.
“The instructions are conditions we designed, not a sample from a population. Their effects are therefore fixed and measured relative to B.”
The authors state this as an assumption.
Reconsideration lands on the correct answer with probability r and on one of the visible incorrect answers with probability 1 - r.
“Reconsideration lands on the correct answer with probability r and on one of the visible incorrect answers with probability 1 -r .”
The authors state this as an assumption.
Disagreement among incorrect answers also triggers reconsideration that reaches the correct answer with probability r.
“Disagreement among incorrect answers also triggers reconsideration that reaches the correct answer with probability r .”
The paper relies on this without support.
An LLM that has just committed to a public answer tends to keep its private answer consistent with it.
“We avoid this joint elicitation because an LLM that has just committed to a public answer tends to keep its private answer consistent with it.”
Cites
Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.
Geman, S., & Geman, D. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767596 · Cited by 17,949 · cited 1× · see citing works
“We sample the posterior by Gibbs sampling (Geman & Geman, 1984) with Pólya–Gamma augmentation (Polson et al., 2013).”
Studies of independence and conformity: I. A minority of one against a unanimous majority
Asch, S. E. (1956) · Psychological Monographs: General and Applied · doi:10.1037/h0093718 · Cited by 4,267 · cited 1× · see citing works
“As members see one another’s views, the majority exerts pressure to conform (Asch, 1956), and the group converges on a shared answer without becoming more accurate (Lorenz et al., 2011).”
Statistical physics of social dynamics
Castellano, C., Fortunato, S., & Loreto, V. (2009) · Reviews of Modern Physics · doi:10.1103/revmodphys.81.591 · Cited by 4,119 · cited 2× · see citing works
“We model opinion dynamics in a team choosing between two alternatives, one correct and one incorrect, which is the standard setting of the Condorcet jury theorem (Grofman et al., 1983; Ladha, 1992) and opinion dynamics (Castellano et al., 2009a).”
Stasser, G., & Titus, W. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.48.6.1467 · Cited by 1,850 · cited 2× · see citing works
“…model, that discussion recovers a team from an incorrect initial majority only while the withholding rate c stays below c ∗ , using the hidden profile task (Stasser & Titus, 1985), where only sharing private information allows a group to find the correct answer (Stasser & Titus, 1985; 2003) (Fig.”
GALTON, F. (1907) · Nature · doi:10.1038/075450a0 · Cited by 1,080 · cited 1× · see citing works
“…explains why majority voting performs surprisingly well: nearly 800 individuals at a livestock exhibition estimated an ox’s weight largely inaccurately, but their median closely matched the true weight (Galton, 1907), because independent errors cancel out (Grofman et al., 1983; Wang et al., 2023).”
How social influence can undermine the wisdom of crowd effect
Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1008636108 · Cited by 1,017 · cited 1× · see citing works
“As members see one another’s views, the majority exerts pressure to conform (Asch, 1956), and the group converges on a shared answer without becoming more accurate (Lorenz et al., 2011).”
Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
Polson, N. G., Scott, J. G., & Windle, J. (2013) · Journal of the American Statistical Association · doi:10.1080/01621459.2013.829001 · Cited by 931 · see citing works
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · cited 1× · see citing works
“…explains why majority voting performs surprisingly well: nearly 800 individuals at a livestock exhibition estimated an ox’s weight largely inaccurately, but their median closely matched the true weight (Galton, 1907), because independent errors cancel out (Grofman et al., 1983; Wang et al., 2023).”
Self-citationPrivate Truths, Public Lies: The Social Consequences of Preference Falsification
Rasmusen, E. (1998) · doi:10.1016/s0167-2681(98)90013-0 · Cited by 794 · cited 1×
“Only public statements are visible to other agents (Kuran, 1995; Garcia-Millan, 2020).”
Minority opinion spreading in random geometry
Galam, S. (2002) · The European Physical Journal B · doi:10.1140/epjb/e20020045 · Cited by 556 · cited 1× · see citing works
“Models that separate private and public opinions (J˛edrzejewski et al., 2018; Gastner et al., 2018; Garcia-Millan, 2020; Kami´nska et al., 2026) and those using majority-rule dynamics (Galam, 2002; Krapivsky & Redner, 2003) demonstrate how public conformity impacts group consensus.”
The Condorcet Jury Theorem, Free Speech and Correlated Votes
Ladha, K. (1992) · American Journal of Political Science · doi:10.2307/2111584 · Cited by 503 · cited 1× · see citing works
“We model opinion dynamics in a team choosing between two alternatives, one correct and one incorrect, which is the standard setting of the Condorcet jury theorem (Grofman et al., 1983; Ladha, 1992) and opinion dynamics (Castellano et al., 2009a).”
Thirteen theorems in search of the truth
Grofman, B., Owen, G., & Feld, S. (1983) · Theory and Decision · doi:10.1007/bf00125672 · Cited by 460 · cited 2× · see citing works
“We model opinion dynamics in a team choosing between two alternatives, one correct and one incorrect, which is the standard setting of the Condorcet jury theorem (Grofman et al., 1983; Ladha, 1992) and opinion dynamics (Castellano et al., 2009a).”
The Minimum Entropy Production Principle
Jaynes, E. (1980) · Annual Review of Physical Chemistry · doi:10.1146/annurev.pc.31.100180.003051 · Cited by 456 · cited 1× · see citing works
“We write the collective dynamics instead as a measure over trajectories, and we fix that measure by maximum caliber, the path counterpart of maximum entropy (Jaynes, 1980; Pressé et al., 2013; Ghosh et al., 2020).”
Dynamics of Majority Rule in Two-State Interacting Spin Systems
Krapivsky, P. L., & Redner, S. (2003) · Physical Review Letters · doi:10.1103/physrevlett.90.238701 · Cited by 452 · cited 1× · see citing works
“Models that separate private and public opinions (J˛edrzejewski et al., 2018; Gastner et al., 2018; Garcia-Millan, 2020; Kami´nska et al., 2026) and those using majority-rule dynamics (Galam, 2002; Krapivsky & Redner, 2003) demonstrate how public conformity impacts group consensus.”
Hidden Profiles: A Brief History
Stasser, G. L., & Titus, W. (2003) · Psychological Inquiry · doi:10.1080/1047840x.2003.9682897 · Cited by 434 · cited 1× · see citing works
“…model, that discussion recovers a team from an incorrect initial majority only while the withholding rate c stays below c ∗ , using the hidden profile task (Stasser & Titus, 1985), where only sharing private information allows a group to find the correct answer (Stasser & Titus, 1985; 2003) (Fig.”
Wittenbaum, G. M., Hollingshead, A. B., & Botero, I. C. (2004) · Communication Monographs · doi:10.1080/0363452042000299894 · Cited by 433 · cited 1× · see citing works
“Groups of people often fail hidden-profile tasks by relying on shared information and seeking confirmation for initial preferences (Stasser & Titus, 1985; Greitemeyer & Schulz-Hardt, 2003; Wittenbaum et al., 2004), and dissent can help them recover (Schulz-Hardt et al., 2006).”
Group decision making in hidden profile situations: Dissent as a facilitator for decision quality.
Schulz‐Hardt, S., Brodbeck, F. C., Mojzisch, A., Kerschreiter, R., & Frey, D. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.91.6.1080 · Cited by 416 · cited 1× · see citing works
“Groups of people often fail hidden-profile tasks by relying on shared information and seeking confirmation for initial preferences (Stasser & Titus, 1985; Greitemeyer & Schulz-Hardt, 2003; Wittenbaum et al., 2004), and dissent can help them recover (Schulz-Hardt et al., 2006).”
Principles of maximum entropy and maximum caliber in statistical physics
Pressé, S., Ghosh, K., Lee, J., & Dill, K. A. (2013) · Reviews of Modern Physics · doi:10.1103/revmodphys.85.1115 · Cited by 399 · cited 1× · see citing works
“We write the collective dynamics instead as a measure over trajectories, and we fix that measure by maximum caliber, the path counterpart of maximum entropy (Jaynes, 1980; Pressé et al., 2013; Ghosh et al., 2020).”
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., & Tu, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.19118 · Cited by 320 · cited 3× · see citing works
“In contrast, other studies report that discussion can outperform individual reasoning or voting (Du et al., 2024; Liang et al., 2024; Chen et al., 2024).”
Self-citationExploring Collaboration Mechanisms for LLM Agents: A Social Psychology ViewPreprint
Zhang, J., Xu, X., Liu, R., & Deng, S.-M. (2023) · arXiv.org · doi:10.48550/arxiv.2310.02124 · Cited by 300 · see citing works
♫ MuSiQue: Multihop Questions via Single-hop Question Composition
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00475 · Cited by 283 · cited 12× · see citing works
“On MuSiQue the relation is unclear (Fig.”
InfluentialGreitemeyer, T., & Schulz‐Hardt, S. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.84.2.322 · Cited by 268 · cited 1× · see citing works
“Groups of people often fail hidden-profile tasks by relying on shared information and seeking confirmation for initial preferences (Stasser & Titus, 1985; Greitemeyer & Schulz-Hardt, 2003; Wittenbaum et al., 2004), and dissent can help them recover (Schulz-Hardt et al., 2006).”
Castellano, C., Muñoz, M. A., & Pastor‐Satorras, R. (2009) · Physical Review E · doi:10.1103/physreve.80.041129 · Cited by 265 · cited 2× · see citing works
“We model opinion dynamics in a team choosing between two alternatives, one correct and one incorrect, which is the standard setting of the Condorcet jury theorem (Grofman et al., 1983; Ladha, 1992) and opinion dynamics (Castellano et al., 2009a).”
Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?Preprint
Wang, Q., Wang, Z., Su, Y., Tong, H.-H., & Song, Y.-Q. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2402.18272 · Cited by 243 · cited 3× · see citing works
“As internalization turns conformed statements into private beliefs, even teams starting with a correct majority can end in an incorrect consensus (Wang et al., 2024b; Wu et al., 2025; Okawa, 2026).”
Sharma, M., Tong, M., Korbak, T., Duperrouzel, D., Was, H., Korbak, J., Cheng, M., Askell, A., Bai, Y., Brain, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.13548 · Cited by 132 · cited 1× · see citing works
“LLM multi-agent systems can fall short (Weng et al., 2025; Zhu et al., 2025; Zhang et al., 2024; Sharma et al., 2024).”
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023) · arXiv · doi:10.48550/arxiv.2305.14325 · Cited by 96 · cited 4× · see citing works
“In contrast, other studies report that discussion can outperform individual reasoning or voting (Du et al., 2024; Liang et al., 2024; Chen et al., 2024).”
InfluentialRöttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H., Schuetze, H., & Hovy, D. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.816 · Cited by 89 · see citing works
Emergent social conventions and collective bias in LLM populations
Ashery, A. F., Aiello, L. M., & Baronchelli, A. (2025) · Science Advances · doi:10.1126/sciadv.adu9368 · Cited by 81 · cited 1× · see citing works
“Above c ∗ , two stable states coexist, and the single-agent accuracy p selects the destination (Abramiuk-Szurlej et al., 2021; Ashery et al., 2025).”
Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?
Wang, Q., Wang, Z., Su, Y., Tong, H., & Song, Y. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.331 · Cited by 80 · see citing works
The Maximum Caliber Variational Principle for Nonequilibria
Ghosh, K., Dixit, P. D., Agozzino, L., & Dill, K. A. (2020) · Annual Review of Physical Chemistry · doi:10.1146/annurev-physchem-071119-040206 · Cited by 74 · cited 1× · see citing works
“We write the collective dynamics instead as a measure over trajectories, and we fix that measure by maximum caliber, the path counterpart of maximum entropy (Jaynes, 1980; Pressé et al., 2013; Ghosh et al., 2020).”
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems
Ren, R., Agarwal, A., Mazeika, M., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2503.03750 · Cited by 66 · cited 1×
“We do not interpret private responses as true beliefs, since they may simply reflect context or question framing (Cummins et al., 2025; Yang & Jia, 2026; Ren et al., 2025).”
Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning StrategiesPreprint
Wang, J., Jain, S., Zhang, D., et al. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2406.06461 · Cited by 65 · cited 5× · see citing works
“The same conformity phenomenon is shown to benefit or harm performance depending Preprint on the LLM and task setting (Zhang et al., 2024).”
InfluentialReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
Chen, J. C.-Y., Saha, S., & Bansal, M. (2023) · arXiv · doi:10.48550/arxiv.2309.13007 · Cited by 64 · cited 3× · see citing works
“In contrast, other studies report that discussion can outperform individual reasoning or voting (Du et al., 2024; Liang et al., 2024; Chen et al., 2024).”
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
Chen, J., Saha, S., & Bansal, M. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.381 · Cited by 64 · see citing works
(2025) · doi:10.52202/085713-4082 · Cited by 51 · see citing works
Consensus time in a voter model with concealed and publicly expressed opinions
Gastner, M. T., Oborny, B., & Gulyás, M. (2018) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/aac14a · Cited by 45 · cited 1× · see citing works
“Models that separate private and public opinions (J˛edrzejewski et al., 2018; Gastner et al., 2018; Garcia-Millan, 2020; Kami´nska et al., 2026) and those using majority-rule dynamics (Galam, 2002; Krapivsky & Redner, 2003) demonstrate how public conformity impacts group consensus.”
Conformity, Confabulation, and Impersonation: Persona Inconstancy in Multi-Agent LLM Collaboration
Baltaji, R., Hemmatian, B., & Varshney, L. (2024) · C3NLP · doi:10.48550/arxiv.2405.03862 · Cited by 37 · cited 1×
“LLM agents tend to conform to majority views (Weng et al., 2025; Zhu et al., 2025; Zhang et al., 2024), yet respond differently when assured of privacy (Baltaji et al., 2024; YS, 2026; Haider & Vosoughi, 2026; Ghaffarizadeh et al., 2026), revealing a public–private gap in answers.”
Large Language Models Cannot Self-Correct Reasoning Yet
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2023) · arXiv · doi:10.48550/arxiv.2310.01798 · Cited by 36 · cited 2× · see citing works
“Detailed experimental settings for every benchmark and figure are in Appendix B.1. fail to correct initial errors (Choi et al., 2025; Estornell & Liu, 2024; Huang et al., 2024; Wang et al., 2024a; Smit et al., 2024; Zhang et al., 2025).”
Think then act or act then think?
Jędrzejewski, A., Marcjasz, G., Nail, P. R., & Sznajd-Weron, K. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206166 · Cited by 31 · cited 1× · see citing works
“Models that separate private and public opinions (J˛edrzejewski et al., 2018; Gastner et al., 2018; Garcia-Millan, 2020; Kami´nska et al., 2026) and those using majority-rule dynamics (Galam, 2002; Krapivsky & Redner, 2003) demonstrate how public conformity impacts group consensus.”
Phase transition and information cascade in a voting model
Hisakado, M., & Mori, S. (2009) · doi:10.1088/1751-8113/43/31/315207 · Cited by 29 · cited 1×
“Voting models with herding also reveal shifts in collective accuracy (Hisakado & Mori, 2010; 2011).”
Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs
Li, Y., Naito, A., & Shirado, H. (2025) · doi:10.48550/arxiv.2505.11556 · Cited by 27 · cited 14×
“8-flash) were tested under instruction B (temperature 0.7) on 120 hidden profile tasks and 65 HiddenBench tasks.”
InfluentialDigital herders and phase transition in a voting model
Hisakado, M., & Mori, S. (2011) · arXiv.org · doi:10.1088/1751-8113/44/27/275204 · Cited by 25 · cited 1×
“Voting models with herding also reveal shifts in collective accuracy (Hisakado & Mori, 2010; 2011).”
Multi-LLM Debate: Framework, Principals, and Interventions
Estornell, A., & Liu, Y. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-0911 · Cited by 23 · cited 2× · see citing works
“Detailed experimental settings for every benchmark and figure are in Appendix B.1. fail to correct initial errors (Choi et al., 2025; Estornell & Liu, 2024; Huang et al., 2024; Wang et al., 2024a; Smit et al., 2024; Zhang et al., 2025).”
Abramiuk-Szurlej, A., Lipiecki, A., Pawłowski, J., & Sznajd-Weron, K. (2021) · Scientific Reports · doi:10.1038/s41598-021-97155-0 · Cited by 18 · see citing works
Understanding the Effects of RLHF on LLM Generalisation and Diversity
Kirk, R., Mediratta, I., Nalmpantis, C., Luketina, J., Hambro, E., Grefenstette, E., & Raileanu, R. (2023) · arXiv · doi:10.48550/arxiv.2310.06452 · Cited by 17 · cited 1× · see citing works
“…reasoning, in contrast, can improve group outcomes, suggesting the need for training methods that incentivize information sharing and transparent reasoning among agents, since post-training with reinforcement learning is known to narrow what a model produces (Kirk et al., 2024; Yue et al., 2025).”
Why Do Multi-Agent LLM Systems Fail?
Cemri, M., Pan, M. Z., Yang, S., Agrawal, L. A., Chopra, B., Tiwari, R., Keutzer, K., Parameswaran, A., Klein, D., Ramchandran, K., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.13657 · Cited by 15 · cited 1× · see citing works
“LLMs similarly withhold unshared information (Li et al., 2026; Zhang et al., 2026; Yadav et al., 2026; Cemri et al., 2025).”
Voting or Consensus? Decision-Making in Multi-Agent Debate
Kaesberg, L. B., Becker, J., Wahle, J. P., Ruas, T., & Gipp, B. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.606 · Cited by 14 · cited 2× · see citing works
“Some studies show that discussion, often run as multi-agent debate, outperforms standalone reasoning or voting (Du et al., 2024; Liang et al., 2024; Chen et al., 2024), whereas others report no effect (Smit et al., 2024; Wang et al., 2024b; Kaesberg et al., 2025).”
Conformity Generates Collective Misalignment in AI Agents Societies
Marzo, G. D., Bellina, A., Castellano, C., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2605.10721 · Cited by 11
Conformity in large language models
Zhu, X., Zhang, C., Stafford, T., Collier, N., & Vlachos, A. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.195 · Cited by 10 · see citing works
LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying CircuitPreprint
Pandey, M. (2026) · arXiv.org · doi:10.48550/arxiv.2604.19117 · Cited by 8 · cited 1× · see citing works
“The first behavior is withholding, in which agents privately dissent while publicly aligning with the majority (Weng et al., 2025; YS, 2026; Pandey, 2026).”
The concealed voter model is in the voter model universality class
Garcia-Millan, R. (2020) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ab7a26 · Cited by 8 · cited 2×
“Models that separate private and public opinions (J˛edrzejewski et al., 2018; Gastner et al., 2018; Garcia-Millan, 2020; Kami´nska et al., 2026) and those using majority-rule dynamics (Galam, 2002; Krapivsky & Redner, 2003) demonstrate how public conformity impacts group consensus.”
Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies
Wang, J., Jain, S., Zhang, D., Ray, B., Kumar, V., & Athiwaratkun, B. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.1112 · Cited by 8 · see citing works
Röttger, P., Hofmann, V., Pyatkin, V., Hinck, M., Kirk, H. R., Schütze, H., & Hovy, D. (2024) · arXiv · doi:10.48550/arxiv.2402.16786 · Cited by 7 · cited 1× · see citing works
“In our study, we define the private answers as operational measurements, which are not guaranteed to reflect a belief held inside the LLM (Song et al., 2025; Rupprecht et al., 2026; Röttger et al., 2024).”
SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses
Jiang, D., Zhang, J., Weller, O., Weir, N., Van Durme, B., & Khashabi, D. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i23.34603 · Cited by 7 · cited 4×
“Detailed experimental settings for every benchmark and figure are in Appendix B.1. fail to correct initial errors (Choi et al., 2025; Estornell & Liu, 2024; Huang et al., 2024; Wang et al., 2024a; Smit et al., 2024; Zhang et al., 2025).”
InfluentialWhen Do LLMs Admit Their Mistakes? Understanding the Role of Model Belief in Retraction
Yang, Y., & Jia, R. (2025) · arXiv.org · doi:10.48550/arxiv.2505.16170 · Cited by 7 · cited 1×
“We do not interpret private responses as true beliefs, since they may simply reflect context or question framing (Cummins et al., 2025; Yang & Jia, 2026; Ren et al., 2025).”
Chen, W.-T., Zhu, Z., Huang, G., et al. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2601.06636 · Cited by 6 · cited 11×
“In MedEInst, zero to three of the five agents are randomly assigned the one crucial clue that changes the answer, while the others receive neutral findings; Fig.”
InfluentialDebate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?
Choi, H. K., Zhu, J., & Li, S. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-3405 · Cited by 6
Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
Smit, A., Duckworth, P., Grinsztajn, N., Barrett, T. D., & Pretorius, A. (2023) · arXiv · doi:10.48550/arxiv.2311.17371 · Cited by 5 · cited 3× · see citing works
“Detailed experimental settings for every benchmark and figure are in Appendix B.1. fail to correct initial errors (Choi et al., 2025; Estornell & Liu, 2024; Huang et al., 2024; Wang et al., 2024a; Smit et al., 2024; Zhang et al., 2025).”
Cognitive dissonance in large language models is neither cognitive nor dissonant
Cummins, J., Elson, M., & Hussey, I. (2025) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2517912122 · Cited by 5 · cited 1× · see citing works
“We do not interpret private responses as true beliefs, since they may simply reflect context or question framing (Cummins et al., 2025; Yang & Jia, 2026; Ren et al., 2025).”
AI agents can coordinate beyond human scale
De Marzo, G., Castellano, C., & Garcia, D. (2024) · arXiv · doi:10.48550/arxiv.2409.02822 · Cited by 4 · see citing works
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Yue, Y., Song, S., & Huang, G. (2025) · arXiv · doi:10.48550/arxiv.2504.13837 · Cited by 3 · cited 1× · see citing works
“…reasoning, in contrast, can improve group outcomes, suggesting the need for training methods that incentivize information sharing and transparent reasoning among agents, since post-training with reinforcement learning is known to narrow what a model produces (Kirk et al., 2024; Yue et al., 2025).”
On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks
Stechly, K., Valmeekam, K., & Kambhampati, S. (2024) · arXiv · doi:10.48550/arxiv.2402.08115 · Cited by 3 · cited 2× · see citing works
“The third is that visible dissent from others prompts agents to reconsider (Stechly et al., 2025; Jiang et al., 2025).”
Everyone Conforms, No One Believes: Pluralistic Ignorance in LLM Agent Populations
Yashwanth, Y. (2026) · doi:10.48550/arxiv.2608.02758 · Cited by 3 · cited 2×
“The first behavior is withholding, in which agents privately dissent while publicly aligning with the majority (Weng et al., 2025; YS, 2026; Pandey, 2026).”
Language Models Fail to Introspect About Their Knowledge of Language
Song, S., Hu, J., & Mahowald, K. (2025) · arXiv · doi:10.48550/arxiv.2503.07513 · Cited by 2 · cited 1× · see citing works
“In our study, we define the private answers as operational measurements, which are not guaranteed to reflect a belief held inside the LLM (Song et al., 2025; Rupprecht et al., 2026; Röttger et al., 2024).”
Competition between private and expressed opinions in binary choice: The α-EPO q-voter model.
Kaminska, B. A., Nowak, B., Lipiecki, A., et al. (2026) · Physical Review E · doi:10.1103/8w9t-jpvp · Cited by 2
Do as We Do, Not as You Think: the Conformity of Large Language Models
Weng, Z., Chen, G., & Wang, W. (2025) · arXiv · doi:10.48550/arxiv.2501.13381 · Cited by 1 · cited 5× · see citing works
“The first behavior is withholding, in which agents privately dissent while publicly aligning with the majority (Weng et al., 2025; YS, 2026; Pandey, 2026).”
InfluentialCan LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning
Wu, H., Li, Z., & Li, L. (2025) · arXiv · doi:10.48550/arxiv.2511.07784 · Cited by 1 · cited 1× · see citing works
“As internalization turns conformed statements into private beliefs, even teams starting with a correct majority can end in an incorrect consensus (Wang et al., 2024b; Wu et al., 2025; Okawa, 2026).”
Ghaffarizadeh, A., Mohaddes, D., Izadkhah, A., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2607.02507 · Cited by 1 · cited 1×
“LLM agents tend to conform to majority views (Weng et al., 2025; Zhu et al., 2025; Zhang et al., 2024), yet respond differently when assured of privacy (Baltaji et al., 2024; YS, 2026; Haider & Vosoughi, 2026; Ghaffarizadeh et al., 2026), revealing a public–private gap in answers.”
Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?
Choi, H. K., Zhu, X., & Li, S. (2025) · arXiv · doi:10.48550/arxiv.2508.17536 · cited 4×
“Detailed experimental settings for every benchmark and figure are in Appendix B.1. fail to correct initial errors (Choi et al., 2025; Estornell & Liu, 2024; Huang et al., 2024; Wang et al., 2024a; Smit et al., 2024; Zhang et al., 2025).”
InfluentialStop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity
Zhang, H., Cui, Z., Chen, J., Wang, X., Zhang, Q., Wang, Z., Wu, D., & Hu, S. (2025) · arXiv · doi:10.48550/arxiv.2502.08788
Self-citationConformity in Large Language Models
Zhu, X., Zhang, C., Stafford, T., Collier, N., & Vlachos, A. (2024) · arXiv · doi:10.48550/arxiv.2410.12428
Zhang, Y., Liu, F., Shan, Y., Huang, X., Yang, X., Zhu, Y., Cheng, X., Liu, C., Zeng, K., Zhang, T. J., et al. (2026) · arXiv · doi:10.48550/arxiv.2603.01045 · cited 1×
“LLMs similarly withhold unshared information (Li et al., 2026; Zhang et al., 2026; Yadav et al., 2026; Cemri et al., 2025).”
When Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMs
Mehdizadeh, A., & Hilbert, M. (2025) · arXiv · doi:10.48550/arxiv.2510.19107 · cited 1×
“Local majority rule buries minority statements (Mehdizadeh & Hilbert, 2025).”
Prompt Perturbations Reveal Human-Like Biases in Large Language Model Survey Responses
Rupprecht, J., Ahnert, G., & Strohmaier, M. (2026) · Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science · doi:10.18653/v1/2026.nlpcss-1.1 · cited 1×
“In our study, we define the private answers as operational measurements, which are not guaranteed to reflect a belief held inside the LLM (Song et al., 2025; Rupprecht et al., 2026; Röttger et al., 2024).”
Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
Hao, X., Wu, Z., Qiu, Y.-X., Xiao, C., Xu, R., Zheng, S., & Qin, J. (2026) · arXiv · doi:10.48550/arxiv.2606.00820 · cited 2×
“The second is internalization, in which agents come to adopt the answer they stated as their own private belief (Weng et al., 2025; Hao et al., 2026).”
More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration
Yadav, A., Black, S., & Sourbut, O. (2026) · arXiv · doi:10.48550/arxiv.2604.07821 · cited 1×
“LLMs similarly withhold unshared information (Li et al., 2026; Zhang et al., 2026; Yadav et al., 2026; Cemri et al., 2025).”
Flag Game: A Toy Model for Mechanistic Swarm Interpretability
Pavlova, E., & Tanaka, H. (2026) · arXiv · doi:10.48550/arxiv.2609.19124 · cited 1×
“Concurrent work extends these models to truth-seeking settings (Fukushima, 2026; Pavlova & Tanaka, 2026), and our model does so based only on four repeatedly observed behaviors of LLM agents.”
Emergence of Biased Consensus in Multi-Agent LLM Debates
Okawa, M. (2026) · arXiv · doi:10.48550/arxiv.2608.02827 · cited 1×
“As internalization turns conformed statements into private beliefs, even teams starting with a correct majority can end in an incorrect consensus (Wang et al., 2024b; Wu et al., 2025; Okawa, 2026).”
Fukushima, M. (2026) · arXiv · doi:10.48550/arxiv.2609.19183 · cited 1×
“Concurrent work extends these models to truth-seeking settings (Fukushima, 2026; Pavlova & Tanaka, 2026), and our model does so based only on four repeatedly observed behaviors of LLM agents.”
Chen, W., Huang, G., Wang, W., et al. (2026) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/2026.acl-long.1847
Flagged first, then by citations
Research works printed but not linked: 1
- Syed Ali Haider and Soroush Vosoughi. Misalignment contagion: Can a misaligned minority shift aligned agents in Multi-Agent LLM deliberation? In Conference on Language Modeling (COLM) , 2026.
- Authors
- Mansuri, C. S., Wang, X., Li, M., Acton, B., Eckardt, R., Patel, D., & Kojaku, S.
- Affiliations
- Department of Physics, Mathematics and Informatics (BY)
- Binghamton University (US)
- IBM (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2609.38324
- OpenAlex
- W7215132816
- Cited by
- 0
- References
- 76
- Type
- posted content
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Opinion Dynamics and Social Influence
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Control Multi-Agent Systems
- Keywords
- majority voting, group decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review