Self-Consistency Improves Chain of Thought Reasoning in Language Models

“…have found that even when the desired answer is fixed, introducing diversity in the reasoning processes can be highly beneficial; therefore we leverage sampling, as commonly used for open-ended text generation (Radford et al., 2019; Brown et al., 2020; Thoppilan et al., 2022), to achieve this goal.”

“In an effort to address this shortcoming, Wei et al. (2022) have proposed chain-of-thought prompting , where a language model is prompted to generate a series of short sentences that mimic the reasoning process a person might employ in solving a task.”

“Prompt with correct chain-of-thought 17.1 Prompt with imperfect chain-of-thought 14.9 + Self-consistency (40 paths) 23.4 Prompt with equations 5.0 + Self-consistency (40 paths) 6.5 PaLM-540B Zero-shot CoT (Kojima et al., 2022) 43.0 + Self-consistency (40 paths) 69.2 Table 8: Self-consistency works under imperfect prom…”

“Here we perform a study using self-consistency to see if it can help fill in the gap, over a set of common NLP tasks, including (1) Closed-Book Question Answering: BoolQ (Clark et al., 2019), HotpotQA (Yang et al., 2018), and (2) Natural Language Inference: e-SNLI (Camburu et al., 2018), ANLI (Nie et al., 2020) and RTE…”

“Large Language Models (LLMs) have achieved remarkable breakthroughs across diverse domains, including conversational systems [1, 16, 32], generative modeling [2, 17, 22], and mathematical reasoning [20, 25, 28].”

“An object displacement can invalidate the coordinates used by an action while leaving the action sequence appropriate.”

“Self-Consistency [Wang et al., 2023] counts unique click clusters across n = 5 samples under a 50 -px tolerance.”

“Chain-of-thought prompting and self-consistency improve reasoning by sampling multiple paths and aggregating final answers [21, 22].”

“Different problem settings often require different patterns of coordination, roles, and interactions among agents (Wang et al., 2023; Du et al., 2023).”

“Prior work has shown that chain-of-thought can improve task accuracy (Wei et al., 2022), and self-consistency can further improve reasoning by aggregating sampled reasoning paths (Wang et al., 2023b).”

“Multi-sampling at every agent step introduces prohibitive latency and cost in long-horizon settings [17], [16].”

“Crucially, while this initial generation process is essentially identical to standard self-consistency sampling (Wang et al., 2022), our framework uniquely repurposes these samples as an explicit seeding context within x Q .”

“Parallel test-time scaling, sampling many reasoning traces and aggregating their answers by majority vote, is among the most reliable ways to improve LLM accuracy on hard problems [29, 3, 26].”

“…tasks where the final answers can be effectively aggregated, e.g., math and short-form question answering, scaling up test-time compute by sampling multiple answers in parallel and then aggregating them has shown promise in significantly improving the model performance, e.g., majority voting [26].”

“A broader thread mitigates hallucinations through direct uncertainty estimation, semantic-disagreement signals across sampled responses, or verifier-based detectors [1, 20, 23, 40, 44, 47, 86], and recent work fine-tunes LLMs for better-calibrated verbalized confidence [34, 43].”

“While effective on complex tasks, these advances substantially increase inference cost: high-resolution images or videos induce large visual contexts, and CoT-based inference often requires long generations or repeated rollouts, such as Best-of-N [7, 8] and self-consistency [9].”

“Another branch of methods directly controls the decoding process and applies selection mechanisms, including self-consistency (Wang et al., 2022), verification (Cobbe et al., 2021), confidence filtering (Fu et al., 2025) and more (Leviathan et al., 2023; Du et al., 2023).”

“Representative manually designed strategies include Chain-of-Thought, Self-Consistency, debate-based collaboration, and iterative refinement, as well as more specialized designs such as MedPrompt, MultiPersona and MetaGPT (Wei et al., 2022, Wang et al., 2022, Du et al., 2024, Madaan et al., 2023, Nori et al., 2023, Wa…”

“An upstream agent may maintain multiple plausible hypotheses, conflicting evidence, or unstable action candidates [31, 58], but the interface requires one action, one query, one answer, or one structured field [57, 59].”

“This places the probe in the broader line of work on CoT reasoning and reasoning-time interventions, including few-shot CoT prompting [2], zero-shot CoT [3], self-consistency decoding [4], least-to-most decomposition [5], tool-or action-augmented reasoning [6], and program-aided reasoning [7].”

“In practice, they vary how long they think from task to task, often producing longer reasoning traces on inputs that appear more difficult [Alomrani et al., 2025, Zhu and Li, 2025].”

“Ensemble meth-ods such as majority voting or confidence averaging [10, 26, 49, 50, 58] provide simple aggregation but remain static, lacking the reasoning refinement and contextual adaptation of collaborative frameworks.”

“Self-consistency [6] samples multiple chains of thought from a single model and marginalises over answers, gaining +17 .”

“The reasoning search paths can take various structures, but can broadly be classified into sequential [21], graph-like [37, 1], or parallel [32, 2].”

“…inference-time scaling methods: • Base model: Single greedy pass without test-time scaling—the minimal compute baseline • Self-Consistency [51]: Samples 32 independent trajectories and selects the final answer via majority voting • Best-of-N (BoN) [4]: Generates 32 complete solutions,…”

“Subsequent work spans self-consistency [29], which aggregates multiple independent samples via majority vote; Reflexion [27], which refines answers through verbal self-reflection; Universal Self-Consistency [2], which uses an LLM aggregator to select the most coherent candidate; and multi-agent…”

“At a higher level, the work sits alongside inference-time-compute methods for language models that sample chains, branch search, or apply verifiers (Wei et al., 2022; Wang et al., 2023; Yao et al., 2023; Lightman et al., 2024; Snell et al., 2025).”

“…confirms that LLMs can generate plausible-sounding reasoning chains causally disconnected from their final answers [6,7], produce substantially different outputs under semantically equivalent inputs [8,9], and exhibit systematic cognitive-bias-like deviations that vary by model architecture [10].”

“Early work elicits such behavior through prompting [Wei et al., 2022, Wang et al., 2022, Yao et al., 2023].”

“Instead of relying on an LLM to pick the best solution as in previous work [9, 60, 61], which is expensive and inefficient, D IFF C ODE G EN selects its final output by distance-based clustering on the dynamic behavior of all generated code candidates.”

“Subsequent methods refine how this buffer is generated, verified, or searched, including self-consistency (Wang et al., 2023), STaR (Zelikman et al., 2022), least-to-most prompting (Zhou et al., 2023), Tree of Thoughts (Yao et al., 2023), and PAL (Gao et al., 2023).”

“…al., 2025; Mathematical Association of America, 2024; He et al., 2024), with gains commonly attributed to test-time scaling (Li et al., 2025; Snell et al., 2025; Wang et al., 2023; Yao et al., 2023a) and reinforcement learning-based post-training such as RLVR (Shao et al., 2024; Wen et al., 2025).”

“Self-consistency [Wang et al., 2022] parallelizes naturally by sampling multiple traces and selecting the majority answer, but is limited to tasks with extractable, votable final answers.”

“A separate line improves reasoning by allocating more inference-time compute, for example through chain-of-thought prompting, self-consistency, and adaptive test-time scaling [3, 31, 4].”

“Another line of work estimates confidence through self-consistency or ensembles by sampling multiple completions and aggregating agreement across responses [33, 31].”

“Recent studies show that self-consistency, where multiple candidate queries are generated and aggregated, can improve performance [7].”

“Following multi-trace test-time inference [26], we sample K independent reasoning traces per side and mean-pool their per-trace vectors into a robust per-side conditional state, (5)”

“Reasoning in large language models has become largely synonymous with the chain-of-thought paradigm [1] and its test-time-compute extensions [2, 3], in which the model generates intermediate reasoning tokens before producing a final answer.”

“…branch multiple reasoning trajectories and select or aggregate final answers, thereby diversifying intermediate reasoning paths during inference (Yao et al., 2022b; Du et al., 2023; Yao et al., 2023; Wang et al., 2023b; Shinn et al., 2023; Besta et al., 2024; Liao et al., 2025; Hua et al., 2026).”

“Wang et al. [42] propose Self-Consistency which samples CoT reasoning paths, selecting the most frequent answer by majority voting.”

“(2) Output Consistency Checking: We compare multiple outputs or reasoning paths to detect contradictions, following the idea of self-consistency [6].”

“Large Language Models for Reasoning Chain-of-thought prompting shows that explicitly decoding intermediate reasoning can improve multi-step problem solving [12, 33, 34].”

“This distinction has driven a surge of interest in Chain-of-Thought (CoT) prompting (Wei et al., 2022) and reasoning-focused Proceedings of the 43 rd International Conference on Machine Learning , Seoul, South Korea.”

“Chain-of-Thought (CoT) reasoning, originally introduced to elicit step-by-step logical inference in large language models, has recently been extended to vision-language models (VLMs) to enhance multimodal reasoning [20, 40, 57, 59, 81].”

“Chain-of-thought (CoT) prompting [Wei et al., 2022] has become a standard technique for improving LLM performance on multi-step reasoning tasks.”

“Chain-of-Thought prompting 40 can improve reasoning by breaking problems into multi-step demonstrations, and self-consistency 41 further aggregates multiple reasoning paths via majority voting.”

“While Self-Consistency with N = 10 samples achieves slightly higher accuracy on some tasks, MGRS uses significantly fewer shows diminishing returns: while moving from N BMC = 1 to N BMC = 4 yields substantial gains (+10.5 AUROC points, +6.7 AUPR points), further doubling to N BMC = 8 provides only marginal improvement…”

“11 These insights suggest that internal signatures of reasoning modes are measurable even when the model does not explicitly represent them.”

“Chain-of-Thought (CoT) prompting [25] encourages sequential deductive processes to enhance model adaptability, while self-consistency (SC) variants [24] aggregate multiple reflections via voting or selection to reduce variance.”

“…assignment; (2) Greedy : each agent joins the coalition maximizing immediate per-capita value; (3) Standard : direct preference query; (4) Vanilla CoT : chain-of-thought prompting [ Wei et al. , 2022 ] ; (5) Self-Consistency : multiple CoT paths with majority voting [ Wang et al. , 2023 ] .”

“T HE advent of Large Reasoning Models (LRMs) [1], [2], [3], [4] has marked a paradigm shift in artificial intelligence, enabling models to solve complex problems through test time compute scaling [5], [6] and extensive Chains of Thought (CoT) [7], [8], [9].”

“Unlike standard SC [Wang et al., 2022], it produces a trajectory { PSC ( f ) } f ∈ F revealing when commitment emerges, enabling the detection–extraction gap analysis (§4.2).”

“Self-consistency further showed that decoding strategy interacts with CoT prompting in important ways, yielding additional gains by aggregating across multiple sampled reasoning paths [7].”

“Since LLM generation is non-deterministic, standard reliability approaches query the model multiple times and apply majority voting [22].”

“Lukasik et al. (2024) show that RAIL improves over greedy and self-consistency decoding (Wang et al., 2023) on semantic text similarity and sentiment analysis tasks.”

“5 Greedy CoT [Wei et al., 2022] 78.5 74.2 65.4 68.4 – Best-of-16 [Wang et al., 2022] 84.2 79.6 68.9 73.1 +4.8 Vanilla ToT [Yao et al., 2023] 87 substrate rather than a superficial addition.”

“…capabilities by allocating additional compute at test time to produce higher-quality outputs, effectively framing generation as a guided search problem (Wang et al., 2022; Wei et al., 2022; Brown et al., 2024; Snell et al., 2024; Beeching et al., 2024; Kang et al., 2025; Dang et al., 2025).”

“The concept of CoT has been improved along different dimensions, including Auto-CoT [62], Complex-CoT [13], Self-Consistency [54], Zero-Shot-CoT [28], Iter-CoT [45], ToT [60], GoT [4], Layered-CoT [42], Multiplex CoT [22], and so on.”

“Despite utilizing an array of agents and self-consistency reasoning [36], these systems enforce premature convergence to deliver a single, frictionless executive summary.”

“Chain-of-Thought (CoT) prompting (Wang et al., 2022; Yao et al., 2023; Kim et al., 2023; Wei et al., 2022) has been widely applied to both LLMs and MLLMs in zero-shot, few-shot, and supervised fine-tuning (SFT) settings (Muen-nighoff et al., 2025), effectively improving answer quality.”

“…2.0 incorporates multiple forms of CoT [127], including Question Decomposition CoT (QDCOT) [128], Iterative Verification and Optimization CoT (ICOT) [129], data augmentation [130], and sampling [59], enhancing the framework’s logical reasoning capabilities and improving usability for non-experts.”

“…patches newly observed jailbreak prompts over time (Wang et al., 2025d), lightweight parameter interventions to modify specific behavioral traits (Wang et al., 2025b), plug-and-play safety enhancement via decoupled alignment and distillation from well-aligned LLMs (Luo et al., 2025c), and…”

“Vanilla steering surpasses self-consistency [35] under a comparable computational budget while requiring fewer decoding operations.”

“We observe that errors in effects ordering and operator generation are greatly reduced through self-consistency [42].”

“However, safety-aligned language models remain susceptible to violating this expectation, being jailbroken (Wei et al., 2023).”

“Flagged items are collected into a disagreement set and routed to an adjudication procedure [26,27].”

“…with a combination of using parallel generations from LLMs (Madaan et al., 2025; Brown et al., 2024), techniques to aggregate such generations (Wang et al., 2023; Fu et al., 2025; Zhao et al., 2025), and ways to prune or compress them to fit in the context window for subsequent model calls…”

“Self-consistency [Wang et al., 2022] demonstrates that sampling diverse reasoning paths improves single-model reliability; we apply the same principle across models, where diversity comes from training rather than sampling.”

“Logit-based methods [17, 19, 21, 25, 46, 53] assume hallucinations correspond to flat token probability distributions, while consistency-based methods [28, 39, 48] measure agreement across multiple generated responses.”

“Beyond single-sample decoding, self-consistency demonstrates that sampling many reasoning paths and selecting the most frequent final answer via majority voting yields substantial gains across reasoning tasks (Wang et al., 2022).”

“To reduce randomness in model judgments, a self-consistency strategy is adopted: multiple reasoning outputs are generated for the same query and voted upon [25].”

“Since ˆ h k provides a reliable proxy for the validity of reasoning steps, we employ the predicted correctness scores to upgrade the Self Consistency(Wang et al., 2023) baseline from a naive, unweighted majority voting to a quality-weighted inference strategy(Liu et al., 2026).”

“This connects to self-consistency research (Wang et al., 2023): multiple sampling provides opportunities to re-interpret complex external context.”

“Self-consistency [Wang et al., 2022], adopted in Gemini’s Deep Think mode [Thang Luong & Edward Lockhart], is a simple and e ff ective approach but can be computationally expensive due to the need to sample many reasoning traces per query.”

“Methods such as SelfCheckGPT [23] or Self-Consistency [41] generate multiple responses for the same input and aggregate them to filter out outliers.”

“…of work has developed various proxies for selecting high-quality reasoning chains, ranging from simple heuristics like ensemble-based agreement (Wang et al., 2023c), to those based on reasoning length and review tokens (Feng et al., 2025), high-entropy “forking” tokens (Wang et al., 2025), and…”

“We employ GPT-4o (Hurst et al., 2024) (2024-08-06) as the backbone LLM for Meta-Cognition Regulator and SCoT reasoning, and GPT-4o-mini (Hurst et al., 2024) (2024-07-18) for KG grounding, both accessed via the OpenAI API.”

“We compute a composite priority score for each sample, defined as the product of a LLM’s self-consistency (Wang et al., 2023) score and biological relevance score.”

“These issues are not necessarily related to factual errors but arise from discourse-level generation problems, particularly in longer or more complex responses, as noted in prior studies on reasoning consistency in large language models [14, 51].”

“Plan and Solve decomposes the task into a plan and execution and reduces logical gaps; self consistency samples multiple reasoning traces and aggregates them to stabilize outputs [37,38].”

“Recent advancements in Large Language Models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks, often mediated by a process known as Chain-of-Thought (CoT) prompting (Wei et al., 2022; Kojima et al., 2022; Wang et al., 2023; Yao et al., 2023).”