Self-Consistency Improves Chain of Thought Reasoning in Language Models

“…have found that even when the desired answer is fixed, introducing diversity in the reasoning processes can be highly beneficial; therefore we leverage sampling, as commonly used for open-ended text generation (Radford et al., 2019; Brown et al., 2020; Thoppilan et al., 2022), to achieve this goal.”

“In an effort to address this shortcoming, Wei et al. (2022) have proposed chain-of-thought prompting , where a language model is prompted to generate a series of short sentences that mimic the reasoning process a person might employ in solving a task.”

“Prompt with correct chain-of-thought 17.1 Prompt with imperfect chain-of-thought 14.9 + Self-consistency (40 paths) 23.4 Prompt with equations 5.0 + Self-consistency (40 paths) 6.5 PaLM-540B Zero-shot CoT (Kojima et al., 2022) 43.0 + Self-consistency (40 paths) 69.2 Table 8: Self-consistency works under imperfect prom…”

“Here we perform a study using self-consistency to see if it can help fill in the gap, over a set of common NLP tasks, including (1) Closed-Book Question Answering: BoolQ (Clark et al., 2019), HotpotQA (Yang et al., 2018), and (2) Natural Language Inference: e-SNLI (Camburu et al., 2018), ANLI (Nie et al., 2020) and RTE…”

“(2) Self-consistency tests agreement across 40 independent samples [43].”

“Single-agent baselines include CoT (Wei et al., 2022), ComplexCoT Fu et al. (2022), Self-Consistency (Wang et al., 2023), and PHP (Zheng et al., 2023).”

“Recent cascade work makes the loop explicit and online, storing the strong model’s deferral-time strategies for reuse [4, 5], the latter deferring by self-consistency [15].”

“…macro-F1 over faulty trajectories and the share of all faults detected and correctly typed; calibration is ten-bin ECE [Pakdaman Naeini et al., 2015] against the binary verdict, coarse for the rule engine’s two constants and the ensemble’s two-valued vote share; cost is seconds per…”

“When agents exchange outputs without a quality filter, stochastic variation can give one rationale an early advantage, a pattern analogous to memetic drift 28 .”

“…can express the same answer; semantic entropy instead clusters sampled generations by meaning and scores cluster diversity Kuhn et al. [2023], Farquhar et al. [2024], and self-consistency decoding marginalises over sampled reasoning paths to pick the most agreed-upon answer [Wang et al., 2023].”

“In multi-sample inference with language models, practitioners rely on the heuristic that “more samples and more agreement mean a more reliable answer” [1].”

“Black-box methods, in contrast, estimate uncertainty by querying the judge repeatedly and measuring how consistent its verdicts are (Wang et al., 2023; 2024).”

“For LLMs that fusion rule is almost always majority voting over sampled chains of thought (self-consistency [1]): draw N candidates, return the answer that appears most often.”

“The comparisons a reviewer would reasonably require — self-consistency over one-shot samples [36], unsupervised Dawid–Skene [8], and a one-shot reader instructed to emit citations [12] — are absent, and Table 2 should be read as an internal design comparison rather than as a positioning against…”

“Self-Consistency (SC), for example, selects the answer receiving the most support across independently sampled trajectories (Wang et al., 2023; Oh and Lee, 2026).”

“TTS began with early breakthroughs like CoT (Wei et al., 2022), and has since diversified into methods including tree-base approaches (Yao et al., 2023; Aygün et al., 2026), iterative feedback loops (Madaan et al., 2023; Shinn et al., 2023), and general agentic harnesses (Ning et al., 2026).”

“Beyond single-chain reasoning, recent methods explore richer deliberation structures, including Tree of Thoughts [61], Algorithm of Thoughts [62], Graph of Thoughts [63], self-consistency [64], and Reflexion [65].”

“Specifically, we employed self-consistency with chain-of-thought (SC-CoT) prompt [45].”

“Post-era models spend more inference compute per attempt, and repeated sampling alone is known to buy coverage on hard problems (Chen et al., 2021; Wang et al., 2023; Brown et al., 2024).”

“Each task is run for several independent traces; every trace is scored by every judge; and a metric’s score is the mode of the pooled votes (majority voting over samples in the spirit of self-consistency [54]), ties broken toward the lower grade.”

“Self-consistency [12] and sample-and-vote scaling [13] aggregate a single model; debate [6, 10, 37], mixture-of-agents [7], and role-structured frameworks [8, 9] aggregate several.”

“For tier k , we draw N independent responses from m k at temperature τ > 0 (the self-consistency regime of Wang et al. (2023)), giving extracted answers When the answer set A ( x ) varies per query (TruthfulQA has |A ( x ) | ∈ { 4 , . . . , 10 } ), all quantities below are defined per-query on A (…”

“…oracle, rather than an independent, answer-key-free gate that demonstrably rejects subtly-wrong candidates (§2.6); the verification principles that gate does build on are classical — differential and metamorphic testing [11, 12], test-based selection [13, 14], and machine-checked proof [10].”

“…and their limits Ring 3’s design draws on an intuition with substantial support: sampling a model repeatedly and taking the majority answer improves reasoning accuracy [102], disagreement between samples signals hallucination [68], and debate between model instances can improve factuality [31].”

“Self-consistency (Wang et al., 2023c) extends this by sampling diverse reasoning paths and selecting answers via majority voting, while Tree of Thoughts (Yao et al., 2023) generalizes CoT into tree-structured deliberate search with self-evaluation and backtracking.”

“Recent reasoning models push this additional test-time computation considerably further, either sequentially, by extending and revising ever longer chains [5]; or in parallel, by searching and exploring among ever more trials at inference [2, 4].”

“Inference-time methods avoid retraining but intervene at coarse granularity: thought-switching penalties Wang et al. [2023] and underthinking-triggered prompt insertions Zhang et al. [2025] act on the surface token stream.”

“Prior work has shown both that structured feedback can improve LLM outputs [4, 5, 8, 6] and, pointedly, that models are poor at correcting their own reasoning without external signals [7].”

“It improves self-consistency (Wang et al., 2023b), simplifies complex problem solving through decomposition (Zhou et al., 2023), facilitates planning (Wang et al., 2023a), automates demonstration generation (Zhang et al., 2023), and enhances supervised fine-tuning (Wei et al., 2022a; Chung et al.,…”

“Our analysis employed GPT-5 for multi-label classification, recognizing that a single workflow may encompass multiple task types (e.g., a workflow that generates an image and then upscales it would be classified as both text_to_image and upscaling ).”

“Test-time reasoning methods such as chain-of-thought prompting, self-consistency, and tree search show that additional inference compute can improve reasoning quality [12, 13, 14].”

“Prior work such as Self-Consistency (Wang et al., 2022) and Tree-of-Thoughts (Yao et al., 2023) shows that generating multiple distinct reasoning paths increases the probability of recovering the correct answer.”

“The verification methods that followed, including self-consistency [241] and iterative refinement [250, 165, 37, 207, 148, 147], presaged the generate-verify loops central to systems like HybridProver [99].”

“…additional inference-time context such as execution feedback, reflection, a longer reasoning budget, or failure lessons, can discover verified solutions that the unaided student currently misses [Wang et al., 2022a, Zelikman et al., 2022, Yao et al., 2023a, Shinn et al., 2023, Yao et al., 2023b].”

“Prompt-based methods steer agents through natural-language interaction patterns, including implicit textual trajectories[6, 7, 18, 24], explicit planning structures[16, 17, 19, 26], and multi-agent coordination[32–37].”

“Self-consistency [26] is an effective candidate selection method that only reports the answer that occurs most often across sampled LLM responses.”

“Smit et al. [7] demonstrate that multi-agent debate systems do not reliably outperform simpler prompting strategies such as self-consistency [8] and often exhibit high sensitivity to hyperparameter settings.”

“Sampling-based uncertainty for M/LLM. Sampling-based methods estimate uncertainty by aggregating multiple outputs, e.g. , quantify the output consistency [10] or their semantic meaning [12, 31].”

“❸ Fine-tuning methods , including Supervised Fine-Tuning (SFT) (Wei et al., 2021) and Knowledgeable-R1 (Lin et al., 2026), which train the model to improve its behavior under contextual knowledge conflicts.”

“Chain-of-thought prompting encourages models to externalize intermediate reasoning steps [28], self-consistency improves accuracy by sampling multiple reasoning traces [27], and broader studies of test-time scaling show that the value of extra compute depends strongly on how it is allocated [24].”

“For example, self-consistency [WWS + 22] can be expressed as π ϕ ( y | x, y 1: n ) = 1 { y = y i for some i ∈ arg max j N ( a ( y j )) } , where N ( a ) = =1 1 { a ( y j ) = a } and a ( y i ) is the final answer extracted from y i .”

“If explanation is necessary, techniques such as answer-consistency checks or self-consistency decoding may be combined with CoT, but they introduce additional inference cost and are therefore less attractive for disconnected or latency-sensitive operation [54].”

“Selective prediction [Geifman and El-Yaniv, 2017, 2019], reasoning-time branch selection [Wang et al., 2022b, Yao et al., 2023, Snell et al., 2025], process-level verifiers [Lightman et al., 2024, Uesato et al., 2022], post-hoc failure attribution [Yeh et al., 2026], and test-time orchestration…”

“Self-improve prompting (e.g., self-consistency [56]), focuses on generating multiple initial outputs first, then iteratively refining them to enhance performance.”

“Chain-of-thought prompting and self-consistency elicit and stabilize intermediate reasoning at inference time [8, 9].”

“However, even with improvements in reasoning approaches like Chain of Thought [9] and Self-Consistency [8], LLMs still struggle with strict framework translation.”

“This mechanism provides an implicit self-consistency check [13] without requiring expensive majority voting over many samples, and significantly reduces the rate of random errors in large-scale evaluation.”

“While explicit, single-step instructions yield low variance in their execution [68], tasks demanding complex inference or the interpretation of implicit physical or technical contexts introduce significant run-to-run variance.”

“Self-consistency samples multiple reasoning paths and aggregates answers to improve reasoning accuracy [24].”

“[37] propose a novel strategy called self-consistency to enhance CoT prompting.”

“Our core method is answer-level self-consistency, inspired by self-consistency decoding for reasoning [7].”

“This issue can be eased via upgraded foundation models and self-consistency methods [5], and our extensible framework supports future LLM upgrades for better precision.”

“Best-of-N (Stiennon et al., 2020; Cobbe et al., 2021), self-consistency (Wang et al., 2023), and retry-style evaluation (Chen et al., 2021), along with recent test-time scaling that allocates more inference compute (Snell et al., 2024; DeepSeek-AI, 2025; Muennighoff et al., 2025), all spend…”

“Existing inference-time interventions—chain-of-thought prompting (Wei et al., 2022), self-consistency (Wang et al., 2023), and tool-augmented prompting (Yao et al., 2023b; Schick et al., 2023)—are commonly used to improve reasoning reliability.”

“…SAPLMA-style linear and MLP probes [2] on per-layer hidden states; (ii) INSIDE EigenScore [4] over multiple sampled completions; (iii) self-consistency [23,18] scored by exact-match plurality and sentence-embedding similarity [21]; and (iv) attention-entropy summaries at three transformer blocks.”

“Chain-of-thought and self-consistency showed that sampled traces expose alternative reasoning paths before answers are aggregated [16, 18].”

“Group Relative Policy Optimization (GRPO) [20] samples multiple trajectories per prompt and estimates group-relative advantages from verifier outcomes, offering a scalable alternative to critic-based policy optimization [24, 3, 8].”

“Scaling test-time compute (OpenAI, 2024; DeepSeek-AI et al., 2025; Snell et al., 2024; Wu et al., 2024) – through self-consistency (Wang et al., 2022), best-of-N (Brown et al., 2024), extending unstructured chains of thought (Setlur et al., 2025), or multi-turn verification (Shao et al., 2025) –…”

“Our key observation is that prefix contamination – the event where a retained prefix contains at least one erroneous step – can be represented as a binary loss that is monotonic with respect to the threshold.”

“Subsequent methods, including self-consistency [36], process supervision [25], and reinforcement learning [10, 43, 45], further improve the robustness and quality of these thinking trajectories.”

“Test-time compute scaling, which generates multiple outputs and aggregates them via self-consistency (Wang et al., 2023) or best-of-N selection (Lightman et al., 2024; Nguyen et al., 2026b), is an effective way to improve accuracy (Snell et al., 2025; Wu et al., 2025).”

“Chain-of-thought prompting and self-consistency established the standard recipe of sampling multiple reasoning traces and aggregating final answers [Wei et al., 2022, Wang et al., 2023].”

“Two major directions include: a) Increasing reasoning trajectory: previous work like Chain-of-Thought (Wei et al., 2022) proposes to scale reasoning steps by lengthening thinking steps in a single reasoning trajectory.”

“By decomposing complex problems into intermediate reasoning steps (Wei et al., 2022; Wang et al., 2023b; Chu et al., 2023), CoT allows models to attain expert-level proficiency in sophisticated domains (Feng et al., 2023).”

“Multiple works in the literature have suggested this as a best practice for improving LLM performance on reasoning tasks [72].”

“…LLM workflows have become a common inference-time way to extract more capability than a single forward pass can deliver, often by decomposing generation, evaluation, aggregation, or delegation across multiple model calls (Yang et al., 2026; Wu et al., 2023; Madaan et al., 2023; Wang et al., 2023).”

“Specifically, we consider Chain-of-Thought (CoT) [105, 106], Tree-of-Thoughts (ToT) [107], and Self-Consistency (SC) [108].”

“The convergence suggests that generate–evaluate–refine is a natural attractor in the space of inference-time strategies discoverable by SLMs, analogous to how self-consistency and chain-of-thought emerge as effective patterns in the prompting literature [Wang et al., 2023, Wei et al., 2022].”

“Under repeated sampling, the same model and prompt that produce an incorrect solution will often produce a correct one [3, 20, 4], suggesting that the capability is present, but its reliable expression is not.”

“Solution traces such as Chain-of-Thought (CoT) (Wei et al., 2022) or Graph-of-Thought Proceedings of the 43 rd International Conference on Machine Learning , Seoul, South Korea.”

“For language tasks, methods such as best-of-N sampling, beam search, self-consistency, and tree search improve accuracy by exploring multiple candidate solutions before selecting one [31,35,36].”

“Building on chain-of-thought reasoning [35], self-consistency decoding [34], and tree search [43, 47], reinforcement learning has become the dominant post-training paradigm for LLM reasoning.”

“…analysis connects to work on test-time compute, where LLM performance is improved by sampling multiple candidate responses, applying self-consistency or voting, or allocating more inference compute to harder instances [Wang et al., 2022, Wu et al., 2024, Snell et al., 2024, Brown et al., 2024].”

“Chain-of-thought prompting [Wei et al., 2022, Kojima et al., 2022] elicits step-by-step reasoning; self-consistency [Wang et al., 2023], least-to-most prompting [Zhou et al., 2023], and Tree of Thoughts [Yao et al., 2023] extend it via marginalisation, decomposition, and branching search.”

“Semantic entropy [17] and self-consistency [24] sample many completions and aggregate, requiring multiple forward passes and producing only trace-level scores.”

“Existing methods use majority vote, self-consistency, reward models, learned verifiers, LLM-as-a-judge, or task-specific checks to choose among sampled candidates (Wang et al., 2023; Chen et al., 2023; Cobbe et al., 2021; Lightman et al., 2024; Jiang et al., 2023; Huang et al., 2025).”

“This phenomenon sharpens under self-consistency decoding with majority voting (Wang et al., 2023), which filters out random errors and reveals inputs that consistently trigger failures across multiple generations.”

“Self-consistency generates multiple reasoning paths and selects the most frequent answer among them (Wang et al., 2022).”

“Chain-of-thought prompting, self-consistency, and STaR demonstrate the value of generating or aggregating step-by-step rationales for stronger final-answer accuracy [11, 17, 18].”