Self-Consistency Improves Chain of Thought Reasoning in Language Models

“…have found that even when the desired answer is fixed, introducing diversity in the reasoning processes can be highly beneficial; therefore we leverage sampling, as commonly used for open-ended text generation (Radford et al., 2019; Brown et al., 2020; Thoppilan et al., 2022), to achieve this goal.”

“In an effort to address this shortcoming, Wei et al. (2022) have proposed chain-of-thought prompting , where a language model is prompted to generate a series of short sentences that mimic the reasoning process a person might employ in solving a task.”

“Prompt with correct chain-of-thought 17.1 Prompt with imperfect chain-of-thought 14.9 + Self-consistency (40 paths) 23.4 Prompt with equations 5.0 + Self-consistency (40 paths) 6.5 PaLM-540B Zero-shot CoT (Kojima et al., 2022) 43.0 + Self-consistency (40 paths) 69.2 Table 8: Self-consistency works under imperfect prom…”

“Here we perform a study using self-consistency to see if it can help fill in the gap, over a set of common NLP tasks, including (1) Closed-Book Question Answering: BoolQ (Clark et al., 2019), HotpotQA (Yang et al., 2018), and (2) Natural Language Inference: e-SNLI (Camburu et al., 2018), ANLI (Nie et al., 2020) and RTE…”

“Subsequent works such as Self-Consistency [32] and Tree-of-Thought [40] encouraged multi-path exploration and reflection.”

“Wang et al. [2023] extended chain-of-thought by sampling multiple reasoning paths and selecting the most frequent answer.”

“Problem-only methods receive only the problem statement: zero-shot chain-of-thought (Wei et al., 2022), self-consistency (Wang et al., 2022), Feedback Descent (Lee et al., 2025), Best-of-N —which selects from N i.i.d. samples via the same pairwise preference mechanism used by D UEL -”

“We compare our injection-based approach against several commonly used prompting alternatives, including few-shot prompting, self-consistency (Wang et al., 2022), and tree-of-thought (Yao et al., 2023).”

“In contrast to previous task decomposition methods (Wei et al., 2022; Besta et al., 2024) that premise on unbounded reasoning expansion, Halo treats reasoning as a resource-constrained optimization problem.”

“Explicit chain-of-thought (CoT) prompting is one of the most effective ways to improve reasoning in language models [Wei et al., 2022, Wang et al., 2022b, Zhou et al., 2022, Khot et al., 2022, Wang et al., 2022a, Sprague et al., 2024], but it is expensive at deployment: output length, latency, and…”

“Q: Does this video contain a humorous punchline?”

“Wang et al. showed that this ensemble-of-thoughts approach significantly improved accuracy on math and commonsense reasoning benchmarks [5].”

“Another common aggregation method is Majority Voting[3, 54, 43], which selects the most frequently generated answer across multiple reasoning paths.”

“Additionally, we also compare with three widely-adapted and advanced reasoning strategies: chain of thought (CoT) (Wei et al., 2022), few-shot reasoning (Alayrac et al., 2022), and self-consistency (Wang et al., 2023).”

“Foundational techniques such as Chain-of-Thought (CoT) (Wei et al., 2023) and Self-Consistency (Wang et al., 2023) demonstrated that linear reasoning traces significantly improve problem-solving capabilities.”

“Different from single-pass decoding, test-time scaling elicits chain-of-thought (CoT) traces and spend additional compute by sampling, searching, or verifying multiple candidates before selecting a final output [21, 27].”

“Post-generation selection strategies Selection heuristics have become a popular approach for improving performance without retraining [Wang et al., 2023].”

“By generating multiple candidate solutions and selecting the best one through verification (Irvine et al., 2023) or majority voting (Wang et al., 2023), models can achieve accuracy far beyond single-run inference.”

“This score function can be constructed in various ways, such as by directly querying an LLM (Tian et al., 2023; Guan et al., 2024) or by capturing frequency (Wang et al., 2023; Manakul et al., 2023).”

“Through prompting step-by-step reasoning, CoT enables LLMs to decompose complex problems into simpler sub-tasks, thus improving their problem-solving capabilities (Yao et al., 2023; Wang et al., 2023; Zhou et al., 2023; Wang et al., 2024).”

“Five different TTS methods have been evaluated: CoT prompting (Wang et al., 2022) and four search against verifiers, as shown in Figure 2.”

“It is supported by empirical observation that LLM errors on atomic tasks tend to be diverse rather than systematic (Wang et al., 2023).”

“Expanding the length of thought allows LRMs to simulate more detailed reasoning paths and iteratively verify answers, but it also introduces unnecessary redundant tokens and inference burden for simple tasks [5]–[7].”

“Reasoning Language Models (RLMs) (OpenAI, 2025b; DeepSeek, 2025) have achieved remarkable performance, but they come with high costs and latency (Wei et al., 2023; Shinn et al., 2023; Wang et al., 2023b; Cao et al., 2025).”

“Simultaneously, test-time strategies such as self-consistency (Wang et al., 2022a), CoT decoding (Wang and Zhou, 2024), ESC (Li et al., 2024) and Pass@k evaluation incur significant computational costs, particularly when sampling multiple trajectories is required to optimize the performance.”

“Alignment techniques such as instruction tuning and chain-of-thought prompting improve their reasoning ability [14–17], but they remain prone to hallucinations and logical errors.”

“SC. Self-Consistency Wang et al. [2022]: to enhance robustness, we aggregate five CoT-generated solutions through an ensemble step over the candidate answers.”

“Self-consistency methods [21] vary sampled reasoning paths to improve accuracy; Semantic Stability varies prompt semantics under deterministic decoding to measure structural disagreement.”

“To enhance the rationality of parameter configurations, we employ chain-of-thought(Wei et al. 2022) prompting strategies to guide LLMs through step-by-step reasoning.”

“Best-of-N and Majority Voting are often referred to as Consistency-based method, as they are a proxy to how consistent the model is in its response.”

“Meanwhile, the self-consistency strategy improves output stability and diversity by generating multiple versions of an anesthesia plan for the same input data [32].”

“This elicitation-only view is empirically supported by test-time scaling studies, where majority voting often matches RLVR peak performance [Wang et al., 2023, Karan and Du, 2026].”

“Sampling-based methods estimate uncertainty through self-consistency (Wang et al., 2023c), semantic entropy (Kuhn et al., 2023), or ensemble disagreement (Chen et al., 2024).”

“Large Language Models perform better on reasoning-heavy tasks such as math, problem-solving, and coding by increasing test-time computation (Wei et al., 2022; Wang et al., 2023; Snell et al., 2025).”

“Self-Consistency [8] addresses reasoning tasks by generating multiple independent samples and selecting the most frequent answer through majority voting.”

“11 shows representative success VLM leads to more robust and precise 3D visual grounding.”

“Diverse agents engage in multi-round discussions where they critique each other’s reasoning and progressively refine their proposals [4, 13, 47].”

“…Setlur et al., 2025; Arora and Zanette, 2025; Wu et al., 2024; Liu et al., 2025; Du et al., 2025; Huang et al., 2025a) or a best-of-k strategy (Wang et al., 2023; Yao et al., 2023a; Brown et al., 2024; Levi, 2024; Schaeffer et al., 2025b; Huang et al., 2025b; Chen et al., 2024b; Du et al.,…”

“Prior committee-style methods (Wang et al., 2023; Brown-Cohen et al., 2024; Wu et al., 2025) often use unstructured deliberation or majority vote among generalist models for evaluation, debate, or tool choice.”

“…over pure transformers (Wang et al., 2025a), reinforcing the value of SSM-based models—especially as inference-time scaling is increasingly used to enhance LM reasoning through techniques like task decomposition (Wei et al., 2022; Yao et al., 2023) and increased sampling (Wang et al., 2023).”

“Pairing temperature ‐ zero decoding with fi xed seeds and majority ‐ vote self ‐ consistency is a possible mitigation route [26, 37].”

“Prompting strategies such as Chain-of-Thought (CoT) encourage models to externalize their reasoning process, generating intermediate “thinking” steps that can extend for hundreds or thousands of tokens to solve a problem (Wei et al., 2022; Wang et al., 2022).”

“Building on CoT, Self-Consistency samples multiple paths and selects a majority answer [37].”

“We further apply self-consistency prompting to evaluate whether VLM’s generated rationales align with its internal decision-making [31, 53, 54].”

“…that the established inference-time mechanisms of dynamic in-context learning (Liu et al., 2021; Zhou et al., 2024; Sarukkai et al., 2025) and self-consistency cascades (Wang et al., 2022b; Yue et al., 2024) can be combined to achieve distillation-like cost reductions while preserving agility.”

“…stray predictions [23]; (ii) self-consistency verification , where multiple prompting or sampling is used and only consensus outputs are accepted [42,5]; and (iii) integrating symbolic consistency checks as a filter layer—e.g., verifying that a flagged vulnerability is consistent with the…”

“Mathematical Reasoning with LLMs LLM reasoning has advanced through chain-of-thought prompting (Wei et al., 2022), self-consistency (Wang et al., 2023), and outcome-level reinforcement learning (DeepSeek-AI, 2025; OpenAI, 2024).”

“This method utilizes expert-annotated chain-of-thought Wei et al. (2022) to structure the LLM’s problem-solving process.”

“Chain-of-Thought (CoT) (Wei et al. 2022) breaks reasoning into sequential steps, while self-consistency (Wang et al. 2022) aggregates multiple reasoning paths.”

“Traditional CoT guides responses step by step (Wei et al., 2023; Savage et al., 2024).”

“Subsequent work has enhanced its reliability through methods like self-consistency (Wang et al. 2023; Yu, He, and Ying 2024) and Tree-of-Thought (Yao et al. 2023).”

“Extensions such as Auto-CoT[38] and Self-Consistency[39] further automate reasoning chain generation and reduce errors by aggregating diverse solutions.”

“To encourage the LLM to articulate its step-by-step reasoning, we apply Chain-of-Thought prompting (CoT) (Wei et al. 2022) leveraging our key observations on the Freebase KG widely used in this study: this KG organizes each relation into a three-level hierarchy, i.e., domain.source type.target type…”

“Self-Consistency generates multiple reasoning chains and selects the most consistent result, improving robustness [27].”

“The textual component is decoded into natural language via a tokenizer, producing human-interpretable reasoning outputs for the target tasks (Wei et al., 2022; Wang et al., 2023b; OpenAI, 2023; Touvron et al., 2023a).”

“Na¨ıve truncation strategies that simply cut reasoning sequences at fixed points disrupt logical coherence, leading to substantial performance loss [17,18].”

“Results can then be aggregated with techniques like self-consistency (Wang et al., 2022; Chen et al., 2023) or verifiers.”

“Unimodal Applications Among the most straight-forward strategies is Self-Consistency (Wang et al., 2023a), which samples diverse chains of thought and selects the most frequent solution as the final output.”

“Linear strategies such as Chain-of-Thought (CoT) [16] and Self-Consistent CoT [15] improved multi-step reasoning but explored limited paths.”

“Among these, self-consistency [4, 21, 26] leverages the stochasticity of LLMs to produce diverse reasoning paths and synthesizes them for more reliable outputs, reducing single-path errors and ensuring alignment with the NLQ.”

“• Self-Consistency (SC, Wang et al., 2023) samples multiple candidate responses given the same input question and selects the most frequently occurring answer as the final output.”

“Complementar-ily, external TTS scales the breadth of inference by generating and evaluating multiple reasoning trajectories.”

“Autoregressive (AR) language models achieve strong reasoning accuracy, especially with chain-of-thought (CoT) prompting and its variants such as zero-shot CoT and self-consistency (Wei et al., 2022b; Kojima et al., 2022; Wang et al., 2023b).”

“Our results also highlight the essential roles of the Decomposer and Critic in the D E CRIM pipeline.”

“Contexts are inter-pretable and explainable for users and developers (Wei et al., 2022; Wang et al., 2022), allow rapid integration of new knowledge at runtime (Lewis et al., 2020; Borgeaud et al., 2022), and can be shared across models or modules in a compound system (Khot et al., 2022).”

“Among them, Chain-of-Thought (CoT) prompting has emerged as a key technique for improving LLM reasoning by breaking problems into intermediate steps, enabling better performance on multiple tasks (Wei et al., 2022; Lyu et al., 2023; Li et al., 2023; Feng et al., 2024).”

“…generated by the LLM, fails to execute successfully 1 , we rerun the model with a slightly increased temperature (with a preset maximum number of attempts) and return the numerical answer returned by the model’s first executable code generation (in contrast to, e.g., majority-vote schemes [38]).”

“This paradigm is widely used in ensemble methods like LLM-Blender (Jiang et al., 2023), which ranks outputs from multiple diverse models, and self-consistency frameworks (Wang et al., 2022) that aggregate multiple reasoning paths.”

“Examples include a re-solve that reaches the same answer (Wang et al., 2022) and a backward probe that recovers an answer from reconstructed reasoning (Weng et al., 2023).”

“Self-consistency (Wang et al., 2023) instead aggregates across multiple sampled CoT paths.”

“Ensemble learning, self-consistency, and mixture-of-agents architectures combine multiple inferences to improve predictive accuracy and variance reduction [3, 22–24].”

“Chain-of-thought prompting and self-consistency showed that eliciting or sampling intermediate reasoning can improve answer accuracy [16, 44, 45].”

“Structured intermediate text—chain-of-thought [74, 38], self-consistency extensions [69, 104, 89], reasoning-trace RL [8], and multimodal CoT [99, 49, 81]—has been the locus of capability gains in both LLMs and VLMs. MM-CoT [93] benchmarks whether visual CoT is grounded in images; Thinking in Space…”

“Several uncertainty scores have been proposed for LLMs. Token-level entropy (Malinin & Gales, 2021), sequence-level NLL, and self-consistency scores (Wang et al., 2023) all proxy model confidence.”

“Subsequent work improves this linguistic interface through sampling and aggregation, as in self-consistency (Wang et al., 2023b), or through explicit planning, as in Plan-and-Solve prompting and Tree of Thoughts (Wang et al., 2023a; Yao et al., 2023).”

“Step-by-step reasoning has thus become a dominant paradigm for tackling complex tasks (Wei et al., 2022; Yao et al., 2023; Huang & Chang, 2023).”

“The premise has support (debate and self-consistency can raise accuracy (Du et al., 2023; Wang et al., 2023; Li et al., 2024)), and it now underwrites production pipelines for reasoning, code, and decision support.”

“For example, Wang et al. [2022] use it to describe a sampling-and-voting procedure to stabilize responses.”

“It includes role prompting, few-shot examples, output formatting, chain-of-thought prompting, self-consistency, and automatic prompt optimization [15, 19, 18, 23].”

“Chain-of-Thought (Wei et al., 2022) and its variants (Wang et al., 2023; Khot et al., 2023; Zhou et al., 2023) improve single-turn reasoning but do not track beliefs across turns.”

“While early advances such as self-consistency [37] and hierarchical task decomposition [55] focused on improving the robustness of internal reasoning, the emergence of VLA models has shifted attention toward grounded decision-making.”

“They have been deployed as open-ended explorers (G. Wang et al., 2023) and as software engineers that resolve real repository issues (Jimenez et al., 2024).”

“To improve reliability, we apply self-consistency [25] by obtaining at least five independent judgments and aggregating them via majority voting, and further validate the results by expert sampling.”