Self-Consistency Improves Chain of Thought Reasoning in Language Models

“…have found that even when the desired answer is fixed, introducing diversity in the reasoning processes can be highly beneficial; therefore we leverage sampling, as commonly used for open-ended text generation (Radford et al., 2019; Brown et al., 2020; Thoppilan et al., 2022), to achieve this goal.”

“In an effort to address this shortcoming, Wei et al. (2022) have proposed chain-of-thought prompting , where a language model is prompted to generate a series of short sentences that mimic the reasoning process a person might employ in solving a task.”

“Prompt with correct chain-of-thought 17.1 Prompt with imperfect chain-of-thought 14.9 + Self-consistency (40 paths) 23.4 Prompt with equations 5.0 + Self-consistency (40 paths) 6.5 PaLM-540B Zero-shot CoT (Kojima et al., 2022) 43.0 + Self-consistency (40 paths) 69.2 Table 8: Self-consistency works under imperfect prom…”

“Here we perform a study using self-consistency to see if it can help fill in the gap, over a set of common NLP tasks, including (1) Closed-Book Question Answering: BoolQ (Clark et al., 2019), HotpotQA (Yang et al., 2018), and (2) Natural Language Inference: e-SNLI (Camburu et al., 2018), ANLI (Nie et al., 2020) and RTE…”

“SFT trains models to replicate explicit reasoning traces collected from annotated solutions [42; 43], whereas RL rewards models for accurate and logically coherent reasoning without requiring explicit intermediate supervision [9; 47].”

“The objective of the cold-start stage is to strengthen the model’s chain-of-thought reasoning and structured response abilities [39, 36], providing a solid foundation for subsequent capability expansion.”

“The first is based on the self-consistency approach, where multiple answers are generated in parallel from one or more LLMs and the final output is chosen by majority vote (Wang et al., 2022; Brown et al., 2024; Chen et al., 2024).”

“…o ff ers another parameter-e ffi cient tuning technique, such as Priming, zero-shot, one-shot, and few-shot [29], chain-of-thought [30], and self-consistency [31], by keeping the model parameters frozen and learning continuous prompts that guide model behavior without modifying the core model.”

“Even with test-time scaling strategies – such as Chain-of-Thought prompting [43], multi-rollout sampling with self-consistency [39], or search-based inference methods including Tree Search and Beam Search [33, 51] – VLMs still struggle to generate reliable spatio-temporal grounding results directly…”

“Self-Consistency: We incorporated the approach from Wang et al. 40 to ensure diversity.”

“Advanced variants include Automatic CoT [17], Self-Consistency [18], and Logical CoT [19].”

“Success under a k -sample budget, defined by whether at least one of the k generations is correct: • Self-consistency (SC) [14].”

“M INDS training framework.”

“The reasoning ability of language models can be enhanced using chain-of-thought (Wei et al., 2022), RLVR (OpenAI, 2024; Guo et al., 2025; Lambert et al., 2025), and self-consistency (Wang et al., 2023a), in math (Hendrycks et al., 2021) and code (Jain et al., 2025; Shi et al., 2024)”

“Chain-of-thought self-consistency prompting (Wang et al., 2023b) improves on the CoT method by proposing different reasoning chains and ensembles on the final result.”

“…(DAG) (Dziri et al., 2023), random walks on reversible/recurrent Markov chains (Kim et al., 2025), or trees of linear solution paths (Shalev-Shwartz & Shashua, 2025), neglect the improvement from diverse sampling (Wang et al., 2023) and ability to connect disparate knowledge (Yin & Wang, 2025).”

“Self-Consistency [50] aggregates multiple sampled rationales to increase reliability.”

“In addition, our runtime analysis contributes to ongoing discussions on the accuracy–efficiency trade–off in large language model reasoning systems (Madaan et al., 2023; Wang et al., 2022).”

“Recent research has shown that self-consistency approaches—wherein multiple reasoning paths are generated and aggregated—can significantly improve response quality (10).”

“…was informed by previous work on self-consistency reasoning, semantic uncertainty estimation and uncertainty-aware diagnostic systems 26,27,42 but evaluates these signals jointly within an autonomous clinical agent workflow and separately for both the final diagnosis (Dx) and reasoning…”

“…selects a verifier-ranked candidate (Cobbe et al., 2021), whereas self-consistency selects an answer by marginalizing over sampled reasoning trajectories (Wang et al., 2023); broader test-time scaling work studies how to allocate inference compute among such strategies (Snell et al., 2025).”

“Prior work established that self-consistency helps on benchmarks where models have higher baseline accuracy (Wang et al., 2023); we show it hurts the majority of problems on a hard one.”

“A large body of work improves performance by (i) more robust decoding/aggregation such as self-consistency (Wang et al., 2025a), (ii) better decomposition protocols such as least-to-most prompting (Zhou et al., 2022) and plan-and-solve prompting (Wang et al., 2023), or (iii) automatic prompt…”

“4○ Self-Consistency [36] samples multiple independent CoTs (i.e., applies 1○ for b runs), producing b different reasoning chains for the same problem, and then selects the most frequent answer as the final response (see Section S11 of the Supp. material for sampling hyperparameters).”

“Ideally, we would retain only those items in E t ′ >t that were already retrievable with the same content at time t : Directly determining whether an item was retriev-able at t is impractical, so we approximate Equation 2 with a binary leakage filter ϕ ( x | q, t, a ⋆ ) , conditioned on the…”

“Self-consistency [22] samples multiple independent reasoning chains at a fixed evidence set and selects the modal answer; agreement across samples signals correctness.”

“Chain-of-thought (CoT) reasoning has emerged as a trans-formative technique for enhancing problem-solving in large language models (Wei et al., 2022; Kojima et al., 2022).”

“…intermediate steps before the answer (Wei et al., 2022; Kojima et al., 2022), with further gains from supervised or RL-based training (Zelikman et al., 2022; Shao et al., 2024) and test-time strategies that sample and score many chains (Wang et al., 2022; Snell et al., 2024; Yao et al., 2023).”

“Self-consistency (Wang et al., 2023) aggregates multiple sampled trajectories via majority voting, while Brown et al. (2024) systematically compare strategies and find no single approach universally dominates.”

“While Chain-of-Thought (CoT) (Wei et al., 2022) and self-consistency (Wang et al., 2022) enhance multi-step reasoning through linear trajectories, Tree-of-Thoughts (ToT) (Yao et al., 2023) and formalize search via branching and backtracking.”

“7 and selecting the majority answer [56].”

“Chain-of-Thought (CoT): A baseline where we allow the model to first generate a reasoning step, then use that reasoning to produce a prediction (Wei et al., 2022).”

“Large language models (LLMs) are increasingly deployed across diverse reasoning, multimodal, and agentic tasks (Wei et al., 2022; Liu et al., 2023; Zhu et al., 2025), often via API-based services.”

“Chain-of-Thought [6], Self-Consistency [7], Least-to-Most prompting [8], and scratchpad-style methods [5] all show gains from exposing intermediate computation.”

“Self-consistency aggregates multiple independent samples per question by majority vote and replaces single-sample greedy decoding with a repeated-sampling majority vote [9].”

“Prior work relaxes the coupling via tagged formats and interleaving protocols (Wei et al., 2022; Xie et al., 2025), pipelined designs that separate latent reasoning from speech (Woo et al., 2025), or specialized streaming mechanisms (Tong et al., 2025).”

“Despite the emergence of reasoning-optimized models (Wei et al., 2022; Wang et al., 2023), a critical “action gap” persists— current frameworks excel at verbalizing medical logic but struggle to maintain stable, tool-augmented trajectories in open-ended clinical environments (Shen et al., 2026;…”

“Prior studies have shown that, even when the target output is fixed, introducing diversity at the reasoning level can improve robustness and decision quality [32].”

“Prior work on uncertainty estimation and self-disambiguation enables models to recognize when user input is incomplete or ambiguous (Lin et al., 2022; Wang et al., 2022; Hou et al., 2024).”

“…outputs that are unfaithful to real-world facts (Ji et al., 2023; Simhi et al., 2025), provided context (Liu et al., 2025), or user instructions (Belem et al., 2025), and arise from mechanisms spanning data, training, and inference (Kalai et al., 2025; Azaria & Mitchell, 2023; Wang et al., 2023b).”

“On the test-quality side, ConVerTest (Taherkhani et al., 2026) applies self-consistency filtering (Wang et al., 2023a) before voting, and TestCase-Eval (Yang et al., 2025) evaluates LLM-generated test reliability; recent systematic studies (Li et al., 2024; Sun et al., 2024) further examine how…”

“We compare several commonly used prompting baselines, including single-agent prompting strategies (zero-shot, zero-shot CoT, few-shot (5-shot), and few-shot CoT) following prior work [8, 16], as well as three advanced multi-agent methods: MAC [7], MDAgents [16], and MedAgents [15].”

“…showing step-level quality diverges from task-completion outcomes; Turpin et al. [23], Lanham et al. [24], and Aravindan and Kejriwal [25] document unfaithful or fragile chain-of-thought reasoning [26, 27], while Jiang et al. [28] show that correct answers can emerge from corrupted reasoning paths.”

“2) ChatGPT and Claude exhib - ited enhanced performance when utilizing Few-Shot + Chain-of-Thought prompting compared to the Zero-Shot mode, while Gemini did not demonstrate such improvement.”

“Those that do not rely directly on answer-token likelihoods, such as agreement-based consistency [27], which measures agreement across sampled answers via a majority vote, remain stable and often improve under reasoning.”

“Similarly, chain-of-thought prompting (Wei et al. 2022) improves transparency in LLMs/VLMs but remains dependent on the model’s internal knowledge, which may be unreliable for specialized domains.”

“We compare C REDAL with the standard aggregation method used in LLM ensembles—utilitarian aggregation—defined as, Taking the argmax over this quantity recovers popular strategies such as majority voting or best-of-N (e.g., [Wang et al., 2022, Si et al., 2023]).”

“Alongside CoT, we evaluate Self-Consistency (SC) [36] to improve robustness.”

“That is, the current study opens up new possibilities for the use of GenAI in the epistemic aspect of learning, particularly extending beyond the typical applications of AI within the cognitive domain (Wang et al., 2022).”

“In chain-of-thought reasoning, we can similarly imagine summing and normalizing probabilities across samples from the system, an approach known as self-consistency (Wang et al., 2023).”

“Complementary inference-time techniques improve reasoning without additional training, including self-consistency [38], least-to-most prompting [51], Plan-and-Solve [37], Program of Thoughts [6], and deliberative search strategies such as Tree of Thoughts [44] and Graph of Thoughts [1] (often…”

“Prior works in MAD (Du et al., 2023; Liang et al., 2024; Chan et al., 2024) demonstrated that iterative debates can induce diverse thoughts with "Society of Minds" and outperform single-agent prompt techniques such as Chain-of-Thought (CoT) (Wei et al., 2022) and Self-Consistency (Wang et al., 2023) in reasoning tasks.”

“For LLMs, chain-of-thought and memory-based methods reuse reasoning traces to guide new problems (Wei et al., 2022; Yan et al., 2025), but traces are usually retrieved as extra context without an explicit procedure, so models can retrieve correct facts yet execute the wrong step.”

“5 A promising solution to addressing this problem is the rapid increase in quality of large language models (LLMs) and our ability to optimize their performance through advanced prompting techniques, such as few-shot, 6,7 chain-of-thought, 7-9 self-consistency, 10,11 and multi-agent collaboration.”

“It builds on VideoJudge [42] and prior work on self-consistency and self-improvement in large language models [7, 27, 44, 47].”

“…on the ways to utilize extra test-time compute, the existing TTS approaches could be divided into three categories: i) self-consistency methods (Wang et al., 2022), which sample multiple reasoning trajectories and aggregate them via majority voting; ii) early stopping methods (Li et al., 2024;…”

“Self-consistency [Wang et al., 2023], Best-of-N [Snell et al., 2024], and Self-Refine [Madaan et al., 2023] underperform outcome-only reprompting (30.8%, 15.9%, and 14.8% vs. 52.9%; Appendix M), confirming that standard test-time methods preserve L 1 shortcuts on causal tasks.”

“Early work demonstrated that visual reasoning in VLMs can be enhanced through Chain-of-Thought finetuning (Zhang et al., 2023; Wei et al., 2022).”

“Our method incorporates Self-Consistency (SC) [25] and Chain-of-Verification (CoVe) [26] to mitigate hallucinations during the generation stage without relying on a ground-truth code solution.”

“This phase is essential for the development of a comprehensive understanding of a variety of mathematical concepts and their respective solutions [22], [23], [24], [25].”

“3) w/ Voting: Voting on the predictions of the different augmented samples and choosing the prediction with the most votes as the final result [69], [70].”

“Notably, self-consistency techniques generate multiple outputs and select the most frequent one, under the assumption that consistency correlates with correctness [2].”

“Subsequent works have extended this paradigm through program-aided reasoning (Chen et al., 2023), self-consistency decoding (Wang et al., 2022), and tree-structured exploration (Yao et al., 2024).”

“Parallel methods like self-consistency (Wang et al., 2023) sample hundreds of reasoning paths and aggregate via majority voting, trading compute for accuracy at pro-portionally scaling costs.”

“Reasoning has become a prevalent paradigm in LLMs, as the thinking process can significantly enhance complex problem solving with structured intermediate inference steps [14,34,35].”

“Subsequent work refined its application through analysis of prompting factors and enhanced reasoning via automatic rationale synthesis, self-consistency decoding, and progressive problem decomposition (Sprague et al., 2024; Zhang et al., 2022; Wang et al., 2022a; Zhou et al., 2022).”

“Solution-level approaches such as best-of-N and self-consistency invest bud-get by generating many complete solutions and aggregating, without explicitly resolving ambiguous local choices along a trajectory (Wang et al., 2022).”

“Chain-of-thought (CoT) has become a standard mechanism for eliciting multi-step reasoning in large language models (LLMs), substantially improving performance on complex tasks (Wei et al., 2022; Yao et al., 2023; Besta et al., 2024; Wang et al., 2022).”

“Chain-of-thought prompting (Wei et al., 2022) showed that intermediate reasoning steps can elicit latent reasoning abilities.”

“Similarly, parallel decoding techniques such as Self-Consistency (Wang et al., 2022) and Best-of-N (Brown et al., 2024) generate multiple reasoning traces concurrently to improve final answer quality through voting.”

“Another work [22] refines the greedy decoding strategy in CoT by generating multiple reasoning paths through the LLM and picking the most consistent response.”

“6 (Z.ai, 2025) as the judge model for both evaluation rubrics, aggregating predictions with majority voting over five samples (Wang et al., 2023).”

“…GRPO-based methods (Li et al., 2025c; Cai et al., 2025; Wu et al., 2025) commonly apply uniform update strength across samples, causing unstable instances to inject noise into gradient updates and undermining optimization reliability (Wang et al., 2022b; Kuhn et al., 2023; Chen et al., 2025a).”

“Methods Exploiting Answer Consistency Self-Consistency (SC) Wang et al. (2023) 2024) halts sampling once a confident majority is detected, avoiding the cost of decoding all N samples.”

“Later studies enhance CoT through self-consistency, which samples and aggregates multiple reasoning paths [49], least-to-most prompting that solves problems via ordered subproblem decomposition [50], and bootstrapped reasoning strategies [51].”

“024 ) indicates that extended reasoning provides greater relative benefits in smaller models, while very large models already capture much of this capability through scale alone.”

“• Exact Match also termed best-of-N in reasoning (Wang et al., 2023) and pass@ N in code (Chen et al., 2024) tasks, we consider it as EM@ N in our time series forecasting task.”

“Historically, AI progressed from task-specific machine learning, optimized for classification or regression in static settings, to foundation and generative models that learn broad representations and exhibit emergent compositional reasoning [Wei et al., 2022, Glanois et al., 2024].”

“…this review, the capability for Deep Reasoning is defined as a model capable of explicit, multistep decomposition, verification, and self-correction during the reasoning process and improving the stability of its conclusions through leveraging test-time compute or self-consistency when necessary (7).”

“We evaluate ConSensus against three representative baselines: (i) Single-Agent , where all modality features are concatenated into a single prompt, (ii) Self-Consistency (SC) (Wang et al., 2023), which samples multiple reasoning paths and selects the most consistent prediction (with temp = 0 .”