CoChat
Sign Up
Cited by 126Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters

Open access
Wang, Boshi, Min, Sewon, Deng, Xiang, Shen, Jiaming, Wu, You, Zettlemoyer, Luke, Sun, Huan·2023·Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
DOI 10.18653/v1/2023.acl-long.153
Open this paper in CoChat — free accountOpen PDF

Abstract

Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, Huan Sun. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

AI snapshot

Written from the paper’s full text.

TL;DR

Chain-of-Thought (CoT) prompting remains effective even when the demonstrated rationales contain invalid reasoning, retaining over 80–90% of CoT performance under various metrics. The key factors for effective CoT are that rationales stay relevant to the query and that their steps are correctly ordered, rather than being logically valid.

Key findings

  • Invalid reasoning in demonstrations still yields high performance: on GSM8K with text-davinci-002, CoT reaches 48.3 Inter. F1 and 48.5 Answer Acc., while invalid reasoning reaches 43.9 Inter. F1 and 39.5 Answer Acc.; on Bamboogle, CoT reaches 45.2 Inter. Recall and 45.2 Answer F1, while invalid reasoning reaches 44.4 Inter. Recall and 39.4 Answer F1.
  • Relevance and coherence are key: the “no relevance” setting (both components irrelevant) underperforms standard prompting on GSM8K (11.0 Answer Acc. vs. 15.4 for STD), and “no coherence for language templates” drops to 28.3 Inter. F1 from CoT’s 48.3.
  • For bridging objects, relevance matters more than coherence: incoherent bridging objects achieve 39.2 Inter. F1 on GSM8K, while irrelevant bridging objects achieve 26.2 Inter. F1.
  • At the instance level, there are cases where CoT is wrong and invalid reasoning is correct: 62/196 for GSM8K and 6/20 for Bamboogle.

Methods

  • The authors run ablation experiments on two multi-step reasoning tasks: arithmetic reasoning on GSM8K (800 of 1319 test examples sampled) and multi-hop factual QA on Bamboogle (all 125 test samples).
  • Main backbone is InstructGPT-175B text-davinci-002 with greedy decoding; additional results are reported for text-davinci-003, PaLM, and Flan-PaLM in the appendix.
  • Ablations alter CoT rationales by invalid reasoning, destroying relevance or coherence of bridging objects and language templates, and combined no-relevance/no-coherence settings; evaluation includes extrinsic answer accuracy/F1 and intrinsic Recall/F1 of bridging objects.

Limitations

  • Stated: experiments cover only arithmetic reasoning and multi-hop QA; symbolic tasks are hard to ablate because their rationales are highly template-based with little variation.
  • Stated: invalid reasoning rationales were manually written, and a more systematic treatment of invalid reasoning is left for future work.
  • Stated: intrinsic evaluation relies on correctness of bridging objects and ground-truth annotations, which may not generalize and are costly to obtain; evident from the checklist: all results are reported with a single run due to limited budgets.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18653/v1/2023.acl-long.153

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 7 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Providing invalid reasoning steps in demonstrations can achieve over 80-90% of the performance obtained using CoT under various metrics.

    “In this paper, we show that CoT reasoning is possible even with invalid demonstrations-prompting with invalid reasoning steps can achieve over 80-90% of the performance obtained using CoT under various metrics, while still generating coherent lines of reasoning during inference.”
    • Also stated in: § Invalid Reasoning · plain
      “We find that valid reasoning for the demonstrations matters only a small portion to the performance of CoT-by providing rationales with invalid reasoning, LLMs can achieve over 80-90% of the performance of CoT under various metrics while performing logically sound and pertinent reasoning.”
    • Also stated in: § Invalid Reasoning · plain
      “by providing rationales with completely invalid reasoning steps, the LLM can still achieve over 80-90% of the performance of CoT under various metrics while generating coherent lines of reasoning towards the answer ( §4).”
    Quote in the paperSubstantial claim
  • Findingown data

    Under intrinsic evaluation (Inter.Recall/Inter.F1), all tested LLMs retain over 90% of the performance achieved under CoT prompting when provided invalid reasoning steps.

    “In particular, under Inter.Recall/Inter.F1, i.e., intrinsic evaluation, which is arguably a more faithful measurement of the rationale quality ( §3.3), all LLMs we tested can retain over 90% of the performance achieved under CoT prompting.”
    Quote in the paperSubstantial claim
  • Findingown data

    Generated rationales under the invalid reasoning setting are overall indistinguishable from rationales generated by CoT prompting.

    “By checking the generated rationales for the invalid reasoning setting, we find that overall they look indistinguishable from the rationales generated by CoT prompting.”
    Quote in the paperSubstantial claim
  • Findingown data

    At the instance level, there is a significant portion of samples where CoT gets the wrong answer and the invalid reasoning setting gets the correct answer (62/196 for GSM8K, 6/20 for Bamboogle).

    “At the instance level, after omitting samples where both settings get the correct/wrong answer, there is a significant portion for the remaining ones (62/196 for GSM8K, 6/20 for Bamboogle) where CoT gets the wrong answer and the invalid reasoning setting gets the correct answer.”
    Quote in the paperSubstantial claim
  • Findingown data

    Providing incoherent bridging objects achieves better performance than providing irrelevant bridging objects, especially on GSM8K (39.2 vs. 26.2 Inter.F1).

    “Providing incoherent bridging objects ( 2 ) achieves better performance than providing irrelevant bridging objects ( 3 ), especially on the more challenging GSM8K dataset (39.2 v.s.26.2 Inter.F1).”
    Quote in the paperSubstantial claim
  • May combine two claims. 15
  • Classified as probably not a substantial claim. 6

  • May combine two claims.

    Findingown data

    Most ablation settings for relevance and coherence (2-7) have rather large performance drops from CoT, with the low-performing ones approaching or even underperforming standard prompting.

    “It can be seen that most of the settings for this section ( 2 -7 ) have rather large performance drops from CoT, where the low-performing ones approach or even underperform standard prompting.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Under the incoherent language templates setting, the LLM generates rationales with incoherent language templates for 14 out of 20 examples, which negatively affects reasoning.

    “By examining the predicted rationales, we find that the LLM is indeed generating rationales with incoherent language templates (14 out of 20 examples), which negatively affects reasoning.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The no relevance setting (7), where both components of the rationale have no relevance, achieves significantly poorer performance than other ablation settings and even underperforms standard prompting on GSM8K.

    “The no relevance setting 7 where both components of the rationale have no relevance achieves significantly poorer performance than other ablation settings, and even underperforms standard prompting (STD) where no rationale is given on GSM8K.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In almost all cases where the predicted final answer is correct under invalid reasoning, the generated rationales reach the answer with valid and sound reasoning steps, drastically different from the demonstrations; when the final answer is wrong, the errors are of the same types as under CoT prompting.

    “In almost all cases where the predicted final answer is correct, the rationales do reach the answer with valid and sound reasoning steps (as in CoT), drastically different from those in the given demonstrations; for cases where the final answer is wrong, the errors the LLM makes are also in the same types with the errors made under CoT prompting.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Even though the reasoning in the invalid reasoning setting is invalid, each step of a rationale still follows the previous steps, so the rationale appears coherent.

    “Each step of a rationale still follows the previous steps.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Even though the reasoning in the invalid reasoning setting is invalid, the rationales still use information from the query and are relevant to the query.

    “The rationales still use information from the query; more specifically, they still start from bridging objects mentioned in the query, and the language templates are related to the query.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Providing invalid reasoning for the rationales overall only marginally harms the performance, and even outperforms CoT on GSM8K under intrinsic evaluation.

    “In particular, providing invalid reasoning for the rationales ( 1 ) overall only marginally harms the performance, and even outperforms CoT on GSM8K under intrinsic evaluation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For the remaining settings where relevance/coherence are ablated (2-7), the same trend can be observed on GSM8K, e.g., the model still suffers a lot when providing rationales that are irrelevant or have incoherent language templates.

    “For the remaining settings where we ablate the relevance/coherence ( 2 -7 ), the same trend can be observed on the challenging GSM8K dataset, e.g., the model still suffers a lot when providing rationales that are irrelevant or have incoherent language templates.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For the relatively easier Bamboogle dataset, the high model capacity has basically erased significant impacts from the ablations, with the only standing observation that the model still needs the rationales to be relevant to maintain its performance.

    “For the relatively easier Bamboogle dataset, the high model capacity indicated by its impressive performance has basically erased significant impacts from the ablations, with the only standing observation that the model still needs the rationales to be relevant to maintain its performance.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    The LLM may express a mathematical equation in different ways (e.g., '5 plus 3 is 8', '5 + 3 = 8'), which is why the authors do not use whole equations.

    “We do not use whole equations since we observe that the LLM may express the mathematical equation in different ways, e.g., “5 plus 3 is 8”, “5 + 3 = 8”.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Relevance and coherence are key for the performance of CoT prompting.

    “Relevance and coherence are key for the performance of CoT prompting.”
    • Also stated in: § PaLM and Flan-PaLM in Appendix A.3. We summarize the main findings in what follows. · hedged
      “This suggests that overall, relevance and coherence are key for the performance of CoT.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    It is important for bridging objects to be relevant, but not as important to have them in the right order to guide the LLM along the reasoning process.

    “which indicates that it is important for the bridging objects to be relevant, but not as important to have them in the right order to guide the LLM along the reasoning process.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Our intrinsic evaluation of the generated rationales may not be a good metric in general cases.

    “Our intrinsic evaluation of the generated rationales is based on the correctness of bridging objects, which, even though is a good indicator of the quality of language templates (Appendix A.2) in our experiments, may not be a good metric in general cases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Most (if not all) tasks in symbolic reasoning are highly template-based and hence the reasoning steps have little variations.

    “However, most (if not all) tasks there are highly templatebased and hence the reasoning steps have little variations, both within each example and across different examples.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The slight editing of the original prompt exemplars only slightly affects the performance of CoT.

    “These edits only slightly affect the performance of CoT; we show our edited demonstration examples and include more details in Appendix A.1.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Due to limited budgets, all results are reported from a single run.

    “Due to limited budgets, we report all results with a single run.This is transparent from the paper.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    LLMs may over-rely on their prior knowledge and ignore important information in the context, including information crucial for specifying task semantics.

    “LLMs may over-rely on their prior knowledge and ignore important information in the context that are presumably rare in the pretraining distribution, including those that are crucial for specifying the task semantics”
    • Also stated in: § A.3 Additional Results & Discussion · hedged
      “LLMs may over-rely on their prior knowledge and ignore important information in the context, including those that are crucial for specifying the task semantics”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Jang et al., 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductnone given

    Our intrinsic evaluation relies on ground truth bridging objects, which are usually not available and costly to annotate.

    “It also relies on ground truth bridging objects, which are usually not available and costly to annotate.”
    Quote in the paperNot a substantial claim
18 of 18

  • The paper relies on this without support.

    There is little understanding of what makes CoT prompting effective and which aspects of the demonstrated reasoning steps contribute to its performance.

    “Despite its success, there is still little understanding of what makes CoT prompting effective and which aspects of the demonstrated reasoning steps contribute to its performance.”

  • The paper relies on this without support.

    The two tasks used, arithmetic reasoning and multi-hop factual question answering, are representative of multi-step reasoning tasks.

    “We experiment on two representative tasks involving multi-step reasoning: arithmetic reasoning & multi-hop factual question answering (QA).”

  • The paper relies on this without support.

    Pretraining corpora are not always accessible.

    “However, the pretraining corpora are not always accessible”

  • The paper relies on this without support.

    Low-level statistics are usually not adequate when the topics of interest are abstract and high-level skills such as reasoning.

    “low-level statistics are usually not adequate when the topics of interest are abstract and highlevel skills such as reasoning”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Self-consistency and ensemble refinement are strategies that sample multiple model answers to arrive at a more consistent response, improving model reliability.

    “Self-consistency [66] and ensemble refinement [9] are strategies that sample multiple model answers to arrive at a more consistent response, improving model reliability [9].”

    From The Opportunities and Risks of Large Language Models in Mental Health (2024) · cites this paper as [66]

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Irving Copi, Carl Cohen, and Victor Rodych. 2016. In- troduction to logic. Routledge.
  2. Shivam Garg, Dimitris Tsipras, Percy S Liang, and Gre- gory Valiant. 2022. What can transformers learn in- context? a case study of simple function classes. In Advances in Neural Information Processing Systems, volume 35, pages 30583-30598.

Cited by

Sorted by citations

Related

Authors
Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L., & Sun, H.
Affiliations
  • The Ohio State University (US)
  • University of Washington (US)
  • Google (United States) (US)
ORCID
Venue
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Publisher
Association for Computational Linguistics
Published
2023
Pages
2717-2739
ISSN
0736-587X (ISSN-L)
DOI
10.18653/v1/2023.acl-long.153
OpenAlex
W4385571045
Cited by
120
References
26
Citations by year
2026: 23 · 2025: 57 · 2024: 22 · 2023: 18
FWCI
21.26
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
Funders
National Science Foundation (1942980, 2112606, 1815674, CAREER)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
chain-of-thought prompting, large language models, prompting strategies, LLM reasoning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review