CoChat
Sign Up
PreprintCited by 93Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing

Open access
Gou, Zhibin, Shao, Zhihong, Gong, Yeyun, Shen, Yelong, Yang, Yujiu, Duan, Nan, Chen, Weizhu·2023·arXiv
DOI 10.48550/arxiv.2305.11738arXiv 2305.11738
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent developments in large language models (LLMs) have been impressive. However, these models sometimes show inconsistencies and problematic behavior, such as hallucinating facts, generating flawed code, or creating offensive and toxic content. Unlike these models, humans typically utilize external tools to cross-check and refine their initial content, like using a search engine for fact-checking, or a code interpreter for debugging. Inspired by this observation, we introduce a framework called CRITIC that allows LLMs, which are essentially "black boxes" to validate and progressively amend their own outputs in a manner similar to human interaction with tools. More specifically, starting with an initial output, CRITIC interacts with appropriate tools to evaluate certain aspects of the text, and then revises the output based on the feedback obtained during this validation process. Comprehensive evaluations involving free-form question answering, mathematical program synthesis, and toxicity reduction demonstrate that CRITIC consistently enhances the performance of LLMs. Meanwhile, our research highlights the crucial importance of external feedback in promoting the ongoing self-improvement of LLMs.

AI snapshot

Written from the paper’s full text.

TL;DR

CRITIC is a training-free framework that lets frozen LLMs verify and revise their own outputs by interacting with external tools such as search engines, code interpreters, and toxicity APIs. Across free-form QA, mathematical program synthesis, and toxicity reduction, it improves ChatGPT, text-davinci-003, and LLaMA-2 models, while showing that self-correction without external feedback is unreliable.

Key findings

  • On ChatGPT, CRITIC reports 7.7 F1 gains across three QA tasks, 7.0% absolute gains on three mathematical reasoning tasks, and a 79.2% reduction in toxicity probability.
  • QA improvements over CoT include ChatGPT AmbigNQ EM 51.8→62.0 and HotpotQA EM 42.8→52.9; text-davinci-003 AmbigNQ EM 44.2→51.4 and TriviaQA EM 67.4→71.2.
  • Math gains include ChatGPT GSM8k PoT 72.5→78.2 (+5.7) and CRITIC* 83.9 (+11.4); TabMWP PoT 75.0→89.0 (+14.0). Scaling is stronger with larger models: TabMWP gains are +4.7 for 7B, +9.4 for 13B, and +16.0 for 70B.
  • Toxicity reduction lowers ChatGPT max toxicity 0.325→0.173 and toxicity probability 0.192→0.040; text-davinci-003 max toxicity 0.344→0.180 and probability 0.210→0.045.
  • Without tool feedback, self-correction is unreliable: CRITIC w/o Tool can degrade performance, e.g. text-davinci-003 GSM8k −1.8, and Self-Refine on Codex drops −44.6. Self-Eval verification reaches only 54% accuracy on HotpotQA and is worse than Only-True on TriviaQA.

Methods

  • CRITIC starts with an initial LLM output, interacts with external tools to generate critiques, then corrects the output conditioned on those critiques; the verify-then-correct cycle can iterate until a stopping criterion. It uses few-shot in-context learning and requires no additional training.
  • Tools are task-specific: Google/web search for QA, a Python interpreter for mathematical program synthesis, and the Perspective API for toxicity reduction.
  • Evaluation spans free-form QA (AmbigNQ, TriviaQA, HotpotQA; 500 validation examples each), mathematical program synthesis (GSM8k, SVAMP, TabMWP official test splits), and toxicity reduction (1k non-toxic prompts from RealToxicityPrompts). Models include ChatGPT gpt-3.5-turbo, text-davinci-003, and LLaMA-2 7B/13B/70B. Metrics include EM/F1, exact match, max toxicity, toxicity probability, perplexity, and dist-2/dist-3.

Limitations

  • Stated: inference latency grows linearly with the number of iterations; prompt engineering depends on appropriate in-context demonstrations; effectiveness on other tasks and LLMs remains uncertain; experiments are limited to the textual modality; explicit language evaluation may not suit all model outputs.
  • Evident from error analysis: CRITIC reduces HotpotQA hallucinations from 36% to 7% but does not eliminate them, and reasoning errors rise from 5% to 10%, refusals from 2% to 12%, and incorrect corrections are 10%. On GSM8k, wrong corrections are 14.3% and accuracy on originally correct outputs drops −4.3%.
  • Evident from QA error analysis: 49% of CRITIC error samples on HotpotQA involve label ambiguity, incorrect labels, or outdated labels, indicating automatic EM/F1 evaluation bias.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2305.11738

Cites

143 of 143 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

23 of 26 cited sources checked · 10 supported · 1 flagged · accuracy not yet measured

  • Findingcitation

    The tested LLMs cannot accurately identify what they know without external tools.

    “Essentially, our tested LLMs are incapable of accurately identifying”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Kadavath et al., 2022 source

      The Introduction (p. 4) says models perform well at predicting whether they know an answer and show some generalization across tasks. The paper reports limitations in out-of-distribution calibration, but it does not conclude that models are incapable of identifying what they know without external tools; its results include predictions on bare questions. (LLM, conf=0.89)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
  • Premisecitation

    AUROC is a better uncertainty measure for free-form generation than calibration metrics such as ECE or Brier score.

    “We use the discrimination metric AUROC as a better measure of uncertainty for free-form generation than calibration metrics ECE or Brier score (Kuhn et al., 2023;Si et al., 2023).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Kuhn et al., 2023 source

      Section 6 (Empirical Evaluation; page unavailable) explicitly says AUROC is a better uncertainty measure for free-form question answering and NLG than calibration measures like the Brier score. The passage does not address ECE specifically, so that named comparison is not verified by our copy. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Si et al., 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingcitation

    Unstable generation-discrimination-critique gaps are especially prominent in tasks requiring external knowledge or complex reasoning, such as QA, commonsense reasoning, and math reasoning.

    “We find such unstable generationdiscrimination-critique gaps (Saunders et al., 2022) becomes particularly prominent in tasks that necessitate external knowledge or intricate reasoning, such as QA, Commonsense Reasoning, and Math reasoning.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Saunders et al., 2022 source
  • Premisecitation

    Most earlier work on natural-language feedback and self-correction prompts or trains language models to revise their initial outputs.

    “Most of them prompt or train language models to correct their initial results.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Shinn et al., 2023 source

      The Related work section describes several earlier approaches that improve initial generations through self-evaluation, prompting, feedback, or debugging. It also describes approaches that score or search among generations rather than correct an initial result, so it does not establish the claim that most earlier work follows the stated pattern. No page number is available in our copy. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
    • cites Madaan et al., 2023 source
    • cites Chen et al., 2023b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation

    High certainty does not imply truthfulness.

    “However, high certainty does not mean truthful (Ott et al., 2018;Xiao & Wang, 2021;Kadavath et al., 2022)”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites Ott et al., 2018 source
    • cites Xiao & Wang, 2021 source
    • cites Kadavath et al., 2022 source
  • May combine two claims. 36
  • Classified as probably not a substantial claim. 11
  • The quote only partly states this claim. 12

  • May combine two claims.

    Premisecitation

    Uncertainty estimation methods for LLMs suffer from poor calibration, difficulty evaluating free-form text, and poor interpretability.

    “these methods suffer from poor calibration of LLMs (Jiang et al., 2020;OpenAI, 2023), difficulty in evaluating free-form text (Kuhn et al., 2023), and poor interpretability.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Jiang et al., 2020 source

      The citation to Jiang et al. is attached to the claim about poor calibration, not the separately cited difficulty of evaluating free-form text. The Introduction (p. 2) discusses how prompt choice affects estimates of what language models know, and §5.3 (p. 9) discusses combining prediction probabilities. Neither addresses whether LLM uncertainty estimates are poorly calibrated; searches for calibration and uncertainty estimation found no such passage. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
    • cites OpenAI, 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Kuhn et al., 2023 source

  • May combine two claims.

    Premisecitation

    Free-form QA is an important open-ended NLG problem with clear ground truth, and hallucination detection for open-ended generation is understudied, especially for LLMs.

    “We evaluate with free-form QA because it's an important open-ended NLG problem with clear ground truth, and hallucination detection for open-ended generation is also insufficiently studied, especially for LLMs (Evans et al., 2021).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Evans et al., 2021 source

      Section 5.1 (p. 60) discusses falsehoods from language models, and Appendix A.5.1 (p. 84) says existing techniques cannot reliably determine an AI system’s truthfulness. The paper does not establish that free-form QA is an important open-ended NLG problem with clear ground truth, or that hallucination detection for open-ended generation is insufficiently studied, especially for LLMs. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Recent follow-up studies support the finding that self-correction in LLMs is unreliable and add further evidence.

    “These studies corroborate the findings regarding the unreliability of self-correction in LLMs and provide additional insights.”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites Huang et al., 2023 source

      The Introduction (p. 1) reports that LLMs often struggle to self-correct reasoning and that performance can deteriorate; the Conclusion (p. 8) says the paper offers further insights. The claim is broader than the source's finding, which concerns reasoning self-correction without external feedback, not self-correction in LLMs generally. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Stechly et al., 2023 source
    • cites Valmeekam et al., 2023 source

      The Introduction reports preliminary evidence questioning LLM self-critiquing in planning, and Section 5.2 identifies false positives that undermine reliability. The paper also examines feedback as an additional insight. Its evidence is limited to GPT-4 on planning tasks, rather than self-correction in LLMs generally. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingcitation

    Without oracle verification, self-correction through self-feedback can degrade performance on these tasks and lead to incorrect changes to initial responses.

    “Without the support of Oracle verification, a technique used in concurrent works like Reflexion (Shinn et al., 2023) and RCI (Kim et al., 2023), self-correction through self-feedback can deteriorate the performance in these tasks, and even lead to incorrect modifications of initial responses.”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Shinn et al., 2023 source

      The Ablation study (page unavailable) reports that removing test generation and execution reduced accuracy from 60% to 52% and led to harmful edits, supporting the claim that self-reflection without verification can worsen results. But Reflexion uses self-generated unit tests, not oracle verification, and the reported result is specifically for a subset of difficult Rust programming problems. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Kim et al., 2023 source

      Appendix C.2 (page unavailable) says RCI uses the correct label to decide when to stop its reasoning-task loop, supporting the reference to verification. But neither that passage nor §2.1 reports that self-feedback without verification degrades performance or incorrectly changes initially correct answers. The paper instead describes RCI as a way to improve outputs. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    RLHF is a white-box alignment method that relies heavily on human annotations to fine-tune a model toward human intentions.

    “RLHF is a white-box alignment technique that heavily depends on human annotations to fine-tune a model, aligning it with human intentions.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Stiennon et al., 2020 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    CRITIC and RLHF target similar objectives but use distinct approaches and can complement each other.

    “While both CRITIC and RLHF (Stiennon et al., 2020) target important objectives for LLMs, such as reducing hallucination and ensuring truthfulness, their approaches are distinct and can complement one another.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Stiennon et al., 2020 source

  • May combine two claims.

    Premisecitation

    Large language models have difficulty verifying truthfulness and the correctness of reasoning.

    “LLMs struggle with verification on truthfulness (Kadavath et al., 2022;Kuhn et al., 2023) and reasoning correctness (Ye & Durrett, 2022;Huang et al., 2022;2023).”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites Kadavath et al., 2022 source

      Kadavath et al. supports the truthfulness-verification part in a limited setting: Section 4.1 (p. 12) says evaluating a model’s own answers may be difficult and that zero-shot P(True) is poorly calibrated. But the Discussion (p. 22) says large models perform well at self-evaluation across a range of subjects, so the unqualified claim that they struggle overstates the finding. The reasoning-correctness clause is attributed to other citations, not Kadavath et al. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Kuhn et al., 2023 source
    • cites Ye & Durrett, 2022 source

      The Introduction (p. 2) reports that LLM-generated explanations can be unreliable and that there is no automated way to assess their reliability perfectly. This supports a difficulty concerning reasoning correctness, but the paper does not show that LLMs themselves struggle when acting as verifiers; the Discussion (p. 10) instead proposes a language model as a possible explanation-assessment model. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premiseown data

    This finding motivated later research to use more reliable LLM-based evaluation for QA tasks.

    “This has motivated subsequent research to adopt a more reliable LLM-based evaluation for QA tasks (Shao et al., 2023).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Shao et al., 2023 source

      The Introduction (page unavailable) says exact-match metrics can underestimate performance on QA tasks and that LLM-based evaluation may be more reliable. Section 4.2 reports that the authors used an LLM to evaluate answer correctness. The paper does not establish that its finding motivated subsequent research to adopt that approach. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    CRITIC markedly reduces toxic outputs while maintaining fluency and diversity comparable to the base LLMs.

    “CRITIC substantially lowers the occurrence of toxic generations, while preserving fluency and diversity as the vanilla LLMs”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.
    • Rests on an assumption: The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

  • May combine two claims.

    Findingown data

    Across varied tasks and LLMs, experiments consistently demonstrate CRITIC's effectiveness, generality, and interoperability.

    “Experiments on diverse tasks and LLMs have consistently shown the effectiveness, generality, and interoperability of CRITIC.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.
    • Rests on an assumption: The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

  • May combine two claims.

    Findingown data

    Repeated correction usually produces steady gains, with a sharp increase when only incorrect samples are corrected.

    “Iterative correction generally leads to continuous improvement, with a notable surge when only modifying erroneous samples (oracle setting).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.
    • Rests on an assumption: The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

  • May combine two claims.

    Findingown data

    Using self-correction without external feedback yields only modest gains or even worse performance across the tested settings.

    “Specifically, modest improvements or even deteriorated performance are observed universally using self-correct without external feedback.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The EM scores in Table 1 range from 30 to over 80 across the three datasets, allowing assessment of generalization across varying difficulty.

    “The EM scores in Table 1 demonstrate a range of 30 to over 80 across the three datasets, enabling an effective assessment of the method's generalization ability across data with varying difficulty.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    CRITIC yields large improvements over initial chain-of-thought outputs for every dataset, setting, and LLM tested, needing just three correction rounds.

    “CRITIC dramatically improves over the model's initial CoT results across all datasets, settings, and LLMs, requiring only three corrections”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.
    • Rests on an assumption: The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

  • May combine two claims.

    Findingown data

    Self-Eval achieves only slightly above random accuracy (54%) on HotpotQA verification and performs worse than the Only-True baseline on TriviaQA.

    “For instance, the Self-Eval approach achieves only slightly better than random guessing accuracy (54%) in verifying answers on HotpotQA, and performs even worse than the Only-True baseline on TriviaQA, despite the fact that Only-True has no discrimination ability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    CRITIC significantly improves fact discernment through tool interaction and outperforms previous estimation methods with generality and interpretability.

    “In contrast, our proposed CRITIC significantly improves the model's ability to discern facts by incorporating tool interaction, outperforming all previous estimation methods while exhibiting strong generality and interpretability.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used by CRITIC, such as web search, Wikipedia, and a code interpreter, provide sufficiently reliable and relevant feedback for verification.

  • May combine two claims.

    Findingown data

    LLMs struggle to distinguish the veracity of their own answers and cannot provide reliable confidence about what they know.

    “Experimental results in Table 5 reveal that LLMs struggle to distinguish the veracity of their own answers and cannot provide reliable confidence regarding”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    CRITIC's toxicity reduction matches supervised state-of-the-art methods without needing additional data or training.

    “CRITIC shows toxicity mitigation capabilities on par with supervised SoTA methods, while not requiring extra data or training”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.
    • Rests on an assumption: The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

  • May combine two claims.

    Findingown data

    CRITIC consistently improves LLM performance on three task types: open-ended question answering, math program generation, and toxicity reduction.

    “Comprehensive evaluations involving free-form question answering, mathematical program synthesis, and toxicity reduction demonstrate that CRITIC consistently enhances the performance of LLMs.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.
    • Rests on an assumption: The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

  • May combine two claims.

    Findingreasoning

    Without oracle verification, self-correction can worsen performance on many tasks and even corrupt initially correct answers.

    “self-correction might surprisingly deteriorate performance for many tasks, even worsening the initial answer”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Additional correction rounds give diminishing returns; most gains come from two to three rounds.

    “The marginal benefits of multiple corrections diminish, and typically, 2-3 rounds of corrections yield most of the benefits.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.
    • Rests on an assumption: The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

  • May combine two claims.

    Findingown data

    CRITIC provides substantial gains over program-of-thought for both LLMs under both correction strategies: correcting all samples or only incorrect programs.

    “CRITIC sizable improves upon the PoT across both LLMs, using either correction strategy: always correcting (CRITIC), or only correcting incorrect programs (CRITIC * ).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.
    • Rests on an assumption: The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The cost of tool use for CRITIC is free.

    “Here we discuss the cost of tool use for CRITIC, which is actually all free.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For LLaMA-2-13B, CRITIC achieves a SVAMP solve rate of 65.7, which is 0.6 points lower than PoT's 66.3.

    “CRITIC | 30.0 (+1.7) | 65.7 (-0.6) | 48.1 (+9.4)”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “PoT | 28.3 | 66.3 | 38.7”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    CRITIC substantially reduces hallucinations (from 36% to 7%) but does not eliminate them.

    “CRITIC can significantly reduce hallucinations (36% vs. 7%), but not all of them.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The external tools used by CRITIC, such as web search, Wikipedia, and a code interpreter, provide sufficiently reliable and relevant feedback for verification.

  • The quote only partly states this claim.

    Findingown data

    For LLaMA-2-70B, CRITIC improves the TabMWP solve rate by 16 points over PoT, reaching 75.0 versus 59.0.

    “CRITIC | 62.3 (+3.0) 84.7 (+2.7) | 75.0 (+16)”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “PoT | 59.3 | 82.0 | 59.0”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Most errors after applying CRITIC come from reasoning mistakes, refusal to answer, and incorrect corrections.

    “Most errors after applying CRITIC arise from reasoning mistakes, refusal to answer, and incorrect corrections.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    CRITIC outperforms Self-Con on AUROC for AmbigNQ, with 0.810 versus 0.760.

    “CRITIC | 0.730 | 0.810 | 0.882 | 0.818 | 0.765 | 0.831”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “Self-Con (Si et al., 2023) | - | 0.760 | - | 0.745 | - | 0.831”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For ChatGPT, CRITIC achieves an F1 of 74.9 on AmbigNQ, which is higher than CoT's F1 of 64.3.

    “ReAct ReAct → CRITIC CRITIC | 52.0 64.8 60.4 72.2 62.0 74.9 | 63.7 69.8 39.1 75.5 81.8 37.9 75.1 81.7 40.3 | 50.2 50.0 52.9”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “CoT Self-Consistency | 51.8 64.3 52.6 65.4 | 72.9 79.2 32.7 75.4 81.3 35.8 | 42.8 47.0”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For ChatGPT, CRITIC achieves an F1 of 52.9 on HotpotQA, which is higher than CoT's F1 of 42.8.

    “ReAct ReAct → CRITIC CRITIC | 52.0 64.8 60.4 72.2 62.0 74.9 | 63.7 69.8 39.1 75.5 81.8 37.9 75.1 81.7 40.3 | 50.2 50.0 52.9”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “CoT Self-Consistency | 51.8 64.3 52.6 65.4 | 72.9 79.2 32.7 75.4 81.3 35.8 | 42.8 47.0”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For ChatGPT, CRITIC without tool use achieves an F1 of 67.3 on AmbigNQ, which is lower than CRITIC's F1 of 74.9.

    “CRITIC w/o Tool | 55.2 67.3 | 73.5 79.9 33.1 | 46.1”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “ReAct ReAct → CRITIC CRITIC | 52.0 64.8 60.4 72.2 62.0 74.9 | 63.7 69.8 39.1 75.5 81.8 37.9 75.1 81.7 40.3 | 50.2 50.0 52.9”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    CRITIC outperforms Self-Con on AUROC for TriviaQA, with 0.818 versus 0.745.

    “CRITIC | 0.730 | 0.810 | 0.882 | 0.818 | 0.765 | 0.831”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “Self-Con (Si et al., 2023) | - | 0.760 | - | 0.745 | - | 0.831”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For ChatGPT, CRITIC achieves an F1 of 81.7 on TriviaQA, which is higher than CoT's F1 of 79.2.

    “ReAct ReAct → CRITIC CRITIC | 52.0 64.8 60.4 72.2 62.0 74.9 | 63.7 69.8 39.1 75.5 81.8 37.9 75.1 81.7 40.3 | 50.2 50.0 52.9”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “CoT Self-Consistency | 51.8 64.3 52.6 65.4 | 72.9 79.2 32.7 75.4 81.3 35.8 | 42.8 47.0”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    In a failure case for CRITIC on GSM8k, CRITIC made a wrong verification, and the initial code was actually wrong.

    “Failure case for CRITIC on GSM8k: Wrong verification, the initial code is actually wrong.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For ChatGPT, CRITIC in the oracle setting achieves an F1 of 79.9 on AmbigNQ, which is higher than CRITIC's F1 of 74.9.

    “CRITIC * | 69.6 79.9 | 80.9 86.6 44.3 | 56.9”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “ReAct ReAct → CRITIC CRITIC | 52.0 64.8 60.4 72.2 62.0 74.9 | 63.7 69.8 39.1 75.5 81.8 37.9 75.1 81.7 40.3 | 50.2 50.0 52.9”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    For ChatGPT, CRITIC improves the GSM8k solve rate by 5.7 points over PoT, reaching 78.2 versus 72.5.

    “CRITIC | 78.2 (+5.7) 83.3 (+1.3) 89.0 (+14.0)”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · plain
      “PoT | 27.9 72.5 | 64.7 82.0 | 46.3 75.0”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    It is unclear whether CRITIC works as well on other tasks and LLMs.

    “the effectiveness of CRITIC on other tasks and LLMs remains uncertain”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    RLHF is not a universal solution to alignment challenges.

    “However, RLHF is not a one-size-fits-all solution to alignment challenges.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    An RLHF model may fail to consistently provide current factual information, generate error-free code, or adapt to a new external environment.

    “For instance, an RLHF model may not consistently provide up-to-date factual information, generate error-free code, or adapt to a new external environment.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Interpretationcitation

    The authors' work improves the reliability of expressed uncertainty by interacting with external tools such as search engines.

    “improve the reliability of expressed uncertainty (Lin et al., 2022a;Kadavath et al., 2022;Zhou et al., 2023) by interacting”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    CRITIC enhances LLMs by enabling self-verification and self-correction through tool interactions, which makes it applicable to black-box models.

    “Naturally, CRITIC enhances LLMs by allowing LLM self-verification and self-correction through tool interactions, making it applicable to black-box models.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The way prompts are constructed can affect the experimental outcomes.

    “different prompt constructions may impact the experimental results.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    LLMs cannot reliably verify themselves, and interacting with external tools may address this.

    “we shed light on the unreliability of LLMs in self-verification, highlighting the potential of external tool interaction to solve this problem.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Hallucinations that remain after CRITIC occur because the search engine fails to find useful evidence or the model misunderstands the evidence.

    “Even after utilizing CRITIC, hallucinations persist due to the inability to find useful evidence via a search engine or misunderstanding the evidence.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The external tools used by CRITIC, such as web search, Wikipedia, and a code interpreter, provide sufficiently reliable and relevant feedback for verification.

  • May combine two claims.

    Interpretationnone given
    § Define the number of employees in each category num_warehouse_workers = 4 num_managers = 2 # Calculate the total number of hours worked in a month num_days = 25 hours_per_day = 8 total_hours = num_days * hours_per_day # Calculate the total wages for the workers warehouse_wages = num_warehouse_workers * warehouse_wage * total_hours manager_wages = num_managers * manager_wage * total_hours total_wages = warehouse_wages + manager_wages # Calculate the total FICA taxes owed fica_rate = 0.10 total_fica_tax = total_wages * fica_rate # Calculate the total amount owed total_owed = total_wages + total_fica_tax answer = total_owed ''' ---Question: A farmer has twice as many pigs as cows, and 4 more cows than goats.If the farmer has 56 animals total, how many pigs does he have?'''python num_of_pigs = 2 * num_of_cows num_of_cows = number_of_goats + 4 num_of_animals = 56 number_of_goats = (num_of_animals -4) / 3 num_of_cows = (number_of_goats + 4) * 2 answer = num_of_cows ''' Execution: NameError("name 'num_of_cows' is not defined") Output: answer = None

    Splitting the original CRITIC prompt for evaluating plausibility before truthfulness can reduce prompt length and decrease inference cost.

    “We split the original CRITIC prompt for evaluating plausibility before truthfulness, which can reduce the length of prompts to decrease the inference cost.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Study design and conductreasoning

    The authors employ a caching mechanism for web search to address temporal inconsistency of the Google API.

    “To address this, we employ a caching mechanism for web search.”
    • Also stated in: § Listing 29: Prompts for CRITIC (plausibility) on hallucination detection with TriviaQA. · hedged
      “A potential concern arises from the temporal inconsistency of the Google API, which may result in unstable evaluations and hinder reproducibility.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The Google API has temporal inconsistency that may cause unstable evaluations and hinder reproducibility.

  • May combine two claims.

    Interpretationreasoning

    Refusal to answer occurs when CRITIC cannot find enough evidence, which the authors consider expected for maintaining truthfulness.

    “The refusal to answer occurs when CRITIC can't find enough evidence to support a response, which we consider an expected behavior to maintain truthfulness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    For mathematical program synthesis, external knowledge is usually unnecessary and a code interpreter acts like a calculator.

    “For mathematical program synthesis tasks, external knowledge is typically unnecessary, and a code interpreter can function equivalently to a calculator.”
    Quote in the paperNot a substantial claim
48 of 48

  • The paper relies on this without support.

    The external tools used (Google search, Python interpreter, PERSPECTIVE API) provide reliable and accurate feedback for verification.

    “we build a web search toolfoot_1 based on Google to search queries generated by LLMs”
    “we utilize the Python interpreter as a tool to get two types of feedback: error messages and execution results.”
    “We use PERSPECTIVE APIfoot_3 as a tool to obtain fine-grained toxicity information.”

  • The paper relies on this without support.

    The few-shot in-context demonstrations are sufficient for LLMs to learn to use tools and correct outputs effectively.

    “CRITIC utilizes the emergent abilities of chain-of-thought reasoning (Wei et al., 2022) and few-shot in-context learning”
    “our experiments rely on appropriate in-context demonstrations.”

  • The paper relies on this without support.

    The external tools used by CRITIC, such as web search, Wikipedia, and a code interpreter, provide sufficiently reliable and relevant feedback for verification.

    “For QA tasks, as mentioned in Sec.4.1, we build a Web Tool for CRITIC to crawl the results of Google Search and web pages like Wikipedia.”
    “For Mathematical program synthesis tasks, we use a local code interpreter, which is free of charge.”

  • The authors state this as an assumption.

    The Google API has temporal inconsistency that may cause unstable evaluations and hinder reproducibility.

    “A potential concern arises from the temporal inconsistency of the Google API, which may result in unstable evaluations and hinder reproducibility.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Modern large language models perform well on general language tasks but still struggle on complex reasoning tasks.

    “Modern large language models (LLMs) like ChatGPT have shown remarkable performance on general language tasks but still struggle on complex reasoning tasks”

    From Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate (2024) · cites this paper as Gou et al., 2023

    The Introduction describes remarkable progress across language tasks, and §4 evaluates models on mathematical and multi-hop reasoning. But the paper’s explicit finding about intricate reasoning concerns models’ difficulty verifying and correcting their own answers (Appendix D.1), not a general inability to perform complex reasoning tasks. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • LLMs' intrinsic self-correction fails without oracle labels as feedback.

    “However, recent works show that LLMs' intrinsic self-correction fails without oracle labels as feedback.”

    From Understanding the Dark Side of LLMs' Intrinsic Self-Correction (2025) · cites this paper as Gou et al., 2023

    The “Intrinsic Self-Correct with NL feedback” section says self-correction without external feedback is unreliable, producing modest improvements or even deteriorated performance. It does not say self-correction always fails, or that oracle labels specifically are necessary: the paper also advocates feedback from external tools. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • CRITIC uses external tools such as search engines and code interpreters to verify and post-correct the initial output.

    “Similarly, CRITIC [88] uses external tools such as search engines and code interpreters to verify and post-correct the initial output.”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [88]

  • A number of tasks that LLMs are evaluated on are problems without a well-defined ground truth, and this makes conclusions much harder to draw.

    “A number of tasks that LLMs are evaluated on (e.g. writing prompts [52], constrained text generation [28], toxicity control [49, 14] , etc.) are problems without a well-defined ground truth.”

    From On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks (2024) · cites this paper as [14]

  • Most existing feedback and verification methods remain focused on result-level correction, turn-level reflection, or auxiliary verification stages, with limited integration of step-level signals into the current action generation process.

    “most existing feedback and verification methods remain focused on result-level correction [8,22], turn-level reflection [7], or auxiliary verification stages [19-21], with limited integration of step-level signals into the current action generation process”

    From A Closed-Loop Modular Language Agent with Step Verification and Local Correction for Multi-Step Task Solving (2026) · cites this paper as [8]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Text-Davinci-003 GSM8k CRITIC score differs between tables

    Major

    Table 2 reports a CRITIC score of 72.2, a 2.1-point gain over PoT's 70.1. Table 10 reports 71.2 and a 1.1-point gain for the same model, dataset, and method.

1 of 1

Cites

Flagged first, then by citations

1 of the 144 references couldn’t be matched to a known paper.

Research works printed but not linked: 7

  1. Sanjoy Dasgupta, Daniel Hsu, Stefanos Poulis, and Xiaojin Zhu. Teaching a black-box learner. In International Conference on Machine Learning, pp. 1547-1555. PMLR, 2019.
  2. Patricia M Greenfield. Language, tools and brain: The ontogeny and phylogeny of hierarchically organized sequential behavior. Behavioral and brain sciences, 14(4):531-551, 1991.
  3. PMLR, 2020.
  4. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feed- back. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 3008-3021.
  5. Curran Asso- ciates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/file/ 1f89885d556929e98d3ef9b86448f951-Paper.pdf.
  6. Krist Vaesen. The cognitive bases of human tool use. Behavioral and brain sciences, 35(4):203-218, 2012.
  7. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum? id=Bct2f8fRd8S.

Cited by

Sorted by citations

Related

Authors
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W.
Affiliations
  • Microsoft Research Asia (China) (CN)
  • Microsoft (United States) (US)
  • Tsinghua University (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2305.11738
arXiv
2305.11738
OpenAlex
W4377372007
Semantic Scholar
bcdaf6c98ddbd6809cf6241aa77200d7394db163
Cited by
57
References
144
Citations by year
2026: 4 · 2025: 19 · 2024: 21 · 2023: 13
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
large language models, self-correction, hallucination mitigation, fact-checking, code generation, toxicity reduction, question answering

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review