CoChat
Sign Up
Cited by 0Topic ModelingMultimodal Machine Learning ApplicationsNatural Language Processing Techniques

Self-Critique Guided Iterative Reasoning for Multi-hop Question Answering

Open access
Chu, Zheng, Fan, Huiming, Chen, Jingchang, Wang, Qianyu, Yang, Mingda, Liang, Jiafeng, Wang, Zhongjie, Li, Hao +3 more·2025·Findings of the Association for Computational Linguistics: ACL 2025
DOI 10.18653/v1/2025.findings-acl.123
Open this paper in CoChat — free accountOpen PDF

Abstract

Zheng Chu, Huiming Fan, Jingchang Chen, Qianyu Wang, Mingda Yang, Jiafeng Liang, Zhongjie Wang, Hao Li, Guo Tang, Ming Liu, Bing Qin. Findings of the Association for Computational Linguistics: ACL 2025. 2025.

AI snapshot

Written from the paper’s full text.

TL;DR

SiGIR uses self-critique feedback to guide iterative question decomposition, retrieval, and reasoning for multi-hop QA, with reward-guided beam search over reasoning trajectories. It reports state-of-the-art results on HotpotQA, 2WikiMQA, and MuSiQue, with an average improvement of 8.6% over prior SOTA.

Key findings

  • SiGIR surpasses previous SOTA on all three datasets, with reported gains of +3.96 on 2WikiMQA, +3.09 on HotpotQA, and +4.85 on MuSiQue; on MuSiQue it reports a 15.0% improvement, including 47.1% and 24.37% on 3-hop and 4-hop questions.
  • Ablations show removing reward-guided search or reward signals hurts performance; removing reward causes a substantial 10% decrease, excluding self-critique data causes a minor decline, and coarse-grained rewards in search are 2.7% lower than fine-grained rewards.
  • Search analysis shows retaining only one candidate drops performance by 2.4%, setting decomposition exploration width to 1 drops it by 13.57%, and cumulative process rewards outperform outcome rewards and random selection.
  • Self-improvement from 40% labeled data reaches the 60% data quantile after one iteration and approaches the 80% quantile after two, with only a 2.2% gap from full labeled fine-tuning; hybrid retrieval improves performance by 5.7%, and the method reduces average reasoning iterations by 26.7%.

Methods

  • Training pipeline: synthesize iterative reasoning rationales with DeepSeek-V2.5, train an iterative reasoner R, train a separate critic C on non-overlapping data, then annotate subprocesses and train the self-critique reasoner R_sc with special tokens using a standard causal language modeling objective.
  • Inference: iteration-level beam search with branching exploration via temperature sampling in question decomposition, retrieval, and reasoning; the model self-evaluates retrieval, reasoning, and overall quality, retains top-k candidates by cumulative process reward, and selects the final answer by cumulative or outcome reward.
  • Evaluation: HotpotQA, 2WikiMQA, and MuSiQue development sets in open-domain setting with token-level F1; Mistral-7B is the default backbone, with additional Qwen2.5 and LLaMA2 experiments; BM25 is default sparse retrieval, with Contriever dense and hybrid retrieval in analysis; LoRA is used, and training takes about 3 hours on 4×A100-SXM4-80GB GPUs.

Limitations

  • Stated: sub-question exploration uses sampling rather than a dedicated question decomposition module, which can limit exploratory breadth; future work may add a query rewriting module.
  • Stated: the method uses synthetic data followed by supervised fine-tuning without reinforcement learning in sub-question reasoning; future work may use Monte Carlo tree search and reinforcement learning.
  • Evident from the text: evaluation is limited to three English multi-hop QA datasets and mainly 7B-scale backbones, and retrieval is conducted within a corpus constructed from training-instance supporting facts and irrelevant documents rather than a full open-domain corpus.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18653/v1/2025.findings-acl.123

Cites

0 citation errors · 14 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

4 of 8 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Iterative retrieval, by alternating between retrieval and generation, can acquire more comprehensive relevant knowledge, improving retrieval performance.

    “By alternating between retrieval and generation, it can acquire more comprehensive relevant knowledge , improving retrieval performance.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Trivedi et al., 2023 source
    • cites Asai et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingown data

    Generative reward modeling outperforms discriminative modeling.

    “Overall, generative reward modeling outperforms discriminative modeling.”
    Quote in the paperSubstantial claim
  • Findingown data

    Exclusion of self-critique data from the training data produces a minor performance decline.

    “Additionally, the exclusion of self-critique data from the training data produces a minor performance decline.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Among generative forms, chain-of-thought (CoT) is preferred over post-hoc explanations.

    “Moreover, among generative forms, chain-of-thought (CoT) is preferred over post-hoc explanations.”
    Quote in the paperSubstantial claim
  • Findingown data

    The generative critic model brings slightly better performance than the separated critic model.

    “The gen-erative critic model brings slightly better performance, but has significant higher inference throughput.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 5
  • May combine two claims. 16
  • The quote only partly states this claim. 2

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Question decomposition breaks down complex multi-hop questions into simpler sub-questions, reducing difficulty of reasoning and planning, and enabling more accurate retrieval.

    “It breaks down the complex multi-hop question into simpler sub-questions, thereby reducing the difficulty of reasoning and planning, and enabling more accurate retrieval.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Cao et al., 2023 source

      The Introduction and §3 Methodology describe decomposing a complex question into a tree of simpler sub-questions, then reasoning from the leaves upward. They do not say that decomposition reduces planning difficulty or makes retrieval more accurate; the Introduction instead discusses how the method handles potentially misleading retrieved knowledge. No page numbers are available in our copy. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Chu et al., 2024a source

      Section 3.1 describes decomposing complex multi-hop questions into simpler sub-questions and solving them bottom-up. Section 5.1 says sub-questions provide clearer queries and that decomposition leads to more accurate retrieval. The paper does not establish the separate claim that decomposition reduces the difficulty of reasoning and planning. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites Su et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Retrieval-augmented generation is a promising strategy for mitigating the knowledge limitations of LLMs.

    “Retrieval-augmented generation is a promising strategy for mitigating the knowledge limitations of LLMs (Jiang et al., 2024).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Jiang et al., 2024 source

  • May combine two claims.

    Findingown data

    SiGIR surpasses previous state-of-the-art methods on all three datasets, achieving improvement of 2WikiMQA (+3.96), HotpotQA (+3.09), and MuSiQue (+4.85).

    “It surpasses the previous state-of-the-art methods on all three datasets, achieving improvement of 2WikiMQA (+3.96),HotpotQA (+3.09), and MuSiQue (+4.85).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the first iteration of self-improvement, the model achieves performance corresponding to the 60% training data quantile; in the second iteration, the model's performance approaches the 80% quantile, with only a 2.2% gap from the performance achieved with the full labeled data fine-tuning.

    “in the first iteration of self-improvement, the model achieves performance corresponding to the 60% training data quantile.In the second iteration, the model's performance approaches the 80% quantile, with only a 2.2% gap from the performance achieved with the full labeled data fine-tuning.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Our method not only surpasses their performance but also reduces the average number of reasoning iterations by 26.7%, demonstrating superior reasoning efficiency.

    “Conversely, our method not only surpasses their performance but also reduces the average number of reasoning iterations by 26.7%, demonstrating superior reasoning efficiency.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    SiGIR achieves an average improvement of 8.6% compared to the SOTA method on three multi-hop reasoning datasets.

    “Extensive experiments on three multi-hop reasoning datasets demonstrate the effectiveness of our proposed method, surpassing the previous SOTA by 8.6%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Without sacrificing reasoning performance, the GenCritic achieves a throughput that is 4.46 times higher.

    “Without sacrificing reasoning performance, the GenCritic achieves a throughput that is 4.46 times higher, demonstrating significantly better reasoning efficiency”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Gen-Critic outperforms separated models in two of the three datasets, achieving an average performance improvement of 0.9%.

    “As shown in Table 6, the Gen-Critic outperforms the separated models in two of the three datasets, achieving an average performance improvement of 0.9%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Starting with 40% of the data, two self-improvement iterations boost performance to nearly 80%, with only a 2% gap with full SFT.

    “Starting with 40% of the data, two self-improvement iterations boost performance to nearly 80%, with only 2% gap with full sft.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    As the amount of labeled data increases, performance improves consistently; the most significant improvement occurs when 40% of the training data is utilized; beyond this threshold, the rate of performance enhancement decreases.

    “Initially, as the amount of labeled data increases, the performance improves consistently.The most significant improvement occurs when 40% of the training data is utilized; beyond this threshold, the rate of performance enhancement decreases.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Dedicated RM exhibits performance variations of -1.6% / +5.6% for retrieval and reasoning tasks, respectively; there is a significant performance decrease of 9.8% in modeling outcome rewards.

    “The dedicated RM exhibits performance variations of -1.6% / +5.6% for retrieval and reasoning tasks, respectively.However, there is a significant performance decrease of 9.8% in modeling outcome rewards.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Using cumulative process rewards for trajectory selection yields the best performance, followed by using outcome rewards; both strategies outperform random selection.

    “Using cumulative process rewards for trajectory selection yields the best performance, followed by using outcome rewards.Both strategies outperform random selection.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In reward-guided greedy inference, fine-grained and coarse-grained rewards have similar effects; when a broader search is required, coarse-grained rewards result in a 2.7% performance decrease compared to fine-grained rewards.

    “In reward-guided greedy inference (e), fine-grained and coarse-grained rewards have similar effects.However, when a broader search is required (d), coarse-grained rewards exhibit certain drawbacks, resulting in a 2.7% performance decrease compared to fine-grained rewards.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On MuSiQue, SiGIR achieves a significant improvement of 15.0% compared to previous methods, with enhancements of 47.1% and 24.37% on 3/4-hop challenging questions, respectively.

    “It achieves a significant improvement of 15.0% on the MuSiQue dataset compared to previous methods, with enhancements of 47.1% and 24.37% on 3/4-hop challenging questions, respectively.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Our method achieves higher F1 than baseline methods.

    “Our method requires fewer iterations to solve one question with higher F1, demonstrating higher reasoning efficiency.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    To prevent the critic from overfitting to the generator's training data, the critic data synthesis uses a small amount of non-overlapping corpus with X.

    “Notably, to prevent the critic from overfitting to the generator's training data, we select a small amount of non-overlapping corpus with X for critic data synthesis, denoted as X critic”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Retaining more candidate reasoning trajectories and exploring different sub-questions help improve performance.

    “Retaining more candidate reasoning trajectories and exploring different sub-questions help improve performance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Our approach can effectively synthesize data through self-critique, yielding competitive performance in situations where labeled data is scarce.

    “This suggests that our approach can effectively synthesize data through self-critique, yielding competitive performance in situations where labeled data is scarce.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The model is trained as a unified model on a mixture of all datasets, enabling it to perform reasoning across diverse multi-hop QA datasets instead of dataset-specific training.

    “It is noteworthy that, to demonstrate the generality of our approach, we train a unified model on a mixture of all datasets, enabling it to perform reasoning across diverse multi-hop QA datasets instead of dataset-specific training.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Our method naturally supports hybrid retrieval (sparse + dense) retrieval, which facilitates better reasoning.

    “Our method naturally supports hybrid retrieval (sparse + dense) retrieval, which facilitates better reasoning.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Sampling ensemble technique scales up but improvement is limited.

    “Wang et al. (2023) employs a sampling ensemble technique to scale up, but the improvement is limited.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wang et al., 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

21 of 21

  • The paper relies on this without support.

    The lack of intermediate guidance often results in inaccurate retrieval and flawed intermediate reasoning, leading to incorrect reasoning.

    “However, the lack of intermediate guidance often results in inaccurate retrieval and flawed intermediate reasoning, leading to incorrect reasoning.”

  • The paper relies on this without support.

    Decomposing a complex question into sub-questions at the beginning of reasoning is highly challenging, and any mistakes can cause subsequent reasoning to deviate from the correct path.

    “First, decomposing a complex question into sub-questions at the beginning of reasoning is highly challenging, and any mistakes can cause subsequent reasoning to deviate from the correct path.”

  • The paper relies on this without support.

    The iterative retrieval approach struggles to handle complex problem planning and articulate retrieval intents clearly, resulting in inaccurate retrieval and reasoning.

    “Second, the iterative retrieval approach struggles to handle complex problem planning and articulate retrieval intents clearly, resulting in inaccurate retrieval and reasoning.”

  • The paper relies on this without support.

    The absence of guidance for intermediate steps makes these methods susceptible to errors during reasoning, resulting in cascading errors.

    “Finally, the absence of guidance for intermediate steps makes these methods susceptible to errors during reasoning, resulting in cascading errors.”

  • The paper relies on this without support.

    Iterative question decomposition facilitates complex question planning and accurate retrieval.

    “iterative question decomposition facilitates complex question planning and accurate retrieval.”

  • The paper relies on this without support.

    Intermediate feedback guided reasoning expands the reasoning space and reduces cascading errors.

    “intermediate feedback guided reasoning expands the reasoning space and reduces cascading errors.”

  • The paper relies on this without support.

    The generator and critic can enhance each other, and merging them into a single model can reduce inference cost.

    “Besides, recent studies have found that the generator and critic can enhance each other, and merging them into a single model can reduce inference cost.”

  • The paper relies on this without support.

    R tends to make mistakes during reasoning, which helps ensure a balance between positive and negative samples.

    “Additionally, we employ R instead of the LLM for reasoning synthesis, as R tends to make mistakes during reasoning.This helps ensure a balance between positive and negative samples.”
8 of 8

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Few-shot explanation contradicts its supplied film document

    Minor

    The example document calls the film Argentine and names Luis Bayón Herrera as its director, but the explanation calls it Spanish and names José María Forqué.

1 of 1

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 5

  1. PMLR.
  2. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
  3. Patrick S. H. Lewis, Ethan Perez, Aleksandra Pik- tus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neu- ral Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
  4. Irrelevant]: The evidence does not relate to the question in any meaningful way. ### Question: In what country was the first Pan-African conference held? ### Evidence: First Pan-African Conference The First Pan-African Conference was held in London from 23 to 25 July 1900 (just prior to the Paris Exhibition of 1900 in order to allow tourists of African descent to attend both events ). Organized primarily by the Trinidadian
  5. barrister Henry Sylvester Williams, it took place in Westminster Town Hall (now Caxton Hall) and was attended by 37 delegates and about 10 other participants and observers from Africa, the West Indies, the US and the UK, including Samuel Coleridge Taylor (the youngest delegate), John Alcindor, Dadabhai Naoroji, John Archer, Henry Francis Downing, and W.E.B. Du Bois, with Bishop Alexander Walters of the AME Zion Church taking the chair. Rating: [Relevant] Explanation: The evidence directly answers the question by stating that the first Pan-African conference was held in London, which is in the United Kingdom. The details provided, such as the dates and location of the conference, further confirm the relevance of the evidence to the question. ### Question: What is the administrative territorial entity for Malta Township? ### Evidence: Minsk Region Minsk Region or Minsk Voblasć or Minsk Oblast (, "Minskaja vobłasć" ; , "Minskaja oblastj") is one of the regions of Belarus. Its administrative center is Minsk, although it is a separate administrative territorial entity of Belarus. As of 2011, the region's population is 1,411,500.

Related

Authors
Chu, Z., Fan, H., Chen, J., Wang, Q., Yang, M., Liang, J., Wang, Z., Li, H., Tang, G., Liu, M., & Qin, B.
Affiliations
Harbin Institute of Technology (CN)
ORCID
Venue
Findings of the Association for Computational Linguistics: ACL 2025
Publisher
Association for Computational Linguistics
Published
2025
Pages
2415-2438
DOI
10.18653/v1/2025.findings-acl.123
OpenAlex
W4412888797
Cited by
0
References
0
FWCI
0.00
Type
conference paper
Language
en
Open access
Licence
cc-by
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
multi-hop question answering, self-criticism, iterative reasoning, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review