Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic Retrieval-Augmented Generation for Industry Challenges

Prompt-based methods leverage in-context reasoning and instruction-following capabilities of pretrained LLMs.

“The first is prompt-based methods, which leverages the in-context reasoning and instructionfollowing capabilities of pretrained LLMs [Yao et al., 2023;Press et al., 2023;Li et al., 2025a].”

• cites Yao et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Press et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Li et al., 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

DeepRetrieval focuses on improving the quality of search queries generated by the LLM agent.

“DeepRe-trieval [Jiang et al., 2025] focuses specifically on improving the quality of the search queries generated by the LLM agent.”

• cites Jiang et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Large Reasoning Models (LRMs) are designed to excel at complex, multi-step reasoning tasks.

“These models, sometimes developed through techniques like large-scale reinforcement learning (e.g., models analogous to OpenAI's o1 [Ope-nAI et al., 2024], DeepSeek-R1 [DeepSeek-AI et al., 2025]), are designed to excel at complex, multi-step reasoning tasks.”

• cites OpenAI et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites DeepSeek-AI et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Early RAG systems based on static pipelines struggle in real-world scenarios requiring complex reasoning, dynamic retrieval, and multi-modal integration.

“While early RAG systems based on static pipelines have shown effectiveness in well-structured tasks, they struggle in real-world scenarios requiring complex reasoning, dynamic retrieval, and multi-modal integration.”

• Also stated in: § 1 Introduction · plain

  “Despite the effectiveness of basic RAG methods, they often struggle when applied to real-world, industrial-scale applications involving complex and heterogeneous data.”

• cites Wang et al., 2025 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

• cites Wang et al., 2024bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Ma et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• AssumptionRests on an assumption: Traditional RAG methods have demonstrated strong performance when queries are well-formed and the necessary information is readily available in the retrieved context.

Search-o1 integrates an Agentic RAG Mechanism where the LRM dynamically triggers search queries based on self-assessed knowledge gaps.

“It integrates two core components: an Agentic RAG Mechanism where the LRM dynamically triggers search queries based on self-assessed knowledge gaps, and a Reason-in-Documents Module that processes retrieved content to dist ill relevant information into a refined format, thereby minimizing noise and maintaining the LRM's reasoning integrity.”

• cites Li et al., 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

DeepResearcher performs end-to-end RL training directly within real-world web environments, moving beyond controlled environments or static corpora.

“DeepResearcher [Zheng et al., 2025] pushes the boundaries of training-based Agentic RAG by moving beyond controlled environments or static corpora to perform end-to-end RL training directly within real-world web environments.”

• cites Zheng et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Search-R1 has shown significant performance improvements over various RAG baselines on question-answering datasets.

“Search-R1 has shown significant performance improvements over various RAG baselines on question-answering datasets.”

• cites Jin et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

DeepRetrieval frames query generation or rewriting as an RL problem, training the LLM to transform an initial user query into a more effective query for downstream retrieval systems.

“It frames the task of query generation or rewriting as an RL problem, training the LLM to transform an initial user query into a more effective query for downstream retrieval systems.”

• cites Jiang et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Search-o1 is specifically designed to enhance LRMs by tackling knowledge insufficiency during long, step-by-step reasoning chains.

“The Search-o1 framework [Li et al., 2025a] is specifically designed to enhance LRMs by tackling knowledge insufficiency during long, step-by-step reasoning chains.”

• cites Li et al., 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

End-to-end training in realistic web environments is crucial for developing robust and capable research agents.

“The results strongly suggest that end-to-end training in realistic web environments is crucial for developing robust and capable research agents, moving closer to the capabilities hinted at by proprietary systems like OpenAI's Deep Research [OpenAI, 2025] or Grok's DeeperSearch.”

• cites OpenAI, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Route-based methods selectively trigger retrieval based on context or model uncertainty.

“route-based methods selectively trigger retrieval based on context or model uncertainty, such as low confidence scores or ambiguous intermediate outputs [Wang et al., 2024a]”

• cites Wang et al., 2024aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

In ReZero, the reward for retrying is conditional upon the agent successfully completing the task, indicated by generating a final answer enclosed in <answer> tags.

“Crucially, this reward for retrying is conditional upon the agent successfully completing the task, indicated by generating a final answer enclosed in <answer> tags.”

• cites Dao and Le, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Search-o1 integrates a Reason-in-Documents Module that processes retrieved content to distill relevant information into a refined format, minimizing noise and maintaining the LRM's reasoning integrity.

“It integrates two core components: an Agentic RAG Mechanism where the LRM dynamically triggers search queries based on self-assessed knowledge gaps, and a Reason-in-Documents Module that processes retrieved content to dist ill relevant information into a refined format, thereby minimizing noise and maintaining the LRM's reasoning integrity.”

• cites Li et al., 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Search-R1 extends RL-based reasoning frameworks by integrating search engine interaction directly into the learning loop.

“It extends RL-based reasoning frameworks (like DeepSeek-R1) by integrating search engine interaction directly into the learning loop.”

• cites Jin et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

DeepRetrieval bypasses the need for expensive and potentially suboptimal supervised query datasets by directly optimizing for retrieval performance.

“By directly optimizing for the end goal (retrieval performance), it bypasses the need for expensive and potentially suboptimal supervised query datasets.”

• cites Jiang et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

DeepRetrieval uses metrics such as recall@k, NDCG, or Hits@N obtained from executing the generated query against a restricted real search engine or document collection to provide feedback to the LLM.

“Metrics such as recall@k, Normalized Discounted Cumulative Gain (NDCG), or evidence-seeking retrieval accuracy (Hits@N) obtained from executing the generated query against a restricted real search engine (like PubMed) or document collection are used to provide feedback to the LLM.”

• cites Jiang et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

ReZero addresses the scenario where an initial search query fails to retrieve necessary information, potentially causing the LLM agent to halt prematurely or generate a suboptimal response.

“It addresses the common scenario where an initial search query might fail to retrieve the necessary information, potentially causing the LLM agent to halt prematurely or generate a suboptimal response.”

• cites Dao and Le, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Search-R1 trains the LLM to autonomously decide when to search and what to search for during a multi-step reasoning process.

“Search-R1 [Jin et al., 2025] tackles a different aspect of agentic search: training the LLM to autonomously decide when to search and what to search for during a multi-step reasoning process.”

• cites Jin et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Agentic reasoning has two primary methods: prompt-based and training-based.

“There are two primary methods for implementing agentic reasoning.”

The field has shifted toward Reasoning Agentic RAG, a paradigm that embeds decision-making and adaptive tool use directly into the retrieval process.

“To address these challenges, the field has shifted toward Reasoning Agentic RAG, a paradigm that embeds decision-making and adaptive tool use directly into the retrieval process.”

• Also stated in: § 1 Introduction · plain

  “To address these limitations of basic RAG in handling complex, real-world tasks, recent research has turned to Agentic RAG [Ravuru et al., 2024], a paradigm that tightly integrates retrieval with reasoning and decision-making.”

• cites Ravuru et al., 2024 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

In ReZero, the conditionality of the retry reward upon task completion prevents the agent from accumulating rewards simply by retrying indefinitely without making progress.

“This conditionality prevents the agent from accumulating rewards simply by retrying indefinitely without making progress.”

R1-Searcher proposes a two-stage, outcome-based RL approach for enhancing search capabilities.

“The related R1-Searcher [Song et al., 2025] framework also proposes a similar two-stage, outcome-based RL approach for enhancing search capabilities.”

• cites Song et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

DeepResearcher demonstrated substantial performance improvements over prompt-engineering baselines and RAG-based RL agents trained on static corpora, particularly on open-domain research tasks.

“Deep-Researcher demonstrated substantial performance improvements over prompt-engineering baselines and RAG-based RL agents trained on static corpora, particularly on open-domain research tasks.”

• cites Zheng et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Retrieval-Augmented Generation (RAG) overcomes the knowledge limitations of Large Language Models (LLMs) by integrating external retrieval with language generation.

“Retrieval-Augmented Generation (RAG) [Chen et al., 2024;Lewis et al., 2020;Gao et al., 2023] has attracted significant attention as a promising approach to overcome the knowledge limitations of LLMs resulting from static pretraining.”

• cites Chen et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Lewis et al., 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Gao et al., 2023 sourceNot checked yet

The progression of training-based methods from Search-R1 to ReZero to DeepRetrieval to DeepResearcher reflects the growing sophistication of RL in agentic search.

“The progression for the training-based methods, from optimizing the decision process of when and what to query (Search-R1), to fostering persistence (ReZero), optimizing query formulation (DeepRetrieval), and managing real-world research workflows (DeepResearcher) reflects the growing sophistication of RL in agentic search.”

ReZero encourages the LLM to explore alternative queries or search strategies if the first attempt proves insufficient by directly rewarding the act of persistence when productive.

“By directly rewarding the act of persistence (when productive), ReZero encourages the LLM to explore alternative queries or search strategies if the first attempt proves insufficient.”

• cites Dao and Le, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

DeepResearcher addresses a key limitation of many existing agents, whether prompt-engineered or trained in simulated/static RAG settings, which often struggle with the complexities of real-world web interaction.

“This addresses a key limitation of many existing agents, whether prompt-engineered or trained in simulated/static RAG settings, which often struggle with the complexities of real-world web interaction.”

• cites Zheng et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

DeepRetrieval's primary focus is on optimizing the content and formulation of the search query itself, compared to other RL methods.

“Compared to other RL methods, DeepRetrieval's primary focus is on optimizing the content and formulation of the search query itself.”

• cites Jiang et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Training-based methods optimize models through reinforcement learning to determine when and how to invoke external tools.

“The second paradigm is trainingbased methods, where models are explicitly optimized through reinforcement learning, to determine when and how to invoke external tools [Jiang et al., 2025;Jin et al., 2025;Zheng et al., 2025].”

• cites Jiang et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Jin et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Zheng et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

ReZero introduces a dimension to RL-based agentic search by focusing on incentivizing persistence.

“ReZero (Retry-Zero) [Dao and Le, 2025] introduces another dimension to RL-based agentic search by specifically focusing on incentivizing persistence.”

• cites Dao and Le, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.