Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

RAG does not seek to adjust the internal parameters (or internal memory) of an LLM.

“In contrast to fine-tuning, RAG does not seek to adjust the internal parameters (or internal memory) of an LLM.”

Standard RAG architectures lack the temporal awareness, version tracking, and memory update mechanisms needed for conversational memory where information evolves over time.

“standard RAG architectures lack the temporal awareness, version tracking, and memory update mechanisms needed for conversational memory where information evolves over time.”

The proposed method differs from Retrieval-Augmented Generation (RAG) by focusing on adaptive, real-time memory management tailored for ongoing dialogues, as opposed to RAG's static knowledge retrieval from fixed databases.

“The proposed memory-augmented architecture differs from Retrieval-Augmented Generation (RAG) [6] by focusing on adaptive, real-time memory management tailored for ongoing dialogues, as opposed to RAG's static knowledge retrieval from fixed databases.”

The retrieval-plus-rerank decomposition adopted descends from RAG.

“The retrieval-plus-rerank decomposition we adopt descends from RAG (Lewis et al., 2020).”

Retrieving relevant knowledge is a widely used strategy to eliminate hallucination.

“Retrieving relevant knowledge is a widely used strategy to eliminate hallucination (Lewis et al., 2020;Li et al., 2023a).”

For AI Agents, RAG has the potential to mitigate hallucinations and can expand static LLM knowledge by grounding outputs in real-time data.

“For AI Agents, RAG has the potential to mitigate hallucinations and can expand static LLM knowledge by grounding outputs in real-time data [216].”

Traditional retrieval-augmented language models only read from human-written corpora.

“While traditional retrieval-augmented language models (Guu et al., 2020; Lewis et al., 2020; Borgeaud et al., 2022) only read from human-written corpora”

RAG incorporation ensures GenAI agents access relevant and accurate knowledge for context-specific prompts.

“RAG incorporation ensures GenAI agents access relevant and accurate knowledge for context-specific prompts (Gao et al., 2023; Lewis et al., 2020).”

Current literature on detecting and mitigating factual errors generated by machine learning models focuses predominantly on a single specific task.

“Current literature on detecting and mitigating factual errors generated by machine learning models focuses predominantly on a single specific task, for example, retrieval-augmented verification models for QA (Lewis et al., 2020), hallucination detection models for text summarization (Fabbri et al., 2022), and execution-based evaluation for code (Shi et al., 2022).”

Prior work optimizes citation accuracy via better retrieval or training but ignores when and how often systems should abstain rather than hallucinate.

“Prior work optimizes citation accuracy via better retrieval [8, 9] or training [10, 11], but ignores when and how often systems should abstain rather than hallucinate .”

Work on retrieval and memory-augmented language models suggests that useful context should often be accessed adaptively rather than monolithically packed into a single prompt.

“More broadly, work on retrieval and memory-augmented language models suggests that useful context should often be accessed adaptively rather than monolithically packed into a single prompt [28; 48; 37; 56].”

RAG uses vector-space semantic search to supply only the most contextually similar passages, reducing input length and theoretically suppressing irrelevant distraction.

“Retrieval-Augmented Generation (RAG) uses vector-space semantic search to supply only the most contextually similar passages, reducing input length and theoretically suppressing irrelevant distraction.[5]”

Lewis et al. (2020) demonstrated that grounding generative outputs in retrieved evidence substantially reduces hallucination in open-domain QA.

“Lewis et al. (2020) established the foundational RAG architecture, demonstrating that grounding generative outputs in retrieved evidence substantially reduces hallucination in open-domain QA. [5]”

Most RAG systems target document corpora rather than the incremental, personalized memories that accumulate during daily assistant interactions.

“While retrieval-augmented generation (RAG) [Lewis et al., 2020] has become the dominant paradigm for grounding LLM outputs in external knowledge, most RAG systems target document corpora rather than the incremental, personalized memories that accumulate during daily assistant interactions.”

RAG is well-suited for medical applications because it enables integration of up-to-date information, enhances transparency, reduces hallucinations, and produces evidence-grounded outputs.

“This architecture enables the integration of up-to-date information, enhances transparency, reduces hallucinations, and produces evidence-grounded outputs, making RAG well-suited for medical applications.”

RAG mitigates limitations of standalone LLMs by combining a retriever that accesses external sources with a generator that conditions responses on query and retrieved content.

“Introduced in 2020, Retrieval-Augmented Generation (RAG) mitigates these limitations by combining a retriever which accesses external sources such as clinical guidelines or EHRs with a generator that conditions responses on both the query and retrieved content [8].”

RAG was introduced as a framework combining generative reasoning explicitly with non-parametric retrieval from verified external sources.

“To address these problems, Retrieval-Augmented Generation (RAG) was introduced as a framework combining generative reasoning explicitly with non-parametric retrieval from verified external sources [6].”

RAG was first to couple a seq2seq generator with a dense document retriever at NeurIPS 2020.

“First to couple a seq2seq generator with a dense docu- ment retriever at NeurIPS 2020.”

RAG substantially improves factual recall in long-horizon settings, but its implications for belief tracking are less understood.

“While RAG substantially improves factual recall in long-horizon settings, its implications for belief tracking are less understood.”

The core limitation of retrieval-augmented approaches to longitudinal memory is that they solve the access problem but not the interpretation problem.

“This is the core limitation of retrieval-augmented approaches to longitudinal memory: they solve the access problem but not the interpretation problem [20, 5].”

Current approaches employ retrieval-augmented generation (RAG) that does not validate whether cited sources are accessible, topically relevant, or factually consistent with the claims they support.

“or employ retrieval-augmented generation (RAG) that does not validate whether cited sources are accessible, topically relevant, or factually consistent with the claims they support (Lewis et al., 2020; Lumer et al., 2025a; Gulati et al., 2026b)”

LLMs can generate responses with inline citations through web search integration and retrieval-augmented generation (RAG).

“LLMs can generate responses with inline citations through web search integration and retrieval-augmented generation (RAG) (Lewis et al., 2020).”

Early RAG implementations often relied on document retrieval methods that introduced noise and excessive context.

“Early RAG implementations, however, often relied on document retrieval methods that introduced noise and excessive context [26, 55, 60, 122].”

Equipping LLM agents with long-term memory significantly enhances their ability to reason over extended horizons.

“Recent work has demonstrated that equipping LLM agents with long-term memory significantly enhances their ability to reason over extended horizons [9, 11].”

Retrieval augmented generation (RAG) architectures directly feed useful information to the QA model, thus grounding the response, reducing factuality hallucinations, and improving explainability of results.

“As opposed to CoT, this architecture directly feeds useful information to the QA model, thus grounding the response, reducing factuality hallucinations, and improving explainability of results.”

Generative search inherits elements from retrieval-augmented generation: retrieve candidate material, ground a response, and provide attribution.

“Generative search inherits elements from retrieval-augmented generation: retrieve candidate material, ground a response, and provide attribution [12,13,14,15].”

Deep search agents conduct tasks through iterative query reasoning, search planning, and reflection on retrieved results, outperforming the traditional single-round retrieval-augmented generation paradigm.

“Deep search agents conduct tasks through iterative query reasoning, search planning, and reflection on retrieved results (Asai et al., 2024), outperforming the traditional single-round retrieval-augmented generation paradigm (Lewis et al., 2020).”

E-commerce generative engines can be viewed as instances of retrieval-augmented generation (RAG).

“At a high level, e-commerce generative engines can be viewed as instances of retrieval-augmented generation (RAG) (Lewis et al., 2020).”

RAG combines a parametric sequence-to-sequence model with a dense non-parametric memory of retrieved passages, making retrieval a first-class component of generation.

“RAG combines a parametric sequence-to-sequence model with a dense non-parametric memory of retrieved passages, making retrieval a first-class component of generation [12].”

Retrieval-Augmented Generation grounds LLM outputs in external evidence retrieved at inference time.

“Retrieval-Augmented Generation grounds LLM outputs in external evidence retrieved at inference time (Lewis et al., 2020; Gao et al., 2023).”

RAG relies on the retrieval of information from an externally-situated store, which functions as a form of non-parametric (or external) memory.

“Instead, RAG relies on the retrieval of information from an externally-situated store, which functions as a form of non-parametric (or external) memory.”

External knowledge stores can be queried dynamically during generation without retraining.

“These systems proved a crucial point: external knowledge stores can be queried dynamically during generation without retraining.”

RAG demonstrated that pairing a generator with a non-parametric retrieval index produces strong results on knowledge-intensive tasks.

“RAG [Lewis et al., 2020] demonstrated that pairing a generator with a non-parametric retrieval index produces strong results on knowledge-intensive tasks.”

Non-parametric memory serves as a supplementary knowledge source that can be dynamically accessed by the large language model, enhancing its ability to retrieve relevant information in real-time, as seen in retrieval-augmented generation (RAG).

“Non-parametric memory serves as a supplementary knowledge source that can be dynamically accessed by the large language model, enhancing its ability to retrieve relevant information in realtime, as seen in retrieval-augmented generation (RAG) [25].”

AI long-term memory can be classified into parametric and non-parametric memory based on the form of storage.

“We then classify AI long-term memory into parametric and non-parametric memory in Sec. 4, based on the form of storage [27, 28].”

AI long-term memory can be categorized into non-parametric memory and parametric memory, following criteria illustrated by Lewis et al. and Mallen et al.

“Based on these observations, we categorize AI long-term memory into non-parametric memory (Sec. 4.1) and parametric memory (Sec. 4.2), following the criteria illustrated by Lewis et al. [27] and Mallen et al. [28] and depicted in Fig. 5.”

The static nature of parametric memory can lead to information lag, posing challenges in maintaining the relevance of stored data as it may not reflect the most current information.

“Additionally, the static nature of parametric memory can lead to information lag, posing challenges in maintaining the relevance of the stored data as it may not reflect the most current information [27, 166].”

Several prior works note the benefits of treating large knowledge sources or long inputs as external resources that a language model accesses adaptively, rather than consuming them in a single pass.

“Several prior works note the benefits of treating large knowledge sources or long inputs as external resources that a language model accesses adaptively, rather than consuming them in a single pass.”

RAG, serving as flat factual memory in the setup, only outperforms on single-hop questions.

“Notably, RAG (Lewis et al., 2020) serves as flat factual memory in our setup, only outperforming on single-hop questions.”

Retrieval-Augmented Generation (RAG) mitigates the hallucination of large language models (LLMs).

“Retrieval-Augmented Generation (RAG) is a promising approach for mitigating the hallucination of large language models (LLMs).”

RAG enhances LLM capabilities by grounding responses in retrieved contextual information.

“Retrieval-Augmented Generation (RAG) has gained attention as a method to enhance LLM capabilities by grounding responses in retrieved contextual information (10).”

RAG enhances large language models by integrating external knowledge retrieval to improve factual consistency and reduce hallucinations.

“Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieval to improve factual consistency and reduce hallucinations.”

Recent studies have started exploring joint modeling of retrieval and generation in knowledge-intensive tasks, such as question answering and dialogue generation.

“Recent studies have started exploring the joint modeling of retrieval and generation in knowledgeintensive tasks, such as question answering (Guu et al., 2020;Lewis et al., 2020;Izacard et al., 2022) and dialogue generation (Zhang et al., 2022).”

Reducing factual errors (hallucinations) is a key challenge for building more reliable language models, even for the most powerful language models.

“Many works have identified reducing factual errors (sometimes called 'hallucinations') as a key challenge for building more reliable language models (Lewis et al., 2020; Kadavath et al., 2022; Zhang et al., 2023), even for the most powerful language models (Bubeck et al., 2023).”

RAG methods as a long-term memory system offer a simple way to update knowledge over time.

“On the other hand, using RAG methods as a long-term memory system offers a simple way to update knowledge over time [36, 42, 66, 73].”

Context adaptation offers advantages over weight updates: contexts are interpretable and explainable, allow rapid integration of new knowledge at runtime, and can be shared across models or modules.

“Adapting through contexts rather than weights offers several key advantages. Contexts are interpretable and explainable for users and developers (Wei et al., 2022; Wang et al., 2023), allow rapid integration of new knowledge at runtime (Lewis et al., 2020; Borgeaud et al., 2022), and can be shared across models or modules in a compound system (Khot et al., 2023).”

Retrieval-Augmented Generation decreases factual errors in knowledge-intensive tasks.

“Retrieval-Augmented Generation (RAG) methods (Figure 1 left; Lewis et al. 2020; Guu et al. 2020) augment the input of LLMs with relevant retrieved passages, reducing factual errors in knowledge-intensive tasks (Ram et al., 2023; Asai et al., 2023a).”

In-context learning methods now routinely invoke tools, leading to LM pipelines that use retrieval models, multimodal foundation models, and more traditional tools like APIs and calculators.

“In-context learning methods now routinely invoke tools, leading to LM pipelines that use retrieval models (Chen et al., 2017; Lewis et al., 2020; Guu et al., 2020; Lazaridou et al., 2022; Izacard et al., 2022), multimodal foundation models, and more traditional tools like APIs (Nakano et al., 2021) and calculators.”

RAG systems typically operate on a finite knowledge base.

“RAG systems typically operate on a finite knowledge base [46].”

The naive RAG paradigm has already been proven to be effective in multiple IA tasks such as question answering.

“This paradigm has already been proven to be effective in multiple IA tasks such as question answering [85].”

M3-Agent employs reinforcement learning to enable multi-turn reasoning and iterative memory retrieval, resulting in higher task success rates than single-turn RAG.

“Instead of using single-turn retrieval-augmented generation (RAG) to load memory into context [22], M3-Agent employs reinforcement learning to enable multi-turn reasoning and iterative memory retrieval, resulting in higher task success rates.”

Lewis et al. (2020) introduced RAG to access specific knowledge bases more precisely when using LLMs and reduce hallucination rates.

“Lewis et al. (2020) introduced RAG to access specific knowledge bases more precisely when using LLMs and reduce the 'hallucination' rates, when a model makes up information in a confident manner.”

Even retrieval-augmented models still do not guarantee attribution.

“Even retrievalaugmented models (Guu et al., 2020;Lewis et al., 2020), which retrieve relevant documents and then condition on them to generate text, still do not guarantee attribution.”

RAG models augment LLMs with an information retrieval mechanism that concatenates input prompts with relevant text retrieved from a vector index.

“RAG models augment LLMs with an information retrieval mechanism that concatenates input prompts with relevant text retrieved from a vector index (Lewis et al., 2020).”

Users can insert custom local modules after some existing blocks, which could allow use-cases like retrieval-augmented generation.

“Finally, users can insert custom local modules after some of the existing blocks, which could allow use-cases like retrieval-augmented generation (Borgeaud et al., 2021;Lewis et al., 2020).”

Current large language models exhibit remarkable single-turn capabilities but suffer from severe limitations in maintaining coherent, personalized interactions over extended temporal horizons (the 'goldfish problem').

“Current large language models (LLMs) exhibit remarkable single-turn capabilities but suffer from severe limitations in maintaining coherent, personalized interactions over extended temporal horizons a phenomenon we term the 'goldfish problem.'”

The strongest non-A-Mem baselines are domain-dependent: rolling Summary on the coding QA axis and Naive RAG on retrieval.

“The strongest non-A-Mem baselines are domain-dependent (rolling Summary on the coding QA axis and Naive RAG on retrieval), both of which A-Mem beats by a comfortable margin under maximum pressure.”

A-Mem's lead over Naive RAG widens with hop count.

“The payoff of memory therefore grows with pressure, and the gap is most visible on retrieval depth: A-Mem's lead over Naive RAG widens with hop count because note-evolution links bridge passages whose isolated query-relevance is low, which is the failure mode of a flat retriever at 5 / 6 -hop.”

HyDE constructs hypothetical documents (assumed answers to the original query) and focuses on embedding similarity from answer to answer rather than seeking embedding similarity for the problem or query.

“HyDE [11] construct hypothetical documents (assumed answers to the original query).It focuses on embedding similarity from answer to answer rather than seeking embedding similarity for the problem or query.”

This structure-aware retrieval enables multi-hop reasoning that flat semantic search cannot support.

“This structure-aware retrieval enables multi-hop reasoning that flat semantic search cannot support [Lewis et al., 2020, Gao et al., 2023].”

Conversational search engines are fundamentally based on the Retrieval-Augmented Generation (RAG) architecture.

“Conversational search engines are fundamentally based on the Retrieval-Augmented Generation (RAG) architecture.”

Naive retrieval-augmented generation lacks temporal awareness and consistency checking, frequently injecting contradictory or outdated context that degrades response quality.

“Information retrieval problems : Naive retrievalaugmented generation [7] lacks temporal awareness and consistency checking, frequently injecting contradictory or outdated context that degrades response quality.”

The typical agentic architecture integrates four components: a planning module, tool use interfaces, short- and long-term memory often implemented through retrieval-augmented generation, and reflection and self-assessment mechanisms.

“The typical architecture integrates four fundamental components: (i) a planning module that decomposes a complex goal into subtasks; (ii) tool use interfaces to query databases, execute bioinformatics pipelines, or control experimental automations; (iii) short- and long-term memory, often implemented through retrieval-augmented generation (RAG), to maintain context and documentary grounding [18];”

Memory-augmented methods retrieve user-specific information from past interactions, addressing the limited context window and enabling integration of past information into the input prompt.

“Memory-augmented methods extend this concept by retrieving user-specific information from past interactions. This addresses the challenge of a limited context window and enables the integration of past information into the input prompt [25, 29, 45].”