Retrieval-Augmented Generation for Large Language Models: A Survey

In the paper

“Coarse-grained retrieval units theoretically can provide more relevant information for the problem, but they may also contain redundant content, which could distract the retriever and language models in downstream tasks [50], [87].”

What the sources say

• ?Could not verify

  The Introduction and §3.1 describe retrieving relevant passages, and the ‘Effect of Number of Retrieved Passages’ section reports benefits from retrieving more passages. None addresses coarse-grained retrieval units or says their redundant content can distract retrievers or language models. Searches for redundancy, passage granularity, and chunk size found no passage supporting that tradeoff. (LLM, conf=0.72)

  cites [50] 50. J. Huang, W. Ping, P. Xu, M. Shoeybi, K. C.-C. Chang, and B. Catan- zaro, "Raven: In-context learning with retrieval augmented encoder- decoder language models," arXiv preprint arXiv:2308.07922, 2023. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Information Retrieval & Knowledge Representation and Retrieval-Augmented Intelligence Report Generation sections describe retrieval and its use in prompting, but do not discuss the size of retrieval units. Searches for coarse-grained units, redundant content, and distraction of retrievers or language models found no passage addressing that trade-off. (LLM, conf=0.74)

  cites [87] 87. P. Ranade and A. Joshi, "Fabula: Intelligence report generation using retrieval-augmented narrative construction," arXiv preprint arXiv:2310.13848, 2023. sourceExperimental — accuracy not yet measured

In the paper

“Structured data, such as knowledge graphs (KGs) [86] , which are typically verified and can provide more precise information.”

What the sources say

• Partly supported

  The Introduction (page unavailable) says task-oriented dialogue systems use external knowledge bases to retrieve relevant entity information and may need localized knowledge for specific tasks. That supports the general usefulness of structured knowledge, but the paper does not say knowledge graphs are typically verified or establish that they provide more precise information. (LLM, conf=0.76)

  cites [86] 86. T. Shi, L. Li, Z. Lin, T. Yang, X. Quan, and Q. Wang, "Dual-feedback knowledge retrieval for task-oriented dialogue systems," arXiv preprint arXiv:2310.14528, 2023. sourceExperimental — accuracy not yet measured

In the paper

“In multiple evaluations of their performance on various knowledge-intensive tasks across different topics, [28] revealed that while unsupervised fine-tuning shows some improvement, RAG consistently outperforms it, for both existing knowledge encountered during training and entirely new knowledge.”

What the sources say

• ?Could not verify

  Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.

  cites [28] 28. Z. Dai, V. Y. Zhao, J. Ma, Y. Luan, J. Ni, J. Lu, A. Bakalov, K. Guu, K. B. Hall, and M.-W. Chang, "Promptagator: Few-shot dense retrieval from 8 examples," arXiv preprint arXiv:2209.11755, 2022. sourceExperimental — accuracy not yet measured

In the paper

“Reranking fundamentally reorders document chunks to highlight the most pertinent results first, effectively reducing the overall document pool, severing a dual purpose in information retrieval, acting as both an enhancer and a filter, delivering refined inputs for more precise language model processing [70].”

What the sources say

• Partly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites [70] 70. P. Sarthi, S. Abdullah, A. Tuli, S. Khanna, A. Goldie, and C. D. Manning, "Raptor: Recursive abstractive processing for tree-organized retrieval," arXiv preprint arXiv:2401.18059, 2024. sourceExperimental — accuracy not yet measured

In the paper

“The bridge model aims to transform the retrieved information into a format that LLMs can work with effectively, allowing it to not only rerank but also dynamically select passages for each query, and potentially employ more advanced strategies like repetition.”

What the sources say

• Partly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites [26] 26. X. Li, E. Nie, and S. Liang, "From classification to generation: Insights into crosslingual retrieval augmented icl," arXiv preprint arXiv:2311.06595, 2023. sourceExperimental — accuracy not yet measured

In the paper

“Reranking can be performed using rule-based methods that depend on predefined metrics like Diversity, Relevance, and MRR, or model-based approaches like Encoder-Decoder models from the BERT series (e.g., SpanBERT), specialized reranking models such as Cohere rerank or bge-raranker-large, and general large language models like GPT [12], [99].”

What the sources say

• ?Could not verify

  Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.

  cites [12] 12. S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Milli- can, G. B. Van Den Driessche, J.-B. Lespiau, B. Damoc, A. Clark et al., "Improving language models by retrieving from trillions of tokens," in International conference on machine learning. PMLR, 2022, pp. 2206-2240. sourceExperimental — accuracy not yet measured
• ?Could not verify

  Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.

  cites [99] 99. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schu- urmans, C. Cui, O. Bousquet, Q. Le et al., "Least-to-most prompting enables complex reasoning in large language models," arXiv preprint arXiv:2205.10625, 2022. sourceExperimental — accuracy not yet measured

In the paper

“HyDE [11] construct hypothetical documents (assumed answers to the original query).It focuses on embedding similarity from answer to answer rather than seeking embedding similarity for the problem or query.”

What the sources say

• ?Could not verify

  The Introduction (p. 1) and §2.2 (p. 3) describe RAG retrieving documents using a query encoder and document embeddings. Searches for HyDE and hypothetical documents found no passage describing HyDE’s construction of assumed answers or its answer-to-answer embedding similarity, so our copy cannot verify the attributed claim. (LLM, conf=0.78)

  cites [11] 11. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel et al., "Retrieval- augmented generation for knowledge-intensive nlp tasks," Advances in Neural Information Processing Systems, vol. 33, pp. 9459-9474, 2020. sourceExperimental — accuracy not yet measured

In the paper

“BGM [26] keeps the retriever and LLM fixed,and trains a bridge Seq2Seq model in between.”

What the sources say

• ?Could not verify

  Sections 3.1–3.2 describe cross-lingual retrieval followed by prompt construction for a multilingual language model. Searches for BGM, a trained bridge Seq2Seq model, and fixed retriever and LLM components found no passage addressing the claimed architecture in our copy; page numbers were not provided. (LLM, conf=0.74)

  cites [26] 26. X. Li, E. Nie, and S. Liang, "From classification to generation: Insights into crosslingual retrieval augmented icl," arXiv preprint arXiv:2311.06595, 2023. sourceExperimental — accuracy not yet measured

In the paper

“(Long) LLMLingua [100], [101] utilize small language models (SLMs) such as GPT-2 Small or LLaMA-7B, to detect and remove unimportant tokens, transforming it into a form that is challenging for humans to comprehend but well understood by LLMs.”

What the sources say

• ?Could not verify

  The Introduction (p. 1) discusses hallucinations, while the Baselines section (p. 6) discusses Llama models and post-processing, not LLMLingua. Searches for LLMLingua, GPT-2 Small, and prompt compression found no passage describing the claimed token-removal method. (LLM, conf=0.76)

  cites [100] 100. S. Dhuliawala, M. Komeili, J. Xu, R. Raileanu, X. Li, A. Celikyilmaz, and J. Weston, "Chain-of-verification reduces hallucination in large language models," arXiv preprint arXiv:2309.11495, 2023. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Discussion and Analysis section describes LLMs labeling data for AnglE training, not LLMLingua or token removal. Searches for LLMLingua, GPT-2 Small, and prompt compression found no passage in our copy addressing the claimed method or its readability. (LLM, conf=0.78)

  cites [101] 101. X. Li and J. Li, "Angle-optimized text embeddings," arXiv preprint arXiv:2309.12871, 2023. sourceExperimental — accuracy not yet measured

In the paper

“Furthermore, PKG introduces an innovative method for integrating knowledge into white-box models via directive fine-tuning [75].”

What the sources say

• ?Could not verify

  The Generative Retrieval and Constrained Decoding and FM-Index sections describe 1-PAGER’s keyword-path retrieval and constrained decoding, not a method called PKG. Searches for PKG, white-box models, and directive fine-tuning found no passage addressing the attributed method. The cited paper therefore does not verify this claim. (LLM, conf=0.76)

  cites [75] 75. P. Jain, L. B. Soares, and T. Kwiatkowski, "1-pager: One pass answer generation and evidence retrieval," arXiv preprint arXiv:2310.16568, 2023. sourceExperimental — accuracy not yet measured

In the paper

“While PRCA [69] add a pluggable reward-driven contextual adapter to enhance performance on specific tasks.”

What the sources say

• ?Could not verify

  Section 3.2.2 describes dynamically introducing retrieved documents based on an LLM’s prior judgment, not PRCA or a reward-driven contextual adapter. Searches for “PRCA” and “reward-driven contextual adapter” found no passage in our copy addressing the attributed method. (LLM, conf=0.76)

  cites [69] 69. R. Ren, Y. Wang, Y. Qu, W. X. Zhao, J. Liu, H. Tian, H. Wu, J.- R. Wen, and H. Wang, "Investigating the factual knowledge boundary of large language models with retrieval augmentation," arXiv preprint arXiv:2307.11019, 2023. sourceExperimental — accuracy not yet measured

In the paper

“AAR (Augmentation-Adapted Retriver) [47] introduces a universal adapter designed to accommodate multiple downstream tasks.”

What the sources say

• ?Could not verify

  Section 3.4 and Chapter 5 describe LMIndexer being fine-tuned for multiple downstream tasks, not an AAR universal adapter. Searches for “AAR,” “adapter,” and “Augmentation-Adapted Retriever” found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)

  cites [47] 47. B. Jin, H. Zeng, G. Wang, X. Chen, T. Wei, R. Li, Z. Wang, Z. Li, Y. Li, H. Lu et al., "Language models as semantic indexers," arXiv preprint arXiv:2310.07815, 2023. sourceExperimental — accuracy not yet measured

In the paper

“PROMPTAGATOR [21] utilizes the LLM as a few-shot query generator to create task-specific retrievers, addressing challenges in supervised fine-tuning, particularly in data-scarce domains.”

What the sources say

• ?Could not verify

  The Introduction describes this paper’s method as ITER-RETGEN, which iterates retrieval and generation and can adapt retrieval using model outputs. Searches for “PROMPTAGATOR” and for few-shot query generation found no passage in our copy addressing the attributed method or its use in data-scarce domains. (LLM, conf=0.76)

  cites [21] 21. Z. Shao, Y. Gong, Y. Shen, M. Huang, N. Duan, and W. Chen, "Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy," arXiv preprint arXiv:2305.15294, 2023. sourceExperimental — accuracy not yet measured

In the paper

“For retrieval tasks that engage with structured data, the SANTA framework [76] implements a tripartite training regimen to effectively encapsulate both structural and semantic nuances.”

What the sources say

• ?Could not verify

  The paper’s Section 3.1 describes a two-stage training strategy for PRCA, not a training regimen for SANTA. Searches for SANTA, tripartite training, and structured-data retrieval found no passage addressing the attributed claim. The difference in stage counts does not contradict a claim about a different framework. (LLM, conf=0.78)

  cites [76] 76. H. Yang, Z. Li, Y. Zhang, J. Wang, N. Cheng, M. Li, and J. Xiao, "Prca: Fitting black-box large language models for retrieval question answer- ing via pluggable reward-driven contextual adapter," arXiv preprint arXiv:2310.18347, 2023. sourceExperimental — accuracy not yet measured

In the paper

“In this approach, the retriever module is directly substituted to generate relevant documents according to a query.”

What the sources say

• Partly supported

  Section 3, “Iterative Corpus Partitioning and Answer Prediction” (page unavailable), says 1-PAGER replaces a separate retrieval-and-answer-generation pipeline with a single sequence-to-sequence model. But the model generates keyword search paths to select relevant documents from a corpus; it does not generate the documents themselves. (LLM, conf=0.87)

  cites [75] 75. P. Jain, L. B. Soares, and T. Kwiatkowski, "1-pager: One pass answer generation and evidence retrieval," arXiv preprint arXiv:2310.16568, 2023. sourceExperimental — accuracy not yet measured

In the paper

“REPLUG [72] utilizes a retriever and an LLM to calculate the probability distributions of the retrieved documents and then performs supervised training by computing the KL divergence.”

What the sources say

• ?Could not verify

  The Introduction and §2.3 describe this paper’s retrieval-augmented event argument extraction method and its negative-log-likelihood training objective. Searches for REPLUG, retrieved-document probability distributions, and KL divergence found no passage addressing how REPLUG is trained. The different objective used by this paper does not establish a contrary claim about REPLUG. (LLM, conf=0.76)

  cites [72] 72. Y. Ren, Y. Cao, P. Guo, F. Fang, W. Ma, and Z. Lin, "Retrieve-and- sample: Document-level event argument extraction via hybrid retrieval augmentation," in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 293-306. sourceExperimental — accuracy not yet measured

In the paper

“The implementation of the query rewrite method in the Taobao, known as BEQUE [9] has notably enhanced recall effectiveness for long-tail queries, resulting in a rise in GMV.”

What the sources say

• ?Could not verify

  The passages from §5.4.2 and §5.5 discuss retrieval and query refinement for reducing LLM hallucinations, not Taobao search. Searches for BEQUE, long-tail queries, recall effectiveness, and GMV found no passage in our copy addressing the claimed implementation or its results. (LLM, conf=0.78)

  cites [9] 9. Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen et al., "Siren's song in the ai ocean: A survey on hal- lucination in large language models," arXiv preprint arXiv:2309.01219, 2023. sourceExperimental — accuracy not yet measured

In the paper

“ITER-RETGEN [14] employs a synergistic approach that leverages”

What the sources say

• ?Could not verify

  The Introduction and §3.1 describe the paper’s Rewrite-Retrieve-Read framework, not ITER-RETGEN. Searches for ITER-RETGEN, the two named generation–retrieval approaches, and the task description found no passage in our copy addressing the attributed claim. (LLM, conf=0.72)

  cites [14] 14. X. Ma, Y. Gong, P. He, H. Zhao, and N. Duan, "Query rewrit- ing for retrieval-augmented large language models," arXiv preprint arXiv:2305.14283, 2023. sourceExperimental — accuracy not yet measured

In the paper

“In the domain of RAG, the standard practice often involves a singular (once) retrieval step followed by generation, which can lead to inefficiencies and sometimes is typically insufficient for complex problems demanding multi-step reasoning, as it provides a limited scope of information [105].”

What the sources say

• Partly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites [105] 105. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, "Lost in the middle: How language models use long contexts," arXiv preprint arXiv:2307.03172, 2023. sourceExperimental — accuracy not yet measured

In the paper

“To overcome challenges, Retrieval-Augmented Generation (RAG) enhances LLMs by retrieving relevant document chunks from external knowledge base through semantic similarity calculation.”

In the paper

“RAG excels in dynamic environments by offering realtime knowledge updates and effective utilization of external knowledge sources with high interpretability.”

In the paper

“Handling semi-structured data poses challenges for conventional RAG systems due to two main reasons.Firstly, text splitting processes may inadvertently separate tables, leading to data corruption during retrieval.Secondly, incorporating tables into the data can complicate semantic similarity searches.”

In the paper

“Advanced RAG introduces specific improvements to overcome the limitations of Naive RAG.Focusing on enhancing retrieval quality, it employs pre-retrieval and post-retrieval strategies.”

In the paper

“This comprehensive review paper offers a detailed examination of the progression of RAG paradigms, encompassing the Naive RAG, the Advanced RAG, and the Modular RAG.”

• Also stated in: § II. OVERVIEW OF RAG · plain

  “The RAG research paradigm is continuously evolving, and we categorize it into three stages: Naive RAG, Advanced RAG, and Modular RAG, as showed in Figure 3.”

In the paper

“RAG relies on external knowledge to enhance LLMs, while the type of retrieval source and the granularity of retrieval units both affect the final generation results.”

In the paper

“Feeding all relevant documents directly into LLMs can lead to information overload, diluting the focus on key details with irrelevant content.”

In the paper

“The modular RAG architecture advances beyond the former two RAG paradigms, offering enhanced adaptability and versatility.”

In the paper

“The process may also encounter redundancy when similar information is retrieved from multiple sources, leading to repetitive responses.”

In the paper

“Facing complex issues, a single retrieval based on the original query may not suffice to acquire adequate context information.”

In the paper

“The retrieval phase often struggles with precision and recall, leading to the selection of misaligned or irrelevant chunks, and the missing of crucial information.”

In the paper

“Despite its distinctiveness, Modular RAG builds upon the foundational principles of Advanced and Naive RAG, illustrating a progression and refinement within the RAG family.”

In the paper

“The main methods in post-retrieval process include rerank chunks and context compressing.”

In the paper

“The development of Advanced RAG and Modular RAG is a response to these specific shortcomings in Naive RAG.”

In the paper

“Large Language Models (LLMs) showcase impressive capabilities but encounter challenges like hallucination, outdated knowledge, and non-transparent, untraceable reasoning processes.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [1] 1. TriviaQA(TQA) [113]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [2] 2. 25], [67] MS MARCO [117]

In the paper

“Unstructured Data, such as text, is the most widely used retrieval source, which are mainly gathered from corpus.”

In the paper

“Moreover, there's a concern that generation models might overly rely on augmented information, leading to outputs that simply echo retrieved content without adding insightful or synthesized information.”

In the paper

“On the other hand, fine-grained retrieval unit granularity increases the burden of retrieval and does not guarantee semantic integrity and meeting the required knowledge.”

In the paper

“RAG synergistically merges LLMs' intrinsic knowledge with the vast, dynamic repositories of external databases.”

In the paper

“However, it comes with higher latency and ethical considerations regarding data retrieval.”

In the paper

“Integrating retrieved information with the different task can be challenging, sometimes resulting in disjointed or incoherent outputs.”

In the paper

“The quality of index construction determines whether the correct context can be obtained in the retrieval phase.”

In the paper

“This phase can also suffer from irrelevance, toxicity, or bias in the outputs, detracting from the quality and reliability of the responses.”

In the paper

“However, both of these methods are not optimal solutions, indicating substantial research opportunities in this area.”

In the paper

“The shift towards a modular RAG approach is becoming prevalent, supporting both sequential processing and integrated end-to-end training across its components.”

In the paper

“It demands significant computational resources for dataset preparation and training, and while it can reduce hallucinations, it may face challenges with unfamiliar data.”

In the paper

“Retrieval-Augmented Generation (RAG) has emerged as a promising solution by incorporating knowledge from external databases.”

In the paper

“While smaller chunks may not fully convey the necessary context, they do have less noise.”

In the paper

“RAG and FT are not mutually exclusive and can complement each other, enhancing a model's capabilities at different levels.”

In the paper

“On the other hand, FT is more static, requiring retraining for updates but enabling deep customization of the model's behavior and style.”

In the paper

“On the contrary, it requires additional effort to build, validate, and maintain structured databases.”

In the paper

“This enhances the accuracy and credibility of the generation, particularly for knowledge-intensive tasks, and allows for continuous knowledge updates and integration of domainspecific information.”

In the paper

“In generating responses, the model may face the issue of hallucination, where it produces content not supported by the retrieved context.”

In the paper

“Despite RAG method are cost-effective and surpass the performance of the native LLM, they also exhibit several limitations.”

In the paper

“Larger chunks can capture more context, but they also generate more noise, requiring longer processing time and higher costs.”

In the paper

“A common misconception in the RAG process is the belief that retrieving as many relevant documents as possible and concatenating them to form a lengthy retrieval prompt is beneficial.”

In the paper

“Sparse and dense embedding approaches capture different relevance features and can benefit from each other by leveraging complementary relevance information.”

In the paper

“Additionally, pre-training language models (PLMs) can be utilized to learn term weights to enhance sparse retrieval.”

In the paper

“Targeted fine-tuning based on the scenario and data characteristics on LLMs can yield better results.”

In the paper

“Language models often struggle when dealing with specialized vocabulary or ambiguous abbreviations with multiple meanings.”

In the paper

“For instance, sparse retrieval models can be used to provide initial search results for training dense retrieval models.”

In the paper

“Certainly, a hybrid routing approach can also be employed, combining both semantic and metadata-based methods for enhanced query routing.”

In the paper

“Consequently, some approaches opt to incorporate an external adapter to aid in alignment.”

In the paper

“Expanding a single query into multiple queries enriches the content of the query, providing further context to address any lack of specific nuances, thereby ensuring the optimal relevance of the generated answers.”

In the paper

“Formulating a precise and clear question is difficult, and imprudent queries result in subpar retrieval effectiveness.”

In the paper

“Specifically, using LLM to generate questions that can be answered by the document, then calculating the similarity between the original question and the hypothetical question during retrieval to reduce the semantic gap between the question and the answer.”

In the paper

“When LLMs lack data in a specific domain, additional knowledge can be provided to the LLM through fine-tuning.”

In the paper

“Another approach, LLM-Embedder [97], exploits LLMs to generate reward signals across multiple downstream tasks.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [97] 97. S. Yang, "Advanced rag 01: Small-to- big retrieval," https://towardsdatascience.com/ advanced-rag-01-small-to-big-retrieval-172181b396d4, 2023.

In the paper

“In this paradigm, SLMs serve as filters, while LLMs function as reordering agents.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [103] 103. BAAI, "Flagembedding," https://github.com/FlagOpen/ FlagEmbedding, 2023.

In the paper

“In instances where the context significantly deviates from pre-training corpus, particularly within highly specialized disciplines such as healthcare, legal practice, and other sectors replete with proprietary jargon, fine-tuning the embedding model on your own domain dataset becomes essential to mitigate such discrepancies.”

In the paper

“Ma et al. [103] propose the”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [103] 103. BAAI, "Flagembedding," https://github.com/FlagOpen/ FlagEmbedding, 2023.

In the paper

“Assigning different weights to document timestamps during retrieval can achieve time-aware RAG, ensuring the freshness of knowledge and avoiding outdated information.”

In the paper

“Utilize KG in constructing the hierarchical structure of documents contributes to maintaining consistency.It delineates the connections between different concepts and entities, markedly reducing the potential for illusions.”

In the paper

“Specifically, it also demonstrates that sparse retrieval models can enhance the zero-shot retrieval capability of dense retrieval models and assist dense retrievers in handling queries containing rare entities, thereby improving robustness.”

In the paper

“This mainly includes a sparse encoder (BM25) and a dense retriever (BERT architecture Pre-training language models).”

In the paper

“After retrieval, it is not a good practice to directly input all the retrieved information to the LLM for answering questions.”

In the paper

“One effective method for enhancing information retrieval is to establish a hierarchical structure for the documents.”

In the paper

“For example, it can enable LLM to adapt to specific data formats and generate responses in a particular style as instructed [37].”

What the sources say

• ?Not checked — no full text of the sourcecites [37] 37. T. Chen, H. Wang, S. Chen, W. Yu, K. Ma, X. Zhao, D. Yu, and H. Zhang, "Dense x retrieval: What retrieval granularity should we use?" arXiv preprint arXiv:2312.06648, 2023. source

In the paper

“Fine-tuning models may present challenges, such as integrating functionality through an API or addressing constraints arising from limited local computational resources.”

In the paper

“where the semantic representation capability of embedding models plays a key role.”

In the paper

“Recent research has introduced prominent embedding models such as AngIE, Voyage, BGE,etc [94]- [96], which are benefit from multi-task instruct tuning.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [94]- [96]

In the paper

“Based on varying queries, routing to distinct RAG pipeline,which is suitable for a versatile RAG system designed to accommodate diverse scenarios.”

In the paper

“Another benefit of fine-tuning is the ability to adjust the model's input and output.”

In the paper

“Iterative retrieval is a process where the knowledge base is repeatedly searched based on the initial query and the text generated so far, providing a more comprehensive knowledge”

In the paper

“However, it may be affected by semantic discontinuity and the accumulation of irrelevant information.”

In the paper

“When circumstances prevent access to powerful proprietary models or larger parameter open-source models, a simple and effective method is to distill the more powerful models(e.g.GPT-4).”

In the paper

“Recursive retrieval is often used in information retrieval and NLP to improve the depth and relevance of search results.”

In the paper

“Aligning LLM outputs with human or retriever preferences through reinforcement learning is a potential approach.”