From Matching to Generation: A Survey on Generative Information Retrieval

It may state two separate claims as one.

On MS MARCO, TSGen achieves Recall@1 of 0.384 and DGR achieves 0.359, surpassing DPR (0.271) and comparable to SPLADEv2 (0.328).

“Specifically, on the MS MARCO dataset, GR models such as TSGen and DGR achieve Recall@1 scores of 0.384 and 0.359 respectively, surpassing dense methods like DPR (0.271) and being comparable to SPLADEv2 (0.328).”

It may state two separate claims as one.

GenRet shows competitive Recall@1 on NQ (0.681), but does not report results on MS MARCO.

“While GenRet shows competitive Recall@1 on NQ (0.681), it does not report results on MS MARCO.”

It may state two separate claims as one.

LTRGR achieves the highest Recall@10 on MS MARCO (0.759) and maintains strong performance across other metrics and on NQ.

“For instance, LTRGR achieves the highest Recall@10 on MS MARCO (0.759) and maintains strong performance across other metrics and on the NQ dataset.”

It may state two separate claims as one.

GENRE achieves the highest scores in several KILT categories, including FEVER (0.847), AY2 (0.928), WnWi (0.877), and WnCw (0.706), outperforming BM25 and MT-DPR.

“Notably, GENRE achieves the highest scores in several categories, including FEVER (0.847), AY2 (0.928), WnWi (0.877), and WnCw (0.706), outperforming the best sparse method BM25 and dense methods like MT-DPR.”

It may state two separate claims as one.

GLEN achieves the highest MRR@100 on NQ (0.754) but lags behind in other metrics.

“GLEN achieves the highest MRR@100 on NQ (0.754) but lags behind in other metrics.”

It may state two separate claims as one.

On NQ, TSGen attains the highest Recall@1 of 0.708, outperforming SPLADEv2 (0.624) and DPR (0.502).

“On the NQ dataset, GR models also show strong performance, with TSGen attaining the highest Recall@1 of 0.708, outperforming both SPLADEv2 (0.624) and DPR (0.502).”

It may state two separate claims as one.

GENRE and BART achieve FEVER scores of 0.847 and 0.821 on KILT.

“For instance, GENRE and BART achieve FEVER scores of 0.847 and 0.821, respectively.”

It may state two separate claims as one.

On MS MARCO, GENRE achieves Recall@1 0.266 and MRR@10 0.361, significantly lower than TSGen (0.384, 0.502) and LTRGR (0.327, 0.463).

“For example, on the MS MARCO dataset, GENRE achieves a Recall@1 of 0.266 and an MRR@10 of 0.361, which are significantly lower than TSGen (Recall@1 = 0.384, MRR@10 = 0.502) and LTRGR (Recall@1 = 0.327, MRR@10 = 0.463).”

It may state two separate claims as one.

SEAL achieves the highest QA scores across several KILT categories, including NQ (0.658), HoPo (0.565), TQA (0.715), and WoW (0.527).

“SEAL achieves the highest QA scores across several categories, including NQ (0.658), HoPo (0.565), TQA (0.715), and WoW (0.527).”

It may state two separate claims as one.

TSGen leads with the highest Recall@1 on MS MARCO and highest MRR@10 on NQ.

“TSGen leads with the highest Recall@1 and MRR@10 on MS MARCO and NQ respectively, indicating robust retrieval capabilities.”

It may state two separate claims as one.

The document ranking list-based IR approach still requires users to spend time summarizing their required answers, which is not ideal enough for information seeking.

“This document ranking list-based IR approach still requires users to spend time summarizing their required answers, which is not ideal enough for information seeking [195].”

• cites [195]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Due to hallucinations, the responses generated by LLMs are often considered unreliable because users are unaware of the sources behind the generated content, making it difficult to verify its accuracy.

“Moreover, due to the aforementioned issue of hallucinations, the responses generated by LLMs are often considered unreliable because users are unaware of the sources behind the generated content, making it difficult to verify its accuracy.”

• cites [143]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [204]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [256]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

Text-based DocIDs have the inherent advantage of effectively leveraging pre-trained language models and offering better interpretability.

“Text-based DocIDs have the inherent advantage of effectively leveraging the strong capabilities of pre-trained language models and offering better interpretability.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Tool augmentation extends the capabilities of LLMs by incorporating specialized tools or APIs that can perform specific functions.

“Tool augmentation, on the other hand, extends the capabilities of LLMs by incorporating specialized tools or APIs that can perform specific functions like mathematical computations, data retrieval, or executing predefined commands [226,245,276].”

• cites [226]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [245]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [276]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Tied-Atomic proposed to link document text with token embeddings and employs contrastive loss for DocID generation.

“To ensure that DocID embeddings can reflect document content, Tied-Atomic [206] proposed to link document text with token embeddings and employs contrastive loss for DocID generation.”

• cites [206]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

MEVI clusters documents using Residual Quantization and utilizes dual-tower and seq2seq model embeddings for balanced performance in large-scale document retrieval.

“MEVI [346] clusters documents using Residual Quantization (RQ) [189] and utilizes dual-tower and seq2seq model embeddings for a balanced performance in large-scale document retrieval.”

• cites [346]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [189]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Unstructured Atomic DocID assigns a unique random integer to each document and contains no structure or semantic information.

“Unstructured Atomic DocID: a unique integer identifier is randomly assigned to each document, containing no structure or semantic information.”

• cites [281]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

Accurate document representations are crucial as they enable the model to more effectively memorize document information, leading to enhanced retrieval performance.

“Accurate document representations are crucial as they enable the model to more effectively memorize document information, leading to enhanced retrieval performance.”

It may state two separate claims as one.

GR eliminates the need for large-scale document indexes in traditional methods, enabling end-to-end training of the model.

“This eliminates the need for large-scale document indexes in traditional methods, enabling end-to-end training of the model.”

It may state two separate claims as one.

The academic community has developed strategies across four key aspects to address LLM reliability challenges: enhancing internal knowledge; augmenting external knowledge; generating responses with citation; and improving personal information assistance.

“To address these challenges, the academic community has developed strategies across four key aspects: enhancing internal knowledge [16,37,56,119,132,193,243,267,285]; augmenting external knowledge [5,113,139,151,204,245,333]; generating responses with citation [129, 142,156,204,314]; and improving personal information assistance [149,172,295,327].”

• cites [16]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [37]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [56]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. It may state two separate claims as one.

Generative information retrieval (GenIR) emerges as a novel paradigm, attracting increasing attention.

“With the advancement of pre-trained language models, generative information retrieval (GenIR) emerges as a novel paradigm, attracting increasing attention.”

• Also stated in: § 1 INTRODUCTION · plain

  “The rapid development of LLMs is transforming IR systems, giving rise to a new paradigm of generative information retrieval (GenIR), which achieves IR goals through generative approaches.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

DynamicRetriever built a model-based IR system based on unstructured atomic DocID.

“Concurrently, DynamicRetriever [370] also built a model-based IR system based on unstructured atomic DocID.”

• cites [370]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

All these methods are effective strategies for improving both the quality and reliability of language model outputs and are essential technologies for building the next generation of generative information retrieval systems.

“All these methods are effective strategies for improving both the quality and reliability of language model outputs and are essential technologies for building the next generation of generative information retrieval systems.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

DSI introduced three numeric DocIDs to represent documents: unstructured atomic, naively structured string, and semantically structured.

“Initially, DSI [281] introduced three numeric DocIDs to represent documents: (1) Unstructured Atomic DocID: a unique integer identifier is randomly assigned to each document, containing no structure or semantic information.”

• cites [281]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

SEAL proposed a substring identifier, representing documents with any N-grams within them.

“To increase the flexibility of DocIDs, SEAL [13] proposed a substring identifier, representing documents with any N-grams within them.”

• cites [13]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

GR assigns a unique identifier to each document, which can be numeric-based or text-based, and then trains a generative retrieval model to learn the mapping from queries to the relevant DocIDs.

“Specifically, GR assigns a unique identifier to each document, which can be numeric-based or text-based, and then trains a generative retrieval model to learn the mapping from queries to the relevant DocIDs.”

• cites [18]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [281]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [307]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Using FM-Index, SEAL could generate N-grams present in the corpus to retrieve all documents containing those N-grams, scoring and ranking documents based on the frequency of N-grams in each document and the importance of N-grams.

“Using FM-Index (a compressed full-text sub-string index) [62], SEAL could generate N-grams present in the corpus to retrieve all documents containing those N-grams, scoring and ranking documents based on the frequency of N-grams in each document and the importance of N-grams.”

• cites [13]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [62]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. It may state two separate claims as one.

The rapid development of LLMs is transforming IR systems, giving rise to a new paradigm of generative information retrieval (GenIR).

“The rapid development of LLMs is transforming IR systems, giving rise to a new paradigm of generative information retrieval (GenIR), which achieves IR goals through generative approaches.”

It may state two separate claims as one.

ASI extends LMIndexer by establishing an end-to-end retrieval framework, incorporating semantic loss functions and reparameterization to enable joint training.

“ASI extends this by establishing an end-to-end retrieval framework, incorporating semantic loss functions and reparameterization to enable joint training.”

• cites [330]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim.

Generative Document Retrieval (GR) leverages the generative model's parameters for memorizing documents, enabling retrieval by directly generating relevant document identifiers without explicit indexing.

“Generative Document Retrieval (GR) leverages the generative model's parameters for memorizing documents, enabling retrieval by directly generating relevant document identifiers without explicit indexing.”

• Also stated in: § 1 INTRODUCTION · plain

  “This approach leverages the parametric memory of generative models to directly generate document identifiers (DocIDs) related to the documents [18,281,307,371].”

• cites [18]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [281]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [307]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Naively Structured String DocID treats random integers as divisible strings and implements character-level DocID decoding to replace large softmax output layers.

“Naively Structured String DocID: treating random integers as divisible strings, implementing character-level DocID decoding to replace large softmax output layers.”

• cites [281]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. It may state two separate claims as one.

GenIR models must possess both sufficient memorized knowledge and the ability to recall the associations between knowledge and source documents.

“To achieve this, GenIR models must possess both sufficient memorized knowledge and the ability to recall the associations between knowledge and source documents, which could be the final goal of GenIR systems.”

• cites [195]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

GenRet used progressive training and diversity clustering for optimization.

“Furthermore, it used progressive training and diversity clustering for optimization.”

• cites [265]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

LLM-URL directly generated URLs using ChatGPT prompts, achieving commendable performance after removing invalid URLs.

“Notably, LLM-URL [376] directly generated URLs using ChatGPT prompts, achieving commendable performance after removing invalid URLs.”

• cites [376]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Semantically Structured DocID introduces semantic structure through hierarchical k-means, allowing semantically similar documents to share prefixes and reducing the search space.

“Semantically Structured DocID: introducing semantic structure through hierarchical 𝑘-means method, allowing semantically similar documents to share prefixes in their identifiers, effectively reducing the search space.”

• cites [281]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

GERE, CorpusBrain, Re3val, and CorpusBrain++ also based their work on title DocIDs for Wikipedia-based tasks.

“Following GENRE, GERE [27], CorpusBrain [28], Re3val [257], and CorpusBrain++ [80] also based their work on title DocIDs for Wikipedia-based tasks.”

• cites [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [28]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [257]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Ultron encoded documents into a latent semantic space using BERT and compressed vectors via Product Quantization, preserving semantic information; each document's PQ code serves as its semantic identifier.

“Subsequently, Ultron [371] encoded documents into a latent semantic space using BERT [121], and compressed vectors into a smaller semantic space via Product Quantization (PQ) [73,102], preserving semantic information.Each document's PQ code serves as its semantic identifier.”

• cites [371]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [121]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [73]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [102]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Generative retrieval relies on pre-trained generative language models to directly generate DocIDs relevant to the query, achieving end-to-end retrieval without relying on large-scale pre-built document indices.

“Unlike traditional index-based retrieval methods, generative retrieval relies on pre-trained generative language models, such as T5 [231] and BART [138], to directly generate document identifiers (DocIDs) relevant to the query, thereby achieving end-to-end retrieval without relying on large-scale pre-built document indices.”

• cites [231]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [138]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

LMIndexer and ASI learned optimal DocIDs through semantic indexing; LMIndexer uses a reparameterization mechanism for unified optimization, facilitating efficient retrieval by aligning semantically similar documents under common DocIDs.

“LMIndexer [112] and ASI [330] learned optimal DocIDs through semantic indexing, with LMIndexer using a reparameterization mechanism for unified optimization, facilitating efficient retrieval by aligning semantically similar documents under common DocIDs.”

• cites [112]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [330]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Traditional sparse and dense retrieval methods rely on large-scale document indices and cannot be optimized in an end-to-end way.

“Although these methods have achieved leaps in accuracy, they rely on large-scale document indices [57,187] and cannot be optimized in an end-to-end way.”

• cites [57]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [187]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim.

Another essential component of generative retrieval is document representation, also known as document identifiers (DocIDs), which act as the target outputs for the GR model.

“Another essential component of generative retrieval is document representation, also known as document identifiers (DocIDs), which act as the target outputs for the GR model.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

RIPOR treats the GR model as a dense encoder to encode document content, splits these representations into vectors via RQ, creating unique DocID sequences.

“Furthermore, RIPOR [344] treats the GR model as a dense encoder to encode document content.It then splits these representations into vectors via RQ [189], creating unique DocID sequences.”

• cites [344]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [189]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim.

Current research in GenIR can be categorized into two aspects: Generative Document Retrieval (GR) and Reliable Response Generation.

“Based on the form of information provided to users, current research in GenIR can be categorized into two aspects: (1) Generative Document Retrieval (GR) leverages the generative model's parameters for memorizing documents, enabling retrieval by directly generating relevant document identifiers without explicit indexing.(2) Reliable Response Generation employs language models to directly generate”

• Also stated in: § 1 INTRODUCTION · plain

  “Currently, research in GenIR primarily focuses on two main patterns: (1) Generative Document Retrieval (GR), which involves retrieving documents by generating their identifiers; and (2) Reliable Response Generation, which entails directly generating user-centric responses with reliability enhancement strategies.”

It may state two separate claims as one.

In the web search scenario, document titles in the corpus often have significant duplication and many meaningless titles, making it unfeasible to use titles alone as DocIDs.

“However, in the web search scenario [205], document titles in the corpus often have significant duplication and many meaningless titles, making it unfeasible to use titles alone as DocIDs.”

• cites [205]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

The most straightforward text-based identifier is the document title, which requires each title to uniquely represent a document in the corpus; otherwise it would not be possible to accurately retrieve a specific document.

“The most straightforward text-based identifier is the document title, which requires each title to uniquely represent a document in the corpus, otherwise, it would not be possible to accurately retrieve a specific document.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

RIPOR implements a prefix-guided ranking optimization, increasing relevance scores for prefixes of pertinent DocIDs through margin decomposed pairwise loss during decoding.

“Furthermore, RIPOR implements a prefix-guided ranking optimization, increasing relevance scores for prefixes of pertinent DocIDs through margin decomposed pairwise loss during decoding.”

• cites [344]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

GenRet proposed learnable document representations, transforming documents into DocIDs through an encoder and reconstructing documents from DocIDs using a decoder trained to minimize reconstruction error.

“Unlike previous static DocIDs, GenRet [265] proposed learnable document representations, transforming documents into DocIDs through an encoder, then reconstructs documents from DocIDs using a decoder, trained to minimize reconstruction error.”

• cites [265]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Dense retrieval methods capture the deep semantic information of documents, significantly improving retrieval precision.

“With the rise of deep learning, dense retrieval methods such as DPR [117] and ANCE [324], based on the bidirectional encoding representations from the BERT model [121], capture the deep semantic information of documents, significantly improving retrieval precision.”

• cites [117]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [324]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [121]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim.

Generative retrieval (GR) has emerged as a promising approach in the field of information retrieval.

“In recent advancements in AIGC, generative retrieval (GR) has emerged as an promising approach in the field of information retrieval, garnering increasing interest from the academic community.”

It may state two separate claims as one.

Retrieval augmentation involves integrating external knowledge sources into the language model's workflow, allowing the model to access up-to-date and accurate information during generation, thereby grounding its responses in verified data and reducing the likelihood of hallucinations.

“Retrieval augmentation involves integrating external knowledge sources into the language model's workflow.This integration allows the model to access up-to-date and accurate information during the generation process, thereby grounding its responses in verified data and reducing the likelihood of hallucinations [139,252,271].”

• cites [139]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [252]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [271]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.

Reliable Response Generation employs language models to directly generate information users seek, breaking the limitations of traditional IR in terms of document granularity and relevance matching while offering flexibility, efficiency, and creativity to meet practical needs.

“Reliable Response Generation employs language models to directly generate information users seek, breaking the limitations of traditional IR in terms of document granularity and relevance matching while offering flexibility, efficiency, and creativity to meet practical needs.”

It may state two separate claims as one.

Ultron addressed the web search title issue by combining URLs and titles as DocIDs, identifying documents through keywords in web page URLs and titles.

“Thus, Ultron [371] effectively addressed this issue by combining URLs and titles as DocIDs, identifying documents through keywords in web page URLs and titles.”

• cites [371]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper may give no reason why it holds.

The progress in GR is shifting retrieval systems from matching to generation.

“The progress in GR is shifting retrieval systems from matching to generation.”

It may state two separate claims as one.

Retrieval augmentation and tool augmentation have emerged as effective strategies to address LLM challenges.

“To address these issues, retrieval augmentation and tool augmentation have emerged as effective strategies.”

• cites [139]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [252]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [271]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

LLMs have made significant progress in natural language tasks, such as dialogue and question answering.

“Based on large pre-training corpora and advanced training techniques like RLHF [36], LLMs [8,105,209,286] have made significant progress in natural language tasks, such as dialogue [209,282] and question answering [174,225].”

• cites [8]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [105]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [209]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

One of the core components of GR is the model training and structure, aiming to enhance the model's ability to memorize documents in the corpus.

“One of the core components of GR is the model training and structure, aiming to enhance the model's ability to memorize documents in the corpus.”

Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

Large language models (LLMs) have brought about revolutionary changes in the field of AI-generated content (AIGC).

“In recent years, large language models (LLMs) have brought about revolutionary changes in the field of AI-generated content (AIGC) [19,359].”

• cites [19]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [359]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

UGR achieves this by letting the GR model learn prompts specific to tasks, generating corresponding document, passage, sentence, or entity identifiers.

“UGR achieves this by letting the GR model learn prompts specific to tasks, generating corresponding document, passage, sentence, or entity identifiers.”

• cites [26]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Re3val first reorders retrieved page titles using context information obtained from a dense retriever, then optimizes the reordering using the REINFORCE algorithm to maximize rewards generated by constrained decoding.

“It first reorders retrieved page titles using context information obtained from a dense retriever, then optimizes the reordering using the REINFORCE algorithm to maximize rewards generated by constrained decoding.”

• cites [257]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

IRGen proposed a semantic image tokenizer, which converts global image features into short sequences capturing high-level semantic information.

“IRGen proposed a semantic image tokenizer, which converts global image features into short sequences capturing high-level semantic information.”

• cites [357]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

ColaRec integrates collaborative filtering signals and content information by deriving generative item identifiers from a pretrained recommendation model and representing users via aggregated item content.

“ColaRec [309] integrates collaborative filtering signals and content information by deriving generative item identifiers from a pretrained recommendation model and representing users via aggregated item content.”

• cites [309]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

SEATER trains a Transformer-based encoder-decoder model and enhances the semantics of each level of indexing through contrastive learning and multi-task learning.

“It then trains a Transformer-based encoder-decoder model and enhances the semantics of each level of indexing through contrastive learning and multi-task learning.”

• cites [254]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

P5, by setting unique training objectives, prompts, and prediction paradigms for each recommendation domain's downstream tasks, serves well as a backbone model, accomplishing various recommendation tasks through generated text.

“This method, by setting unique training objectives, prompts, and prediction paradigms for each recommendation domain's downstream tasks, serves well as a backbone model, accomplishing various recommendation tasks through generated text.”

• cites [74]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

DSI++ aims to address the incremental learning challenges encountered by DSI.

“At first, DSI++ [192] aims to address the incremental learning challenges encountered by DSI [281].”

• cites [192]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [281]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

CorpusLM designs a series of auxiliary DocID understanding tasks to deepen the model's understanding of DocID semantics.

“In addition, CorpusLM designs a series of auxiliary DocID understanding tasks to deepen the model's understanding of DocID semantics.”

• cites [152]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

GRACE explores various types of image identifiers, including strings, numbers, semantic and atomic identifiers, to adapt to different memory and retrieval requirements.

“GRACE explores various types of image identifiers, including strings, numbers, semantic and atomic identifiers, to adapt to different memory and retrieval requirements.”

• cites [178]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

TIGER then trains a transformer-based encoder-decoder model with this semantically informative indexing identifier sequence to generate item identifiers for recommending the next item based on historical sequences.

“It then trains a transformer-based encoder-decoder model with this semantically informative indexing identifier sequence to generate item identifiers for recommending the next item based on historical sequences.”

• cites [233]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

GCoQA focuses on key information in the dialogue context at each decoding step, achieving more precise and efficient passage retrieval and answer generation, thereby improving retrieval performance and overall system efficiency.

“This method focuses on key information in the dialogue context at each decoding step, achieving more precise and efficient passage retrieval and answer generation, thereby improving retrieval performance and overall system efficiency.”

• cites [158]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

DSI++ employs DocT5Query to generate pseudo queries for documents in the existing corpus as training data augmentation, mitigating forgetting of GR models.

“It also employs DocT5Query [208] to generate pseudo queries for documents in the existing corpus as training data augmentation, mitigating the forgetting issue of GR models.”

• cites [192]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [208]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

UniGen proposes a unified generation framework to integrate retrieval and question answering tasks, bridging the gap between query input and generation targets using connectors generated by large language models.

“UniGen [155] proposes a unified generation framework to integrate retrieval and question answering tasks, bridging the gap between query input and generation targets using connectors generated by large language models.”

• cites [155]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

AutoTSG proposed a term set-based document representation, using keywords extracted from titles and content rather than predefined sequences, allowing retrieval of the target document as long as the generated term set is included in the extracted keywords.

“Unlike the sequential DocIDs described earlier, AutoTSG [352] proposed a term set-based document representation, using keywords extracted from titles and content, rather than predefined sequences, allowing for retrieval of the target document as long as the generated term set is included in the extracted keywords.”

• cites [352]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

NOVO proposed learnable continuous N-grams constituting term-set DocIDs.

“Similarly based on term-sets, NOVO [311] proposed learnable continuous N-grams constituting term-set DocIDs.”

• cites [311]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

DearDR samples sentences from Wikipedia documents as input and trains a self-regressive model to decode page titles or hyperlinks, or both, without the need for manually labeled data.

“The model samples sentences from Wikipedia documents as input and trains a self-regressive model to decode page titles or hyperlinks, or both, without the need for manually labeled data.”

• cites [283]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

CorpusBrain++ introduces the KILT++ benchmark for continuously updated KILT tasks and designs a dynamic architecture paradigm with a backbone-adapter structure.

“CorpusBrain++ [80] introduces the KILT++ benchmark for continuously updated KILT [218] tasks and designs a dynamic architecture paradigm with a backbone-adapter structure.”

• cites [80]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [218]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

CodeDSI explores different DocID representation strategies, including direct and clustered representation, as well as numerical and character representations.

“CodeDSI explores different DocID representation strategies, including direct and clustered representation, as well as numerical and character representations.”

• cites [203]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

P5 transforms various recommendation tasks into different natural language sequences, designing a universal, shared framework for recommendation completion.

“P5 [74] transforms various recommendation tasks into different natural language sequences, designing a universal, shared framework for recommendation completion.”

• cites [74]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

ColaRec uses an item indexing generation loss and contrastive loss to align content-based semantic spaces with collaborative interaction spaces, enhancing the model's ability to recommend items in an end-to-end framework.

“Then it uses an item indexing generation loss and contrastive loss to align content-based semantic spaces with collaborative interaction spaces, enhancing the model's ability to recommend items in an end-to-end framework.”

• cites [309]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

GMR proposed to employ language model memory and multi-hop memory to train a generative retrieval model, enabling it to memorize the target corpus and simulate real retrieval scenarios through constructing pseudo multi-hop query data, achieving dynamic stopping and efficient performance in multi-hop retrieval tasks.

“GMR [131] proposed to employ language model memory and multi-hop memory to train a generative retrieval model, enabling it to memorize the target corpus and simulate real retrieval scenarios through constructing pseudo multi-hop query data, achieving dynamic stopping and efficient performance in multi-hop retrieval tasks.”

• cites [131]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Based on the learned item identifiers, LC-Rec integrates semantic and collaborative information, enabling large language models to better adapt to sequence recommendation tasks.

“Based on the learned item identifiers, it integrates semantic and collaborative information, enabling large language models to better adapt to sequence recommendation tasks.”

• cites [360]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. It may be too vague to check against evidence.

Focusing solely on semantic information and overlooking collaborative filtering information under the recommendation context might limit the further development of generative models.

“Focusing solely on semantic information and overlooking the collaborative filtering information under the recommendation context might limit the further development of generative models.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The multi-view DocID was also used in LTRGR and DGR.

“This multi-view DocID was also used in LTRGR [159] and DGR [161].”

• cites [159]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [161]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Generative retrieval methods have been tailored to various downstream generative tasks, including fact verification, entity linking, open-domain QA, dialogue, slot filling, knowledge-intensive tasks, code, conversational QA, and multi-modal retrieval, demonstrating superior performance and efficiency.

“Generative retrieval methods, apart from addressing retrieval tasks individually, have been tailored to various downstream generative tasks.These include fact verification [284], entity linking [86], open-domain QA [126], dialogue [51], slot filling [137], among others, as well as knowledgeintensive tasks [218], code [179], conversational QA [3], and multi-modal retrieval scenarios [165], demonstr”

• cites [284]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [86]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [126]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

GeMKR first guides multi-granularity visual learning using object-aware prefix tuning techniques to align visual features with LLMs' text feature space, achieving cross-modal interaction.

“It first guides multi-granularity visual learning using object-aware prefix tuning techniques to align visual features with LLMs' text feature space, achieving cross-modal interaction.”

• cites [178]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

LC-Rec, after generating semantic indexing identifiers similar to TIGER using a residual quantized autoencoder with uniform semantic mapping, engages in a series of alignment tasks, including sequential item prediction, explicit index-language alignment, and recommendation-oriented implicit alignment.

“Therefore, after generating semantic indexing identifiers similar to TIGER using a residual quantized autoencoder with uniform semantic mapping, LC-Rec [360] also engages in a series of alignment tasks, including sequential item prediction, explicit index-language alignment, and recommendation-oriented implicit alignment.”

• cites [360]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [233]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

GLEN uses dynamic lexical DocIDs and follows a two-phase index learning strategy: first assigns DocIDs by extracting keywords from documents using self-supervised signals, then refines DocIDs by integrating query-document relevance through two loss functions.

“Later, GLEN [135] uses dynamic lexical DocIDs and follows a two-phase index learning strategy.First, it assigns DocIDs by extracting keywords from documents using self-supervised signals.Then, it refines DocIDs by integrating query-document relevance through two loss functions.”

• cites [135]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

GERE proposed using an encoder-decoder-based GR model to replace traditional indexing-based methods for fact verification.

“For fact verification tasks [284], which involve determining the correctness of input claims, GERE [27] proposed using an encoder-decoder-based GR model to replace traditional indexing-based methods.”

• cites [27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [284]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

GeMKR employs a two-step retrieval process: generating knowledge clues closely related to the query and then retrieving corresponding documents based on these clues.

“GeMKR then employs a two-step retrieval process: generating knowledge clues closely related to the query and then retrieving corresponding documents based on these clues.”

• cites [178]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

During inference, GLEN ranks documents using DocID weights without additional overhead.

“During inference, GLEN ranks documents using DocID weights without additional overhead.”

• cites [135]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

UGR proposed using different granularities of N-gram DocIDs to adapt to various downstream tasks, unifying different retrieval tasks into a single generative form.

“Similarly, UGR [26] proposed using different granularities of N-gram DocIDs to adapt to various downstream tasks, unifying different retrieval tasks into a single generative form.”

• cites [26]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

IncDSI views the addition of new documents as a constrained optimization problem to find optimal representations for new documents, aiming to ensure new documents can be correctly retrieved by relevant queries and maintain retrieval performance of existing documents unaffected.

“IncDSI [124] views the addition of new documents as a constrained optimization problem to find optimal representations for the new documents.This approach aims to (1) ensure new documents can be correctly retrieved by their relevant queries, and (2) maintain the retrieval performance of existing documents unaffected.”

• cites [124]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

MINDER proposed multi-view identifiers, including generated pseudo queries from document content via DocT5Query, titles, and sub-strings.

“For a more comprehensive representation of documents, MINDER [160] proposed multi-view identifiers, including generated pseudo queries from document content via DocT5Query [208], titles, and sub-strings.”

• cites [160]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [208]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

Due to the inherent nature of GR models as generative models, a natural advantage lies in their ability to be jointly trained with downstream generators to obtain a unified model for retrieval and generation tasks.

“However, due to the inherent nature of GR models as generative models, a natural advantage lies in their ability to be jointly trained with downstream generators to obtain a unified model for retrieval and generation tasks.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

DearDR utilizes remote supervision and self-supervised learning techniques, using Wikipedia page titles and hyperlinks as training data.

“Futhermore, DearDR [283] utilizes remote supervision and self-supervised learning techniques, using Wikipedia page titles and hyperlinks as training data.”

• cites [283]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Re3val proposes a retrieval framework combining generative reordering and reinforcement learning.

“Re3val [257] proposes a retrieval framework combining generative reordering and reinforcement learning.”

• cites [257]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

TIGER initially learns a residual quantized autoencoder to generate semantically informative indexing identifiers for different items.

“Similarly, TIGER [233] initially learns a residual quantized autoencoder to generate semantically informative indexing identifiers for different items.”

• cites [233]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

IPQ reduces computational costs and allows the system to adapt flexibly to new documents.

“This method reduces computational costs and allows the system to adapt flexibly to new documents.”

• cites [25]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

GRACE training includes learning to memorize images and their corresponding identifiers, and learning to generate the target image identifiers from textual queries.

“The training process includes (1) learning to memorize images and their corresponding identifiers, and (2) learning to generate the target image identifiers from textual queries.”

• cites [178]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

CorpusBrain introduced three pre-training tasks to enhance the model's understanding of query-document relationships at various granularities: Internal Sentence Selection, Leading Paragraph Selection, and Hyperlink Identifier Prediction.

“For Knowledge-Intensive Language Tasks (KILT) [218], CorpusBrain [28] introduced three pre-training tasks to enhance the model's understanding of query-document relationships at various granularities: Internal Sentence Selection, Leading Paragraph Selection, and Hyperlink Identifier Prediction.”

• cites [28]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [218]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

Challenges remain in creating a grounded system that delivers faithful answers, such as hallucination, prolonged inference time, and high operational costs.

“However, challenges remain in creating a grounded system that delivers faithful answers, such as hallucination, prolonged inference time, and high operational costs.”

It may state two separate claims as one.

Compared to traditional PQ methods, IPQ designs two adaptive thresholds to update only a subset of centroids instead of all, maintaining the indices of updated centroids constant.

“Compared to traditional PQ methods, IPQ designs two adaptive thresholds to update only a subset of centroids instead of all, maintaining the indices of updated centroids constant.”

• cites [25]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

UniGen employs shared encoders and task-specific decoders for retrieval and question answering, introducing iterative enhancement strategies to continuously improve the performance of both tasks.

“UniGen employs shared encoders and task-specific decoders for retrieval and question answering, introducing iterative enhancement strategies to continuously improve the performance of both tasks.”

• cites [155]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

In the GPT series and LLaMA family, larger models tend to perform better on diverse downstream tasks, including few-shot learning, language understanding, and generation.

“For instance, in the GPT [2,16,228,229] series and LLaMA [285,286] family, larger models tend to perform better on diverse downstream tasks, including few-shot learning, language understanding, and generation [34].”

• cites [2]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [16]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [228]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [229]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [285]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [286]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [34]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

GeMKR combines LLMs' generation capabilities with visual-text features, designing a generative knowledge retrieval framework.

“Later, GeMKR [178] combines LLMs' generation capabilities with visual-text features, designing a generative knowledge retrieval framework.”

• cites [178]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Through denoising query modeling, NOVO learned to generate queries from documents with noise, thereby implicitly learning to filter out document N-grams more relevant to queries.

“Through denoising query modeling, the model learned to generate queries from documents with noise, thereby implicitly learning to filter out document N-grams more relevant to queries.”

• cites [311]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper may give no reason why it holds.

Large language models have been positioned as a novel form of IR system, capable of generating reliable responses directly aligned with users' informational needs.

“The rapid advancement of large language models has positioned them as a novel form of IR system, capable of generating reliable responses directly aligned with users' informational needs.”

It may state two separate claims as one.

Generative recommendation systems do not require the computation of ranking scores for each item followed by database indexing, but instead accomplish item recommendations through the direct generation of IDs.

“Generative recommendation systems do not require the computation of ranking scores for each item followed by database indexing, but instead accomplish item recommendations through the direct generation of IDs.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

SEATER designs balanced k-ary tree-structured indexes, using a constrained k-means clustering method to recursively cluster vectors encoded from item texts, obtaining equal-length identifiers.

“SEATER [254] designs a balanced k-ary tree-structured indexes, using a constrained k-means clustering method to recursively cluster vectors encoded from item texts, obtaining equal-length identifiers.”

• cites [254]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

During training, CorpusBrain++ generates pseudo queries for new document sets and continues to pre-train adapters for specific tasks.

“During training, CorpusBrain++ generates pseudo queries for new document sets and continues to pre-train adapters for specific tasks.”

• cites [80]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

CorpusLM adopts a multi-task learning approach and introduces ranking-guided DocID decoding strategies and continuous generation strategies to improve retrieval and generation performance.

“The model adopts a multi-task learning approach and introduces ranking-guided DocID decoding strategies and continuous generation strategies to improve retrieval and generation performance.”

• cites [152]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

IDGenRec includes a text ID generator trained on item metadata with a diversified ID generation algorithm, and an alternating training strategy that optimizes both the ID generator and the LLM-based recommendation model for improved performance and accuracy in sequential recommendations.

“The framework includes a text ID generator trained on item metadata with a diversified ID generation algorithm, and an alternating training strategy that optimizes both the ID generator and the LLM-based recommendation model for improved performance and accuracy in sequential recommendations.”

• cites [273]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

PAG constructed DocIDs based on sets of key terms, disregarding the order of terms, which is utilized for approximating document relevance in decoding.

“Recently, PAG [345] also constructed DocIDs based on sets of key terms, disregarding the order of terms, which is utilized for approximating document relevance in decoding.”

• cites [345]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

CodeDSI includes indexing and retrieval stages, learning to map code samples and real queries to their respective DocIDs.

“Similar to DSI [281], it includes indexing and retrieval stages, learning to map code samples and real queries to their respective DocIDs.”

• cites [203]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [281]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Incremental learning refers to the ability of machine learning models to continuously learn new skills and tasks while retaining previously acquired knowledge.

“Incremental learning refers to the ability of machine learning models to continuously learn new skills and tasks while retaining previously acquired knowledge [301, Table 2.”

• cites [301]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Conditional RAG adapts to various query types through distinct processes, improving the system's flexibility.

“The Conditional RAG framework adapts to various query types through distinct processes, improving the system's flexibility.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Jang et al. challenge traditional multitask prompt fine-tuning by refining expert models on individual tasks.

“Jang et al. [99] challenge traditional multitask prompt fine-tuning by refining expert models on individual tasks.”

• cites [99]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

O-LoRA achieves superior continual learning by training new tasks in orthogonal low-rank subspaces, significantly minimizing task interference.

“O-LoRA [305] achieves superior continual learning by training new tasks in orthogonal low-rank subspaces, significantly minimizing task interference.”

• cites [305]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

There are primarily three paradigms for internal knowledge editing within language models: adding trainable parameters, locate-then-edit, and meta-learning.

“There are primarily three paradigms for internal knowledge editing within language models: adding trainable parameters, locate-then-edit, and meta-learning.”

It may state two separate claims as one.

The quality of training data substantially affects the reliability of model outputs; noise, misinformation, and incomplete information can disrupt the learning process, leading to hallucinations and other issues.

“The quality of training data substantially affects the reliability of model outputs.Noise, misinformation, and incomplete information can disrupt the learning process, leading to hallucinations and other issues.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Based on the interaction flow between retriever and generator, RAG methods can be divided into four categories.

“Based on the interaction flow between these two, RAG methods can be divided into four categories [72].”

• cites [72]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Discrete code-book modules are introduced in the middle layers of the language model, as in GRACE, to adjust and correct information.

“Alternatively, discrete code-book modules are introduced in the middle layers of the language model, as in GRACE [83], to adjust and correct information.”

• cites [83]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Wan et al. proposed a faithfulness-aware decoding method to enhance the faithfulness of the beam-search approach by incorporating a Ranker to reorder generated sequences and a lookahead method to avoid unfaithful tokens.

“Wan et al. [292] proposed a faithfulnessaware decoding method to enhance the faithfulness of the beam-search approach by incorporating a Ranker to reorder generated sequences and a lookahead method to avoid unfaithful tokens.”

• cites [292]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Meta-Learning uses hyper-networks to generate the necessary updates for model editing.

“Meta-Learning, another paradigm, uses hyper-networks to generate the necessary updates for model editing.”

• cites [17]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Loop RAG involves deep interactions between the retriever and generator components, and owing to multi-turn retrieval and generation processes, it excels at handling complex and diverse input queries, yielding superior results in response generation.

“Loop RAG involves deep interactions between the retriever and generator components.Owing to multi-turn retrieval and generation processes, accompanied by comprehensive interactions, it excels at handling complex and diverse input queries, yielding superior results in response generation.”

It may state two separate claims as one.

The strongest result in Pradeep et al.'s experiments was achieved by using a training task that only utilized synthetic queries to Naive DocIDs, expanding the model to T5-XL (3B parameters), achieving an MRR@10 of 26.7.

“The strongest result in the experiments was achieved by using a training task that only utilized synthetic queries to Naive DocIDs, expanding the model to T5-XL (3B parameters) to achieve an MRR@10 of 26.7.”

• cites [220]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

LLM-Blender employs a ranker and a fuser to combine answers from various LLMs, including black-box models, but faces high deployment costs.

“Alternatively, the LLM-Blender framework [107] employs a ranker and a fuser to combine answers from various LLMs, including black-box models, but faces high deployment costs.”

• cites [107]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Lee et al. combined suffix arrays and MinHash to approximate matching and deduplicate the training dataset, reducing direct reproduction from the same source.

“To reduce redundancies in crawled internet data, Lee et al. [132] combined suffix arrays [188] and MinHash [15] to approximate matching and deduplicate the training dataset, reducing direct reproduction from the same source.”

• cites [132]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [188]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [15]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

The Chain-of-Thought prompting method guides the model to explicitly decompose the question into a reasoning chain during decoding, improving response accuracy by grounding the final answer on accurate intermediate steps.

“The Chain-of-Thought (CoT) [313] prompting method guides the model to explicitly decompose the question into a reasoning chain during decoding, improving response accuracy by grounding the final answer on accurate intermediate steps.”

• cites [313]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Inference-Time Intervention identifies attention heads strongly correlated with response correctness, adjusts their orientations, and moderates their activation, achieving more truthful generation with minimal model interference.

“Inference-Time Intervention (ITI) [147] identifies attention heads strongly correlated with response correctness, adjusts their orientations, and moderates their activation, achieving more truthful generation with minimal model interference.”

• cites [147]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Jang et al. introduced Continual Knowledge Learning to explore how LLMs update and retain knowledge amidst rapidly changing information, creating benchmarks like FUAR.

“Jang et al. [100] introduced Continual Knowledge Learning (CKL) to explore how LLMs update and retain knowledge amidst rapidly changing information, creating benchmarks like FUAR.”

• cites [100]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

DynamicIR revealed that GR systems are superior in adapting to evolving knowledge, handling temporally informed data, and are more efficient in terms of memory, indexing time, and FLOPs compared to dense retrieval systems.

“Through experimental analysis, DynamicIR revealed that GR systems are superior in adapting to evolving knowledge, handling temporally informed data, and are more efficient in terms of memory, indexing time, and FLOPs compared to dense retrieval systems.”

• cites [337]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

To better understand user needs, researchers have explored two main approaches: personalized prompt design and model fine-tuning.

“To better understand user needs, researchers have explored two main approaches: personalized prompt design and model fine-tuning.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Progressive Prompts appends new soft prompts for each new task, facilitating knowledge transfer and reducing forgetting.

“Progressive Prompts [236] appends new soft prompts for each new task, facilitating knowledge transfer and reducing forgetting.”

• cites [236]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Nucleus Sampling samples within a set probability range for tokens, ensuring better diversity while balancing variety and reliability.

“Nucleus Sampling [133] samples within a set probability range for tokens, ensuring better diversity while balancing variety and reliability.”

• cites [133]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

MoE utilizes a gating mechanism to selectively activate sections of network parameters during inference, greatly increasing the effective parameters without inflating inference costs.

“The predominant approach is the Mixture of Experts (MoE) [96], which utilizes a gating mechanism to selectively activate sections of network parameters during inference, greatly increasing the effective parameters without inflating inference costs [58,61,106,136].”

• cites [96]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [58]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [61]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [106]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [136]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Ke et al. proposed Domain Adaptive Pre-training to improve the model's adaptability to new domains while preventing forgetting using techniques like soft masking and contrastive learning.

“Additionally, Ke et al. [119] proposed Domain Adaptive Pre-training (DAP-training) to improve the model's adaptability to new domains while preventing forgetting using techniques like soft masking and contrastive learning.”