CoChat
Sign Up
Cited by 174Information Retrieval and Search BehaviorTopic ModelingRecommender Systems and Techniques

The DOI matches its Crossref record; 439 of the 439 references it cites were found in a registry or an index; none of the 275 sources its claims cite is checked for support yet.

From Matching to Generation: A Survey on Generative Information Retrieval

Open access
Li, Xiaoxi, Jin, Jiajie, Zhou, Yujia, Zhang, Yuyao, Zhang, Peitian, Zhu, Yutao, Dou, Zhicheng·2025·ACM Transactions on Information Systems
DOI 10.1145/3722552
Open this paper in CoChat — free accountOpen PDF

Abstract

Information Retrieval (IR) systems are crucial tools for users to access information, which have long been dominated by traditional methods relying on similarity matching. With the advancement of pre-trained language models, Generative Information Retrieval (GenIR) emerges as a novel paradigm, attracting increasing attention. Based on the form of information provided to users, current research in GenIR can be categorized into two aspects: (1) Generative Retrieval ( GR ) leverages the generative model’s parameters for memorizing documents, enabling retrieval by directly generating relevant document identifiers without explicit indexing. (2) Reliable Response Generation employs language models to directly generate information users seek, breaking the limitations of traditional IR in terms of document granularity and relevance matching while offering flexibility, efficiency, and creativity to meet practical needs. This article aims to systematically review the latest research progress in GenIR. We will summarize the advancements in GR regarding model training and structure, document identifier, incremental learning, and so on, as well as progress in reliable response generation in aspects of internal knowledge memorization, external knowledge augmentation, and so on. We also review the evaluation, challenges, and future developments in GenIR systems. This review aims to offer a comprehensive reference for researchers, encouraging further development in the GenIR field (Github Repository: https://github.com/RUC-NLPIR/GenIR-Survey ).

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This survey reviews generative information retrieval (GenIR), organizing the field into generative document retrieval (GR) and reliable response generation. It synthesizes methods, evaluation, challenges, and prospects, and reports comparative results showing GR can be competitive with sparse and dense retrieval on selected benchmarks while facing scalability and reliability issues.

Key findings

  • GenIR is categorized into two main patterns: GR, which retrieves documents by directly generating document identifiers (DocIDs) from model parameters, and reliable response generation, which directly generates user-centric answers with reliability enhancement strategies.
  • In the survey’s reported experiments, GR methods are competitive: on MS MARCO 300K, TSGen achieves Recall@1 0.384 and DGR 0.359, surpassing DPR (0.271); on NQ320K, TSGen achieves Recall@1 0.708, outperforming SPLADEv2 (0.624) and DPR (0.502). On KILT, GENRE with title DocIDs reaches FEVER 0.847, AY2 0.928, WnWi 0.877, and WnCw 0.706; SEAL with sub-string DocIDs reaches NQ 0.658, HoPo 0.565, TQA 0.715, and WoW 0.527.
  • A large-scale study on 8.8M MS MARCO passages found synthetic-query training with Naive DocIDs and T5-XL (3B) reached MRR@10 26.7, while scaling to T5 XXL (11B) did not improve and led to a decline. DynamicIR found GR superior in adapting to evolving knowledge and more efficient in memory, indexing time, and FLOPs compared to dense retrieval.
  • Key challenges and prospects include GR scalability, dynamic corpora, DocID design, efficiency, and multi-modal retrieval; for response generation, accuracy/factuality, real-time properties, bias/fairness, privacy/security; and a unified retrieval-generation framework.

Methods

  • The paper is a survey/systematic review of GenIR research, proposing a taxonomy and reviewing GR (model training and structure, document identifiers, incremental learning, downstream task adaptation, multi-modal GR, generative recommendation) and reliable response generation (internal knowledge memorization, external knowledge augmentation, citation generation, personal information assistants), plus evaluation and challenges.
  • It includes comparative experiments on MS MARCO 300K (320k documents, 360k training instances, 772 testing instances), NQ320K (109k documents, 320k training instances, 7,830 testing instances), and KILT standard development sets, using Recall@{1,10,100}, MRR@{10,100}, and R-Precision, comparing sparse, dense, and GR methods.
  • The analysis draws on results reported in existing papers and provides own implementations where settings are not aligned for fair comparison.

Limitations

  • The paper does not state a formal limitations section.
  • The text notes field limitations: GR accuracy drops on million-level corpora and increasing model size does not reliably improve performance; constrained beam search causes high inference latency; DocID uniqueness and quality are difficult in general web corpora; human evaluation is costly and subjective.
  • Evident from the text: the survey’s scope is GenIR and selected representative methods; its own experiments are limited to MS MARCO 300K, NQ320K, and KILT development sets rather than full web-scale; comparisons mix reported results and own implementations due to dataset, corpus size, and metric misalignment.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3722552

Cites

439 of 439 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 275 cited sources checked · 0 supported · 0 flagged

Show all 70 claims

  • It may state two separate claims as one. 129
  • Probably not a substantial claim. 124
  • The quote only partly states this claim. 1
198 of 198

  • The paper relies on this without support.

    There is currently no comprehensive review that systematically organizes the research, challenges, and prospects of GenIR.

    “However, there is currently no comprehensive review that systematically organizes the research, challenges, and prospects of this emerging field.”

  • The paper relies on this without support.

    There is still a lack of a systematic review that organizes the existing research under the new paradigm of generative information access.

    “Despite these efforts, there is still a lack of a systematic review that organizes the existing research under this new paradigm of generative information access.”

  • The paper relies on this without support.

    The documents available for retrieval are continuously updated and expanded.

    “However, in reality, the documents available for retrieval are continuously updated and expanded.”

  • The paper relies on this without support.

    Information is constantly evolving in real-life scenarios.

    “In real-life scenarios, information is constantly evolving, and therefore, the GenIR system needs to continuously acquire the latest knowledge to meet users' information needs.”

  • The paper relies on this without support.

    The model's knowledge storage is limited.

    “Since the model's knowledge storage is limited, knowledge updating is necessary to ensure more reliable generated responses.”

  • The paper relies on this without support.

    Large language models have issues such as susceptibility to hallucinations, difficulty handling in-domain knowledge, and challenges with knowledge updating.

    “Although large language models have demonstrated significant effectiveness in response generation, issues such as susceptibility to hallucinations, difficulty handling in-domain knowledge, and challenges with knowledge updating persist.”

  • The paper relies on this without support.

    Evaluating the effectiveness of GR methods relies on high-quality and challenging benchmark datasets.

    “Evaluating the effectiveness of GR methods relies on high-quality and challenging benchmark datasets.”

  • The paper relies on this without support.

    Wikipedia documents have unique manually annotated titles that represent the core entity discussed in the page.

    “This is because each document in Wikipedia has a unique manually annotated title that represents the core entity discussed in that page.”

  • The paper relies on this without support.

    In web search scenarios such as MS MARCO, many documents lack a unique title, are overlapping, and titles do not accurately represent the core content of the documents.

    “However, in the web search scenario, such as in the MS MARCO dataset [205], many documents lack a unique title, are overlapping, and the titles do not accurately represent the core content of the documents.”

  • The paper relies on this without support.

    For language models, indexing new documents inevitably leads to forgetting old ones.

    “However, for language models, indexing new documents inevitably leads to forgetting old ones, posing a challenge for GR systems.”

  • The paper relies on this without support.

    Most current GR methods use a unique DocID to uniquely identify a document.

    “Most current GR methods use a unique DocID to uniquely identify a document.”

  • The paper relies on this without support.

    As the number of documents in a corpus increases, maintaining a unique DocID becomes increasingly challenging.

    “However, as the number of documents in a corpus increases, maintaining a unique DocID becomes increasingly challenging.”

  • The paper relies on this without support.

    Even if a unique DocID is maintained, it is difficult to differentiate significantly from other DocIDs semantically, leading to reduced retrieval precision.

    “Even if a unique DocID is maintained, it is difficult to differentiate significantly from other DocIDs semantically, leading to reduced retrieval precision.”

  • The paper relies on this without support.

    Current GR methods generally rely on constrained beam search to generate multiple DocID sequences during inference, resulting in high latency.

    “Current GR methods generally rely on constrained beam search to generate multiple DocID sequences during inference, resulting in high latency.”

  • The paper relies on this without support.

    Existing multi-modal generative retrieval models aim to retrieve images by converting each image in the collection into a unique sequence that serves as its identifier.

    “Existing multi-modal generative retrieval models aim to retrieve images by converting each image in the collection into a unique sequence that serves as its identifier.”

  • The paper relies on this without support.

    A language model is then employed to predict these image identifiers, enabling effective image retrieval.

    “A language model is then employed to predict these image identifiers, enabling effective image retrieval.”

  • The paper relies on this without support.

    Existing methods perform image representation and image identifier prediction separately for generative retrieval.

    “Existing methods perform image representation and image identifier prediction separately for generative retrieval.”

  • The paper relies on this without support.

    Current multi-modal generative retrieval methods predominantly focus on text and image modalities.

    “Current multi-modal generative retrieval methods predominantly focus on text and image modalities.”

  • The paper relies on this without support.

    The mechanisms by which language models recall knowledge during inference are not yet clear.

    “The mechanisms by which language models recall knowledge during inference are not yet clear and need to be fully understood to better serve user information needs.”

  • The paper relies on this without support.

    Existing work mainly focuses on task accuracy.

    “Existing work mainly focuses on task accuracy, but studying the cost-effectiveness of these methods is also a valuable topic.”

  • The paper relies on this without support.

    Research on real-time knowledge acquisition remains limited.

    “Research on real-time knowledge acquisition remains limited, making it a valuable area for investigation.”

  • The paper relies on this without support.

    Continually relying on outdated knowledge from language models is inadequate, as models cannot comprehend the significance of given contexts or backgrounds in the current era, thus reducing the reliability of the generated content.

    “Moreover, continually relying on outdated knowledge from language models is inadequate, as models cannot comprehend the significance of given contexts or backgrounds in the current era, thus reducing the reliability of the generated content.”

  • The paper relies on this without support.

    Researchers have explored various methods to enhance the fairness of generated content during training data selection, training methods, generation techniques, and rewriting phases.

    “Researchers have explored various methods to enhance the fairness of generated content during training data selection, training methods, generation techniques, and rewriting phases.”

  • The paper relies on this without support.

    Biases have not been eradicated.

    “However, biases have not been eradicated and require a thorough understanding of the mechanisms by which generative models produce biases, to design methods to solve them and build fair GenIR systems that further the practical application of GenIR.”

  • The paper relies on this without support.

    Due to the unclear mechanisms of memory and generation in pre-trained language models, GenIR systems inevitably return unsafe content.

    “Moreover, due to the unclear mechanisms of memory and generation in pre-trained language models, GenIR systems inevitably return unsafe content.”

  • The paper relies on this without support.

    Facing a broader internet corpus presents significant challenges.

    “Facing a broader internet corpus presents significant challenges.”
26 of 26

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • GLEN does not have the highest NQ MRR@100

    Minor

    The discussion calls GLEN's 0.754 the highest NQ MRR@100, but Table 3 reports 0.774 for TSGen and 0.767 for NOVO.

  • The cited Equation (2) does not describe seq2seq training

    Minor

    The model-training section directs readers to Equation (2) for standard sequence-to-sequence training, but Equation (2) defines document and query encoder outputs, not a training method.

  • DSI citation points to the Llama reference

    Minor

    The text identifies [281] as DSI, but reference 281 is a Llama paper. The bibliography lists "Transformer Memory as a Differentiable Search Index" under reference 277.

  • SEAL does not have the highest KILT WoW score

    Minor

    The discussion lists WoW among the categories where SEAL scores highest. Table 4 gives SEAL 0.527 on WoW, versus 0.563 for GENRE and 0.538 for CorpusBrain.

  • LTRGR is not the highest on MS MARCO Recall@10

    Minor

    Table 3 gives LTRGR a Recall@10 of 0.759, below TSGen's 0.781 and GTR-Base's 0.793. It is not the highest even among the generative retrieval methods shown.

  • BART's stated FEVER score differs from Table 4

    Minor

    The discussion reports a FEVER score of 0.821 for title-based BART, whereas the BART title row in Table 4 reports 0.819. The 0.821 score belongs to CorpusBrain.

6 of 6

Citations

Cites

Flagged first, then by citations

Show all 309 references

Research works printed but not linked: 49

  1. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization@ACL 2005. Association for Computational Linguistics, 65–72. Retrieved from https://aclanthology.org/W05-0909/
  2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Proceedings of the Advances in Neural Information Processing Systems 33, pp. 1877–1901.
  3. Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2021. Autoregressive entity retrieval. In Proceedings of the 9th International Conference on Learning Representations (ICLR ’21), Virtual Event. OpenReview.net. Retrieved from https://openreview.net/forum?id=5k8F6UU39V
  4. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In Proceedings of the 28th USENIX Security Symposium (USENIX Security ’19). USENIX Association, 267–284. Retrieved from https://www.usenix.org/conference/usenixsecurity19/presentation/carlini
  5. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Úlfar Erlingsson, et al. 2021. Extracting training data from large language models. In Proceedings of the 30th USENIX Security Symposium (USENIX Security ’21). USENIX Association, 2633–2650. Retrieved from https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting
  6. 10.18653/v1/2023.findings-acl.681 · DOI
  7. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4299–4307. Retrieved from https://proceedings.neurips.cc/paper/2017/hash/d5e2c0adad503c91f91df240d0cd4e49-Abstract.html
  8. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. Wizard of Wikipedia: Knowledge-powered conversational agents. In Proceedings of the International Conference on Learning Representations.
  9. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In Proceedings of the International Conference on Machine Learning. PMLR, 3929–3938.
  10. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In Proceedings of the 9th International Conference on Learning Representations (ICLR ’21), Virtual Event. OpenReview.net. Retrieved from https://openreview.net/forum?id=d7KBjmI3GmQ
  11. Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen Fürstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, and Gerhard Weikum. 2011. Robust disambiguation of named entities in text. In Proceedings of the Empirical Methods in Natural Language Processing (EMNLP), 782–792.
  12. Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023. Transformer-Patcher: One mistake worth one neuron. In Proceedings of the 11th International Conference on Learning Representations (ICLR ’23). OpenReview.net. Retrieved from https://openreview.net/pdf?id=4oYUGeGBPm
  13. Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2023. Exploring the benefits of training expert language models over instruction tuning. In Proceedings of the International Conference on Machine Learning. PMLR, 14702–14729.
  14. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023. BeaverTails: Towards improved safety alignment of LLM via a human-preference dataset. In Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023 (NeurIPS ’23), 24678–24704. Retrieved from http://papers.nips.cc/paper_files/paper/2023/hash/4dbb61cb68671edc4ca3712d70083b9f-Abstract-Datasets_and_Benchmarks.html
  15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 4171–4186.
  16. Muhammad Khalifa David Wadden Emma Strubell Honglak Lee Lu Wang Iz Beltagy and Hao Peng. 2024. Source-Aware Training Enables Knowledge Attribution in Language Models. Retrieved from https://api.semanticscholar.org/CorpusID:268819100
  17. Varsha Kishore, Chao Wan, Justin Lovelace, Yoav Artzi, and Kilian Q. Weinberger. 2023. IncDSI: Incrementally updatable document retrieval. In Proceedings of the International Conference on Machine Learning (ICML ’23). Proceedings of Machine Learning Research, Vol. 202, PMLR, 17122–17134. Retrieved from https://proceedings.mlr.press/v202/kishore23a.html
  18. Tian Lan, Deng Cai, Yan Wang, Heyan Huang, and Xian-Ling Mao. 2023. Copy is all you need. In Proceedings of the 11th International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=CROlOA9Nd8C
  19. Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 (NeurIPS ’20), Virtual, 9459–9474. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
  20. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Proceedings of the International Conference on Machine Learning. PMLR, 12888–12900.
  21. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023. Inference-Time Intervention: Eliciting truthful answers from a language model. In Proceedings of the 37th Conference on Neural Information Processing Systems. Retrieved from https://openreview.net/forum?id=aLLuYpn83y
  22. Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024. PMET: Precise model editing in a transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, 18564–18572.
  23. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. Retrieved from https://api.semanticscholar.org/CorpusID:964287
  24. Adam Liska, Tomás Kociský, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, Cyprien de Masson d’Autume, Tim Scholtes, Manzil Zaheer, Susannah Young, et al. 2022. StreamingQA: A benchmark for adaptation to new knowledge over time in question answering models. In Proceedings of the International Conference on Machine Learning (ICML ’22). Proceedings of Machine Learning Research, Vol. 162, PMLR, 13604–13622. Retrieved from https://proceedings.mlr.press/v162/liska22a.html
  25. Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1 (NeurIPS Datasets and Benchmarks ’21), Virtual. Retrieved from https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/c16a5320fa475530d9583c34fd356ef5-Abstract-round1.html
  26. Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2024. Reasoning on graphs: Faithful and interpretable large language model reasoning. In Proceedings of the International Conference on Learning Representations.
  27. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. In Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022 (NeurIPS ’22), 17359–17372. Retrieved from http://papers.nips.cc/paper_files/paper/2022/hash/6f1d43d5a82a37e89b0665b33bf3a182-Abstract-Conference.html
  28. Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In Proceedings of the 11th International Conference on Learning Representations (ICLR ’23). OpenReview.net. Retrieved from https://openreview.net/pdf?id=MkbcAHIYgyS
  29. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP ’23). Association for Computational Linguistics, 12076–12100. Retrieved from https://aclanthology.org/2023.emnlp-main.741
  30. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2022. Fast model editing at scale. In Proceedings of the 10th International Conference on Learning Representations (ICLR ’22), Virtual Event. OpenReview.net. Retrieved from https://openreview.net/forum?id=0DcZxeWfOPt
  31. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated MAchine Reading COmprehension dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS ’16). CEUR Workshop Proceedings, Vol. 1773, CEUR-WS.org. Retrieved from https://ceur-ws.org/Vol-1773/CoCoNIPS_2016_paper9.pdf
  32. Rodrigo Nogueira and Jimmy Lin. 2019. From doc2query to docTTTTTquery. Online Preprint 6 (2019), 2.
  33. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. In Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022 (NeurIPS ’22). Retrieved from http://papers.nips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html
  34. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Hamza Alobeidli, Alessandro Cappelli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The RefinedWeb dataset for Falcon LLM: Outperforming curated corpora with web data only. In Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023 (NeurIPS ’23), 79155–79172. Retrieved from http://papers.nips.cc/paper_files/paper/2023/hash/fa3ed726cc5073b9c31e3e49a807789c-Abstract-Datasets_and_Benchmarks.html
  35. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.
  36. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Proceedings of the 37th Conference on Neural Information Processing Systems. Retrieved from https://openreview.net/forum?id=HPuSIXJaa9
  37. Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, et al. 2023. Recommender systems with generative retrieval. In Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023 (NeurIPS ’23), 10299–10315. Retrieved from http://papers.nips.cc/paper_files/paper/2023/hash/20dcab0f14046a5c6b02b61da9f13229-Abstract-Conference.html
  38. Juan Ramos. 2003. Using TF-IDF to determine word relevance in document queries. In Proceedings of the 1st Instructional Conference on Machine Learning, Vol. 242. Citeseer, 29–48.
  39. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. HuggingGPT: Solving AI tasks with ChatGPT and its friends in hugging face. In Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023 (NeurIPS ’23), 38154–38180. Retrieved from http://papers.nips.cc/paper_files/paper/2023/hash/77c33e6a367922d003ff102ffb92b658-Abstract-Conference.html
  40. Alane Suhr and Yoav Artzi. 2024. Continual learning for instruction following from realtime feedback. In Proceedings of the Advances in Neural Information Processing Systems 36, 32340–32359.
  41. Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Prakash Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. 2022. Transformer memory as a differentiable search index. In Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022 (NeurIPS ’22), 21831–21843. Retrieved from http://papers.nips.cc/paper_files/paper/2022/hash/892840a6123b5ec99ebaab8be1530fba-Abstract-Conference.html
  42. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 5998–6008. Retrieved from https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
  43. Tongtong Wu, Massimo Caccia, Zhuang Li, Yuan-Fang Li, Guilin Qi, and Gholamreza Haffari. 2022. Pretrained language model in continual learning: A comparative study. In Proceedings of the 10th International Conference on Learning Representations (ICLR ’22), Virtual Event. OpenReview.net. Retrieved from https://openreview.net/forum?id=figzpGMrdD
  44. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2020. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In Proceedings of the International Conference on Learning Representations (ICLR).
  45. Shunyu Yao, Jeffrey Zhao, Dian Yu, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2022. ReAct: Synergizing reasoning and acting in language models. In Proceedings of the NeurIPS 2022 Foundation Models for Decision Making Workshop. Retrieved from https://openreview.net/forum?id=tvI4u1ylcqs
  46. Soyoung Yoon Chaeeun Kim Hyunji Lee Joel Jang and Minjoon Seo. 2023. Exploring the practicality of generative retrieval on dynamic corpora. Retrieved from https://api.semanticscholar.org/CorpusID:258967398
  47. Kui Zhang, Guangquan Lu, Guixian Zhang, Zhi Lei, and Lijuan Wu. 2022. Personalized headline generation with enhanced user interest perception. In Proceedings of the International Conference on Artificial Neural Networks. Springer, 797–809.
  48. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2024. LIMA: Less is more for alignment. In Proceedings of the Advances in Neural Information Processing Systems 36, 55006–55021.
  49. Yujia Zhou, Zheng Liu, and Zhicheng Dou. 2024. Boosting the potential of large language models with an intelligent information assistant. In Proceedings of the Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024 (NeurIPS ’24). Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (Eds.). Retrieved from http://papers.nips.cc/paper_files/paper/2024/hash/28d38c036365420f61ce03300418e44a-Abstract-Conference.html

Cited by

Sorted by citations

Items 101–174 of 174

Related

See all

Metadata

Authors
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z.
Affiliations
  • Renmin University of China (CN)
  • Tsinghua University (CN)
ORCID
Venue
ACM Transactions on Information Systems
Publisher
Association for Computing Machinery (ACM)
Published
2025-05-09
Volume
43
Issue
3
Pages
1-62
ISSN
  • 1046-8188 (ISSN-L)
  • 1558-2868
DOI
10.1145/3722552
OpenAlex
W4408326921
Cited by
172
References
309
Citations by year
2026: 70 · 2025: 89
FWCI
95.77
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Transactions on Information Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
  • Beijing Natural Science Foundation (L233008)
  • National Natural Science Foundation of China (62272467)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
Keywords
generative retrieval, pre-trained language models, incremental learning, knowledge memorization, similarity matching

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review