CoChat

Recent Advances in Generative Information Retrieval

Open access
Tang, Yubao, Zhang, Ruqing, Guo, Jiafeng, de Rijke, Maarten · 2023 · Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region

The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

Generative retrieval (GR) has become a highly active area of information retrieval (IR) that has witnessed significant growth recently. Compared to the traditional “index-retrieve-then-rank” pipeline, the GR paradigm aims to consolidate all information within a corpus into a single model. Typically, a sequence-to-sequence model is trained to directly map a query to its relevant document identifiers (i.e., docids). This tutorial offers an introduction to the core concepts of the GR paradigm and a comprehensive overview of recent advances in its foundations and applications. We start by providing preliminary information covering foundational aspects and problem formulations of GR. Then, our focus shifts towards recent progress in docid design, training approaches, inference strategies, and the applications of GR. We end by outlining remaining challenges and issuing a call for future GR research. This tutorial is intended to be beneficial to both researchers and industry practitioners interested in developing novel GR solutions or applying them in real-world scenarios.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3624918.3629547

Cites

34 of 35 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 35 references

Research works printed but not linked: 15

  1. Michele Bevilacqua Giuseppe Ottaviano Patrick Lewis Wen-tau Yih Sebastian Riedel and Fabio Petroni. 2022. Autoregressive Search Engines: Generating Substrings as Document Identifiers. In NeurIPS. 31668–31683.
  2. Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Yixing Fan, and Xueqi Cheng. 2022. GERE: Generative Evidence Retrieval for Fact Verification. In SIGIR. 2184–2189.
  3. Nicola De Cao Gautier Izacard Sebastian Riedel and Fabio Petroni. 2021. Autoregressive Entity Retrieval. In ICLR.
  4. Jianfeng Gao Xiaodong He and Jian-Yun Nie. 2010. Clickthrough-based Translation Models for Web Search: From Word Models to Phrase Models. In CIKM.
  5. Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat and Mingwei Chang. 2020. Retrieval Augmented Language Model Pre-training. In ICML. 3929–3938.
  6. Hyunji Lee, Jaeyoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vladimir Karpukhin, Yi Lu, and Minjoon Seo. 2023. Nonparametric Decoding for Generative Retrieval. In Findings of the ACL 2023. 12642–12661.
  7. Yongqi Li Nan Yang Liang Wang Furu Wei and Wenjie Li. 2023. Multiview Identifiers Enhanced Generative Retrieval. In ACL. 6636–6648.
  8. Yu-An Liu Ruqing Zhang Jiafeng Guo Wei Chen and Xueqi Cheng. 2023. On the Robustness of Generative Retrieval Models: An Out-of-Distribution Perspective. In Gen-IR@SIGIR.
  9. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches.
  10. Thong Nguyen and Andrew Yates. 2023. Generative Retrieval as Dense Retrieval. In Gen-IR@SIGIR.
  11. Daniel E. Rose and Danny Levinson. 2004. Understanding User Goals in Web Search. In WWW.
  12. James Thorne. 2022. Data-efficient Autoregressive Document Retrieval for Fact Verification. In Workshop on SENLP.
  13. Zihan Wang, Yujia Zhou, Yiteng Tu, and Zhicheng Dou. 2023. NOVO: Learnable and Interpretable Document Identifiers for Model-Based IR. In CIKM.
  14. Caiming Xiong Victor Zhong and Richard Socher. 2017. DCN+: Mixed Objective And Deep Residual Coattention for Question Answering. In ICLR.
  15. Shengyao Zhuang Houxing Ren Linjun Shou Jian Pei Ming Gong Guido Zuccon and Daxin Jiang. 2023. Bridging the Gap between Indexing and Retrieval for Differentiable Search Index with Query Generation. In Gen-IR@SIGIR.

Cited by

See all 35

Metadata

Authors
Tang, Y., Zhang, R., Guo, J., & de Rijke, M.
Affiliations
  • Institute of Computing Technology (CN)
  • Chinese Academy of Sciences (CN)
  • University of Amsterdam, The Netherlands
Venue
Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region
Publisher
ACM
Published
2023-11-26
Pages
294-297
OpenAlex
W4388955883
Cited by
22
References
19
Citations by year
2026: 13 · 2025: 6 · 2024: 3
FWCI
5.66
Type
conference paper
Language
en
Open access
Locations
Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) (publishedVersion, cc-by)
Funders
  • Lenovo-CAS Joint Lab Youth Scientist Project
  • Hybrid Intelligence Center, a 10-year program funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research
  • NONE (No. JCKY2022130C039)
  • the National Natural Science Foundation of China (NSFC) (No. 62006218)
  • project LESSEN (NWA.1389.20.183 of the research program NWA ORC 2020/21)
  • the China Scholarship Council (No. 202104910234)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
Keywords
generative retrieval, document retrieval, sequence-to-sequence models, information retrieval

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Recent Advances in Generative Information Retrieval. https://app.cochat.ai/papers/10.1145/3624918.3629547
The paper
Tang, Y., Zhang, R., Guo, J., & de Rijke, M. (2023). Recent Advances in Generative Information Retrieval. Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region. https://doi.org/10.1145/3624918.3629547

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.