Learning to Rank in Generative Retrieval
Open accessAbstract
Generative retrieval stands out as a promising new paradigm in text retrieval that aims to generate identifier strings of relevant passages as the retrieval target. This generative paradigm taps into powerful generative language models, distinct from traditional sparse or dense retrieval methods. However, only learning to generate is insufficient for generative retrieval. Generative retrieval learns to generate identifiers of relevant passages as an intermediate goal and then converts predicted identifiers into the final passage rank list. The disconnect between the learning objective of autoregressive models and the desired passage ranking target leads to a learning gap. To bridge this gap, we propose a learning-to-rank framework for generative retrieval, dubbed LTRGR. LTRGR enables generative retrieval to learn to rank passages directly, optimizing the autoregressive model toward the final passage ranking target via a rank loss. This framework only requires an additional learning-to-rank training phase to enhance current generative retrieval systems and does not add any burden to the inference stage. We conducted experiments on three public benchmarks, and the results demonstrate that LTRGR achieves state-of-the-art performance among generative retrieval methods. The code and checkpoints are released at https://github.com/liyongqi67/LTRGR.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575 · see citing works
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836 · see citing works
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · see citing works
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · see citing works
Reading Wikipedia to answer open-domain questions
Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1171 · Cited by 1,546 · see citing works
Opportunistic data structures with applications
Ferragina, P., & Manzini, G. (2000) · Proceedings 41st Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.2000.892127 · Cited by 1,325 · see citing works
MS MARCO: A Human Generated MAchine Reading COmprehension DatasetPreprint
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T. G., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2025.5 · Cited by 1,190 · see citing works
An Efficient Boosting Algorithm for Combining Preferences
Freund, Y., Iyer, R. D., Schapire, R. E., & Singer, Y. (1998) · Cited by 806 · see citing works
Latent retrieval for weakly supervised open domain question answering
Lee, K., Chang, M.-W., & Toutanova, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1612 · Cited by 771 · see citing works
Autoregressive Entity RetrievalPreprint
Cao, N. D., Izacard, G., Riedel, S., & Petroni, F. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2010.00904 · Cited by 655 · see citing works
Nogueira, R., & Cho, K. (2019) · arXiv · doi:10.48550/arxiv.1901.04085 · Cited by 615 · see citing works
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset.
Nguyen, T. G., Rosenberg, M., Xia, S., Gao, J., Tiwary, S., Majumder, R., & Deng, L. (2016) · neural information processing systems · Cited by 537 · see citing works
McRank: Learning to Rank Using Multiple Classification and Gradient Boosting
Li, P., Burges, C. J. C., & Wu, Q. (2007) · Cited by 441 · see citing works
A Short Introduction to Learning to Rank
Li, H. (2011) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.e94.d.1854 · Cited by 357 · see citing works
Subset Ranking Using Regression
Cossock, D., & Zhang, T. (2006) · Lecture notes in computer science · doi:10.1007/11776420_44 · Cited by 241 · see citing works
Generation-Augmented Retrieval for Open-Domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.316 · Cited by 146 · see citing works
Transformer Memory as a Differentiable Search IndexPreprint
Tay, Y., Tran, V. Q., Dehghani, M., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.06991 · Cited by 80 · see citing works
Autoregressive Search Engines: Generating Substrings as Document Identifiers
Bevilacqua, M., Ottaviano, G., Lewis, P., Yih, W.-T., Riedel, S., & Petroni, F. (2022) · arXiv · doi:10.48550/arxiv.2204.10628 · Cited by 80 · see citing works
A Neural Corpus Indexer for Document RetrievalPreprint
Wang, Y., Hou, Y., Wang, H., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.02743 · Cited by 47 · see citing works
How Does Generative Retrieval Scale to Millions of Passages?
Pradeep, R., Hui, K., Gupta, J. P., Lelkes, A. D., Zhuang, H., Lin, J., Metzler, D., & Tran, V. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.83 · Cited by 41 · see citing works
Multiview Identifiers Enhanced Generative Retrieval
Li, Y., Yang, N., Wang, L., Wei, F., & Li, W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.366 · Cited by 37 · see citing works
Self-citationGeneration-Augmented Retrieval for Open-domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2020) · arXiv · doi:10.48550/arxiv.2009.08553 · Cited by 31 · see citing works
Generative retrieval for conversational question answering
Li, Y., Yang, N., Wang, L., Wei, F., & Li, W. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103475 · Cited by 30 · see citing works
Self-citationTOME: A Two-stage Approach for Model-based Retrieval
Ren, R., Zhao, W. X., Liu, J., Wu, H., Wen, J.-R., & Wang, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.336 · Cited by 14 · see citing works
Multiview Identifiers Enhanced Generative RetrievalPreprint
Li, Y., Yang, N., Wang, L., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.16675 · Cited by 4 · see citing works
Self-citationHow Does Generative Retrieval Scale to Millions of Passages?Preprint
Pradeep, R., Hui, K., Gupta, J. P., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.11841 · Cited by 3 · see citing works
TOME: A Two-stage Approach for Model-based RetrievalPreprint
Ren, R., Zhao, W. X., Liu, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.11161
Flagged first, then by citations
Research works printed but not linked: 5
- [3]3. Cao, Z.; Qin, T.; Liu, T.-Y.; Tsai, M.-F.; and Li, H. 2007. Learning to rank: from pairwise approach to listwise ap- proach. In Proceedings of the 24th international conference on Machine learning, 129-136.
- [6]6. Crammer, K.; and Singer, Y. 2001. Pranking with ranking. Advances in neural information processing systems, 14.
- [12]12. ACL.
- [20]20. ACL.
- [27]27. Xia, F.; Liu, T.-Y.; Wang, J.; Zhang, W.; and Li, H. 2008. List- wise approach to learning to rank: theory and algorithm. In Proceedings of the 25th international conference on Machine learning, 1192-1199.
Cited by
Sorted by citations
Lightweight and Direct Document Relevance Optimization for Generative Information Retrieval
Mekonnen, K. A., Tang, Y., & de Rijke, M. (2025) · Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3726302.3730023 · Cited by 3 · see citing works
- Authors
- Li, Y., Yang, N., Wang, L., Wei, F., & Li, W.
- Affiliations
- Hong Kong Polytechnic University (HK)
- Microsoft (United States) (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2024-03-24
- Volume
- 38
- Issue
- 8
- Pages
- 8716-8723
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W4393153307
- Cited by
- 27
- References
- 27
- Citations by year
- 2026: 7 · 2025: 15 · 2024: 4 · 2023: 1
- FWCI
- 9.90
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62076212)
- Hong Kong Polytechnic University
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- generative retrieval, learning to rank, passage ranking, rank loss, autoregressive language models, text retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
