The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Continual Learning for Generative Retrieval over Dynamic Corpora
Open accessAbstract
Generative retrieval (GR) directly predicts the identifiers of relevant documents (i.e., docids) based on a parametric model. It has achieved solid performance on many ad-hoc retrieval tasks. So far, these tasks have assumed a static document collection. In many practical scenarios, however, document collections are dynamic, where new documents are continuously added to the corpus. The ability to incrementally index new documents while preserving the ability to answer queries with both previously and newly indexed relevant documents is vital to applying GR models. In this paper, we address this practical continual learning problem for GR. We put forward a novel Continual-LEarner for generatiVE Retrieval (CLEVER) model and make two major contributions to continual learning for GR: (i) To encode new documents into docids with low computational cost, we present Incremental Product Quantization, which updates a partial quantization codebook according to two adaptive thresholds; and (ii) To memorize new documents for querying without forgetting previous knowledge, we propose a memory-augmented learning mechanism, to form meaningful connections between old and new documents. Empirical results demonstrate the effectiveness and efficiency of the proposed model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We use a learning rate of 5e−5 and Adam optimizer [22] with a linear warmup over the first 10% steps.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 6×
“After applying the above update strategy for all 𝑀 sub-codebooks, we obtain the specific codebook 𝑍 at session 𝑡 : (i) for new documents in D , we obtain their PQ codes U D 𝑡 based on the 𝑍 as the docids, and (ii) for old documents, their PQ codes will not be affected since we only operate on…”
InfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“Given a large-scale base document set D0 and sufficiently many labeled query-document pairs PD0 0 , we can train an initial GR model f (·) via a standard seq2seq objective [42].”
MethodDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“Existing CL approaches [13] can be categorized into: (i) replay methods, maintaining a subset of previous samples and training models together with samples in the new session [18, 52]; (ii) regularization-based methods, regularizing the model parameters to enable important parameters concerning the previous tasks to b…”
MethodIntroduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 3×
“We initialize the parameters of the encoder-decoder architecture from the official checkpoint of T5base [38].”
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · cited 1×
“We feed the [CLS] representation h0 into a projector network [7, 8], which is a feed-forward neural network with a non-linear activation function (i.”
MethodOvercoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · cited 2×
“For the proposed CLEVER method, exploring the joint optimization of quantization methods in IPQ and GR models using supervised labels, and devising advanced thresholds for adaptively updating PQ codes, hold great potential for enhancing retrieval effectiveness.”
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · cited 1×
“Recently, CL has been considered in computer vision [40, 45] and natural language processing [1, 19], but few efforts have been devoted to IR so far.”
BackgroundDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
Research works printed but not linked: 3
- Xinlei Chen and Kaiming He. 2021. Exploring Simple Siamese Representation Learning. CVPR ( 2021 ), 15745--15753. Xinlei Chen and Kaiming He. 2021. Exploring Simple Siamese Representation Learning. CVPR (2021), 15745--15753.
- Herbert H Clark and Susan E Haviland . 1974. Psychological Processes as Linguistic Explanation. Explaining linguistic phenomena ( 1974 ), 91--124. Herbert H Clark and Susan E Haviland. 1974. Psychological Processes as Linguistic Explanation. Explaining linguistic phenomena (1974), 91--124.
- Shicheng Xu , Liang Pang , Huawei Shen , and Xueqi Cheng . 2022. Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning. arXiv preprint arXiv:2204.02725 ( 2022 ). Shicheng Xu, Liang Pang, Huawei Shen, and Xueqi Cheng. 2022. Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning. arXiv preprint arXiv:2204.02725 (2022). · arXiv
Cited by
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
“CLEVER [109], based on Product Quantization (PQ) [206], proposes In-cremental Product Quantization (IPQ) for generating PQ codes as DocIDs for documents.”
Scalable and Effective Generative Information Retrieval
Zeng, H., Luo, C., Jin, B., Sarwar, S. M., Wei, T., & Zamani, H. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645477 · Cited by 32
“These results can be found on Table 4 in Appendix A.”
BackgroundKnow where to go: Make LLM a relevant, responsible, and trustworthy searchers
Shi, X. D., Liu, J., Liu, Y., Cheng, Q., & Lu, W. P. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114354 · Cited by 72
RAGVA: Engineering retrieval augmented generation-based virtual assistants in practice
Yang, R., Fu, M. C., Tantithamthavorn, C. K., Arora, C., Vandenhurk, L., & Chua, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112436 · Cited by 47
TravelRAG: A Tourist Attraction Retrieval Framework Based on Multi-Layer Knowledge Graph
Song, S., Yang, C., Li, X., Shang, H., Li, Z., & Chang, Y. (2024) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi13110414 · Cited by 40
Metadata
- Authors
- Chen, J., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan, Y., & Cheng, X.
- Affiliations
- Institute of Computing Technology (CN)
- University of Amsterdam, Amsterdam, Netherlands
- Venue
- Proceedings of the 32nd ACM International Conference on Information and Knowledge Management
- Publisher
- ACM
- Published
- 2023-10-21
- Pages
- 306-315
- ISSN
- 2155-0751 (ISSN-L)
- OpenAlex
- W4386302269
- Cited by
- 40
- References
- 59
- Citations by year
- 2026: 7 · 2025: 17 · 2024: 15 · 2023: 1
- FWCI
- 10.29
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM International Conference on Information and Knowledge Management (CIKM) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- LESSEN (NWA.1389.20.183)
- CAS Project for Young Scientists in Basic Research (YSBR-034)
- Project (JCKY2022130C039, 2021QY1701)
- Lenovo-CAS Joint Lab Youth Scientist Project
- National Natural Science Foundation of China (NSFC) (62006218, 61902381)
- China Scholarship Council (202104910234)
- 10-year program
- Youth Innovation Promotion Association CAS (2021100)
- Innovation Project of ICT CAS (E261090)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- generative retrieval, continual learning, incremental indexing, product quantization, forgetting, document encoding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Continual Learning for Generative Retrieval over Dynamic Corpora. https://app.cochat.ai/papers/10.1145/3583780.3614821
- The paper
- Chen, J., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan, Y., & Cheng, X. (2023). Continual Learning for Generative Retrieval over Dynamic Corpora. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. https://doi.org/10.1145/3583780.3614821
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.