Continual Learning for Generative Retrieval over Dynamic Corpora
Open accessThe DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Abstract
Generative retrieval (GR) directly predicts the identifiers of relevant documents (i.e., docids) based on a parametric model. It has achieved solid performance on many ad-hoc retrieval tasks. So far, these tasks have assumed a static document collection. In many practical scenarios, however, document collections are dynamic, where new documents are continuously added to the corpus. The ability to incrementally index new documents while preserving the ability to answer queries with both previously and newly indexed relevant documents is vital to applying GR models. In this paper, we address this practical continual learning problem for GR. We put forward a novel Continual-LEarner for generatiVE Retrieval (CLEVER) model and make two major contributions to continual learning for GR: (i) To encode new documents into docids with low computational cost, we present Incremental Product Quantization, which updates a partial quantization codebook according to two adaptive thresholds; and (ii) To memorize new documents for querying without forgetting previous knowledge, we propose a memory-augmented learning mechanism, to form meaningful connections between old and new documents. Empirical results demonstrate the effectiveness and efficiency of the proposed model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We use a learning rate of 5e−5 and Adam optimizer [22] with a linear warmup over the first 10% steps.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 6×
“After applying the above update strategy for all 𝑀 sub-codebooks, we obtain the specific codebook 𝑍 at session 𝑡 : (i) for new documents in D , we obtain their PQ codes U D 𝑡 based on the 𝑍 as the docids, and (ii) for old documents, their PQ codes will not be affected since we only operate on…”
InfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“Given a large-scale base document set D0 and sufficiently many labeled query-document pairs PD0 0 , we can train an initial GR model f (·) via a standard seq2seq objective [42].”
MethodDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“Existing CL approaches [13] can be categorized into: (i) replay methods, maintaining a subset of previous samples and training models together with samples in the new session [18, 52]; (ii) regularization-based methods, regularizing the model parameters to enable important parameters concerning the previous tasks to b…”
MethodIntroduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 3×
“We initialize the parameters of the encoder-decoder architecture from the official checkpoint of T5base [38].”
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · cited 1×
“We feed the [CLS] representation h0 into a projector network [7, 8], which is a feed-forward neural network with a non-linear activation function (i.”
MethodOvercoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · cited 2×
“For the proposed CLEVER method, exploring the joint optimization of quantization methods in IPQ and GR models using supervised labels, and devising advanced thresholds for adaptively updating PQ codes, hold great potential for enhancing retrieval effectiveness.”
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · cited 1×
“Recently, CL has been considered in computer vision [40, 45] and natural language processing [1, 19], but few efforts have been devoted to IR so far.”
BackgroundDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
The probabilistic relevance framework: BM25 and beyond
Robertson, S., & Zaragoza, H. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000019 · Cited by 3,374 · cited 5×
“We observe that: (i) BM25 exhibits a swift training process.”
BackgroundMethodInfluentialProduct Quantization for Nearest Neighbor Search
Jégou, H., Douze, M., & Schmid, C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.57 · Cited by 2,956 · cited 4×
“First, how to incrementally index new documents with low computational and memory costs? We introduce incremental product quantization (IPQ) based on product quantization (PQ) methods [20] to generate PQ codes for documents as docids, which can represent large volumes of documents via a small number of quantization ce…”
BackgroundMethodInfluentialMS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270 · cited 1×
“Recently, CL has been considered in computer vision [40, 45] and natural language processing [1, 19], but few efforts have been devoted to IR so far.”
BackgroundNatural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 4×
“To facilitate the study of continual document learning for GR, we build two benchmark datasets, i.e., CDI-MS and CDI-NQ, from MS MARCO 300k [33, 36, 54] and Natural Questions (NQ) [25], respectively.”
InfluentialDanielsson, P.-E. (1980) · Computer Graphics and Image Processing · doi:10.1016/0146-664x(80)90054-4 · Cited by 1,998 · cited 1×
“Then, for each sub-codebook, we compute the Euclidean distance [11] between the newly arrived sub-vector x m and its nearest centroid zt−1 m,km based on the last session t − 1, i.”
MethodTutorial on maximum likelihood estimation
Myung, I. J. (2003) · Journal of Mathematical Psychology · doi:10.1016/s0022-2496(02)00028-7 · Cited by 1,802 · cited 3×
“To memorize information about each new document, we leverage maximum likelihood estimation (MLE) [35] to maximize the likelihood of a docid conditioned on the corresponding document, i.”
MethodInfluentialContinual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647 · cited 1×
“Existing CL approaches [13] can be categorized into: (i) replay methods, maintaining a subset of previous samples and training models together with samples in the new session [18, 52]; (ii) regularization-based methods, regularizing the model parameters to enable important parameters concerning the previous tasks to b…”
MethodA continual learning survey: Defying forgetting in classification tasks
De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2019) · arXiv · doi:10.48550/arxiv.1909.08383 · Cited by 1,647
Gradient Episodic Memory for Continual Learning
Lopez-Paz, D., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1706.08840 · Cited by 1,403 · cited 1×
“Besides VERT, following [26, 33], we apply (i) average performance (AP) to measure the average performance by the end of training”
MethodWhat's new? Acquiring New information as a process in comprehension
Haviland, S. E., & Clark, H. H. (1974) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(74)80003-4 · Cited by 1,151 · cited 2×
“, given, similar information, in their memory to enhance a mental model of the information as a whole [9, 10, 17].”
Discourse Production and Comprehension
Wertsch, J. V., & Freedle, R. O. (1978) · The American Journal of Psychology · doi:10.2307/1421524 · Cited by 905 · cited 1×
“The elastic weight consolidation (EWC) [23] loss L is used to regularize the model parameters, via the weighted distance between Θ 𝑡 − 1 and Θ , where 𝐹 is the Fisher information matrix [23], and 𝐹 𝑙 denotes each model parameter.”
A Deep Relevance Matching Model for Ad-hoc Retrieval
Guo, J., Fan, Y., Ai, Q., & Croft, W. B. (2016) · Proceedings of the 25th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2983323.2983769 · Cited by 806 · cited 1×
“For traditional pipeline frameworks for IR [16, 21, 41], indexing and retrieval are two separate modules.”
MethodSelf-citationExpert Gate: Lifelong Learning with a Network of Experts
Aljundi, R., Chakravarty, P., & Tuytelaars, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.753 · Cited by 619 · cited 2×
“Recently, CL has been considered in computer vision [40, 45] and natural language processing [1, 19], but few efforts have been devoted to IR so far.”
BackgroundMethodFew-Shot Class-Incremental Learning
Tao, X., Hong, X., Chang, X., Dong, S., Wei, X., & Gong, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01220 · Cited by 499
Maintaining Discrimination and Fairness in Class Incremental Learning
Zhao, B., Xiao, X., Gan, G., Zhang, B., & Xia, S.-T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01322 · Cited by 494 · cited 1×
“Existing CL approaches [13] can be categorized into: (i) replay methods, maintaining a subset of previous samples and training models together with samples in the new session [18, 52]; (ii) regularization-based methods, regularizing the model parameters to enable important parameters concerning the previous tasks to b…”
MethodOnline continual learning in image classification: An empirical survey
Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., & Sanner, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.10.021 · Cited by 410
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2020) · arXiv · doi:10.48550/arxiv.2011.10566 · Cited by 381
On Tiny Episodic Memories in Continual Learning
Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P. K., Torr, P. H. S., & Ranzato, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10486 · Cited by 298
Optimizing Dense Retrieval Model Training with Hard Negatives
Zhan, J., Mao, J., Liu, Y., Guo, J., Zhang, M., & Ma, S. (2021) · arXiv · doi:10.48550/arxiv.2104.08051 · Cited by 251 · cited 1×
“In particular, we have presented a continual learning method to alleviate possible high computational costs for generating new docids, and leverage both past similar documents and pseudo-queries for consolidating knowledge.”
Self-citationOptimizing Dense Retrieval Model Training with Hard Negatives
Zhan, J., Mao, J., Liu, Y., Guo, J., Zhang, M., & Ma, S. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462880 · Cited by 248 · cited 1×
“In particular, we have presented a continual learning method to alleviate possible high computational costs for generating new docids, and leverage both past similar documents and pseudo-queries for consolidating knowledge.”
Self-citationDense Text Retrieval Based on Pretrained Language Models: A Survey
Zhao, W. X., Liu, J., Ren, R., & Wen, J.-R. (2024) · ACM Transactions on Information Systems · doi:10.1145/3637870 · Cited by 211 · cited 1×
“One is to generate docids for documents [53] including atomic identifiers (e.”
BackgroundAutoregressive Entity Retrieval
Cao, N. D., Izacard, G., Riedel, S., & Petroni, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00904 · Cited by 193 · cited 3×
“At inference time, we adopt constrained beam search [12] to decode the docids with 24 timesteps and 15 beams.”
BackgroundMethodDocument Expansion by Query Prediction
Nogueira, R. F., Yang, W., Lin, J., & Cho, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.08375 · Cited by 191 · cited 1×
“We finetune the T5 model [38] based on the query-document pairs PD0 0 in the initial session, by taking the document terms as input and producing a query following [37].”
MethodTransformer Memory as a Differentiable Search Index
Tay, Y., Tran, V., Dehghani, M., Ni, J., Bahri, D., Mehta, H., Qin, Z., Hui, K., Zhao, Z., Gupta, J., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1587 · Cited by 128
PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval
Ma, X., Guo, J., Zhang, R., Fan, Y., Ji, X., & Cheng, X. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441777 · Cited by 122 · cited 1×
“In this work, we have focused on a critical requirement for generative retrieval (GR) models to be usable in practical scenarios, where new documents are continuously added to the corpus.”
Self-citationRethinking search: making domain experts out of dilettantes
Metzler, D., Tay, Y., Bahri, D., & Najork, M. (2021) · ACM SIGIR Forum · doi:10.1145/3476415.3476428 · Cited by 112
Kousha, K., & Thelwall, M. (2020) · Quantitative Science Studies · doi:10.1162/qss_a_00066 · Cited by 103
Ultron: An Ultimate Retriever on Corpus with a Model-based IndexerPreprint
Zhou, Y., Yao, J., Dou, Z., Wu, L., Zhang, P., & Wen, J.-R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.09257 · Cited by 102 · cited 8×
“Recently, some work has been adopted to generate pseudo-queries [47, 54] and designed pre-training tasks [6] to tackle the limited availability of labeled data.”
BackgroundMethodInfluentialWitten, I. H., Bainbridge, D., & Nichols, D. M. (2010) · Thammasat University Digital Collections · Cited by 87
Transformer Memory as a Differentiable Search IndexPreprint
Tay, Y., Tran, V. Q., Dehghani, M., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.06991 · Cited by 80 · cited 18×
“When we look at variants of CLEVER in terms of IPQ, we find that: (i) CLEVER performs worse than CLEVER which updates the embeddings for each individual docid (also used in DSI++), showing that it is difficult for the GR models to accommodate to new documents without updating docids.”
BackgroundMethodInfluentialAutoregressive Search Engines: Generating Substrings as Document Identifiers
Bevilacqua, M., Ottaviano, G., Lewis, P., Yih, W.-T., Riedel, S., & Petroni, F. (2022) · arXiv · doi:10.48550/arxiv.2204.10628 · Cited by 80 · cited 1×
“, titles [5, 12, 44], n-grams [2, 4] or URLs [54]) and semantically structured identifiers (e.”
BackgroundAutoregressive Search Engines: Generating Substrings as Document Identifiers
Bevilacqua, M., Ottaviano, G., Lewis, P., Yih, S., Riedel, S., & Petroni, F. (2022) · neural information processing systems · doi:10.52202/068431-2296 · Cited by 76
A Neural Corpus Indexer for Document Retrieval
Wang, Y., Hou, Y., Wang, H., Miao, Z., Wu, S., Chen, Q., Xia, Y., Chi, C., Zhao, G., Liu, Z., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1856 · Cited by 69
GERE: Generative Evidence Retrieval for Fact Verification
Chen, J., Zhang, R., Guo, J., Fan, Y., & Cheng, X. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531827 · Cited by 66
Self-citationCorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks
Chen, J., Zhang, R., Guo, J., Liu, Y., Fan, Y., & Cheng, X. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · doi:10.1145/3511808.3557271 · Cited by 57
Self-citationDynamicRetriever: A Pre-trained Model-based IR System Without an Explicit Index
Zhou, Y., Yao, J., Dou, Z., Wu, L., & Wen, J.-R. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1373-9 · Cited by 52
Model Zoo: A Growing "Brain" That Learns ContinuallyPreprint
Ramesh, R., & Chaudhari, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03027 · Cited by 50 · cited 1×
“CL scenarios [30, 39] can be divided into task increment, domain increment and class increment.”
BackgroundMulti-Task Retrieval for Knowledge-Intensive Tasks
Maillard, J., Karpukhin, V., Petroni, F., Yih, W.-T., Oguz, B., Stoyanov, V., & Ghosh, G. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.89 · Cited by 47 · cited 1×
“In this work, we have focused on a critical requirement for generative retrieval (GR) models to be usable in practical scenarios, where new documents are continuously added to the corpus.”
A Neural Corpus Indexer for Document RetrievalPreprint
Wang, Y., Hou, Y., Wang, H., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.02743 · Cited by 47 · cited 4×
“Recently, some work has been adopted to generate pseudo-queries [47, 54] and designed pre-training tasks [6] to tackle the limited availability of labeled data.”
InfluentialGraph Few-shot Class-incremental Learning
Tan, Z., Ding, K., Guo, R., & Liu, H. (2022) · Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3488560.3498455 · Cited by 42 · cited 1×
“Continual learning (CL) has been a longstanding research topic to overcome the catastrophic forgetting problem of previously acquired knowledge, while continuously learning new knowledge from few labeled samples [43].”
Robust Retrieval Augmented Generation for Zero-shot Slot Filling
Glass, M., Rossiello, G., Chowdhury, M. F. M., & Gliozzo, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.148 · Cited by 38
B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval
Ma, X., Guo, J., Zhang, R., Fan, Y., Li, Y., & Cheng, X. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462869 · Cited by 36
Self-citationPre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction
Ma, X., Guo, J., Zhang, R., Fan, Y., & Cheng, X. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531772 · Cited by 35 · cited 1×
“words and thus is not differential with other representations [27, 49].”
BackgroundSelf-citationA Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning
Chen, J., Zhang, R., Guo, J., de Rijke, M., Liu, Y., Fan, Y., & Cheng, X. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591631 · Cited by 32
Self-citationSemantic-Enhanced Differentiable Search Index Inspired by Learning Strategies
Tang, Y., Zhang, R., Guo, J., Chen, J., Zhu, Z., Wang, S., Yin, D., & Cheng, X. (2023) · Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3580305.3599903 · Cited by 29 · cited 1×
“, titles [5, 12, 44], n-grams [2, 4] or URLs [54]) and semantically structured identifiers (e.”
BackgroundSelf-citationZhuang, S., Ren, H., Shou, L., Pei, J., Gong, M., Zuccon, G., & Jiang, D. (2022) · arXiv · doi:10.48550/arxiv.2206.10128 · Cited by 22 · cited 1×
“(ii) DSI-QG [56] leverages a query generation model to augment the document collection at indexing.”
MethodDSI++: Updating Transformer Memory with New Documents
Mehta, S., Gupta, J., Tay, Y., Dehghani, M., Tran, V., Rao, J., Najork, M., Strubell, E., & Metzler, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.510 · Cited by 21
B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc RetrievalPreprint
Ma, X., Guo, J., Zhang, R., Fan, Y., Li, Y., & Cheng, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09791 · Cited by 20
Self-citationDense Text Retrieval based on Pretrained Language Models: A SurveyPreprint
Zhao, W. X., Liu, J., Ren, R., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.14876 · Cited by 19 · cited 1×
“One is to generate docids for documents [53] including atomic identifiers (e.”
BackgroundGraph Few-shot Class-incremental LearningPreprint
Tan, Z. W., Ding, K., Guo, R., & Liu, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.12819 · Cited by 10 · cited 1×
“Continual learning (CL) has been a longstanding research topic to overcome the catastrophic forgetting problem of previously acquired knowledge, while continuously learning new knowledge from few labeled samples [43].”
DSI++: Updating Transformer Memory with New DocumentsPreprint
Mehta, S. V., Gupta, J. P., Tay, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.09744 · Cited by 3 · cited 6×
“We also compare with DSI++ [33], which continuously fine-tunes DSI over new documents by directly assigning each new document an atomic docid, i.”
BackgroundMethodResultInfluentialSemantic-Enhanced Differentiable Search Index Inspired by Learning StrategiesPreprint
Tang, Y., Zhang, R., Guo, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.15115 · Cited by 2 · cited 1×
“, titles [5, 12, 44], n-grams [2, 4] or URLs [54]) and semantically structured identifiers (e.”
BackgroundSelf-citation
Research works printed but not linked: 4
- 10. Herbert H Clark and Susan E Haviland. 1974. Psychological Processes as Lin- guistic Explanation. Explaining linguistic phenomena (1974), 91-124.
- 12. Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2020. Au- toregressive Entity Retrieval. In ICLR.
- 32. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Intro- duction to Information Retrieval. Cambridge University Press.
- 49. Shicheng Xu, Liang Pang, Huawei Shen, and Xueqi Cheng. 2022. Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning. arXiv preprint arXiv:2204.02725 (2022). · arXiv
Cited by
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
“CLEVER [109], based on Product Quantization (PQ) [206], proposes In-cremental Product Quantization (IPQ) for generating PQ codes as DocIDs for documents.”
Scalable and Effective Generative Information Retrieval
Zeng, H., Luo, C., Jin, B., Sarwar, S. M., Wei, T., & Zamani, H. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645477 · Cited by 32
“These results can be found on Table 4 in Appendix A.”
BackgroundKnow where to go: Make LLM a relevant, responsible, and trustworthy searchers
Shi, X. D., Liu, J., Liu, Y., Cheng, Q., & Lu, W. P. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114354 · Cited by 72
RAGVA: Engineering retrieval augmented generation-based virtual assistants in practice
Yang, R., Fu, M. C., Tantithamthavorn, C. K., Arora, C., Vandenhurk, L., & Chua, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112436 · Cited by 49
TravelRAG: A Tourist Attraction Retrieval Framework Based on Multi-Layer Knowledge Graph
Song, S., Yang, C., Li, X., Shang, H., Li, Z., & Chang, Y. (2024) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi13110414 · Cited by 40
Metadata
- Authors
- Chen, J., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan, Y., & Cheng, X.
- Affiliations
- Institute of Computing Technology (CN)
- University of Amsterdam, Amsterdam, Netherlands
- Venue
- Proceedings of the 32nd ACM International Conference on Information and Knowledge Management
- Publisher
- ACM
- Published
- 2023-10-21
- Pages
- 306-315
- ISSN
- 2155-0751 (ISSN-L)
- OpenAlex
- W4386302269
- Cited by
- 40
- References
- 59
- Citations by year
- 2026: 7 · 2025: 17 · 2024: 15 · 2023: 1
- FWCI
- 10.29
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM International Conference on Information and Knowledge Management (CIKM) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- LESSEN (NWA.1389.20.183)
- CAS Project for Young Scientists in Basic Research (YSBR-034)
- Project (JCKY2022130C039, 2021QY1701)
- Lenovo-CAS Joint Lab Youth Scientist Project
- National Natural Science Foundation of China (NSFC) (62006218, 61902381)
- China Scholarship Council (202104910234)
- 10-year program
- Youth Innovation Promotion Association CAS (2021100)
- Innovation Project of ICT CAS (E261090)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- generative retrieval, continual learning, incremental indexing, product quantization, forgetting, document encoding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.