The DOI matches its DataCite record; the references it cites are not checked yet.
Unsupervised Dense Information Retrieval with Contrastive Learning
Open accessAbstract
Recently, information retrieval has seen the emergence of dense retrievers, using neural networks, as an alternative to classical sparse methods based on term-frequency. These models have obtained state-of-the-art results on datasets and tasks where large training sets are available. However, they do not transfer well to new applications with no training data, and are outperformed by unsupervised term-frequency methods such as BM25. In this work, we explore the limits of contrastive learning as a way to train unsupervised dense retrievers and show that it leads to strong performance in various retrieval settings. On the BEIR benchmark our unsupervised model outperforms BM25 on 11 out of 15 datasets for the Recall@100. When used as pre-training before fine-tuning, either on a few thousands in-domain examples or on the large MS~MARCO dataset, our contrastive model leads to improvements on the BEIR benchmark. Finally, we evaluate our approach for multi-lingual retrieval, where training data is even scarcer than for English, and show that our approach leads to strong unsupervised performance. Our model also exhibits strong cross-lingual transfer when fine-tuned on supervised English data only and evaluated on low resources language such as Swahili. We show that our unsupervised models can perform cross-lingual retrieval between different scripts, such as retrieving English documents from Arabic queries, which would not be possible with term matching methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Based on our review of existing work, we identify two primary retrieval methods suitable for non-parametric memory including sparse methods based on bag-of-words models and dense methods based on deep representation learning for vector encoding and relevance score calculation [269, 270, 232, 271].”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [232]
The cited source says what the claim attributes to it.
In the citing paper
“The retrieval of non-parametric memory involves locating the most relevant K memory fragments. This process consists of two main steps. It first calculates the relevance score between the query q and each fragment m in the non-parametric memory, and then ranks the fragments m based on their relevance scores and selects the topK fragments m with the highest scores to form the retrieval results [267”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [232]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Offdiagonal entries show that records optimized for one encoder largely retain effectiveness on others, with most cross-transfer pairs exceeding 70% AHR.”
From MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning (2026) · cites this paper as Izacard et al., 2021
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The similarity score of HotpotQA distractors is not substantially larger than that of the least similar supporting passages, meaning that these distractors are not very effective.”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [35]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, it can be seen that the distribution of the distractor scores in HotpotQA is much closer to the lower bound of the support passage scores compared to the other two datasets.”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [35]
Experimental — accuracy not yet measured
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We compare HippoRAG with existing RAG methods on multi-hop question answering (QA) and show that our method outperforms the state-of-the-art methods remarkably, by up to 20 %.”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [35]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 58
- Akari Asai, Xinyan Yu, Jungo Kasai, and Hannaneh Hajishirzi. One question answering model for many languages with cross-lingual dense passage retrieval. CoRR , abs/2107.11976, 2021. URL https://arxiv. org/abs/2107.11976 . 10, 11 · arXiv
- Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 , 2016. 3, 6 · arXiv
- Adam Berger, Rich Caruana, David Cohn, Dayne Freitag, and Vibhu Mittal. Bridging the lexical chasm: statistical approaches to answer-finding. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval , pp. 192-199, 2000. 1
- Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. arXiv preprint arXiv:2104.14294 , 2021. 2 · arXiv
- Wei-Cheng Chang, Felix X Yu, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. Pre-training tasks for embedding-based large-scale retrieval. arXiv preprint arXiv:2002.03932 , 2020. 2, 3 · arXiv
- Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading Wikipedia to answer open-domain questions. In Proc. ACL , 2017a. 1
- Ting Chen, Yizhou Sun, Yue Shi, and Liangjie Hong. On sampling strategies for neural network-based collaborative filtering. arXiv preprint arXiv:1706.07881 , 2017b. 5 · arXiv
- Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning , pp. 1597-1607. PMLR, 2020. 2, 4, 5
- Xilun Chen, Kushal Lakhotia, Barlas Oğuz, Anchit Gupta, Patrick Lewis, Stan Peshterliev, Yashar Mehdad, Sonal Gupta, and Wen-tau Yih. Salient phrase aware dense retrieval: Can a dense retriever imitate a sparse one?, 2021. URL https://arxiv.org/abs/2110.06918 . 3 · arXiv
- Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. Tydi QA: A benchmark for information-seeking question answering in typologically diverse languages. CoRR , abs/2003.05002, 2020. URL https://arxiv.org/abs/2003.05002 . 9 · arXiv
- Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learning at scale. CoRR , abs/1911.02116, 2019. URL http://arxiv.org/abs/1911.02116 . 9, 10, 19 · arXiv
- Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harshman. Indexing by latent semantic analysis. Journal of the American society for information science , 41(6):391-407, 1990. 2
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. NAACL , 2019. 3, 4
- Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie. Cert: Contrastive self-supervised learning for language understanding. arXiv preprint arXiv:2005.12766 , 2020. 3 · arXiv
- Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. Splade v2: Sparse lexical and expansion model for information retrieval. arXiv preprint arXiv:2109.10086 , 2021. 7 · arXiv
- Luyu Gao and Jamie Callan. Unsupervised corpus aware language model pre-training for dense passage retrieval. arXiv preprint arXiv:2108.05540 , 2021. 3 · arXiv
- Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 , 2021. 5, 7 · arXiv
- Daniel Gillick, Alessandro Presta, and Gaurav Singh Tomar. End-to-end retrieval in continuous space. arXiv preprint arXiv:1811.08008 , 2018. 3 · arXiv
- John M Giorgi, Osvald Nitski, Gary D Bader, and Bo Wang. Declutr: Deep contrastive learning for unsupervised textual representations. arXiv preprint arXiv:2006.03659 , 2020. 3 · arXiv
- Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. Realm: Retrieval-augmented language model pre-training. arXiv preprint arXiv:2002.08909 , 2020. 3, 5, 7 · arXiv
- Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 9729-9738, 2020. 2, 6, 17
- Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. Efficiently teaching an effective dense retriever with balanced topic aware sampling. arXiv preprint arXiv:2104.06967 , 2021. 7 · arXiv
- Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management , pp. 2333-2338, 2013. 2, 4
- Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. arXiv preprint arXiv:1905.01969 , 2019. 3 · arXiv
- Gautier Izacard and Edouard Grave. Distilling knowledge from reader to retriever for question answering, 2020a. URL https://arxiv.org/abs/2012.04584 . 3, 7 · arXiv
- Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282 , 2020b. 6 · arXiv
- Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Sebastian Riedel, and Edouard Grave. A memory efficient baseline for open domain question answering. arXiv preprint arXiv:2012.15156 , 2020. 3 · arXiv
- Yacine Jernite, Samuel R Bowman, and David Sontag. Discourse-based objectives for fast unsupervised sentence representation learning. arXiv preprint arXiv:1705.00557 , 2017. 3 · arXiv
- Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data , 2019. 3
- Karen Sparck Jones. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation , 1972. 2
- Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proc. ACL , 2017. 6, 7
- Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906 , 2020. 3, 4, 5, 7 · arXiv
- Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics , 7:453-466, 2019. 1, 6, 7
- Jungmin Kwon, Jeongseop Kim, Hyunseo Park, and In Kwon Choi. ASAM: adaptive sharpness-aware minimization for scale-invariant learning of deep neural networks. arXiv preprint arXiv:2102.11600 , 2021. 17 · arXiv
- Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 , 2019. 3 · arXiv
- Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. In Proc. ACL , 2019. 2, 3, 4, 5, 6
- Mike Lewis, Marjan Ghazvininejad, Gargi Ghosh, Armen Aghajanyan, Sida Wang, and Luke Zettlemoyer. Pre-training via paraphrasing. arXiv preprint arXiv:2006.15020 , 2020. 3 · arXiv
- Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. Pretrained transformers for text ranking: Bert and beyond. arXiv preprint arXiv:2010.06467 , 2020. 2 · arXiv
- Shayne Longpre, Yi Lu, and Joachim Daiber. MKQA: A linguistically diverse benchmark for multilingual open domain question answering. CoRR , abs/2007.15207, 2020. URL https://arxiv.org/abs/2007.15207 . 9 · arXiv
- Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. Sparse, dense, and attentional representations for text retrieval. arXiv preprint arXiv:2005.00181 , 2020. 3 · arXiv
- Christopher D Manning, Hinrich Schütze, and Prabhakar Raghavan. Introduction to information retrieval . Cambridge university press, 2008. 2
- Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, and Xia Song. Coco-lm: Correcting and contrasting text sequences for language model pretraining. arXiv preprint arXiv:2102.08473 , 2021. 3 · arXiv
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems , pp. 3111-3119, 2013. 3
- Bhaskar Mitra, Nick Craswell, et al. An introduction to neural information retrieval. Foundations and Trends® in Information Retrieval , 13(1):1-126, 2018. 2
- Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. Ms marco: A human generated machine reading comprehension dataset. In CoCo@ NIPS , 2016. 1
- Rodrigo Nogueira and Kyunghyun Cho. Passage re-ranking with bert. arXiv preprint arXiv:1901.04085 , 2019. 2, 4 · arXiv
- Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and Rabab Ward. Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval. IEEE/ACM Transactions on Audio, Speech, and Language Processing , 24(4):694-707, 2016. 2
- Ori Ram, Gal Shachaf, Omer Levy, Jonathan Berant, and Amir Globerson. Learning to retrieve passages without supervision, 2021. URL https://arxiv.org/abs/2112.07708 . 3 · arXiv
- Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 , 2019. 3, 4 · arXiv
- Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. Learning semantic representations using convolutional neural networks for web search. In Proceedings of the 23rd international conference on world wide web , pp. 373-374, 2014. 2
- Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint arXiv:2104.08663 , 2021. 2, 6, 18, 19 · arXiv
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. Fever: a large-scale dataset for fact extraction and verification. arXiv preprint arXiv:1803.05355 , 2018. 1 · arXiv
- Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. CCNet: Extracting high quality monolingual datasets from web crawl data. In Proceedings of the 12th Language Resources and Evaluation Conference , 2020. 6, 9, 17
- Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 3733-3742, 2018. 2, 4, 6
- Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. Clear: Contrastive learning for sentence representation. arXiv preprint arXiv:2012.15466 , 2020. 3 · arXiv
- Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv preprint arXiv:2007.00808 , 2020. 3, 4, 7 · arXiv
- Sohee Yang and Minjoon Seo. Is retriever merely an approximator of reader? arXiv preprint arXiv:2010.10999 , 2020. 3 · arXiv
- Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. Mr. tydi: A multi-lingual benchmark for dense retrieval. CoRR , abs/2108.08787, 2021. URL https://arxiv.org/abs/2108.08787 . 9, 10 · arXiv
Cited by
Sorted by citations
Items 101–200 of 203
PolEval 2022/23 Challenge Tasks and Results
Kobyliński, Ł., Ogrodniczuk, M., Rybak, P., Przybyła, P., Pęzik, P., Mikołajczyk, A., Janowski, W., Marcińczuk, M., & Smywiński-Pohl, A. (2023) · Annals of Computer Science and Information Systems · doi:10.15439/2023f5627 · Cited by 4
SUGAR: Leveraging Contextual Confidence for Smarter Retrieval
Zubkova, H., Park, J.-H., & Lee, S.-W. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890064 · Cited by 9
Unsupervised Legal Evidence Retrieval via Contrastive Learning with Approximate Aggregated Positive
Yao, F., Zhang, J., Zhang, Y., Liu, X., Sun, C., Liu, Y., & Shen, W. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25603 · Cited by 4
Detrimental Contexts in Open-Domain Question Answering
Oh, P., & Thorne, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.776 · Cited by 7
LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., & Yu, D. (2024) · arXiv · doi:10.48550/arxiv.2410.10813 · Cited by 493
“In Table 9, we compare the performance of Stella V5 with two alternative popular retrievers: BM25 (Robertson & Zaragoza, 2009) and Contriever (Izacard et al., 2022).”
Human-inspired Perspectives: A Survey on AI Long-term Memory
He, Z., Lin, W., Zheng, H., Zhang, F., Jones, M. W., Aitchison, L., Xu, X., Liu, M., Kristensson, P. O., & Shen, J. (2024) · arXiv · doi:10.48550/arxiv.2411.00489 · Cited by 3
“The ranking score can be obtained by calculating the Euclidean distance [223] or by performing a dot product [226] between the query vector and the memory fragment vector.”
PeerQA: A Scientific Question Answering Dataset from Peer Reviews
Baumgärtner, T., Briscoe, T., & Gurevych, I. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.22 · Cited by 3
Mo, Y., Tang, M., Lin, R., Zhou, B., & Li, X. (2025) · IEEE Access · doi:10.1109/access.2025.3582519 · Cited by 5
Exploring ℓ 0 Sparsification for Inference-free Sparse Retrievers
Shen, X., Geng, Z., & Yang, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730192 · Cited by 3
Less LLM, More Documents: Searching for Improved RAG
Ning, J., Kong, Y., Long, Y., & Callan, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_38 · Cited by 9
RobustQA: Benchmarking the Robustness of Domain Adaptation for Open-Domain Question Answering
Han, R., Qi, P., Zhang, Y., Liu, L., Burger, J., Wang, W. Y., Huang, Z., Xiang, B., & Roth, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.263 · Cited by 5
Query Expansion and Verification with Large Language Model for Information Retrieval
Zhang, W., Liu, Z., Wang, K., & Lian, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5672-8_29 · Cited by 3
Early Exit Strategies for Approximate k -NN Search in Dense Retrieval
Busolin, F., Lucchese, C., Nardini, F. M., Orlando, S., Perego, R., & Trani, S. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679903 · Cited by 6
Beyond decomposition: Hierarchical dependency management in multi‐document question answering
Zheng, X., Li, Z., Chen, Q., & Zhang, Y. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24971 · Cited by 3
Chatbot Dialog Design for Improved Human Performance in Domain Knowledge Discovery
Oruche, R., Cheng, X., Zeng, Z., Vazzana, A., Goni, M. A., Shibo, B. W., Goruganthu, S. K., Kee, K. F., & Calyam, P. P. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2024.3514742 · Cited by 3
The 2nd Futuredial Challenge: Dialog Systems With Retrieval Augmented Generation (Futuredial-RAG)
Cai, Y., Chen, S., Wu, Y., Huang, Y., Feng, J., & Ou, Z. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832299 · Cited by 3
Santra, P., Ghosh, M., Ganguly, D., Basuchowdhuri, P., & Naskar, S. K. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70021 · Cited by 5
Variational Open-Domain Question Answering
Liévin, V., Motzfeldt, A. G., Jensen, I. R., & Winther, O. (2022) · Research at the University of Copenhagen (University of Copenhagen) · doi:10.48550/arxiv.2210.06345 · Cited by 4
Dua, D., Strubell, E., Singh, S., & Verga, P. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.807 · Cited by 6
Cross-document Event Coreference Search: Task, Dataset and Modeling
Eirew, A., Caciularu, A., & Dagan, I. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.58 · Cited by 3
Wen, J., Zhou, H., Guan, J., Zhou, J., & Huang, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.612 · Cited by 3
Medical Document Embedding Enhancement with Heterogeneous Mixture-of-Experts
Liu, X., Zhu, Y., Pang, T., Xue, K., Zhang, X., & Fan, C. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822374 · Cited by 4
Measuring Actual Privacy of Obfuscated Queries in Information Retrieval
Faveri, F. L. D., Faggioli, G., & Ferro, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_4 · Cited by 3
Attribute or Abstain: Large Language Models as Long Document Assistants
Buchmann, J., Liu, X., & Gurevych, I. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.463 · Cited by 3
From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Gutiérrez, B. J., Shu, Y., Qi, W., Zhou, S., & Su, Y. (2025) · arXiv · doi:10.48550/arxiv.2502.14802 · Cited by 2
“Large embedding models (7B) significantly outperform classic smaller LM-based models like Contriever and GTR, achieving at least a 9 .”
Huang, H., DeLucia, A., Tiyyala, V. M., & Dredze, M. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.693 · Cited by 2
“In addition, doctor responses are not themselves complete and might have information omissions due to subjective writing preferences.”
Enhanced Privacy Policy Comprehension via Pre-trained and Retrieval-Augmented Models
Zhang, X., Zhang, B., Zhang, C., & Wei, L. (2024) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom63139.2024.00094 · Cited by 2
Cai, J., & Zhu, F. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129379 · Cited by 2
Fajčík, M., Dočekal, M., Doležal, J., Ondrej, K., Beneš, K., Kapsa, J., Smrž, P., Polok, A., Hradiš, M., Nevěřilová, Z., et al. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.32 · Cited by 14
Xu, C., Chen, Q., Feng, Y., & Cheng, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-09530-5_1 · Cited by 7
Zhong, Z., Yuan, W., Qu, L., Chen, T., Wang, H., Zhao, X., & Yin, H. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3779452 · Cited by 5
A General-Purpose Multilingual Document Encoder
Litschko, O. G. R., Litschko, R., & Glavaš, G. (2023) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2023.mrl-1.4 · Cited by 3
Document Embeddings Enhance Biomedical Retrieval-Augmented Generation
Kong, Y., Yang, Z., Luo, L., Ding, Z., Wang, L., Liu, W., Zhang, Y., Xu, B., Wang, J., Sun, Y., et al. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822781 · Cited by 2
Mallapragada, S., Xie, Y., Chawan, V. R., Hailat, Z., & Wang, Y. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826020 · Cited by 2
Improving Knowledge Base Question Answering via Retrieval Enhancement and Stepwise Reasoning
Huang, D. K., Gao, J., Luo, X., & Wu, H. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890582 · Cited by 4
Exploring the Effectiveness of Multi-stage Fine-Tuning for Cross-Encoder Re-rankers
Pezzuti, F., MacAvaney, S., & Tonellotto, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_45 · Cited by 2
Reply With Sticker: New Dataset and Model for Sticker Retrieval
Liang, B., Wang, B., Bai, Z., Lang, Q., Sun, M., Hou, K., Zhou, L., Xu, R., & Wong, K. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3587415 · Cited by 8
Zhang, Z., Yu, W., Zheng, N., Ju, M., & Jiang, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00591 · Cited by 3
Rosati, D. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.46 · Cited by 3
rT5: A Retrieval-Augmented Pre-trained Model for Ancient Chinese Entity Description Generation
Hu, M., Zhao, X., Wei, J., Wu, J., Sun, X., Li, Z., Wu, Y., Sun, Y., & Zhang, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44693-1_57 · Cited by 2
Improving Sequential Model Editing with Fact Retrieval
Han, X., Li, R., Tan, H., Wang, Y., Chai, Q., & Pan, J. Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.749 · Cited by 5
Relevance-assisted Generation for Robust Zero-shot Retrieval
Kim, J., Kim, M., Park, J., & Hwang, S.-W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.67 · Cited by 3
DESIRE-ME: Domain-Enhanced Supervised Information Retrieval Using Mixture-of-Experts
Kasela, P., Pasi, G., Perego, R., & Tonellotto, N. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56060-6_8 · Cited by 2
Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
Hu, Y., Wang, Y., & McAuley, J. (2025) · arXiv · doi:10.48550/arxiv.2507.05257 · Cited by 1
“Early methods like DPR [20] and Contriever [17] are based on BERT [9], while more recent models such as NV-Embed-v2 [22] utilize decoder-only backbones and achieve significantly improved retrieval performance.”
Unsupervised Dense Retrieval Training with Web Anchors
Xie, Y., Liu, X., & Xiong, C. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592080 · Cited by 3
NAIL: Lexical Retrieval Indices with Efficient Non-Autoregressive Decoders
Soares, L. B., Gillick, D., Cole, J. R., & Kwiatkowski, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.156 · Cited by 2
Simple Domain Adaptation for Sparse Retrievers
Vast, M., Zong, Y., Piwowarski, B., & Soulier, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56063-7_32 · Cited by 1
: A Voting-Based Paradigm for Enhancing Retrieval Augmented Generation
Guan, W., Li, X., Lu, J., & Zhou, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78119-3_9 · Cited by 1
Streamlining conformity assessment of software applying large language models
Litzinger, J., Neumann, J., Peters, D., & Thiel, F. (2025) · Measurement Sensors · doi:10.1016/j.measen.2024.101792 · Cited by 2
Anichkov, Y., Popov, V., & Bolovtsov, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-80853-1_31 · Cited by 2
Barbiero, L., Agostini, F., Baci, E., Frigo, F., Vianello, M., Pozza, D., & Campese, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-00071-2_32 · Cited by 1
Li, R., Cui, M., Wang, B., Wang, Y., & Miao, Q. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112812 · Cited by 3
Legal Case Entailment via Optimal Transport–Enhanced Retrieval and Large Language Model Ranking
Tran, T. N., Nguyen, P. M., & Nguyen, M. L. (2026) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3789254 · Cited by 1
Wang, Z., He, D., Zhang, Z., Liu, Y., Liu, J., Zeng, Z., Qin, Z., Li, Z., Li, X., Yao, H., et al. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792200 · Cited by 1
Dissecting GraphRAG: A Modular Analysis of Knowledge Structuring for Factoid Question Answering
Nishida, N., Munne, R. F., Liu, S., Tokunaga, N., Yamagata, Y., Cheng, F., Kozaki, K., & Matsumoto, Y. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.615 · Cited by 1
mAggretriever: A Simple yet Effective Approach to Zero-Shot Multilingual Dense Retrieval
Lin, S.-C., Ahmad, A., & Lin, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.715 · Cited by 1
Hybrid Feature Fusion for Enhancing Medical Document Embedding
Zhu, Y., Liu, X., Pang, T., Xiao, X., Zhang, X., & Fan, C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888956 · Cited by 1
External Knowledge Is Not Always Needed: An Adaptive Retrieval Augmented Generation Method
Guan, W., Lu, J., Feng, Q., Li, X., & Zhou, J. (2024) · International Symposium on Artificial Intelligence and Intelligent Manufacturing (AIIM) · doi:10.1109/aiim64537.2024.10934284 · Cited by 1
In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
Tan, Z., Yan, J., Hsu, I.-H., Han, R., Wang, Z., Le, L. T., Song, Y., Chen, Y., Palangi, H., Lee, G., et al. (2025) · arXiv · Cited by 54
“• Contriever ( facebook/contriever ) (Izacard et al., 2022): A dense retriever optimized for semantic search leveraging contrastive learning.”
MemMorph: Tool Hijacking in LLM Agents via Memory Poisoning
Zhang, X., Zheng, Y., Xu, Z., Zhou, K., Shen, B., Ou, H., Zhang, T., & Lam, K.-Y. (2026) · arXiv · doi:10.48550/arxiv.2605.26154
“Figure 4 reports the attack transferability across four widely-used embedding models: E5 (Wang et al., 2022), BGE (Xiao et al., 2024), Contriever (Izacard et al., 2021), and MiniLM (Wang et al., 2020).”
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Hsieh, C.-Y., Chuang, Y.-S., Li, C.-L., Wang, Z., Le, L. T., Kumar, A., Glass, J., Ratner, A., Lee, C.-Y., Krishna, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.16008
“The k − 1 distractor passages are Wikipedia chunks retrieved by Contriever (Izacard et al., 2022a) that are most relevant to the query but do not contain any of the annotated answers in NaturalQuestions.”
Adapting Language Models to Compress Contexts
Chevalier, A., Wettig, A., Ajith, A., & Chen, D. (2023) · arXiv · Cited by 59
Self-Critique Guided Iterative Reasoning for Multi-hop Question Answering
Chu, Z., Fan, H., Chen, J., Wang, Q., Yang, M., Liang, J., Wang, Z., Li, H., Tang, G., Liu, M., et al. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.123 · Cited by 2
Unsupervised Domain Adaption for Neural Information RetrievalPreprint
Dominguez, C., Campos, J. A., Agirre, E., & Azkune, G. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4457167
Multi-index Retrieve and Rerank with Sequence-to-Sequence Model
Wojtasik, K. (2023) · Annals of Computer Science and Information Systems · doi:10.15439/2023f3900
Lossy Compression Options for Dense Index Retention
Mackenzie, J., & Moffat, A. (2023) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3624918.3625316
Fu, X., li, J., & Li, B. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4647617
Balepur, N., Huang, J., & Chang, K. C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.288 · Cited by 1
DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework
Kim, K., & Lee, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.150
Boot and Switch: Alternating Distillation for Zero-Shot Dense Retrieval
Jiang, F., Xu, Q., Drummond, T. W., & Cohn, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.65 · Cited by 3
Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words
Kurita, H., Kobayashi, G., Yokoi, S., & Inui, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.729
DocSplit: Simple Contrastive Pretraining for Large Document Embeddings
Wang, Y., & Izbicki, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.945
Noisy Self-Training with Synthetic Queries for Dense Retrieval
Jiang, F., Drummond, T. W., & Cohn, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.803 · Cited by 1
Varanasi, S., Butt, U., & Neumann, G. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_124
Generate to Understand for Representation in One Pre-training Stage
Xue, C., Zhong, X., & Liu, X. (2023) · International Conference on Information Systems and Computing Technology (ISCTech) · doi:10.1109/isctech60480.2023.00054
On Interfacing Tip-of-the-tongue References
Kim, J.-H., Hong, S., & Hwang, S.-W. (2023) · doi:10.18653/v1/2023.nlint-1.3
A Data-Centric Contrastive Embedding Framework for Contextomized Quote Detection
Song, S., Han, J., & Park, K. (2024) · IEEE Access · doi:10.1109/access.2024.3377227
Enhancing Machine Translation Experiences with Multilingual Knowledge Graphs
Conia, S., Lee, D. C., Li, M., Minhas, U. F., & Li, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30563
I3: I ntent- I ntrospective Retrieval Conditioned on I nstructions
Pan, K., Li, J., Wang, W., Fei, H., Song, H., Ji, W., Lin, J., Liu, X., Chua, T., & Tang, S. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657745 · Cited by 6
Enhancing Retrieval-Augmented LMs with a Two-Stage Consistency Learning Compressor
Xu, C., Zhao, D., Wang, B., & Xing, H. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5669-8_42 · Cited by 1
Bridge the Query and Document: Contrastive Learning for Generative Document Retrieval
Zhao, Y., Ni, Z., Hu, Z., Xu, S., & Xu, B. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650777
Are Dense Retrieval Models Few-Shot Learners?
Cheng, H., Sun, S., Yang, S., Zheng, B., Yang, W., & Hu, H. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0119-6_39
Zero-shot Document Retrieval with Hybrid Pseudo-document Retriever
Sun, D. S., Guo, W., Liu, X., Zhang, Y., Hou, Z., & Li, Z. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889897
Advancing sentiment understanding and safety in NLP systems
Deng, Y. (2025) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/183610
Few-Shot Multilingual Open-Domain QA from Five Examples
Jiang, F., Drummond, T. W., & Cohn, T. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00750
Wang, L., Wang, T., Zhou, T. H., Jia, X. T., & Zhao, H. L. (2025) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd64889.2025.11033377
Елисеев, В. О., Максимова, А. Ю., & Иванович, Б. В. (2025) · System Informatics · doi:10.31144/si.2307-6410.2025.n27.p77-92
Lin, B., & Liu, T. (2025) · International Conference on Inventive Research in Computing Applications (ICIRCA) · doi:10.1109/icirca65293.2025.11089532
Enabling retrieval-augmented generation and knowledge graphs with discourse analysis
Galitsky, B. A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-30046-2.00003-x
A Model-agnostic Pre-training Framework for Search Result Diversification
Deng, Z., Dou, Z., Zhu, Y., & Wen, J.-R. (2025) · ACM Transactions on Information Systems · doi:10.1145/3764662 · Cited by 1
Heo, S., Choi, W., & Lee, S. (2025) · Journal of the Computational Structural Engineering Institute of Korea · doi:10.7734/coseik.2025.38.4.257
RM2: Retrieve-More-Then-Read-More for Multi-Hop Question Answering
Zhao, F., Chen, S., Hou, J., Sun, H., Zhang, Q., Yao, T., Yuan, C., & Wang, X. (2025) · International Conference on Communication and Information Processing (ICCIP) · doi:10.1145/3784833.3784928
LLM-Based Listwise Reranking Under the Effect of Positional Bias
Qiao, J., Huang, J., Ma, X., Wang, S., Yin, D., Kanoulas, E., & Yates, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_9
CompactRAG: Reducing LLM Calls and Token Overhead in Multi-Hop Question Answering
Yang, H., Yang, Z., Zhang, X., Wei, W., Zhang, Y., & Yang, L. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792512 · Cited by 2
Yuan, L., Liu, S., Wang, Y., Zhu, S., Wang, T., Mao, T., & Shao, S. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10838-y
Nazi, Z. A., Hristidis, V., McLean, A. L., Meem, J. A., & Chowdhury, M. T. A. (2026) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-026-01220-z · Cited by 1
UNICS: Multilingual Code Search via Unified Pseudocode and Contrastive Transfer Learning
Ye, F., Ge, J., Li, C., Huang, L., & Luo, B. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797067
Reading Between the Lines: A Text-based Deep Learning Approach for Understanding Company Dynamics
Duan, H., Yang, Y., & Tam, K. Y. (2026) · MIS Quarterly · doi:10.25300/misq/2026/19345
Reproducing Complex Set-Compositional Information Retrieval
Degenhart, V., Timman, D., de Vries, A. P., Hasibi, F., & Hoveyda, M. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808555
A Multi-stage Hybrid Retrieval Framework for Knowledge-Augmented Question Answering
Veningston, K., Selvan, C., Ronalda, M., Chaudhuri, B. B., & Tatwal, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39379-1_38
Metadata
- Authors
- Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., & Grave, E.
- Affiliations
- Université Paris Sciences et Lettres (FR)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2112.09118
- OpenAlex
- W4320813768
- Cited by
- 172
- References
- 0
- Citations by year
- 2026: 14 · 2025: 44 · 2024: 29 · 2023: 79 · 2022: 6
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- contrastive learning, pre-training, cross-lingual transfer, low-resource languages, Swahili language, BEIR benchmark, MS MARCO
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
