The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Sparse, Dense, and Attentional Representations for Text Retrieval
Open accessAbstract
Abstract Dual encoders perform retrieval by encoding documents and queries into dense low-dimensional vectors, scoring each document by its inner product with the query. We investigate the capacity of this architecture relative to sparse bag-of-words models and attentional neural networks. Using both theoretical and empirical analysis, we establish connections between the encoding dimension, the margin between gold and lower-ranked documents, and the document length, suggesting limitations in the capacity of fixed-length encodings to support precise retrieval of long documents. Building on these insights, we propose a simple neural model that combines the efficiency of dual encoders with some of the expressiveness of more costly attentional architectures, and explore sparse-dense hybrids to capitalize on the precision of sparse retrieval. These models outperform strong alternatives in large-scale retrieval.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Based on our review of existing work, we identify two primary retrieval methods suitable for non-parametric memory including sparse methods based on bag-of-words models and dense methods based on deep representation learning for vector encoding and relevance score calculation [269, 270, 232, 271].”
From Human-inspired Perspectives: A Survey on AI Long-term Memory (2024) · cites this paper as [270]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Self-citationAdvances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,043
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Decoding by Linear Programming
Candès, E. J., & Tao, T. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.858979 · Cited by 7,134
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
A statistical interpretation of term specificity and its application in retrieval
Jones, K. S. (2021) · J. Documentation · doi:10.1108/eb026526 · Cited by 5,541
Approximate nearest neighbors: towards removing the curse of dimensionality
Indyk, P., & Motwani, R. (1998) · Proceedings of the thirtieth annual ACM symposium on Theory of computing - STOC '98 · doi:10.1145/276698.276876 · Cited by 4,188
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
The probabilistic relevance framework: BM25 and beyond
Robertson, S., & Zaragoza, H. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000019 · Cited by 3,374
Cited by
Sorted by citations
Qu, Y., Ding, Y., Liu, J., Liu, K., Ren, R., Zhao, W. X., Dong, D., Wu, H. R., & Wang, H. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.466 · Cited by 597
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Xiong, L., Xiong, C., Li, Y., Tang, K., Liu, J., Bennett, P. N., Ahmed, J., & Overwikj, A. (2021) · International Conference on Learning Representations · Cited by 376
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., & Zaharia, M. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.272 · Cited by 353
Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling
Hofstätter, S., Lin, S.-C., Yang, J.-H., Lin, J., & Hanbury, A. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462891 · Cited by 374
Pretrained Transformers for Text Ranking: BERT and Beyond
Lin, J., Nogueira, R. F., & Yates, A. (2021) · Synthesis lectures on human language technologies · doi:10.2200/s01123ed1v01y202108hlt053 · Cited by 328
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval
Gao, L., & Callan, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.203 · Cited by 251
Optimizing Dense Retrieval Model Training with Hard Negatives
Zhan, J., Mao, J., Liu, Y., Guo, J., Zhang, M., & Ma, S. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462880 · Cited by 248
Dense Text Retrieval Based on Pretrained Language Models: A Survey
Zhao, W. X., Liu, J., Ren, R., & Wen, J.-R. (2024) · ACM Transactions on Information Systems · doi:10.1145/3637870 · Cited by 211
“A straightforward approach is to aggregate the retrieval scores of the two approaches [1], [96].”
BackgroundMethodInfluentialCOIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List
Gao, L., Dai, Z., & Callan, J. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.241 · Cited by 265
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Xiong, L., Xiong, C., Li, Y., Tang, K.-F., Liu, J., Bennett, P., Ahmed, J., & Overwijk, A. (2020) · arXiv · doi:10.48550/arxiv.2007.00808 · Cited by 204
Condenser: a Pre-training Architecture for Dense Retrieval
Gao, L., & Callan, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.75 · Cited by 256
(2022) · doi:10.18653/v1/2022.emnlp-main.669 · Cited by 177
Self-citationInformation Retrieval: Recent Advances and Beyond
Hambarde, K., & Proença, H. P. M. C. (2023) · IEEE Access · doi:10.1109/access.2023.3295776 · Cited by 212
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking
Ren, R., Qu, Y., Liu, J., Zhao, W. X., She, Q., Wu, H. R., Wang, H., & Wen, J.-R. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.224 · Cited by 177
Generation-Augmented Retrieval for Open-Domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.316 · Cited by 146
PARADE: Passage Representation Aggregation forDocument Reranking
Li, C., Yates, A., MacAvaney, S., He, B., & Sun, Y. (2023) · ACM Transactions on Information Systems · doi:10.1145/3600088 · Cited by 184
Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges
Wang, J., Huang, J. X., Tu, X., Wang, J., Huang, A. J., Laskar, M. T. R., & Bhuiyan, A. (2024) · ACM Computing Surveys · doi:10.1145/3648471 · Cited by 156
COCO-LM: Correcting and Contrasting Text Sequences for Language Model PretrainingPreprint
Yu, M., Xiong, C., Bajaj, P., Tiwary, S., Bennett, P. N., Han, J., & Song, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08473 · Cited by 143
Semantic Models for the First-Stage Retrieval: A Comprehensive Review
Guo, J., Cai, Y., Fan, Y., Sun, F., Zhang, R., & Cheng, X. (2022) · ACM Transactions on Information Systems · doi:10.1145/3486250 · Cited by 153
In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval
Lin, S.-C., Yang, J.-H., & Lin, J. (2021) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2021.repl4nlp-1.17 · Cited by 156
RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder
Xiao, S., Liu, Z., Shao, Y., & Cao, Z. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.35 · Cited by 175
What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs?
Gao, S., Wen, X.-C., Gao, C., Wang, W., Zhang, H., & Lyu, M. R. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00109 · Cited by 106
Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation
Ma, J., Korotkov, I., Yang, Y., Hall, K., & McDonald, R. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.92 · Cited by 104
Distilling Knowledge from Reader to Retriever for Question Answering
Izacard, G., & Grave, E. (2020) · arXiv · doi:10.48550/arxiv.2012.04584 · Cited by 102
Few-Shot Conversational Dense Retrieval
Yu, S., Liu, Z., Xiong, C., Feng, T., & Liu, Z. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462856 · Cited by 116
SparTerm: Learning Term-based Sparse Representation for Fast Text RetrievalPreprint
Bai, Y., Li, X., Wang, G., Zhang, C., Shang, L., Xu, J., Wang, Z., Wang, F., & Liu, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00768 · Cited by 154
Distilling Dense Representations for Ranking using Tightly-Coupled TeachersPreprint
Lin, S.-C., Yang, J.-H., & Lin, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11386 · Cited by 141
Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance
Zhan, J., Mao, J., Liu, Y., Guo, J., Zhang, M., & Ma, S. (2021) · Proceedings of the 30th ACM International Conference on Information & Knowledge Management · doi:10.1145/3459637.3482358 · Cited by 91
Complement Lexical Retrieval Model with Semantic Residual Embeddings
Gao, L., Dai, Z., Chen, T., Fan, Z., Durme, B. V., & Callan, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72113-8_10 · Cited by 100
An Analysis of Fusion Functions for Hybrid Retrieval
Bruch, S., Gai, S., & Ingber, A. (2023) · ACM Transactions on Information Systems · doi:10.1145/3596512 · Cited by 118
GraphFormers: GNN-nested Transformers for Representation Learning on Textual GraphPreprint
Yang, J., Liu, Z., Xiao, S., Li, C., Lian, D., Agrawal, S., Singh, A., Sun, G., & Xie, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.02605 · Cited by 89
Pseudo-Relevance Feedback for Multiple Representation Dense Retrieval
Wang, X., Macdonald, C., Tonellotto, N., & Ounis, I. (2021) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3471158.3472250 · Cited by 82
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
Yu, H., Xiong, C., & Callan, J. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482124 · Cited by 64
Pre-training Methods in Information Retrieval
Fan, Y., Xie, X., Cai, Y., Chen, J., Ma, X., Li, X., Zhang, R., & Guo, J. (2022) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000100 · Cited by 69
Complementing Lexical Retrieval with Semantic Residual EmbeddingPreprint
Gao, L., Dai, Z., Chen, T., Fan, Z., Van Durme, B., & Callan, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.13969 · Cited by 73
PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval
Ren, R., Lv, S., Qu, Y., Liu, J., Zhao, W. X., She, Q., Wu, H., Wang, H., & Wen, J.-R. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.191 · Cited by 76
End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question AnsweringPreprint
Sachan, D. S., Reddy, S., Hamilton, W. L., Dyer, C., & Yogatama, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05346 · Cited by 71
Adversarial Retriever-Ranker for dense text retrievalPreprint
Zhang, H., Gong, Y., Shen, Y., Lv, J., Duan, N., & Chen, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03611 · Cited by 61
Pre-trained Language Model for Web-scale Retrieval in Baidu Search
Liu, Y., Lu, W., Cheng, S., Shi, D., Wang, S., Cheng, Z., & Yin, D. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467149 · Cited by 68
Deep Cross-Modal Hashing Based on Semantic Consistent Ranking
Liu, X., Zeng, H., Shi, Y., Zhu, J., Hsia, C., & Ma, K. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3254199 · Cited by 58
CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning
Wu, Z., Luan, Y., Rashkin, H., Reitter, D., Hajishirzi, H., Mari Ostendorf, E., & Tomar, G. S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.679 · Cited by 73
Self-citationThe Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes
Reimers, N., & Gurevych, I. (2021) · doi:10.18653/v1/2021.acl-short.77 · Cited by 71
Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder
Lu, S., He, D., Xiong, C., Ke, G., Malik, W., Dou, Z., Bennett, P., Liu, T.-Y., & Overwijk, A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.220 · Cited by 49
Multi-Task Retrieval for Knowledge-Intensive Tasks
Maillard, J., Karpukhin, V., Petroni, F., Yih, W.-T., Oğuz, B., Stoyanov, V., & Ghosh, G. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.89 · Cited by 67
CoSENT: Consistent Sentence Embedding via Similarity Ranking
Huang, X., Peng, H., Zou, D., Liu, Z., Li, J., Liu, K., Wu, J. X., Su, J., & Yu, P. S. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3402087 · Cited by 53
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., & Gurevych, I. (2021) · arXiv · doi:10.48550/arxiv.2104.08663 · Cited by 45
Multi-View Document Representation Learning for Open-Domain Dense Retrieval
Zhang, S., Liang, Y., Gong, M., Jiang, D., & Duan, N. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.414 · Cited by 54
ColBERT-PRF: Semantic Pseudo-Relevance Feedback for Dense Passage and Document Retrieval
Wang, X., Macdonald, C., Tonellotto, N., & Ounis, I. (2022) · ACM Transactions on the Web · doi:10.1145/3572405 · Cited by 45
Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation
Hofstätter, S., Althammer, S., Schröder, M., Sertkan, M., & Hanbury, A. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4068215 · Cited by 45
Query Embedding Pruning for Dense Retrieval
Tonellotto, N., & Macdonald, C. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482162 · Cited by 46
Understanding Hard Negatives in Noise Contrastive Estimation
Zhang, W., & Stratos, K. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.86 · Cited by 43
Promptagator: Few-shot Dense Retrieval From 8 Examples
Dai, Z., Zhao, V. Y., Ma, J., Luan, Y., Ni, J., Lu, J., Bakalov, A., Guu, K., Hall, K. B., & Chang, M.-W. (2022) · arXiv · doi:10.48550/arxiv.2209.11755 · Cited by 43
Self-citationDealing with Typos for BERT-based Passage Retrieval and Ranking
Zhuang, S., & Zuccon, G. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.225 · Cited by 49
AutoKG: Efficient Automated Knowledge Graph Generation for Language Models
Chen, B., & Bertozzi, A. L. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386454 · Cited by 55
Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection
Arabzadeh, N., Yan, X., & Clarke, C. L. A. (2021) · Proceedings of the 30th ACM International Conference on Information & Knowledge Management · doi:10.1145/3459637.3482159 · Cited by 41
Curriculum Contrastive Context Denoising for Few-shot Conversational Dense Retrieval
Mao, K., Dou, Z., & Qian, H. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531961 · Cited by 40
Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?
Chen, X., Lakhotia, K., Oğuz, B., Gupta, A., Lewis, P., Peshterliev, S., Mehdad, Y., Gupta, S., & Yih, W.-T. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.19 · Cited by 40
Phrase Retrieval Learns Passage Retrieval, Too
Lee, J., Wettig, A., & Chen, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.297 · Cited by 41
On Approximate Nearest Neighbour Selection for Multi-Stage Dense Retrieval
Macdonald, C., & Tonellotto, N. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482156 · Cited by 37
Magnani, A., Liu, F., Chaidaroon, S., Yadav, S., Suram, P. R., Puthenputhussery, A., Chen, S., Xie, M., Kashi, A., Lee, T., et al. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539164 · Cited by 47
R2-D2: A Modular Baseline for Open-Domain Question Answering
Fajcik, M., Docekal, M., Ondrej, K., & Smrz, P. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.73 · Cited by 36
Out-of-Domain Semantics to the Rescue! Zero-Shot Hybrid Retrieval Models
Chen, T., Zhang, M., Lu, J. R., Bendersky, M., & Najork, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99736-6_7 · Cited by 43
Learning to retrieve passages without supervision
Ram, O., Shachaf, G., Levy, O., Berant, J., & Globerson, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.193 · Cited by 35
Questions Are All You Need to Train a Dense Passage Retriever
Sachan, D. S., Lewis, M., Yogatama, D., Zettlemoyer, L., Pineau, J., & Zaheer, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00564 · Cited by 35
PARM: A Paragraph Aggregation Retrieval Model for Dense Document-to-Document Retrieval
Althammer, S., Hofstätter, S., Sertkan, M., Verberne, S., & Hanbury, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99736-6_2 · Cited by 36
A Confusion Method for the Protection of User Topic Privacy in Chinese Keyword-based Book Retrieval
Wu, Z., Xie, J., Shen, S., Lin, C., Xu, G., & Chen, E. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3571731 · Cited by 34
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingPreprint
Ren, R., Qu, Y., Liu, J., Zhao, W. X., She, Q., Wu, H. R., Wang, H., & Wen, J.-R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07367 · Cited by 34
A Memory Efficient Baseline for Open Domain Question AnsweringPreprint
Izacard, G., Petroni, F., Hosseini, L., Cao, N. D., Riedel, S., & Grave, É. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15156 · Cited by 47
Passage Retrieval for Outside-Knowledge Visual Question Answering
Qu, C., Zamani, H., Yang, L., Croft, W. B., & Miller, E. G. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462987 · Cited by 38
Generation-Augmented Retrieval for Open-domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2020) · arXiv · doi:10.48550/arxiv.2009.08553 · Cited by 31
Uni-Retriever: Towards Learning the Unified Embedding Based Retriever in Bing Sponsored Search
Zhang, J., Liu, Z., Han, W., Xiao, S., Zheng, R., Shao, Y., Sun, H., Zhu, H., Srinivasan, P., Deng, W., et al. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539212 · Cited by 36
Introducing Neural Bag of Whole-Words with ColBERTer
Hofstätter, S., Khattab, O., Althammer, S., Sertkan, M., & Hanbury, A. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557367 · Cited by 44
Learning To Retrieve: How to Train a Dense Retrieval Model Effectively and EfficientlyPreprint
Zhan, J., Mao, J., Liu, Y., Zhang, M., & Ma, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10469 · Cited by 30
Semantic-Enhanced Differentiable Search Index Inspired by Learning Strategies
Tang, Y., Zhang, R., Guo, J., Chen, J., Zhu, Z., Wang, S., Yin, D., & Cheng, X. (2023) · Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3580305.3599903 · Cited by 29
Scaling Laws For Dense Retrieval
Fang, Y., Zhan, J., Ai, Q., Mao, J., Su, W., Chen, J., & Liu, Y. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657743 · Cited by 28
More Robust Dense Retrieval with Contrastive Dual Learning
Li, Y., Liu, Z., Xiong, C., & Liu, Z. (2021) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3471158.3472245 · Cited by 28
Liu, J., Kang, Y., Tang, D., Song, K., Sun, C., Wang, X., Lu, W. P., & Liu, X. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560683 · Cited by 28
RetGen: A Joint Framework for Retrieval and Grounded Text Generation Modeling
Zhang, Y., Sun, S., Gao, X., Fang, Y., Brockett, C., Galley, M., Gao, J., & Dolan, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21429 · Cited by 27
Kong, W., Khadanga, S., Li, C., Gupta, S. K., Zhang, M., Xu, W., & Bendersky, M. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539137 · Cited by 27
Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networks
Louis, A., DIJCK, G. V., & Spanakis, G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.203 · Cited by 37
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Zhou, K., Gong, Y., Liu, X., Zhao, W. X., Shen, Y., Dong, A., Lü, J., Majumder, R., Wen, J.-R., & Duan, N. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-industry.56 · Cited by 29
Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval
Tang, H., Sun, X., Jin, B., Wang, J., Zhang, F., & Wu, W. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.392 · Cited by 26
Challenges in Generalization in Open Domain Question Answering
Liu, L., Lewis, P., Riedel, S., & Stenetorp, P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.155 · Cited by 34
A Free Format Legal Question Answering System
Khazaeli, S., Punuru, J. R., Morris, C., Sharma, S., Staub, B., Cole, M., Chiu-Webster, S., & Sakalley, D. (2021) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/2021.nllp-1.11 · Cited by 29
Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering
Chuang, Y.-S., Fang, W., Li, S.-W. D., Yih, W.-T., & Glass, J. R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.768 · Cited by 43
Joint Passage Ranking for Diverse Multi-Answer Retrieval
Min, S., Lee, K., Chang, M.-W., Toutanova, K., & Hajishirzi, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.560 · Cited by 39
Self-citationTowards Effective and Efficient Sparse Neural Information Retrieval
Formal, T., Lassance, C., Piwowarski, B., & Clinchant, S. (2023) · ACM Transactions on Information Systems · doi:10.1145/3634912 · Cited by 30
TravelRAG: A Tourist Attraction Retrieval Framework Based on Multi-Layer Knowledge Graph
Song, S., Yang, C., Li, X., Shang, H., Li, Z., & Chang, Y. (2024) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi13110414 · Cited by 29
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations
Xin, J., Xiong, C., Srinivasan, A., Sharma, A., Jose, D., & Bennett, P. N. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.316 · Cited by 33
LFR-GAN: Local Feature Refinement based Generative Adversarial Network for Text-to-Image Generation
Deng, Z., He, X., & Peng, Y. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3589002 · Cited by 25
Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity
Mysore, S., Cohan, A., & Hope, T. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.331 · Cited by 24
A Dense Representation Framework for Lexical and Semantic Matching
Lin, S.-C., & Lin, J. (2023) · ACM Transactions on Information Systems · doi:10.1145/3582426 · Cited by 23
MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering ModelsPreprint
Guo, M., Yang, Y., Cer, D. M., Shen, Q., & Constant, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02507 · Cited by 26
Retrieval And Structuring Augmented Generation with Large Language Models
Jiang, P., Ouyang, S., Jiao, Y., Zhong, M., Tian, R., & Han, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736557 · Cited by 28
Zerveas, G., Rekabsaz, N., Cohen, D., & Eickhoff, C. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531891 · Cited by 22
Persona-Based Conversational AI: State of the Art and Challenges
Liu, J., Symons, C. T., & Vatsavai, R. R. (2022) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw58026.2022.00129 · Cited by 25
Improving Wikipedia verifiability with AI
Petroni, F., Broscheit, S., Piktus, A., Lewis, P., Izacard, G., Hosseini, L., Dwivedi-Yu, J., Lomelí, M., Schick, T., Bevilacqua, M., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00726-1 · Cited by 31
Analysing the Robustness of Dual Encoders for Dense Retrieval Against Misspellings
Sidiropoulos, G., & Kanoulas, E. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531818 · Cited by 22
Trial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision
Wang, Z., & Sun, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.476 · Cited by 29
Metadata
- Authors
- Luan, Y., Eisenstein, J., Toutanova, K., & Collins, M.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Luan, Yi
- Eisenstein, Jacob
- Toutanova, Kristina
- Collins, Michael
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2021
- Volume
- 9
- Pages
- 329-345
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W3157758108
- Cited by
- 363
- References
- 61
- Citations by year
- 2026: 46 · 2025: 48 · 2024: 39 · 2023: 81 · 2022: 69 · 2021: 61
- FWCI
- 56.52
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- text retrieval, dual-encoder models, sparse retrieval, bag-of-words model, document length, retrieval precision, large-scale retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
