The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking
Open accessAbstract
In neural Information Retrieval, ongoing research is directed towards improving the first retriever in ranking pipelines. Learning dense embeddings to conduct retrieval using efficient approximate nearest neighbors methods has proven to work well. Meanwhile, there has been a growing interest in learning sparse representations for documents and queries, that could inherit from the desirable properties of bag-of-words models such as the exact matching of terms and the efficiency of inverted indexes. In this work, we present a new first-stage ranker based on explicit sparsity regularization and a log-saturation effect on term weights, leading to highly sparse representations and competitive results with respect to state-of-the-art dense and sparse methods. Our approach is simple, trained end-to-end in a single stage. We also explore the trade-off between effectiveness and efficiency, by controlling the contribution of the sparsity regularization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Khattab, O., & Zaharia, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401075 · Cited by 1,301
Nogueira, R., & Cho, K. (2019) · arXiv · doi:10.48550/arxiv.1901.04085 · Cited by 615
Overview of the TREC 2019 deep learning track
Craswell, N., Mitra, B., Yilmaz, E., Campos, D., & Voorhees, E. M. (2020) · arXiv · doi:10.48550/arxiv.2003.07820 · Cited by 68
Research works printed but not linked: 21
- Yang Bai, Xiaoguang Li, Gang Wang, Chaoliang Zhang, Lifeng Shang, Jun Xu, Zhaowei Wang, Fangshan Wang, and Qun Liu. 2020. SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval. arxiv: 2010.00768 [cs.IR] · arXiv
- Zhuyun Dai and Jamie Callan. 2019. Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval. arxiv: 1910.10687 [cs.IR] · arXiv
- 10.1145/3366423.3380258 · DOI
- 10.1145/3397271.3401204 · DOI
- 10.1145/1008992.1009004 · DOI
- Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrieval-Augmented Language Model Pre-Training. arxiv: 2002.08909 [cs.CL] · arXiv
- Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arxiv: 2010.02666 [cs.IR] · arXiv
- Vladimir Karpukhin, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. arxiv: 2004.04906 [cs.CL] · arXiv
- 10.1145/2833157.2833162 · DOI · Paper page
- Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2020. Distilling Dense Representations for Ranking using Tightly-Coupled Teachers. arxiv: 2010.11386 [cs.IR] · arXiv
- 10.1145/3397271.3401262 · DOI
- Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document Expansion by Query Prediction. arxiv: 1904.08375 [cs.IR] · arXiv
- Biswajit Paria, Chih-Kuan Yeh, Ian E. H. Yen, Ning Xu, Pradeep Ravikumar, and Barnabás Póczos. 2020. Minimizing FLOPs to Learn Efficient Sparse Representations. arxiv: 2004.05665 [cs.LG] · arXiv
- Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury Gregory Chanan Trevor Killeen Zeming Lin Natalia Gimelshein Luca Antiga et al. 2019. PyTorch: An Imperative Style High-Performance Deep Learning Library.. In NeurIPS .
- 10.1145/3409256.3409818 · DOI
- Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. HuggingFace's Transformers: State-of-the-art Natural Language Processing. arxiv: 1910.03771 [cs.CL] · arXiv
- Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. arxiv: 2007.00808 [cs.IR] · arXiv
- Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwikj. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. In International Conference on Learning Representations. https://openreview.net/forum?id=zeFrfgyZln
- 10.1145/3331184.3331340 · DOI
- 10.1145/3269206.3271800 · DOI
- Tiancheng Zhao, Xiaopeng Lu, and Kyusong Lee. 2020. SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval. arxiv: 2009.13013 [cs.CL] · arXiv
Cited by
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., & Zaharia, M. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.272 · Cited by 353
Query2doc: Query Expansion with Large Language Models
Wang, L., Yang, N., & Wei, F. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.585 · Cited by 214
Dense Text Retrieval Based on Pretrained Language Models: A Survey
Zhao, W. X., Liu, J., Ren, R., & Wen, J.-R. (2024) · ACM Transactions on Information Systems · doi:10.1145/3637870 · Cited by 211
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
A survey on knowledge-oriented retrieval-augmented generation
Cheng, M., Luo, Y., Ouyang, J., Liu, Q., Liu, H., Li, L., Yu, S., Zhang, B., Cao, J., Ma, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.10677 · Cited by 140
Scalable and Effective Generative Information Retrieval
Zeng, H., Luo, C., Jin, B., Sarwar, S. M., Wei, T., & Zamani, H. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645477 · Cited by 32
Scaling Laws For Dense Retrieval
Fang, Y., Zhan, J., Ai, Q., Mao, J., Su, W., Chen, J., & Liu, Y. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657743 · Cited by 28
RA-DIT: Retrieval-Augmented Dual Instruction Tuning
Lin, X. V., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01352 · Cited by 26
Zeng, H., Luo, C., & Zamani, H. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657746 · Cited by 22
“To filter out those useful tokens, the L1 [71] or FLOPs [18, 19, 46] regular-izer will be incorporated into the objective function to sparsify the high-dimension vectors.”
BackgroundMethodInfluentialSparseEmbed: Learning Sparse Lexical Representations with Contextual Embeddings for Retrieval
Kong, W., Dudek, J. M., Li, C., Zhang, M., & Bendersky, M. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3592065 · Cited by 14
Kazi, S., Khoja, S., & Daud, A. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100756 · Cited by 9
LLM-augmented multi-agent cooperative framework for medical case retrieval in cardiology
Deng, L., Hu, H., Lu, K., & He, P. (2025) · Journal of King Saud University Computer and Information Sciences · doi:10.1007/s44443-025-00311-z · Cited by 8
Gilani, K., Verket, M., Peters, C., Dumontier, M., Rocca, H.-P. B.-L., & Urovi, V. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110745 · Cited by 5
Li, Y., Fu, X., Verma, G., Buitelaar, P., & Liu, M. (2025) · arXiv · doi:10.48550/arxiv.2510.24476 · Cited by 4
Lightweight and Direct Document Relevance Optimization for Generative Information Retrieval
Mekonnen, K. A., Tang, Y., & de Rijke, M. (2025) · Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3726302.3730023 · Cited by 3
“Building on the success of transformer-based pre-trained language models, recent research has explored various neural retrieval approaches [61]: learned sparse retrieval [15, 16], dense retrieval [20, 60, 64, 65], and cross-encoders [37].”
Metadata
- Authors
- Formal, T., Piwowarski, B., & Clinchant, S.
- Venue
- Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2021-07-11
- Pages
- 2288-2292
- OpenAlex
- W3154280800
- Cited by
- 394
- References
- 6
- FWCI
- 40.89
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
