Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations
Open accessThe DOI matches its Crossref record; 39 of the 41 references it cites were found in a registry or an index.
Abstract
Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations. It aims to provide effective, reproducible, and easy-to-use first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. Around this toolkit, our group has built a culture of reproducibility through shared norms and tools that enable rigorous automated testing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Interestingly, the classic BM25 model [18] remains competitive, surpassing some semantic models trained on search behavior datasets.”
From Generative Agents Navigating Digital Libraries (2025) · cites this paper as [18]
In the citing paper
“Sparse lexical methods, from classical BM25 [Lin et al., 2021] to lightweight full-text indexes such as SQLite FTS5, complement dense retrieval by matching exact terms.”
From MemX: A Local-First Long-Term Memory System for AI Assistants (2026) · cites this paper as Lin et al., 2021
Citations
Cites
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“This is particularly the case for work on deep learning today, since the major toolkits (PyTorch [29] and Tensorflow [1]) have both adopted Python as their front-end This work is licensed under a Creative Commons Attribution International 4.”
BackgroundTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“also provides document expansion baselines using our doc2query method [28]; the latest model uses T5 [33] as described in Nogueira and Lin [26].”
MethodTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140 · cited 3×
“Row (2a) refers to brute-force scans over the document vectors in Faiss [12], which provides exact nearest-neighbor search.”
BackgroundMethodDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Malkov, Y. A., & Yashunin, D. A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889473 · Cited by 1,904 · cited 3×
“To support dense and hybrid retrieval, Pyserini integrates Facebook’s Faiss library for efficient similarity search over dense vectors [12], which in turns integrates the HNSW library [23] to support low-latency querying.”
MethodColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Khattab, O., & Zaharia, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401075 · Cited by 1,301 · cited 8×
“Row (3a) shows the results of combining TCT-ColBERT with BM25 bag-of-words search over the original texts, while row (3b) shows results that combine document expansion using doc2query with the T5 model.”
MethodInfluential
Research works printed but not linked: 6
- Sebastian Hofstatter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2020). · arXiv
- Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2021. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2021). · arXiv
- Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, and Nazli Goharian. 2021. Simplified Data Wrangling with textttir_datasets. arXiv:2103.02280 (2021). · arXiv
- Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. 2021 c. The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models. arXiv:2101.05667 (2021). · arXiv
- Chenyan Xiong, Zhenghao Liu, Si Sun, Zhuyun Dai, Kaitao Zhang, Shi Yu, Zhiyuan Liu, Hoifung Poon, Jianfeng Gao, and Paul Bennett. 2020 a. CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search. arXiv:2011.01580 (2020). · arXiv
- Yongze Yu, Jussi Karlgren, Hamed Bonab, Ann Clifton, Md Iftekhar Tanveer, and Rosie Jones. 2020. Spotify at the TREC 2020 Podcasts Track: Segment Retrieval. In Proceedings of the Twenty-Ninth Text REtrieval Conference (TREC 2020).
Cited by
Items 101–200 of 809
Improving Wikipedia verifiability with AI
Petroni, F., Broscheit, S., Piktus, A., Lewis, P., Izacard, G., Hosseini, L., Dwivedi-Yu, J., Lomelí, M., Schick, T., Bevilacqua, M., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00726-1 · Cited by 44
“For example, verification requires the ability to detect logical entailment in natural language and to convert claims and their context to the best search term for finding evidence—two long-standing problems that have been primarily investigated in somewhat synthetic settings (Bowman et al., 2015;…”
BackgroundRewriting the Code: A Simple Method for Large Language Model Augmented Code Search
Li, H., Zhou, X., & Shen, Z. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.75 · Cited by 43
Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modality
Ma, X., Gao, L., Zhuang, S., Zhan, J. S., Callan, J., & Lin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730135 · Cited by 45
Self-citationir_metadata: An Extensible Metadata Schema for IR Experiments
Breuer, T., Keller, J., & Schaer, P. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531738 · Cited by 42
“Pyserini was introduced for a better integration of Anserini and modern deep learning frameworks that are usually implemented with Python [44].”
MethodLi, H., Zhuang, S., Mourad, A., Ma, X., Lin, J., & Zuccon, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99736-6_40 · Cited by 42
Self-citationZero-Shot Cross-Lingual Reranking with Large Language Models for Low-Resource Languages
Adeyemi, M., Oladipo, A., Pradeep, R., & Lin, J. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-short.59 · Cited by 41
Self-citationBASES: Large-scale Web Search User Simulation with Large Language Model based Agents
Ren, R., Qiu, P., Qu, Y., Liu, J., Zhao, X., Wu, H. Y., Wen, J.-R., & Wang, H. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.50 · Cited by 41
Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers?
Li, M., Zhuang, H., Hui, K., Qin, Z., Lin, J., Jagerman, R., Wang, X., & Bendersky, M. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657979 · Cited by 40
“The improvement on TREC DL and Wiki+News BEIR datasets are more significant compared to other datasets in BEIR, which results from the instruction fine-tuning step of PaLM2 as previously mentioned.”
MethodSelf-citationAn Investigation of Prompt Variations for Zero-Shot LLM-Based Rankers
Sun, S., Zhuang, S., Wang, S., & Zuccon, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_12 · Cited by 40
“Task Instruction and Output Type: We analyse task instructions (TI) and output types (OT) together due to their inherent interrelation; OT largely depends on the ranking algorithm that the TI implements.”
Ma, X., Li, M., Sun, K., Xin, J., & Lin, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.227 · Cited by 40
“All experiments are implemented using the Pyserini IR toolkit (Lin et al., 2021).”
MethodSelf-citationMa, N., Zhang, Q., Murai, F., Braham, W. W., & Samuelson, H. W. (2023) · Building and Environment · doi:10.1016/j.buildenv.2023.110319 · Cited by 40
Kim, Y., Choi, Y., Choi, E., Choi, J., Park, H. J., & Hwang, W. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.319 · Cited by 40
Self-Calibrated Listwise Reranking with Large Language Models
Ren, R., Wang, Y., Zhou, K., Zhao, W. X., Wang, W., Liu, J., Wen, J.-R., & Chua, T. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714658 · Cited by 39
“Here, we compare our proposed approach against numerouscompetitivererankingbaselines,includingasparsereranker BM25, zero-shot rerankers, and fine-tuned rerankers.”
Overview of Touché 2022: Argument Retrieval
Bondarenko, A., Fröbe, M., Kiesel, J., Syed, S., Gurcke, T., Beloucif, M., Panchenko, A., Biemann, C., Stein, B., Wachsmuth, H., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13643-6_21 · Cited by 39
Tevatron: An Efficient and Flexible Toolkit for Neural Retrieval
Gao, L., Ma, X., Lin, J., & Callan, J. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591805 · Cited by 39
“For sparse retrieval, Tevatron directly utilizes Pyserini [15] to build inverted indexes for conducting efficient lexical search.”
MethodSelf-citationAccelerating Learned Sparse Indexes Via Term Impact Decomposition
Mackenzie, J., Mallia, A., Moffat, A., & Petri, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.205 · Cited by 38
“Sparse term importance representations such as DeepImpact (Mallia et al., 2021) and uniCOIL (Gao et al., 2021; Lin and Ma, 2021) have enabled the use of effective transformer-based text representations that can match the effectiveness of recent dense text representations (Karpukhin et al., 2020; Qu…”
BackgroundMethodAfriCLIRMatrix: Enabling Cross-Lingual Information Retrieval for African Languages
Ogundepo, O., Zhang, X., Sun, S., Duh, K., & Lin, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.597 · Cited by 38
“The use of multilingual models for CLIR often builds on techniques that have previously been applied to monolingual retrieval (Lin et al., 2021b).”
MethodSelf-citationListwise Generative Retrieval Models via a Sequential Learning Process
Tang, Y., Zhang, R., Guo, J., de Rijke, M., Chen, W., & Cheng, X. (2024) · ACM Transactions on Information Systems · doi:10.1145/3653712 · Cited by 37
“For BM25, we use the Pyserini [59] implementation for this baseline.”
MethodAbdallah, A., Piryani, B., Mozafari, J., Herzinger, A., Holdcroft, J., & Jatowt, A. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-demo.21 · Cited by 37
“Built in Python, it leverages machine learning libraries such as TensorFlow [1], PyTorch [79], Spacy [38], and Pyserini [58] .”
Vera: Prediction Techniques for Reducing Harmful Misinformation in Consumer Health Search
Pradeep, R., Ma, X., Nogueira, R. F., & Lin, J. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3463120 · Cited by 37
“All our experiments used the Pyserini IR toolkit [2, 10], which provides a Python interface to Anserini [24, 25], itself built on the popular open-source Lucene search engine.”
MethodSelf-citationEvaluating Embedding APIs for Information Retrieval
Kamalloo, E., Zhang, X., Ogundepo, O., Thakur, N., Alfonso-Hermelo, D., Rezagholizadeh, M., & Lin, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.50 · Cited by 37
Self-citationAn in-depth investigation on the behavior of measures to quantify reproducibility
Maistro, M., Breuer, T., Schaer, P., & Ferro, N. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103332 · Cited by 36
Askari, A., Aliannejadi, M., Abolghasemi, A., Kanoulas, E., & Verberne, S. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3614812 · Cited by 36
“𝑘 1 and 𝑏 are parameters with default values [28, 29].”
BackgroundEfficient Document-at-a-Time and Score-at-a-Time Query Evaluation for Learned Sparse Representations
Mackenzie, J., Trotman, A., & Lin, J. (2022) · ACM Transactions on Information Systems · doi:10.1145/3576922 · Cited by 36
“68, after tuning on a selection of training instances on the MS MARCO passage ranking test collection [40].”
BackgroundSelf-citationJina-ColBERT-v2: A General-Purpose Multilingual Late Interaction Retriever
Xiao, H., Wang, B., & Jha, R. (2024) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2024.mrl-1.11 · Cited by 35
Generating Multi-Aspect Queries for Conversational Search
Abbasiantaeb, Z., Lupart, S., & Aliannejadi, M. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.383 · Cited by 35
“For the first-stage retrieval we employ the BM25 model from Py-serini (Lin et al., 2021a) using the default values for the parameters.”
MethodMoro, G., & Valgimigli, L. (2021) · Sensors · doi:10.3390/s21196430 · Cited by 35
Cross-lingual Open-Retrieval Question Answering for African Languages
Ogundepo, O., Gwadabe, T., Rivera, C., Clark, J. R., Ruder, S., Adelani, D. I., Dossou, B. F. P., Diop, A., Sikasote, C., Hacheme, G., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.997 · Cited by 35
CodeRAG-Bench: Can Retrieval Augment Code Generation?
Wang, Z. Z., Asai, A., Yu, X. V., Xu, F. F., Xie, Y., Neubig, G., & Fried, D. (2024) · arXiv · doi:10.48550/arxiv.2406.14497 · Cited by 34
“Experimental setup For retrieval, we implement BM25 retrievers using pyserini (Lin et al., 2021) with parameter k 1 = 1 .”
Optimizing Query Generation for Enhanced Document Retrieval in RAGPreprint
Koo, H., Kim, M., & Hwang, S. J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.12325 · Cited by 34
“…q i , and documents set D = { d j } Jj =1 the BM25 alignment score is as follow, where f ( q i , D ) is frequency of query terms in the document D , | D | is the length of the document, AVG DL is average document length, and k 1 and b are default hyper-parameters from Pyserini (Lin et al., 2021).”
BackgroundInfluentialDoes RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation SystemsPreprint
Wu, X., Li, S. C., Wu, H.-T., Tao, Z., & Fang, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.19804 · Cited by 34
“In contrast, sparse retriever BM25, tends to introduce a larger bias towards female, suggesting that BM25’s sparse retrieval is more prone to favoring female content.”
InfluentialKostric, I., & Balog, K. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657933 · Cited by 33
“For sparse retrieval, following [2], we employ the Pyserini [17] toolkit and use BM25 for retreival with hyparameters 𝑘 1 = 0 .”
Document Expansion Baselines and Learned Sparse Lexical Representations for MS MARCO V1 and V2
Ma, X., Pradeep, R., Nogueira, R. F., & Lin, J. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531749 · Cited by 32
“This is accomplished via Pyserini, our Python toolkit for reproducible information retrieval research with sparse and dense representations [13].”
MethodSelf-citationSLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes
Li, M., Lin, S.-C., Ma, X., & Lin, J. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591977 · Cited by 32
“These models leverage current inverted index libraries such as Pyserini [20] and Elasticsearch to perform efficient sparse retrieval or even hybrid with dense retrieval [12, 22, 35].”
MethodSelf-citationUnderstanding Differential Search Index for Text Retrieval
Chen, X., Liu, Y., He, B., Sun, L., & Sun, Y. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.681 · Cited by 13
DMDD: A Large-Scale Dataset for Dataset Mentions Detection
Pan, H., Zhang, Q., Dragut, E. C., Caragea, C., & Latecki, L. J. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00592 · Cited by 32
“In Pyserini, sparse retrieval is based on BM25 and uses bag of word representations, while dense retrieval employs transformer-encoded representations, with the encoder being ColBERTv2 (Santhanam et al., 2021).”
MethodREANO: Optimising Retrieval-Augmented Reader Models through Knowledge Graph Generation
Fang, J., Meng, Z., & MacDonald, C. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.115 · Cited by 32
AfriQA: Cross-lingual Open-Retrieval Question Answering for African LanguagesPreprint
Ogundepo, O., Gwadabe, T., Rivera, C. E., Clark, J. H., Ruder, S., Adelani, D. I., Dossou, B. F. P., Abdou Aziz, D. I. O. P., Sikasote, C., Hacheme, G., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06897 · Cited by 32
“Retrieval is performed using the Faiss Flat Index implementation provided by Pyserini.”
MethodInfluentialMeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation
Wang, Z. J., & Chau, D. H. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657662 · Cited by 31
To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers
Li, H., Wang, S., Zhuang, S., Mourad, A., Ma, X., Lin, J., & Zuccon, G. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531884 · Cited by 30
“We use the model implemented in Pyserini 4 [15] by the original authors; • DistilBERT KD TASB + VPRF (DBB+VPRF) : The application of the Rocchio VPRF from Li et al. [12] on top of DistilBERT KD TASB dense retriever.”
MethodInfluentialSelf-citationAnother Look at DPR: Reproduction of Training and Replication of Retrieval
Ma, X., Sun, K., Pradeep, R., Li, M., & Lin, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99736-6_41 · Cited by 30
“Code associated with our retrieval experiments is packaged in the Pyserini IR toolkit 2 [10] and code associated with our end-to-end QA experiments is part of the PyGaggle toolkit 3 for neural text ranking.”
MethodResultSelf-citationGenerative and Pseudo-Relevant Feedback for Sparse, Dense and Learned Sparse RetrievalPreprint
Mackie, I., Chatterjee, S., & Dalton, J. (2023) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.48550/arxiv.2305.07477 · Cited by 30
“We index the term vectors using Pyserini [20] and use their “impact” searcher for max-passage aggregation.”
MethodInfluentialConversational Gold: Evaluating Personalized Conversational Search System Using Gold Nuggets
Abbasiantaeb, Z., Lupart, S., Azzopardi, L. A., Dalton, J., & Aliannejadi, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730316 · Cited by 29
“The iKAT 2023 dataset included a Pyserini [27] index for the collection.”
R2MED: A Benchmark for Reasoning-Driven Medical Retrieval
Zhang, X., Li, L., Liu, Z., & Zhou, X. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.2138 · Cited by 29
FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents
Thakur, N., Lin, J., Havens, S., Carbin, M., Khattab, O., & Drozdov, A. (2025) · neural information processing systems · doi:10.52202/085713-5255 · Cited by 30
Self-citationPaRaDe: Passage Ranking using Demonstrations with LLMs
Drozdov, A., Zhuang, H., Dai, Z., Qin, Z., Rahimi, R., Wang, X., Alon, D., Iyyer, M., McCallum, A., Metzler, D., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.950 · Cited by 29
Retrieval augmented scientific claim verification
Liu, H., Soroush, A., Nestor, J. G., Park, E., Idnay, B., Fang, Y., Pan, J. Y., Liao, S., Bernard, M., Peng, Y., et al. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae021 · Cited by 30
Shaar, S., Haouari, F., Mansour, W. A., Hasanain, M., Babulkov, N., Alam, F., Martino, G. D. S., Elsayed, T., & Nakov, P. (2021) · Qatar University QSpace (Qatar University) · doi:10.48550/arxiv.2109.12987 · Cited by 29
RankMamba: Benchmarking Mamba's Document Ranking Performance in the Era of TransformersPreprint
Xu, Z. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.18276 · Cited by 29
ThinkQE: Query Expansion via an Evolving Thinking Process
Lei, Y., Shen, T., & Yates, A. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.965 · Cited by 28
Efficient Neural Ranking using Forward Indexes
Leonhardt, J., Rudra, K., Khosla, M., Anand, A., & Anand, A. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511955 · Cited by 28
Overview of the TREC 2024 NeuCLIR TrackPreprint
Lawrie, D., MacAvaney, S., Mayfield, J., McNamee, P., Oard, D. W., Soldaini, L., & Yang, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.14355 · Cited by 28
“The Patapsco framework [5] supports CLIR lexical retrieval through Pyserini [14].”
ShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based AgentsPreprint
Wang, J., Xiao, K., Sun, Q., Zhao, H., Luo, T., Zhang, J. D., & Zeng, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.04266 · Cited by 28
“We use Pyserini(Lin et al. 2021) to build a product search engine, utilizing the BM25 sparse retrieval model to construct the index offline.”
DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Search
Lupart, S., Aliannejadi, M., & Kanoulas, E. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3729966 · Cited by 27
Neural Query Synthesis and Domain-Specific Ranking Templates for Multi-Stage Clinical Trial Matching
Pradeep, R., Li, Y., Wang, Y., & Lin, J. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531853 · Cited by 27
“We accomplish this using the default parameters of the Pyserini IR Toolkit [5].”
MethodSelf-citationptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Language
Piau, M., Lotufo, R., & Nogueira, R. F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79032-4_23 · Cited by 28
Self-citationTowards Unified Alignment Between Agents, Humans, and EnvironmentPreprint
Yang, Z., Liu, A., Liu, Z., Liu, K., Xiong, F., Wang, Y., Yang, Z., Hu, Q., Chen, X., Zhang, Z., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.07744 · Cited by 28
“Specifically, on top of the Pyserini (Lin et al., 2021) search engine used in the original Webshop, we integrate a Collaborative Filtering (CF) (Sarwar et al., 2001) algorithm and a Determinantal Point Process (DPP) based algorithm (Chen et al., 2018) for fine-grained personalized re-ranking.”
MethodAlignment for Efficient Tool Calling of Large Language Models
Xu, H., Wang, Z., Zhu, Z., Pan, L., Chen, X., Fan, S., Chen, L., & Yu, K. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.898 · Cited by 26
Ye, Y., Feng, X., Feng, X., Ma, W., Qin, L., Xu, D., Yang, Q. K., Liu, H., & Qin, B. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.603 · Cited by 26
InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers
Boytsov, L., Patel, P., Sourabh, V., Nisar, R., Kundu, S., Ramanathan, R., & Nyberg, E. (2023) · Transactions on machine learning research · doi:10.48550/arxiv.2301.02998 · Cited by 27
“Comparing effectiveness of FlexNeuART [5] BM25 with effectiveness of Pyserini [23] BM25—used the InPars study [4]—we can see that on all datasets except TREC DL 2020 we closely match (within 1.5”
MethodInfluentialElmobark, N. (2025) · International Journal of Environment Engineering and Education · doi:10.55151/ijeedu.v7i1.188 · Cited by 25
“…flexibility and efficiency in handling large data volumes by dynamically allocating computational resources — more recent research by Lin et al. [74] has also proposed hybrid indexing techniques that combine traditional inverted indexes with deep learning-based retrieval mechanisms to further…”
Question Rewriting? Assessing Its Importance for Conversational Question Answering
Raposo, G., Ribeiro, R., Martins, B., & Coheur, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99739-7_23 · Cited by 25
“In order to incorporate relevant knowledge when answering the questions, our system uses a passage retrieval module built with Pyserini [6], i.e., an easy-to-use Python toolkit that allows searching over a document collection using sparse and dense representations.”
MethodTowards Query Performance Prediction for Neural Information Retrieval: Challenges and Opportunities
Faggioli, G., Formal, T., Lupart, S., Marchesin, S., Clinchant, S., Ferro, N., & Piwowarski, B. (2023) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3578337.3605142 · Cited by 25
“We then replace it with the SPLADE BoW representations of queries and documents, using the strategy proposed by [60] to use SPLADE together with classical index-based”
MethodResources for Brewing BEIR: Reproducible Reference Models and an Official LeaderboardPreprint
Kamalloo, E., Thakur, N., Lassance, C., Ma, X., Yang, J.-H., & Lin, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.07471 · Cited by 25
Self-citationJina-ColBERT-v2: A General-Purpose Multilingual Late Interaction RetrieverPreprint
Jha, R., Wang, B., Günther, M., Mastrapas, G., Sturua, S., Mohr, I., Koukounas, A., Akram, M. K., Wang, N., & Xiao, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.16672 · Cited by 25
In-Storage Acceleration of Retrieval Augmented Generation as a Service
Mahapatra, R., Santhanam, H., Priebe, C., Xu, H., & Esmaeilzadeh, H. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731032 · Cited by 24
Optimizing Guided Traversal for Fast Learned Sparse Retrieval
Qiao, Y., Yang, Y., Lin, H., & Yang, T. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583497 · Cited by 24
“For MS MARCO, the contents in the document collections are segmented during indexing and re-grouped after retrieval using “max-passage” strategy following [23].”
MethodRaLLe: A Framework for Developing and Evaluating Retrieval-Augmented Large Language Models
Hoshi, Y., Miyashita, D., Ng, Y., Tatsuno, K., Morioka, Y., Torii, O., & Deguchi, J. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.4 · Cited by 24
Huly, O., Pogrebinsky, I., Carmel, D., Kurland, O., & Maarek, Y. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657935 · Cited by 24
“The retrieval system (the Retriever) is based on the Pyserini package [16].”
Quati: A Brazilian Portuguese Information Retrieval Dataset from Native Speakers
Bueno, M. C., de Oliveira, E. S., Nogueira, R. F., Lotufo, R., & Pereira, J. (2024) · Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL) · doi:10.5753/stil.2024.245426 · Cited by 24
Self-citationQuestion Rewriting for Open-Domain Conversational QA: Best Practices and Limitations
Tredici, M. D., Barlacchi, G., Shen, X., Cheng, W., & de Gispert, A. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482164 · Cited by 24
“BERTserini is an end-to-end open-domain QA system that leverages BERT architecture for reading, and implements the retrieval with the open-source Pyserini toolkit [11].”
MethodNatural Logic-guided Autoregressive Multi-hop Document Retrieval for Fact Verification
Aly, R., & Vlachos, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.411 · Cited by 24
Query Expansion Using Contextual Clue Sampling with Language ModelsPreprint
Liu, L., Li, M., Lin, J., Riedel, S., & Stenetorp, P. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.07093 · Cited by 24
“…based on semantic matching for open-domain question answering such as DPR (Karpukhin et al., 2020), approaches based on lexical matching (e.g., BM25) remain important due to their space-efficiency and can serve as input to hybrid methods (Gao et al., 2021; Formal et al., 2021; Lin and Ma, 2021).”
MethodSelf-citationRaLLe: A Framework for Developing and Evaluating Retrieval-Augmented Large Language ModelsPreprint
Hoshi, Y., Miyashita, D., Ng, Y., Tatsuno, K., Morioka, Y., Torii, O., & Deguchi, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.10633 · Cited by 25
“We use the Pyserini (Lin et al., 2021) implementation of unigram BM25 with the default parameters of k 1 = 0 .”
MethodRAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented GenerationPreprint
Fleischer, D., Berchansky, M., Wasserblat, M., & Izsak, P. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.02545 · Cited by 24
“We note that different retrieval processes or frameworks (Liu, 2022; Chase, 2022; Lin et al., 2021) can be used in retrieval steps.”
MethodDrowning in Documents: Consequences of Scaling Reranker InferencePreprint
Jacob, M., Lindgren, E., Zaharia, M., Carbin, M., Khattab, O., & Drozdov, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.11767 · Cited by 27
“With this in mind, we develop a fair and simple protocol for comparing retrievers and rerankers: Rather than apply the reranker only on the top documents provided by the retriever, apply the reranker on all of the documents that are being retrieved from.”
Hybrid Retrieval for Hallucination Mitigation in Large Language Models: A Comparative AnalysisPreprint
Mala, C. S., Gezici, G., & Giannotti, F. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.05324 · Cited by 25
“…and semantic searches through query expansion and dynamic weighting as illustrated in Figure 1, aiming to maximize both precision and recall [37] and further examine its influence on LLM generated responses as illustrated in Figure 2 Hybrid retrieval approach Our hybrid retrieval process…”
Sun, S., Zhang, K., Li, J., Yu, M., Hou, K., Wang, Y., & Cheng, X. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104147 · Cited by 24
MedCite: Can Language Models Generate Verifiable Text for Medicine?
Wang, X., Tan, M., Jin, Q., Xiong, G., Hu, Y., Zhang, A., Lu, Z., & Zhang, M. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.967 · Cited by 23
Cherumanal, S. P., Tian, L., Abushaqra, F. M., de Paula, A. F. M., Ji, K., Ali, H., Hettiachchi, D., Trippas, J. R., Scholer, F., & Spina, D. (2024) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3627508.3638309 · Cited by 23
“75) and (ii) dense retrieval using Dense Passage Retrieval (DPR) [10] implementations in the pyserini toolkit [13].”
MethodYan, H., Wang, S., Liu, H., Mamitsuka, H., & Zhu, S. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae401 · Cited by 23
“Iterating this step, we obtain the set of candidates In order to reduce the time and space cost, we use a classical IR approach of BM25 (Lin et al. 2021) for measuring textual similarity.”
MethodEcoRank: Budget-Constrained Text Re-ranking Using Large Language Models
Rashid, M., Meem, J. A., Dong, Y., & Hristidis, V. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.773 · Cited by 23
Understanding and Mitigating the Threat of Vec2Text to Dense Retrieval Systems
Zhuang, S., Koopman, B., Chu, X., & Zuccon, G. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698414 · Cited by 24
Learning to Rank in the Age of Muppets: Effectiveness–Efficiency Tradeoffs in Multi-Stage Ranking
Zhang, Y., Hu, C., Liu, Y., Fang, H., & Lin, J. (2021) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2021.sustainlp-1.8 · Cited by 23
“The retrieved candidate passages are then sent to feature extraction through Anserini’s Python interface, Pyserini (Lin et al., 2021).”
MethodSelf-citationEfficient Neural Ranking Using Forward Indexes and Lightweight Encoders
Leonhardt, J., Müller, H., Rudra, K., Khosla, M., Anand, A., & Anand, A. (2023) · ACM Transactions on Information Systems · doi:10.1145/3631939 · Cited by 23
“We use the Pyserini [49] toolkit, which provides a number of pre-trained encoders (available on the HuggingFace Hub 1 ) and corresponding indexes (see Table 1), for our retrieval experiments.”
MethodZhuang, S., Ren, H., Shou, L., Pei, J., Gong, M., Zuccon, G., & Jiang, D. (2022) · arXiv · doi:10.48550/arxiv.2206.10128 · Cited by 22
“We also use the Pyserini implementation for this baseline.”
MethodShoppingBench: A Real-World Intent-Grounded Shopping Benchmark for LLM-based Agents
Wang, J., Xiao, K., Sun, Q., Zhao, H., Luo, T., Zhang, J. D., & Zeng, X. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i39.40640 · Cited by 22
How to Leverage Personal Textual Knowledge for Personalized Conversational Information Retrieval
Mo, F., Zhao, L., Huang, K., Dong, Y., Huang, D., & Nie, J. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679939 · Cited by 22
Consolidating Ranking and Relevance Predictions of Large Language Models through Post-Processing
Yan, L., Qin, Z., Zhuang, H., Jagerman, R., Wang, X., Bendersky, M., & Oosterhuis, H. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.25 · Cited by 22
AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark
Chen, J., Wang, N., Li, C., Wang, B., Xiao, S., Xiao, H., Liao, H., Lian, D., & Liu, Z. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.982 · Cited by 22
Validating Simulations of User Query Variants
Breuer, T., Fuhr, N., & Schaer, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99736-6_6 · Cited by 22
Bergen: A benchmarking library for retrieval-augmented generation
Rau, D., Déjean, H., Chirkova, N., Formal, T., Wang, S., Clinchant, S., & Nikoulina, V. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.449 · Cited by 20
Who can verify this? Finding authorities for rumor verification in Twitter
Haouari, F., Elsayed, T., & Mansour, W. A. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103366 · Cited by 20
Estimating Query Performance Through Rich Contextualized Query Representations
Ebrahimi, S., Khodabakhsh, M., Arabzadeh, N., & Bagheri, E. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56066-8_6 · Cited by 20
“For the retrieval method, we estimate BM25 implemented by Pyserini [27].”
MethodDyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities
Nguyen, T. D.-M., Chatterjee, S., MacAvaney, S., Mackie, I., Dalton, J., & Yates, A. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.45 · Cited by 20
WikiWhy: Answering and Explaining Cause-and-Effect QuestionsPreprint
Ho, M., Sharma, A., Chang, J. S., Saxon, M., Levy, S., Lu, Y., & Wang, W. Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.12152 · Cited by 20
“We experiment with BM25 (Robertson et al., 2009) and Dense Passage Retriever (DPR) (Karpukhin et al., 2020) as our document retriever, using their Pyserini implementations (Lin et al., 2021).”
MethodLEANN: A Low-Storage Vector IndexPreprint
Wang, Y., Li, Z., Liu, S., Wu, Y., Mao, Z., Zhao, Y., Xiao, Y., Xu, Z., Zhou, Y., Stoica, I. G., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.08276 · Cited by 20
Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval
Park, S.-W., Kim, T.-L., & Ko, Y. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.1345 · Cited by 19
An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
Liu, Q., Guo, G., Mao, J., Dou, Z., Wen, J.-R., Jiang, H., Zhang, X., & Cao, Z. (2024) · ACM Transactions on Information Systems · doi:10.1145/3639818 · Cited by 19
ESPN: Memory-Efficient Multi-vector Information Retrieval
Shrestha, S., Reddy, N., & Li, Z. (2024) · ACM SIGPLAN International Symposium on Memory Management (ISMM) · doi:10.1145/3652024.3665515 · Cited by 19
“We evaluated ColBERTv2 with different first stage retrievers using the models from Pyserini (Lin et al., 2021).”
Method
Metadata
- Authors
- Lin, J., Ma, X., Lin, S.-C., Yang, J.-H., Pradeep, R., & Nogueira, R.
- Affiliations
- University of Waterloo (CA)
- Venue
- Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2021-07-11
- Pages
- 2356-2362
- OpenAlex
- W3180230246
- Cited by
- 360
- References
- 41
- Citations by year
- 2026: 67 · 2025: 72 · 2024: 61 · 2023: 97 · 2022: 44 · 2021: 18
- FWCI
- 45.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion, cc-by)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Waterloo-Huawei Joint Innovation Laboratory
- Canada First Research Excellence Fund
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- reproducibility, sparse retrieval, dense retrieval, hybrid retrieval, BM25, transformer representations, nearest neighbor search, neural ranking, relevance assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.