Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations
Open accessThe DOI matches its Crossref record; 39 of the 41 references it cites were found in a registry or an index.
Abstract
Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations. It aims to provide effective, reproducible, and easy-to-use first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. Around this toolkit, our group has built a culture of reproducibility through shared norms and tools that enable rigorous automated testing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Interestingly, the classic BM25 model [18] remains competitive, surpassing some semantic models trained on search behavior datasets.”
From Generative Agents Navigating Digital Libraries (2025) · cites this paper as [18]
In the citing paper
“Sparse lexical methods, from classical BM25 [Lin et al., 2021] to lightweight full-text indexes such as SQLite FTS5, complement dense retrieval by matching exact terms.”
From MemX: A Local-First Long-Term Memory System for AI Assistants (2026) · cites this paper as Lin et al., 2021
Citations
Cites
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“This is particularly the case for work on deep learning today, since the major toolkits (PyTorch [29] and Tensorflow [1]) have both adopted Python as their front-end This work is licensed under a Creative Commons Attribution International 4.”
BackgroundTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“also provides document expansion baselines using our doc2query method [28]; the latest model uses T5 [33] as described in Nogueira and Lin [26].”
MethodTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140 · cited 3×
“Row (2a) refers to brute-force scans over the document vectors in Faiss [12], which provides exact nearest-neighbor search.”
BackgroundMethodDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Malkov, Y. A., & Yashunin, D. A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889473 · Cited by 1,904 · cited 3×
“To support dense and hybrid retrieval, Pyserini integrates Facebook’s Faiss library for efficient similarity search over dense vectors [12], which in turns integrates the HNSW library [23] to support low-latency querying.”
MethodColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Khattab, O., & Zaharia, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401075 · Cited by 1,301 · cited 8×
“Row (3a) shows the results of combining TCT-ColBERT with BM25 bag-of-words search over the original texts, while row (3b) shows results that combine document expansion using doc2query with the T5 model.”
MethodInfluential
Research works printed but not linked: 6
- Sebastian Hofstatter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2020). · arXiv
- Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2021. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2021). · arXiv
- Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, and Nazli Goharian. 2021. Simplified Data Wrangling with textttir_datasets. arXiv:2103.02280 (2021). · arXiv
- Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. 2021 c. The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models. arXiv:2101.05667 (2021). · arXiv
- Chenyan Xiong, Zhenghao Liu, Si Sun, Zhuyun Dai, Kaitao Zhang, Shi Yu, Zhiyuan Liu, Hoifung Poon, Jianfeng Gao, and Paul Bennett. 2020 a. CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search. arXiv:2011.01580 (2020). · arXiv
- Yongze Yu, Jussi Karlgren, Hamed Bonab, Ann Clifton, Md Iftekhar Tanveer, and Rosie Jones. 2020. Spotify at the TREC 2020 Podcasts Track: Segment Retrieval. In Proceedings of the Twenty-Ninth Text REtrieval Conference (TREC 2020).
Cited by
Items 301–400 of 809
Zero-Shot Dense Retrieval with Embeddings from Relevance FeedbackPreprint
Jedidi, N., Chuang, Y.-S., Shing, L., & Glass, J. R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.21242 · Cited by 10
“We use the hybrid retrieval implementation from Pyserini, which scores the document by a weighted average of the sparse retrieval and dense retrieval scores.”
MethodAttacking Open-domain Question Answering by Injecting Misinformation
Pan, L., Chen, W., Kan, M.-Y., & Wang, W. Y. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.35 · Cited by 9
“For document retrieval, we apply the widely-used sparse retrieval based on BM25, implemented with the Pyserini toolkit (Lin et al., 2021).”
MethodNonparametric Decoding for Generative Retrieval
Lee, H., Kim, J., Chang, H., Oh, H., Yang, S., Karpukhin, V., Lu, Y., & Seo, M. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.801 · Cited by 9
“For BM25, we use pyserini (Lin et al., 2021) where the corpus is the same as in DPR.”
MethodExploring the Representation Power of SPLADE Models
Mackenzie, J., Zhuang, S., & Zuccon, G. (2023) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3578337.3605129 · Cited by 9
“We use the Pyserini [14] toolkit to run BM25 with the tuned parameters 𝑘 1 = 0 .”
MethodDemographically-Inspired Query Variants Using an LLM
Alaofi, M., Ferro, N., Thomas, P., Scholer, F., & Sanderson, M. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744608 · Cited by 9
Bag-of-Words Baselines for Semantic Code Search
Zhang, X., Xin, J., Yates, A., & Lin, J. (2021) · doi:10.18653/v1/2021.nlp4prog-1.10 · Cited by 9
“The underlying implementation of BM25 and RM3 are provided by the Pyserini toolkit (Lin et al., 2021), which in turn is built on the Lucene open-source search library, but Capreolus provides simplified mechanisms for parameter tuning and other useful features for end-to-end experiments.”
MethodSelf-citationShen, T., Liu, M., Zhou, M., & Xiong, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.730 · Cited by 9
“The sparse baselines are implemented with Pyserini (Lin et al., 2021) using BM25 (Robertson et al., 2009), along with a “tuned” BM25 baseline which optimizes MRR@100 by tuning the default BM25 parameters k 1 and b on the development set.”
MethodRau, D., & Kamps, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56066-8_19 · Cited by 9
“We follow the original work and use the publicly available Pyserini Python toolkit [8] for initial retrieval with BM25 using the default parameters.”
Integrating Open-domain Knowledge via Large Language Model for Multimodal Fake News Detection
Xie, A., Zhu, F., Han, J., & Hu, S. (2024) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd61410.2024.10580818 · Cited by 10
“The BM25 [20] algorithm is then utilized to retrieve open-domain knowledge from a local snapshot of Wikipedia [3].”
MethodDense Retrieval with Continuous Explicit Feedback for Systematic Review Screening Prioritisation
Mao, X., Zhuang, S., Koopman, B., & Zuccon, G. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657921 · Cited by 9
Chandradevan, R., Dhole, K., & Agichtein, E. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.413 · Cited by 9
Milanese, G. C., Peikos, G., Pasi, G., & Viviani, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_17 · Cited by 9
Evaluating Large Language Models for Cross-Lingual Retrieval
Zuo, L., Hong, P., Kraus, O., Plank, B., & Litschko, R. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.612 · Cited by 9
Efficient and Accurate Contextual Re-Ranking for Knowledge Graph Question Answering
Sun, K., Jedema, N., Sharma, K., Janssen, R., Pujara, J., Szekely, P. A., & Moschitti, A. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/33r8uqf98tgy · Cited by 9
“We evaluate 3 retrieval techniques: BM25 (Robertson and Zaragoza, 2009), DPR (Karpukhin et al., 2020), and a balanced hybrid that sums BM25 and DPR scores, all implemented using Pyserini (Lin et al., 2021).”
MethodSPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval
Thakur, N., Wang, K., Gurevych, I., & Lin, J. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591902 · Cited by 9
“Here, Pyserini [23] which is based on Apache Lucene is becoming the de-facto standard.”
MethodSelf-citationHybrid and Collaborative Passage Reranking
Zhang, Z., Zhou, W., Shi, J., & Li, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.880 · Cited by 9
“These methods are implemented using RocketQA toolkit 4 and Pyserini toolkit (Lin et al., 2021a) which is built on Lucene and FAISS (Johnson et al., 2021).”
MethodInfluentialLearn to be Fair without Labels: A Distribution-based Learning Framework for Fair Ranking
Chen, F. L., & Fang, H. (2023) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3578337.3605132 · Cited by 9
“As all models are built on retrieval and re-rank, we acquire the initial ranking with top 500 documents per query utilizing the BM25 [24] model implemented by Pyserini 8 , a Python toolkit for reproducible information retrieval research [17].”
MethodKnowledge Rumination for Pre-trained Language Models
Yao, Y., Wang, P., Mao, S., Tan, C., Huang, F., Chen, H., & Zhang, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.206 · Cited by 9
Yang, J., Baldwin, T., & Cohn, T. (2023) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2023.mrl-1.21 · Cited by 9
“For this experiment, we use the default whitespace tokenizer in the Pyserini library.”
MethodSelf-citationTemplate Filling for Controllable Commonsense Reasoning
Rajagopal, D., Khetan, V., Sacaleanu, B., Gershman, A. V., Fano, A., & Hovy, E. H. (2023) · doi:10.18653/v1/2023.findings-ijcnlp.23 · Cited by 9
UPV at TREC Health Misinformation Track 2021 Ranking with SBERT and Quality EstimatorsPreprint
Schlicht, I. B., de Paula, A. F. M., & Rosso, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.06080 · Cited by 9
“This index was composed of 10k documents for each query, which were retrieved by the Pyserini implementation of BM25 (Yilmaz et al., 2020; Lin et al., 2021).”
MethodEnd-to-End Retrieval with Learned Dense and Sparse Representations Using LucenePreprint
Chen, H., Lassance, C., & Lin, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.18503 · Cited by 9
“In truth, both dense and sparse representations are indispensable, as the combination of evidence from both approaches leads to higher-quality output than either alone [Ma et al., 2021, Kamalloo et al., 2023].”
BackgroundSelf-citationCross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance LanguagesPreprint
Litschko, R., Kraus, O., Blaschke, V., & Plank, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.12806 · Cited by 9
Access Paths for Efficient Ordering with Large Language ModelsPreprint
Zhao, F., Chen, J., Pan, Y., Rabbani, T., Sohaib, Agrawal, D., Abbadi, A. E., Aggarwal, P., Datta, A., & Tsirogiannis, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.00303 · Cited by 9
Alignment for Efficient Tool Calling of Large Language ModelsPreprint
Xu, H., Wang, Z., Zhu, Z., Pan, L., Chen, X., Chen, L., & Yu, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.06708 · Cited by 9
“We follow the Pyserini setup for TriviaQA and utilize a sparse retriever to retrieve the top 100 highest-scoring passages.”
Melz, E. (2023) · arXiv · doi:10.48550/arxiv.2311.04177 · Cited by 8
“The retrieval model used is Pyserini (Lin et al., 2021), a Python information retrieval library that employs dense representations supported by the Faiss library (Meta, 2017).”
MethodGAR-meets-RAG Paradigm for Zero-Shot Information Retrieval
Arora, D., Kini, A., Chowdhury, S. R., Natarajan, N., Sinha, G., & Sharma, A. (2023) · arXiv · doi:10.48550/arxiv.2310.20158 · Cited by 8
Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion
Yoon, Y., Jung, J., Yoon, S., & Park, K. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.980 · Cited by 8
ICR: Iterative Clarification and Rewriting for Conversational Search
Cao, Z., Li, P., & Zhu, Q. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.496 · Cited by 8
On the Use of LLMs for Relevance Labelling
Alaofi, M., Thomas, P., Scholer, F., & Sanderson, M. (2026) · ACM Transactions on Information Systems · doi:10.1145/3788872 · Cited by 8
“Retrieval was conducted using Pyterrier [26], except for Doc2Query for which Pyserini [24] was used over a pre-built augmented corpus with doc2query-T5 expansions.”
Addressing Corpus Knowledge Poisoning Attacks on RAG Using Sparse Attention
Dekel, S., Tennenholtz, M., & Kurland, O. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1399 · Cited by 8
Yu, Z., Xiao, K., Zhao, H., Luo, T., & Zeng, X. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1669 · Cited by 8
Kim, J., Korikov, A., Liang, J., Cui, J., Liu, Y. S., Wen, Q., Zhao, M., & Sanner, S. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.481 · Cited by 9
ReAlign: Optimizing the Visual Document Retriever with Reasoning-Guided Fine-Grained Alignment
Yang, H., Ji, Y., Xu, Z., Liu, Z., Yan, Y., Chen, Z., Wang, S., Gu, Y., & Yu, G. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809602 · Cited by 8
Efficient Text-to-Code Retrieval with Cascaded Fast and Slow Transformer Models
Gotmare, A. D., Li, J., Joty, S., & Hoi, S. C. H. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616369 · Cited by 8
“MRR scores on the test set for the CodeBERT baseline ( fast encoder) along with several other baselines including sparse meth-ods like BM25 (implemented using Pyserini [31]), fine-tuned CNN, BiRNN, multi-head attention models are shown in Table 3.”
MethodContrastiveMix: Overcoming Code-Mixing Dilemma in Cross-Lingual Transfer for Information Retrieval
Do, J., Lee, J., & Hwang, S.-W. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-short.17 · Cited by 8
“We trained models based on Tevatron (Gao et al., 2023) 4 and evaluated them with Pyserini (Lin et al., 2021) 5 .”
MethodEnhancing Cross-Domain Recommender Systems with LLMs: Evaluating Bias and Beyond-Accuracy Measures
Kolb, T. E. (2024) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3640457.3688027 · Cited by 8
“We plan to evaluate various classes of recommendation algorithms, with a focus on content-based algorithms like BM-25 [45], by integrating Pyserini [36] and JinaAI [42].”
Informational ontological modeling
Tsvetkov, V. Y., & Сергеевич, К. Н. (2025) · Russian Technological Journal · doi:10.32362/2500-316x-2025-13-2-18-26 · Cited by 8
RankFlow: A Multi-Role Collaborative Reranking Workflow Utilizing Large Language Models
Jin, C., Peng, H., Zhang, A., Chen, N., Zhao, J., Xie, X., Li, K., Feng, S., Zhong, K.-W., Ding, C., et al. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3717575 · Cited by 8
“It not only surpasses all supervised methods but also outperforms SoTA zero-shot unsupervised methods, including the pointwise method RG, the pairwise method PRP, and the listwise method RankGPT-4.”
ir_explain: A Python Library of Explainable IR Methods
Saha, S., Agarwal, H., Venktesh, V., Anand, A., Mohanty, S., Majumdar, D., & Mitra, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730343 · Cited by 8
“Anserini [47], Pyserini [19], PyTerrier [25] and PyGaggle [27] are examples of modern IR toolkits that support both sparse and dense retrieval models, and provide easy access to the standard retrieve-and-rerank pipeline on TREC collections.”
Efficiency-Effectiveness Reranking FLOPs for LLM-based Rerankers
Peng, Z., Wei, T.-R., Song, T., & Zhao, Y. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-industry.186 · Cited by 8
Investigating LLM Variability in Personalized Conversational Information Retrieval
Lupart, S., Dijk, D. V., Langezaal, E., van Dort, I., & Aliannejadi, M. (2025) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3767695.3769502 · Cited by 8
Ward, A. R., Avula, S., Cheng, H.-F., Sarwar, S. M., Murdock, V., & Agichtein, E. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592057 · Cited by 8
“serini framework [13]; 2) a search result diversification component that we implemented the based on the Maximal Marginal Relevance (MMR) algorithm [2]; and 3) a Query Expansion (QE) component based on the double-metaphone algorithm [16].”
MethodGAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration
Piktus, A., Ogundepo, O., Akiki, C., Oladipo, A., Zhang, X., Schoelkopf, H., Biderman, S., Potthast, M., & Lin, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.57 · Cited by 8
Self-citationCertified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking
Li, M., Zhang, X., Xin, J., Zhang, H., & Lin, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.23 · Cited by 8
Self-citationAuRED: Enabling Arabic Rumor Verification using Evidence from Authorities over Twitter
Haouari, F., Elsayed, T., & Suwaileh, R. (2024) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/2024.arabicnlp-1.3 · Cited by 8
“Using Pyserini (Lin et al., 2021), we constructed an index for all tweets from all authorities for a each rumor.”
MethodOverview of LiLAS 2021 – Living Labs for Academic Search
Schaer, P., Breuer, T., Castro, L. J., Wolff, B., Schaible, J., & Tavakolpoursaleh, N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85251-1_25 · Cited by 8
How Train–Test Leakage Affects Zero-Shot Retrieval
Fröbe, M., Akiki, C., Potthast, M., & Hagen, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20643-6_11 · Cited by 8
Bojić, I., Ong, Q. C., Joty, S., & Car, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.19707 · Cited by 8
“Lucene BM25 + BERT SQuAD2 QA 0.30 PubMedBERT + BioBERT BioASQ 0.24 Although our initial fine-tuned QA system did not outperform the baseline on the test labels, re-sults from human evaluation of the two QA systems on real-world questions suggested otherwise.”
InfluentialCitations as Queries: Source Attribution Using Language Models as RerankersPreprint
Muther, R., & Smith, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.17322 · Cited by 8
“[5]ForWikipedia, the baseline retrieval model attains a recall at 10 of .64 and MRR of .478.”
MethodAbonizio, H., Bonifacio, L., Jeronymo, V. A., Lotufo, R., Zavrel, J., & Nogueira, R. F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.04601 · Cited by 8
“The dataset_source argument is designed to integrate with twowidely useddatasetinterfaces in the IRcommunity:Pyserini [7], a toolkit for conducting reproducible IR research with sparse and dense representations, and ir_datasets [8], a commonplace for several IR ad-hoc ranking benchmarks.”
MethodInfluentialSelf-citationLai, Y., Wu, J., Zhang, C., Sun, H., & Zhou, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.01965 · Cited by 8
“CDR (Qu et al., 2020; Yu et al., 2021; Lin et al., 2021b) aims to improve the representation of the current query along with its historical context by training dense retrievers.”
BackgroundMethodInfluentialMessIRve: A Large-Scale Spanish Information Retrieval DatasetPreprint
Valentini, F., Cotik, V., Furman, D., Bercovich, I., Altszyler, E., & Pérez, J. M. (2024) · Conicet · doi:10.48550/arxiv.2409.05994 · Cited by 8
“We index the corpus and run retrieval using Pyserini with default parameters and Spanish analyzer (Lin et al., 2021).”
PyTerrier-GenRank: The PyTerrier Plugin for Reranking with Large Language ModelsPreprint
Dhole, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.05339 · Cited by 8
Repository-level Code Search with Neural Retrieval MethodsPreprint
Gandhi, S., Gao, L., & Callan, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.07067 · Cited by 8
Retro*: Optimizing LLMs for Reasoning-Intensive Document RetrievalPreprint
Lan, J., Chen, J., Liu, Z., Li, C., Bao, S., & Lian, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.24869 · Cited by 8
Text Information Retrieval in Tetun
de Jesús, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28241-6_48 · Cited by 7
Limitations of Open-Domain Question Answering Benchmarks for Document-level Reasoning
Kamalloo, E., Clarke, C. L. A., & Rafiei, D. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592011 · Cited by 7
“We also consider a simple dense document retrieval, known as MaxP [15], that estimates document relevance from a ranked list of passages by taking the score of the top-ranking passage in the document.”
Nguyen, D.-H., Le, N.-K., & Nguyen, L.-M. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103514 · Cited by 7
Leveraging multiple control codes for aspect-controllable related paper recommendation
Long, K., Li, S., Tang, J., & Wang, T. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103879 · Cited by 7
Efficient Dynamic Hard Negative Sampling for Dialogue Selection
Han, J., Lee, D., Shin, J., Bae, H., Bang, J., Kim, S.-H., Choi, S. J., & Lee, H. (2024) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2024.nlp4convai-1.6 · Cited by 7
“The problem of selecting the most suitable answer from multiple candidates has been extensively explored in the field of natural language processing, particularly within selection tasks (Lowe et al., 2015; Wu et al., 2018a; Zhang et al., 2018a; Kim et al., 2020, 2021).”
BackgroundMethodLuo, C. F., Shayanfar, R., Bhambhoria, R. V., Dahan, S., & Zhu, X. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.924 · Cited by 7
SearchLab: Exploring Conversational and Traditional Search Interfaces in Information Retrieval
Zerhoudi, S., & Granitzer, M. (2025) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3698204.3716475 · Cited by 7
“With the rise of machine learning and Python, new wrappers such as Pyserini [13] and PyTerrier [16] have emerged.”
Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling
Zhang, H., Bi, K., Guo, J., Sun, X., Liu, S. B., Shi, D., Yin, D., & Cheng, X. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3839865 · Cited by 7
Constructing Set-Compositional and Negated Representations for First-Stage Ranking
Krasakis, A. M., Yates, A., & Kanoulas, E. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761238 · Cited by 7
Türkmen, M. D., Kutlu, M., ALTUN, B., & Cosgun, G. (2026) · ACM Transactions on Information Systems · doi:10.1145/3841465 · Cited by 7
“To assess the evaluation reliability of GenTREC , we implemented various retrieval systems using the Pyserini library [47].”
Lovón-Melgarejo, J., Mouysset, M., Oleiwan, J., Moreno, J. G., Damase‐Michel, C., & Tamine, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_10 · Cited by 7
Noise-Reduction for Automatically Transferred Relevance Judgments
Fröbe, M., Akiki, C., Potthast, M., & Hagen, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13643-6_4 · Cited by 7
“We conduct our experiments with the PyGaggle 9 implementation [18] of monoT5 as this model shows state-of-the-art effectiveness in a range of retrieval experiments [36].”
MethodXpmIR: A Modular Library for Learning to Rank and Neural IR Experiments
Zong, Y., & Piwowarski, B. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591818 · Cited by 7
“More precisely, as of today, XpmIR provides components for ir-datasets, ir-measures, Pyserini, and FAISS.”
MethodInfluentialBai, Y., Colas, A., & Wang, D. Z. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591907 · Cited by 7
“We use the Pyserini IR toolkit [24] to implement both BM25 and DPR tweet retrievers.”
MethodSCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examples
Fu, D., Godbole, A., & Jia, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.485 · Cited by 7
Analysis of Data Engineering Techniques With Data Quality in Multilingual Information Recovery
Rangineni, S., Information Technology, I. R., Information Technology, I. R., Information Technology, I. R., Information Technology, I. R., Information Technology, I. R., Bhanushali, A., Marupaka, D., Venkata, S., & Suryadevara, M. (2023) · International Journal of Computer Sciences and Engineering · doi:10.26438/ijcse/v11i10.2936 · Cited by 7
Penha, G., & Hauff, C. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28241-6_9 · Cited by 7
Overview of the TREC 2023 Product Product Search TrackPreprint
Campos, D., Kallumadi, S. T., Rosset, C., Zhai, C. X., & Magnani, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.07861 · Cited by 7
“In providing a simple-to-use dataset, we believe broad experimentation using popular retrieval libraries Lin et al. [2021] Gao et al. [2022] can lead to broad improvements in retrieval performance.”
SleepCoT: A Lightweight Personalized Sleep Health Model via Chain-of-Thought DistillationPreprint
Zheng, H., Xing, X., & Xu, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.16924 · Cited by 7
“Extractive QA system name EM Lucene BM25[17] + BERT SQuAD2 QA[23] The evaluation results of SleepCoT are presented in table 6.”
MethodLikert or Not: LLM Absolute Relevance Judgments on Fine-Grained Ordinal ScalesPreprint
Godfrey, C., Nie, P., Ostapuk, N., Ken, D., Gao, S., & Inati, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.19334 · Cited by 7
“For each query, we collect the top 100 BM25 hits us-ing Pyserini Lin et al. [2021].”
Kumar, S., Pande, M., & Damle, A. Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.18297 · Cited by 7
“• First-Stage Retrieval: For TREC DL evaluation, we retrieve the top 1000 candidate passages per query us-ing a standard BM25 baseline via Pyserini [28] on the ‘msmarco-v1-passage‘ index.”
A Graph-based Verification Framework for Fact-CheckingPreprint
Huang, Y., Zhang, R., Nie, Z., Chen, J., & Zhang, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.07282 · Cited by 7
“In the Open Book setting (open), we follow ProgramFC (Pan et al., 2023), using BM25 (Robertson et al., 2009) retriever via Pyserini (Lin et al., 2021) for all methods.”
Distillation versus Contrastive Learning: How to Train Your RerankersPreprint
Xu, Z., Huang, Z., Zhuang, S., & Srikumar, V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.08336 · Cited by 7
“The retriever — named RepQwen — is trained on Tevatron/MSMARCO-passage-aug trainset, which consists of 486k ( q i , d + i ) pairs with hard negatives mined from BM25 and CoCon-denser (Gao and Callan, 2022).”
Facilitating Cross-Lingual Information Retrieval Evaluations for African Languages
Adeyemi, M. (2024) · UWSpace (University of Waterloo) · Cited by 7
(2022) · doi:10.1145/3477495.3531658 · Cited by 6
A Quantitative Analysis of Noise Impact on Document Ranking
Giamphy, E., Sanchis, K., Dashyan, G., Guillaume, J., Hamdi, A. T. E., Sanselme, L., & Doucet, A. (2023) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53992.2023.10394665 · Cited by 6
“We use the Pyserini python toolkit to run our experiments [35].”
MethodBrowsing and Searching Metadata of TREC
Breuer, T., Voorhees, E. M., & Soboroff, I. M. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657873 · Cited by 6
Integration of biomedical concepts for enhanced medical literature retrieval
Frihat, S., & Fuhr, N. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00724-z · Cited by 6
“Several tools have been developed to enable easy implementation of this retrieval architecture, e.g., Pyserini [51].”
Artifact Sharing for Information Retrieval Research
MacAvaney, S. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730147 · Cited by 6
“For example, PyTerrier [18] and Pyserini [10] platforms include repositories for pre-built indexes 1 .”
TITE: Token-Independent Text Encoder for Information Retrieval
Schlatt, F., Hagen, T., Potthast, M., & Hagen, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730094 · Cited by 6
μDS: Multi-Objective Data Snippet Extraction for Dataset Search
Zhou, X., Chen, Q., Chen, J., & Cheng, G. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730120 · Cited by 6
Axioms for Retrieval-Augmented Generation
Reimer, J. H., Fröbe, M., Stein, B., Potthast, M., & Hagen, M. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744601 · Cited by 6
ReFormeR: Learning and Applying Explicit Query Reformulation Patterns
Bigdeli, A., Incesu, M., Arabzadeh, N., Clarke, C. L. A., & Bagheri, E. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21300-6_30 · Cited by 6
“Retrieval was carried out using BM25 implemented by Pyserini [13], selected for its robust integration with large-scale document collections.”
RankEvolve: Automating the Discovery of Retrieval Algorithms via LLM-Driven Evolution
Nian, J., Li, F., Park, D. H., & Fang, Y. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809962 · Cited by 6
“4 following Pyserini [8] defaults.”
Pseudo-Relevance Feedback with Dense Retrievers in Pyserini
Li, H., Zhuang, S., Ma, X., Lin, J., & Zuccon, G. (2022) · Australasian Document Computing Symposium · doi:10.1145/3572960.3572982 · Cited by 6
“In this context, we present a new PRF framework that implemented as a new feature in Pyserini [11] – a popular DR Python toolkit.”
MethodSelf-citationConversational Query Reformulation with the Guidance of Retrieved Documents
Park, J., & Lee, H. (2026) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2026.081336 · Cited by 6
SqCLIRIL: Spoken query cross-lingual information retrieval in Indian languages
Dave, B., & Majumder, P. (2025) · Pattern Recognition Letters · doi:10.1016/j.patrec.2025.08.022 · Cited by 6
A Hybrid Text Generation-Based Query Expansion Method for Open-Domain Question Answering
Zhu, W., Zhang, X., Zhai, Q., & Liu, C. (2023) · Future Internet · doi:10.3390/fi15050180 · Cited by 6
pybool_ir: A Toolkit for Domain-Specific Search Experiments
Scells, H., & Potthast, M. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591819 · Cited by 6
“No separate search engine written in another programming language needs to be maintained, like in the case of pyserini [7] and pyterrier [8], both of which require programming in Java to implement additional functionality.”
BackgroundYu, C.-J., Lee, W., Ke, L. T., Guo, S.-W., & Fan, Y.-C. (2024) · International Conference on Natural Language Generation · doi:10.18653/v1/2024.inlg-main.16 · Cited by 6
“Using these 12 keywords, 12 relevant sentences were retrieved with Pyserini (Lin et al., 2021) from our testing corpus.”
MethodDetect, Retrieve, Comprehend: A Flexible Framework for Zero-Shot Document-Level Question AnsweringPreprint
McDonald, T., Tsan, B., Saini, A., Ordoñez, J., Gutierrez, L., Nguyen, P., Mason, B., & Ng, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.01959 · Cited by 6
“For BM25, we create an inverted index on QASPER validation and test sets using Pyserini (Lin et al. 2021) with default parameters ( k 1 =0.9, b =0.4).”
MethodInfluentialRankFlow: A Multi-Role Collaborative Reranking Workflow Utilizing Large Language ModelsPreprint
Jin, C., Peng, H., Zhang, A., Chen, N., Zhao, J., Xie, X., Li, K., Feng, S., Zhong, K., Ding, C., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.00709 · Cited by 6
“It not only surpasses all supervised methods but also outperforms SoTA zero-shot unsupervised methods, including the pointwise method RG, the pairwise method PRP, and the listwise method RankGPT-4.”
Leveraging Reference Documents for Zero-Shot Ranking via Large Language ModelsPreprint
Li, J., Hu, X., Yang, Z., Zhuang, S., & Zhang, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.11452 · Cited by 6
“All queries are derived from the initial retrieval results obtained through the BM25 method (Lin et al., 2021).”
Influential
Metadata
- Authors
- Lin, J., Ma, X., Lin, S.-C., Yang, J.-H., Pradeep, R., & Nogueira, R.
- Affiliations
- University of Waterloo (CA)
- Venue
- Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2021-07-11
- Pages
- 2356-2362
- OpenAlex
- W3180230246
- Cited by
- 360
- References
- 41
- Citations by year
- 2026: 67 · 2025: 72 · 2024: 61 · 2023: 97 · 2022: 44 · 2021: 18
- FWCI
- 45.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion, cc-by)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Waterloo-Huawei Joint Innovation Laboratory
- Canada First Research Excellence Fund
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- reproducibility, sparse retrieval, dense retrieval, hybrid retrieval, BM25, transformer representations, nearest neighbor search, neural ranking, relevance assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.