Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations
Open accessThe DOI matches its Crossref record; 39 of the 41 references it cites were found in a registry or an index.
Abstract
Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations. It aims to provide effective, reproducible, and easy-to-use first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. Around this toolkit, our group has built a culture of reproducibility through shared norms and tools that enable rigorous automated testing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Interestingly, the classic BM25 model [18] remains competitive, surpassing some semantic models trained on search behavior datasets.”
From Generative Agents Navigating Digital Libraries (2025) · cites this paper as [18]
In the citing paper
“Sparse lexical methods, from classical BM25 [Lin et al., 2021] to lightweight full-text indexes such as SQLite FTS5, complement dense retrieval by matching exact terms.”
From MemX: A Local-First Long-Term Memory System for AI Assistants (2026) · cites this paper as Lin et al., 2021
Citations
Cites
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“This is particularly the case for work on deep learning today, since the major toolkits (PyTorch [29] and Tensorflow [1]) have both adopted Python as their front-end This work is licensed under a Creative Commons Attribution International 4.”
BackgroundTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“also provides document expansion baselines using our doc2query method [28]; the latest model uses T5 [33] as described in Nogueira and Lin [26].”
MethodTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140 · cited 3×
“Row (2a) refers to brute-force scans over the document vectors in Faiss [12], which provides exact nearest-neighbor search.”
BackgroundMethodDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Malkov, Y. A., & Yashunin, D. A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889473 · Cited by 1,904 · cited 3×
“To support dense and hybrid retrieval, Pyserini integrates Facebook’s Faiss library for efficient similarity search over dense vectors [12], which in turns integrates the HNSW library [23] to support low-latency querying.”
MethodColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Khattab, O., & Zaharia, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401075 · Cited by 1,301 · cited 8×
“Row (3a) shows the results of combining TCT-ColBERT with BM25 bag-of-words search over the original texts, while row (3b) shows results that combine document expansion using doc2query with the T5 model.”
MethodInfluential
Research works printed but not linked: 6
- Sebastian Hofstatter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2020). · arXiv
- Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2021. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2021). · arXiv
- Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, and Nazli Goharian. 2021. Simplified Data Wrangling with textttir_datasets. arXiv:2103.02280 (2021). · arXiv
- Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. 2021 c. The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models. arXiv:2101.05667 (2021). · arXiv
- Chenyan Xiong, Zhenghao Liu, Si Sun, Zhuyun Dai, Kaitao Zhang, Shi Yu, Zhiyuan Liu, Hoifung Poon, Jianfeng Gao, and Paul Bennett. 2020 a. CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search. arXiv:2011.01580 (2020). · arXiv
- Yongze Yu, Jussi Karlgren, Hamed Bonab, Ann Clifton, Md Iftekhar Tanveer, and Rosie Jones. 2020. Spotify at the TREC 2020 Podcasts Track: Segment Retrieval. In Proceedings of the Twenty-Ninth Text REtrieval Conference (TREC 2020).
Cited by
Items 401–500 of 809
Enhancing Sparse Retrieval via Unsupervised Learning
Ma, X., Fun, H., Yin, X., Mallia, A., & Lin, J. (2023) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3624918.3625334 · Cited by 5
Self-citationRefined Medical Search via Dense Retrieval and User Interaction
Goli, R., Moffat, A., & Buchanan, G. R. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730292 · Cited by 5
“We have also compared our results with sparse retrieval methods using Pyserini [16], specifically evaluating performance against BM25 and RM3-PRF .”
Mitigating Source Bias with LLM Alignment
Dai, S., Zhou, Y., Pang, L., Li, Z., Du, Z., Wang, G., & Xu, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730038 · Cited by 5
“Considering the implementation of 𝑁 rephrasing instructions designed to produce semantically equivalent outputs, we further expand the alignment dataset by pairing each instruction with the constructed preference quadruples, thereby rapidly increasing the dataset size by a factor of 𝑁 .”
OPOR-Bench: Evaluating Large Language Models on Online Public Opinion Report Generation
Yu, J., Xu, Y., Li, H., Li, J., Zhu, L., Shen, H., & Shi, L. (2026) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.073771 · Cited by 5
“Relevance Reranking: In the refinement phase, we employ the BM25 retriever (Lin et al., 2021) to rerank the collected articles.”
SciNUP: Natural Language User Interest Profiles for Scientific Literature Recommendation
Arustashvili, M., & Balog, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_51 · Cited by 5
Predicting the Benefit of Retrieval Augmentation in Open-Domain Question Answering
Dado, O., Carmel, D., & Kurland, O. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840689 · Cited by 5
“For sparse retrieval we use Okapi BM25 [31] via the Pyserini implementation [47] with default parameter values.”
Model Analysis & Evaluation for Ambiguous Question Answering
Papakostas, K., & Papadopoulou, I. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.279 · Cited by 5
Resources for Combining Teaching and Research in Information Retrieval Coursework
Fröbe, M., Scells, H., Elstner, T., Akiki, C., Gienapp, L., Reimer, J. H., MacAvaney, S., Stein, B., Hagen, M., & Potthast, M. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657886 · Cited by 5
“Pyserini [37] and PyTerrier [43] have also been used in teaching university and summer school courses [70].”
Discovering Biases in Information Retrieval Models Using Relevance Thesaurus as Global Explanation
Kim, Y., Rahimi, R., & Allan, J. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1089 · Cited by 5
Evaluating D-MERIT of Partial-annotation on Information Retrieval
Rassin, R., Fairstein, Y., Kalinsky, O., Kushilevitz, G., Cohen, N., Libov, A., & Goldberg, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.171 · Cited by 5
Ferrara, A., Pomo, C., & Tonellotto, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88720-8_42 · Cited by 5
Lighting the Way for BRIGHT: Reproducible Baselines with Anserini, Pyserini, and RankLLM
Sharifymoghaddam, S., Ge, Y., Vasudeva, R., & Lin, J. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808570 · Cited by 5
“To facilitate further research with the BRIGHT benchmark, we have integrated reproducible baselines into two popular information retrieval (IR) toolkits, Anserini [37, 16] and Pyserini [17], for convenient first-stage retrieval.”
Self-citationTypos Correction Training against Misspellings from Text-to-Text Transformers
Xie, G., Zhang, K., Duan, L., Zhang, W., & Huang, Z. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5q26k2bvsx49 · Cited by 5
“Among all baselines, we use the Pyserini (Lin et al., 2021) implementation to produce baseline BM25 results on all datasets.”
MethodRosati, D. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.46 · Cited by 5
“The 6 sentence chunks were indexed with Pyserini (Lin et al., 2021).”
MethodInfluentialRetrieving Webpages Using Online Discussions
Ros, K., Jin, M., Levine, J., & Zhai, C. (2023) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3578337.3605139 · Cited by 5
“WeimplementedBM25[34]andBM25withRM3pseudo-feedback[1] via Pyserini [25], an open-source retrieval toolkit built on Lucene, for the initial baselines.”
MethodVILE: Block-Aware Visual Enhanced Document Retrieval
Yuan, H., Dou, Z., Zhou, Y., Guo, Y., & Wen, J.-R. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615107 · Cited by 5
“To replicate the baselines, we utilize open-source repositories [18, 22] for sparse retrieval models.”
MethodArTrivia: Harvesting Arabic Wikipedia to Build A New Arabic Question Answering Dataset
Alrowili, S., & Vijay‐Shanker, K. (2023) · Arabic Natural Language Processing Conference (ArabicNLP) · doi:10.18653/v1/2023.arabicnlp-1.17 · Cited by 5
“Then, we use the sparse-based retrieval BM25 with the Pyserini tool (Lin et al., 2021) to build our indexed Arabic Wikipedia 3 .”
MethodNoisy Self-Training with Synthetic Queries for Dense Retrieval
Jiang, F., Drummond, T. W., & Cohn, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.803 · Cited by 5
SCC - A Test Collection for Search in Chat Conversations
Sabei, I., Mourad, A., & Zuccon, G. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557692 · Cited by 5
Shared Tasks as Tutorials: A Methodical Approach
Elstner, T., Loebe, F., Ajjour, Y., Akiki, C., Bondarenko, A., Fröbe, M., Gienapp, L., Kolyada, N., Mohr, J., Sandfuchs, S., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26877 · Cited by 5
“2021) and Pyserini (Lin et al. 2021) simplify hands-on exercises by enabling researchers and students alike to implement AI-based retrieval pipelines in Python and run experiments directly on Google Colab.”
MethodAligning Documentation and Q&A Forum through Constrained Decoding with Weak Supervision
Pudari, R., Zhou, S., Ahmed, I., Dai, Z., & Zhou, S. (2023) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58846.2023.00043 · Cited by 5
“Thus, we include a SOTA retrieval-based baseline, Pyserini [33] , which is a widely-used reproducible Information Retrieval (IR) toolkit in the field of NLP for indexing a collection of documents and querying the index to retrieve relevant documents.”
MethodLike a Skilled DJ - an Expert Study on News Recommendations Beyond Accuracy
Kolb, T. E., Nalis-Neuner, I., & Neidhardt, J. (2023) · reposiTUm (TU Wien) · doi:10.34726/5332 · Cited by 5
GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data ExplorationPreprint
Piktus, A., Ogundepo, O., Akiki, C., Oladipo, A., Zhang, X., Schoelkopf, H., Biderman, S., Potthast, M., & Lin, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.01481 · Cited by 5
“The lack of stan-dardisation of IR evaluation was the main motiva-tor behind creating Anserini (Yang et al., 2017), a Lucene 6 -based toolkit for reproducible IR research, and the follow-up Pyserini (Lin et al., 2021)—a convenient Python API to the underlying Java-based implementation of Anserini.”
MethodInfluentialSelf-citationAbdallah, A., Mozafari, J., Piryani, B., Abdelgwad, M. M., & Jatowt, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.00600 · Cited by 5
“It includes several datasets covering various retrieval tasks such as fact-checking, question answering, and others.”
CLIRudit: Cross-Lingual Information Retrieval of Scientific DocumentsPreprint
Valentini, F., Kozlowski, D., & Larivière, V. (2025) · Conicet · doi:10.48550/arxiv.2504.16264 · Cited by 5
Cai, Y., Zhang, Y., Long, D., Li, M., Xie, P., & Zheng, W. (2025) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.48550/arxiv.2509.00520 · Cited by 5
“Despite using a point-wise paradigm, ER ANK -4B achieves a notable nDCG@10 of 38.7 with the BM25 hybrid on documents retrieved by ReasonIR-8B.”
InfluentialHow Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking ModelsPreprint
Abdallah, A., Piryani, B., Mozafari, J., Ali, M., & Jatowt, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.16757 · Cited by 5
“For the initial retrieval, we used BM25 to pull the top 100 documents per query, which the rerankers then reordered using Pyserini (Lin et al., 2021).”
TASR: Training-Free Adaptive Stopping for Iterative RetrievalPreprint
Kieback, A., Amadasun, U. P., Chadha, A. R., & Elkins, A. C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.13814 · Cited by 5
“…beyond the curated distractor pool, we re-run the same pipeline on three open-domain BM25 corpora via prebuilt Pyserini Lucene indexes [14]: HotpotQA-fullwiki (same 300 eval questions, retrieved from the full Wikipedia index), NQ-Open [11] (300 questions subsampled with seed 42 from…”
Gosling Grows Up: Retrieval with Learned Dense and Sparse Representations Using Anserini
Lin, J., Chen, A. H., Lassance, C., Ma, X., Pradeep, R., Teofili, T., Xian, J., Yang, J.-H., Zhong, B., & Zhong, V. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730281 · Cited by 4
“In 2019, our group began building a Python toolkit called Pyserini [34] that provides convenient bindings to Anserini as well as additional functionalities, chief among them support for dense retrieval models via the Faiss toolkit [22].”
Self-citationOn Listwise Reranking for Corpus Feedback
Yoon, S., Kim, J., Kwon, D., Anand, A., & Hwang, S.-W. (2026) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3773966.3779404 · Cited by 4
“…TREC [3] DL’19/’20 (43/54 queries) and a subset of BEIR [23] collections (TREC-COVID, Signal, TREC-NEWS, and Touche) with query size less than 100, and report nDCG@10 [8] under reranking budget 𝑐 (c= 100 or 1000) rounded to the nearest tenth, mostly prepared with the pre-built Pyserini index [10].”
Huynh, T.-L., Vu, T.-T., Wang, W., Le, T., Gašević, D., Li, Y.-F., & Do, T.-T. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.20 · Cited by 4
Efficient Temporal-aware Matryoshka Adaptation for Temporal Information Retrieval
Huynh, T.-L., Wang, W., Le, T., Vu, T.-T., Gašević, D., Li, Y.-F., & Do, T.-T. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.249 · Cited by 4
Evaluating the Efficiency and Effectiveness of Learned Sparse Retrieval with the lsr_benchmark
Fröbe, M., Schlatt, F., Rulli, C., Hagen, T. H., Reimer, J. H., Hendriksen, G., Lassance, C., Nardini, F. M., Venturini, R., & Potthast, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_57 · Cited by 4
A Parametric Memory Head for Continual Generative Retrieval
Mekonnen, K. A., Tang, Y., & de Rijke, M. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809725 · Cited by 4
A Reproducibility Study of LLM-Based Query Reformulation
Bigdeli, A., Rad, R. H., Le, H. S., Incesu, M., Arabzadeh, N., Clarke, C. L. A., & Bagheri, E. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808560 · Cited by 4
Do Agents Need to Plan Step-by-Step? Rethinking Planning Horizon in Data-Centric Tool Calling
Otani, N., Bhutani, N., Kim, H., Zhang, D., & Junior, E. R. H. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813129 · Cited by 4
“5 [20].”
Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning
Fang, W., & Glass, J. R. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.2090 · Cited by 4
DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment
Geng, X., He, X., Chen, S., Xiao, Y., Zhang, F., Huang, S., Mi, H., Liang, Z., Fang, T., & Fung, Y. R. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1117 · Cited by 4
Zekarias, A., Meldau, E.-L., Bista, S., China, J. F., & Sandberg, L. (2025) · Drug Safety · doi:10.1007/s40264-025-01529-6 · Cited by 4
“The narrative search engine was built using the Apache Lucene open-source search engine software library through the python package Pyserini [32] version 0.18.0.”
InfluentialOn Reference (In-)Determinacy in Natural Language Inference
Chen, S., Malaviya, C., Fabrikant, A., Taitelbaum, H., Schuster, T., Buthpitiya, S., & Roth, D. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.450 · Cited by 4
Detecting Frozen Phrases in Open-Domain Question Answering
Yadegari, M., Kamalloo, E., & Rafiei, D. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531793 · Cited by 4
VILT: Video Instructions Linking for Complex Tasks
Fischer, S., Gemmell, C., Mackie, I., & Dalton, J. (2022) · doi:10.1145/3552467.3554794 · Cited by 4
“We note that not all steps have techniques and that some techniques are not covered by the video corpus due to limited scope.”
BackgroundMethodKALE: Using a K-Sparse Projector for Lexical Expansion
Borges, L., Martins, B., & Callan, J. (2023) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3578337.3605131 · Cited by 4
“Indexing and BM25 search were done with the Pyserini toolkit [19], with 12 threads and a batch size of 64.”
MethodMTGER: Multi-view Temporal Graph Enhanced Temporal Reasoning over Time-Involved Document
Chu, Z., Wang, Z., Liang, J., Liu, M., & Qin, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1016 · Cited by 4
An intent-enhanced feedback extension model for code search
Hu, H., Fang, M., & Liu, J. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107589 · Cited by 4
No Query Left Behind: Query Refinement via Backtranslation
Rajaei, D., Taheri, Z., & Fani, H. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679729 · Cited by 4
“We have applied two sparse informationretrievalmethods,namely bm25 [69]and qld [67], using pyserini [52] to retrieve relevant content for the original queries as well as the backtranslated versions.”
Reproducible Hybrid Time-Travel Retrieval in Evolving Corpora
Staudinger, M., Piroi, F., & Rauber, A. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698421 · Cited by 4
“We mention, for example, ir_datasets [18], pybool_ir [29], Pyserini [17], TIREx [8], and ir-metadata [6].”
A Fact Checking and Verification System for FEVEROUS Using a Zero-Shot Learning Approach
Temiz, O., Kılıç, Ö. O., Kızıldağ, A. O., & Temizel, T. T. (2021) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2021.fever-1.13 · Cited by 4
“Then, we indexed it using Anserini toolkit with the help of Pyserini (Lin et al., 2021), a Python interface for Anserini.”
MethodA2A-API: A Prototype for Biomedical Information Retrieval Research and Benchmarking
Rybinski, M., Watts, L., & Karimi, S. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531667 · Cited by 4
“Pyserini [5] is a Python package over Anserini.”
MethodOffline Pseudo Relevance Feedback for Efficient and Effective Single-pass Dense RetrievalPreprint
Wen, X., Chen, X., Chen, X., He, B., & Sun, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.10191 · Cited by 4
“The aforementioned models are reproduced by Pyserini [38] with default parameter settings, except that the results of CROW reported in [11] are compared.”
MethodAugmenting Ad-Hoc IR Dataset for Interactive Conversational SearchPreprint
Erbacher, P., Nie, J., Preux, P., & Soulier, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.06119 · Cited by 4
“At test time, we perform a first-stage retrieval on the initial query using the pyserini (Lin et al., 2021) implementation of BM25.”
MethodSemi-Parametric Retrieval via Binary Bag-of-Tokens Index
Zhou, J., Li, D., Wei, F., & Chen, L. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2405.01924 · Cited by 4
“For BM25 retrieval, we utilize Pyserini [71], a library based on a Java implementation developed around Lucene.”
CacheFocus: Dynamic Cache Re-Positioning for Efficient Retrieval-Augmented GenerationPreprint
Lee, K.-H., Park, E., Han, D., & Na, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.11101 · Cited by 4
OrdRankBen: A Novel Ranking Benchmark for Ordinal Relevance in NLPPreprint
Wang, Y., Qian, L., Peng, X., Huang, J., & Feng, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.00674 · Cited by 4
“Ranking methods have been extensively studied as machine learning problems due to their significance in various tasks [6, 7, 12, 13, 25].”
Breuer, T. (2023) · DuEPublico (University of Duisburg-Essen) · doi:10.17185/duepublico/78449 · Cited by 4
Search, Inspect, Fetch: Exploiting Structure-Aware Boolean Retrieval for Deep-Search AgentsPreprint
Wang, S., Chen, H., Yin, Y., Zhuang, S., Koopman, B., & Zuccon, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.02751 · Cited by 4
“Sparse retrieval uses Pyserini/Lucene BM25 (Lin et al., 2021; Yang et al., 2018a); dense retrieval uses exact inner-product search with BAAI/bge-base-en-v1.”
BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
Chen, Z., Ma, X., Zhuang, S., Nie, P., Zou, K., Liu, A., Green, J., Patel, K., Meng, R., Su, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.06600 · Cited by 3
“We use the Pyserini IR toolkit [38] to serve the BM25 retriever, and the Tevatron dense retrieval toolkit [39] to serve Qwen3-Embedding and ReasonIR.”
Self-citationEnhancing Retrieval-Augmented Generation for Text Completion Through Query Selection
Pogrebinsky, I., Carmel, D., & Kurland, O. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744610 · Cited by 3
“The retrieval system is based on Pyserini [13].”
CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents
Valentini, F., Kozlowski, D., & Larivière, V. (2025) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2025.mrl-main.16 · Cited by 3
Multi-model pseudo-document generation and reconstruction for hybrid query expansion
Wu, Y., & Wang, K. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104543 · Cited by 3
LLM-Based Adversarial Persuasion Attacks on Fact-Checking Systems
Leite, J. A., Razuvayevskaya, O., Bontcheva, K., & Scarton, C. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.86 · Cited by 3
Improving Arabic Information Retrieval and Reranking Performance Using Knowledge Distillation
Hatem, M. A., Azouaou, F., & Batata, S. (2026) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3796229 · Cited by 3
“…the 1,000 candidate passage retrieved by BM25 for each query from the official validation set of 6,980 queries using Pyserini, a Python wrapper over the Anserini [26] information retrieval ( IR ) toolkit for reproducible research with sparse and dense representations developed by Lin et al. [17].”
How Do LLM-Generated Texts Impact Term-Based Retrieval Models?
Huang, W., Bi, K., Cai, Y., Chen, W., Guo, J., & Cheng, X. (2026) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3773966.3777988 · Cited by 3
RAC: Retrieval-Augmented Clarification for Faithful Conversational Search
Kebir, A. R., Guigue, V., Lhadj, L. S., & Soulier, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_39 · Cited by 3
On the Challenges and Opportunities of Learned Sparse Retrieval for Code
Lupart, S., Louis, M., Formal, T., Déjean, H., & Clinchant, S. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.193 · Cited by 3
Retrieval-Augmented Diffusion Language Model for Generative Commonsense Reasoning
FANG, Y., Yu, H.-T., Joho, H., & Fujita, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0369-7_22 · Cited by 3
Reformulate, Retrieve, Localize: Agents for Repository-Level Bug Localization
Caumartin, G., & Melo, G. (2026) · doi:10.1145/3786161.3788460 · Cited by 4
“Earlyapproachesfocusedon enhancinglexical andstructuralmatch-ing between bug reports and source code; subsequent work concentrated on automating query reformulation to improve retrieval accuracy.”
Layer-wise Token Compression for Efficient Document Reranking
Zhuang, S., Xu, Z., & Lauriola, I. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809871 · Cited by 3
Multi-Faceted Self-Consistent Preference Alignment for Query Rewriting in Conversational Search
Cao, Z., Li, P., & Zhu, Q. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.638 · Cited by 3
When Rules Learn: A Self-Evolving Agent for Legal Case Retrieval
Tao, M., Hu, J., Zhou, X., Hu, W., Cheng, J., Cao, Y., Luo, Z., & Geng, G. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.2152 · Cited by 3
“Following previous work (Li et al., 2024), we employ Pyserini Lucene (Lin et al., 2021) to implement BM25 retrieval.”
Yin, Y., Wang, S., Koopman, B., & Zuccon, G. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808553 · Cited by 3
“We evaluate a two-stage cascade retrieval pipeline initiated by a BM25 first-stage retriever (via Pyserini [13]), which identifies the top-1,000 candidates.”
Fernandes, L. C., de Castro, M. V. B., Ribeiro, L. D. S., Pacheco, L. A. D. S., & Sandes, E. F. O. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105151 · Cited by 3
“As averaging may result in fractional values and standard evaluation tools generally assume integer labels (e.g., Pyserini [23] and pytrec eval [24]), the final scores were rounded using Banker’s rounding.”
POISS: A Large-Scale Multilingual Dataset for Point-of-Interest Search
Maritan, N., Moschitti, A., Borazio, F., Zhou, X., & Bai, Z. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.2497 · Cited by 3
Matsubara, Y. (2023) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/2023.nlposs-1.18 · Cited by 3
“Pyserini (Lin et al., 2021) is a Python toolkit built on PyTorch (Paszke et al., 2019) and Faiss (Johnson et al., 2019) for reproducible information retrieval research with sparse and dense representations, and the sparse representation-based retrieval support comes from Lucene via Anserini.”
MethodMeasuring Bias in Search Results Through Retrieval List Comparison
Ratz, L., Schedl, M., Kopeinik, S., & Rekabsaz, N. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56069-9_2 · Cited by 3
GSM-EL: A Generalizable Symbol-Manipulation Approach for Entity Linking
Cheng, X., Wang, Y. P., Fan, Y., Guo, J., Zhang, R., & Bi, K. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3523399 · Cited by 3
Nakata, K., Miyashita, D., Ng, Y., Hoshi, Y., & Deguchi, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91585-7_2 · Cited by 3
Multi Stage Retrieval for Web Search During Crisis
Tcaciuc, C. C., Cambrin, D. R., & Garza, P. (2025) · Future Internet · doi:10.3390/fi17060239 · Cited by 3
“Hybrid approaches, which combine traditional lexical-based methods with neural retrieval models, have been explored to balance between efficiency and accuracy [26].”
FlashBack: Efficient Retrieval-Augmented Language Modeling for Fast Inference
Liu, R., Xiao, X., Huang, H., Chi, Z., & Wu, Z. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.33 · Cited by 3
“In our implementation, we use the Transform-ers library (Wolf et al., 2020) and BM25 retrieval from the Pyserini library (Lin et al., 2021).”
Joint Inference of Retrieval and Generation for Passage Re-ranking
Fang, W., Chuang, Y.-S., & Glass, J. R. (2024) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2024.findings-eacl.151 · Cited by 3
“J PR (LLM) can mitigate the worst cases, however it mostly does not 2 Pyserini reproductions for BEIR can be found at https: //castorini.github.io/pyserini/2cr/beir.html . outperform J PR that uses the considerably smaller generative model.”
MethodAugmenting Small Language Model for Better Medical Question Answering through Source Authentication
Das, S., Chatterji, S., & Mukherjee, I. (2026) · ACM Transactions on Information Systems · doi:10.1145/3797887 · Cited by 3
GENRA: Enhancing Zero-shot Retrieval with Rank Aggregation
Katsimpras, G., & Παλιούρας, Γ. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.431 · Cited by 3
“In our experiments, we utilize the pre-built indexes of the aforementioned datasets, extracted from the Pyserini toolkit (Lin et al., 2021).”
MethodBai, Y., Colas, A., Grant, C., & Wang, Z. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4mw5yrky2v7p · Cited by 3
Taxation Perspectives from Large Language Models: A Case Study on Additional Tax Penalties
Choi, E., Suh, Y. J., Lee, S., Oh, H., Kang, J., Hur, W., Park, H., & Hwang, W. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.218 · Cited by 3
RePair: An Extensible Toolkit to Generate Large-Scale Datasets for Query Refinement via Transformers
Narayanan, Y. L., & Fani, H. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615129 · Cited by 3
Integration of text and geospatial search for hydrographic datasets using the lucene search library
Yang, M. Y. R., Yang, S., & Lin, J. (2022) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3529372.3533280 · Cited by 3
“Therefore, we built a sparse Lucene inverted index using Pyserini [4], with the unique river id as the document id and the river name as the “contents” field we are searching over.”
MethodInfluentialSelf-citationZhang, Q., Yang, Z., Huang, Y., Chen, Z., Cai, Z., Wang, K., Zheng, J., He, J., & Gao, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.07010 · Cited by 3
“Our retrieval work is primarily reliant on Pyserini[17], a userfriendly and powerful Python toolkit for information retrieval research using sparse and dense representations.”
MethodSequencing Matters: A Generate-Retrieve-Generate Model for Building Conversational AgentsPreprint
Patwardhan, Q., & Yang, G. H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.09513 · Cited by 3
Corpus-informed Retrieval Augmented Generation of Clarifying QuestionsPreprint
Krasakis, A. M., Yates, A., & Kanoulas, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.18575 · Cited by 3
“For 𝐵𝑀 25 search, we use the Pyserini toolkit [17] using the default parameters.”
MethodDirected Information $γ$-covering: An Information-Theoretic Framework for Context EngineeringPreprint
Huang, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.00079 · Cited by 3
“We use pyserini (Lin et al., 2021) with its prebuilt BM25 index (Robertson & Zaragoza, 2009) over HotpotQA (Yang et al., 2018), specifically the beir-v1.0.0-hotpotqa.flat index.”
InfluentialKo, T.-W., Jiang, J., & Cheng, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.04069 · Cited by 3
“We implement it using the Pyserini (Lin et al., 2021) library, which provides efficient and effective tools for information retrieval.”
Huynh, T.-L., Vu, T.-T., Wang, W., Le, T., Gašević, D., Li, Y.-F., & Do, T.-T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.09924 · Cited by 3
“RQ-based docid is the first docid that is effective for large-scale standard IR datasets (Bajaj et al., 2016). while most GR works have only shown competitive performance on smaller-scale benchmarks (Pradeep et al., 2023).”
Enabling Cross-lingual Information Retrieval for African Languages
Ogundepo, O. (2023) · UWSpace (University of Waterloo) · Cited by 3
Information Retrieval with Dense and Sparse Representations
Chuang, Y.-S. (2024) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 3
Lu, Q., & Sidiropoulos, N. D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.02554 · Cited by 3
“To construct the candidate pools, we first use BM25 sparse retrieval (via Pyserini (Lin et al., 2021)) to fetch the top-3000 passages from a corpus of approximately 21 million distinct 100-word Wikipedia passages for each query in the development set.”
DiffRetriever: Parallel Representative Tokens for Retrieval with Diffusion Language ModelsPreprint
Wang, S., Yu, Y., Zhuang, S., Koopman, B., & Zuccon, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.07210 · Cited by 3
“We compare DiffRetriever against four main baselines: (1) BM25 uses the Pyserini (Lin et al., 2021) default hyperparameters and index for each dataset.”
ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance FeedbackPreprint
Bigdeli, A., Arabzadeh, N., Rad, R. H., Ebrahimi, S., Clarke, C. L. A., & Bagheri, E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.13905 · Cited by 3
“Following prior reformulation studies (Bigdeli et al., 2025, 2026b), all methods use BM25 as the first-stage retriever, with retrieval performed using the Pyserini toolkit (Lin et al., 2021).”
Exact Adaptive Hybrid Retrieval Without Fixed Top-L CutoffsPreprint
Zhang, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.07152 · Cited by 3
“We generate it with Pyserini 2.3.0/Anserini on JDK 21.0.10 [18].”
InfluentialBrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents
Chen, Z., Ma, X., Zhuang, S., Nie, P., Zou, K., Sharifymoghaddam, S., Liu, A., Green, J., Patel, K., Meng, R., et al. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.1023 · Cited by 2
“We use Pyserini (Lin et al., 2021) to serve BM25, Tevatron (Ma et al., 2025a) to serve Qwen3-Embedding and ReasonIR, along with Py-Late (Chaffin and Sourty, 2024) to serve Jina-ColBERT-v2.”
Self-citation
Metadata
- Authors
- Lin, J., Ma, X., Lin, S.-C., Yang, J.-H., Pradeep, R., & Nogueira, R.
- Affiliations
- University of Waterloo (CA)
- Venue
- Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2021-07-11
- Pages
- 2356-2362
- OpenAlex
- W3180230246
- Cited by
- 360
- References
- 41
- Citations by year
- 2026: 67 · 2025: 72 · 2024: 61 · 2023: 97 · 2022: 44 · 2021: 18
- FWCI
- 45.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion, cc-by)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Waterloo-Huawei Joint Innovation Laboratory
- Canada First Research Excellence Fund
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- reproducibility, sparse retrieval, dense retrieval, hybrid retrieval, BM25, transformer representations, nearest neighbor search, neural ranking, relevance assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.