Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations
Open accessThe DOI matches its Crossref record; 39 of the 41 references it cites were found in a registry or an index.
Abstract
Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations. It aims to provide effective, reproducible, and easy-to-use first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. Around this toolkit, our group has built a culture of reproducibility through shared norms and tools that enable rigorous automated testing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Interestingly, the classic BM25 model [18] remains competitive, surpassing some semantic models trained on search behavior datasets.”
From Generative Agents Navigating Digital Libraries (2025) · cites this paper as [18]
In the citing paper
“Sparse lexical methods, from classical BM25 [Lin et al., 2021] to lightweight full-text indexes such as SQLite FTS5, complement dense retrieval by matching exact terms.”
From MemX: A Local-First Long-Term Memory System for AI Assistants (2026) · cites this paper as Lin et al., 2021
Citations
Cites
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“This is particularly the case for work on deep learning today, since the major toolkits (PyTorch [29] and Tensorflow [1]) have both adopted Python as their front-end This work is licensed under a Creative Commons Attribution International 4.”
BackgroundTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“also provides document expansion baselines using our doc2query method [28]; the latest model uses T5 [33] as described in Nogueira and Lin [26].”
MethodTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140 · cited 3×
“Row (2a) refers to brute-force scans over the document vectors in Faiss [12], which provides exact nearest-neighbor search.”
BackgroundMethodDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Malkov, Y. A., & Yashunin, D. A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889473 · Cited by 1,904 · cited 3×
“To support dense and hybrid retrieval, Pyserini integrates Facebook’s Faiss library for efficient similarity search over dense vectors [12], which in turns integrates the HNSW library [23] to support low-latency querying.”
MethodColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Khattab, O., & Zaharia, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401075 · Cited by 1,301 · cited 8×
“Row (3a) shows the results of combining TCT-ColBERT with BM25 bag-of-words search over the original texts, while row (3b) shows results that combine document expansion using doc2query with the T5 model.”
MethodInfluential
Research works printed but not linked: 6
- Sebastian Hofstatter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2020). · arXiv
- Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2021. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2021). · arXiv
- Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, and Nazli Goharian. 2021. Simplified Data Wrangling with textttir_datasets. arXiv:2103.02280 (2021). · arXiv
- Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. 2021 c. The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models. arXiv:2101.05667 (2021). · arXiv
- Chenyan Xiong, Zhenghao Liu, Si Sun, Zhuyun Dai, Kaitao Zhang, Shi Yu, Zhiyuan Liu, Hoifung Poon, Jianfeng Gao, and Paul Bennett. 2020 a. CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search. arXiv:2011.01580 (2020). · arXiv
- Yongze Yu, Jussi Karlgren, Hamed Bonab, Ann Clifton, Md Iftekhar Tanveer, and Rosie Jones. 2020. Spotify at the TREC 2020 Podcasts Track: Segment Retrieval. In Proceedings of the Twenty-Ninth Text REtrieval Conference (TREC 2020).
Cited by
Items 201–300 of 809
DAPR: A Benchmark on Document-Aware Passage Retrieval
Wang, K., Reimers, N., & Gurevych, I. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.236 · Cited by 19
Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
Weir, N., Sanders, K., Weller, O., Sharma, S., Jiang, D., Jiang, Z., Mishra, B. D., Tafjord, O., Jansen, P. A., Clark, P. A., et al. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.531 · Cited by 19
Designing Safe and Relevant Generative Chats for Math Learning in Intelligent Tutoring Systems
Levonian, Z., Henkel, O., Li, C., & Postle, M.-E. (2025) · Journal of Educational Data Mining · doi:10.5281/zenodo.14751364 · Cited by 20
Patapasco: A Python Framework for Cross-Language Information Retrieval Experiments
Costello, C. J., Yang, E., Lawrie, D., & Mayfield, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-99739-7_33 · Cited by 19
Re-Ranking Step by Step: Investigating Pre-Filtering for Re-Ranking with Large Language ModelsPreprint
Nouriinanloo, B., & Lamothe, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.18740 · Cited by 19
“Generally, all of the retrieved passages from the retriever (typically BM25 (Lin et al., 2021)) are passed to the LLM-based re-ranker, to generate the ranked list of passages based on their relevance to the query.”
BackgroundMethodInfluentialEnronQA: Towards Personalized RAG over Private DocumentsPreprint
Ryan, M. J., Xu, D., Nivera, C., & Campos, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.00263 · Cited by 20
“We test BM25 using the PySerini implementation [44] and ColBERTv2 [64] over the full set of 103,638 emails.”
CsFEVER and CTKFacts: acquiring Czech data for fact verification
Ullrich, H., Drchal, J., Rýpar, M., Vincourová, H., & Moravec, V. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09654-3 · Cited by 18
“Inspired by the criticism of choosing weak baselines presented in [39], we decided to validate our TF-IDF baseline against the proposed Anserini toolkit implemented by Pyserini [40].”
MethodEntity Retrieval for Answering Entity-Centric Questions
Shavarani, H. S., & Sarkar, A. (2025) · International Workshop on Knowledge-Augmented Methods for Natural Language Processing (KnowledgeNLP) · doi:10.18653/v1/2025.knowledgenlp-1.1 · Cited by 18
Building a Culture of Reproducibility in Academic ResearchPreprint
Lin, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.13534 · Cited by 18
“I believe that my research group—with the help of external collaborators—has achieved this “aspirational ideal” in many parts of our two IR toolkits, Anserini (Yang et al., 2017, 2018) and Pyserini (Lin et al., 2021a).”
BackgroundSelf-citationPaRaDe: Passage Ranking using Demonstrations with Large Language ModelsPreprint
Drozdov, A., Zhuang, H., Dai, Z., Qin, Z., Rahimi, R., Wang, X., Alon, D., Iyyer, M., McCallum, A., Metzler, D., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.14408 · Cited by 19
“For each dataset, we retrieve the top-100 documents using BM25 from Pyserini (Lin et al., 2021).”
MethodLRAGE: Legal Retrieval Augmented Generation Evaluation ToolPreprint
Park, M., Oh, H., Choi, E., & Hwang, W. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.01840 · Cited by 18
“LRAGE extends Language Model Evaluation Harness (Gao et al., 2024b) by integrating it with pyserini (Lin et al., 2021) for information retrieval, while allowing easy control over individual components.”
How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking Models
Abdallah, A., Piryani, B., Mozafari, J., Ali, M. E., & Jatowt, A. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.305 · Cited by 17
Zhang, W., Huang, J.-H., Vakulenko, S., Xu, Y., Rajapakse, T. C., & Kanoulas, E. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78495-8_6 · Cited by 18
exHarmony: Authorship and Citations for Benchmarking the Reviewer Assignment Problem
Ebrahimi, S., Salamat, S., Arabzadeh, N., Bashari, M., & Bagheri, E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_1 · Cited by 17
BiDeV: Bilateral Defusing Verification for Complex Claim Fact-Checking
Liu, Y., Sun, H., Guo, W., Xiao, X., Mao, C., Yu, Z., & Yan, R. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i1.32034 · Cited by 17
Liu, L., & Zhang, M. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.9 · Cited by 17
Nguyen, K. V., Nguyen-Thuan, P., Nguyen, N. D., Nguyen, A. G.-T., & Nguyen, N. L.-T. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-022-01735-z · Cited by 17
“We compare our retriever with different retrievers TF-IDF, Pyserini [45], BM25 [68], BM25 Plus [68], and BERT-Ranker [53], widely used in state-of-the-art QA systems.”
ResultCODEC: Complex Document and Entity Collection
Mackie, I., Owoicho, P., Gemmell, C., Fischer, S., MacAvaney, S., & Dalton, J. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531712 · Cited by 17
“T5 [29] is state-of-the-art LM re-ranker that casts text re-ranking into a sequence-to-sequence setting and has shown impressive results.”
BackgroundMethodInfluentialLarge Language Models are Effective Text Rankers with Pairwise Ranking Prompting
Qin, Z., Jagerman, R., Hui, K., Zhuang, H., Wu, J., Yan, L., Shen, J., Liu, T., Liu, J., Metzler, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.17563 · Cited by 16
“This is because the metrics using BM25 reported in (Sun et al., 2023b) on NFCorpus does not match ours and the public consensus numbers (while the numbers match for all selected datasets), so we exclude NFCorpus to avoid unfair comparisons possibly due to errors during their evaluation.”
MethodInfluentialUnify Graph Learning with Text: Unleashing LLM Potentials for Session Search
Wu, S., Tu, Q., Liu, H., Xu, J., Liu, Z., Zhang, G., Wang, R., Chen, X., & Yan, R. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645574 · Cited by 16
“We use the pyserini [24] tool to calculate the BM25 scores.”
MethodEnhancing Biomedical Question Answering with Large Language Models
Yang, H., Li, S., & Gonçalves, T. (2024) · Information · doi:10.3390/info15080494 · Cited by 16
Simple Transformers: Open-source for All
Rajapakse, T. C., Yates, A., & de Rijke, M. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698412 · Cited by 16
Decoy Effect in Search Interaction: Understanding User Behavior and Measuring System Vulnerability
Chen, N., Liu, J., Fang, H., Luo, Y., Sakai, T., & Wu, X.-M. (2025) · ACM Transactions on Information Systems · doi:10.1145/3708884 · Cited by 16
FACTors: A New Dataset for Studying the Fact-checking Ecosystem
Altuncu, E., Başķent, C., Bhattacherjee, S., Li, S., & Roy, D. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730339 · Cited by 16
“For researchers preferring Python-based text processing, tools like Pyserini [27] can be used to interact with the index, providing a familiar interface for information retrieval and analysis tasks.”
MoZIP: A Multilingual Benchmark to Evaluate Large Language Models in Intellectual Property
Ni, S., Tan, M., Bai, Y., Niu, F., Yang, M., Zhang, B., Xu, R., Chen, X., Li, C., & Hu, X. X. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4bjfznjvz9xa · Cited by 16
Hybrid Retrieval Architectures: Integrating Vector Search into Production Systems
Adepoju, S. A. (2022) · International Journal of Scientific Research in Mechanical and Materials Engineering · doi:10.32628/ijsrmme22615 · Cited by 16
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual RetrievalPreprint
Litschko, R., Vulić, I., & Glavaš, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.02292 · Cited by 16
“This could explain why interpolating between the preranking and reranking (ENS, last column) yields further gains with DISTIL DmBERT as the preranker (Table 1), but not when we prerank with NMT+BM25 (Table 2).”
MethodInfluentialBenchmarking Information Retrieval Models on Complex Retrieval TasksPreprint
Killingback, J., & Zamani, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.07253 · Cited by 16
“For BM25, we used the implementation from the Pyserini library [36] with default parameters.”
Large Language Models Empowered Personalized Web Agents
Cai, H., Li, Y., Wang, W., Zhu, F., Shen, X., Li, W., & Chua, T.-S. (2025) · Proceedings of the ACM on Web Conference 2025 · doi:10.1145/3696410.3714842 · Cited by 15
“Therefore, we define two metrics as follows: • Function accuracy (function acc): This metric assesses the agent’s ability to select the correct function and provide parameters in the correct format.”
WebFAQ: A Multilingual Collection of Natural Q&A Datasets for Dense Retrieval
Dinzinger, M., Caspari, L., Dastidar, K. G., Mitrović, J., & Granitzer, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3731934 · Cited by 15
Anserini Gets Dense Retrieval: Integration of Lucene's HNSW Indexes
Ma, X., Teofili, T., & Lin, J. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615112 · Cited by 15
“With this addition, Anserini provides a unified interface for dense and sparse retrieval.”
BackgroundSelf-citationTriviaHG: A Dataset for Automatic Hint Generation from Factoid Questions
Mozafari, J., Jangra, A., & Jatowt, A. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657855 · Cited by 15
“In this paper, we present a framework for constructing datasets tailored to hint generation for factoid questions, consisting of two key modules: 1) Question Sampling Module and 2) Hint Generation Module.”
MethodInjecting the score of the first-stage retriever as text improves BERT-based re-rankers
Askari, A., Abolghasemi, A., Pasi, G., Kraaij, W., & Verberne, S. (2024) · Discover Computing · doi:10.1007/s10791-024-09435-8 · Cited by 15
“…of the words that overlap between the query and the document, BM25 calculates a score for the pair: where t is a term, tf t , d is the frequency of t in document d , rsj t is the Robertson-Spärck Jones weight [2] of t , and l is the average document length. k 1 and b are parameters [78, 79].”
AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking
Yoon, S.-Y., Kim, G., Cho, G.-H., & Hwang, S.-W. (2025) · neural information processing systems · doi:10.52202/085713-0890 · Cited by 15
A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage
Xin, R., Mireshghallah, N., Li, S. S., Duan, M., Kim, H., Choi, Y., Tsvetkov, Y., Oh, S., & Koh, P. W. (2026) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml68715.2026.00065 · Cited by 15
“Our analysis contrasts GRIT, a semantic retriever, with BM25 [56], a sparse retriever, while also examining different approaches to construct the query for the retriever.”
AutoMIR: Effective Zero-Shot Medical Information Retrieval without Relevance Labels
Li, L., Zhang, X., Zhou, X., & Liu, Z. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.1305 · Cited by 15
Measuring Bias in a Ranked List Using Term-Based Representations
Abolghasemi, A., Azzopardi, L. A., Askari, A., de Rijke, M., & Verberne, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56069-9_1 · Cited by 15
“For sparse and dense retrieval models we employ the pre-built indexes, and their corresponding query encoders provided by the Pyserini toolkit [25].”
MethodData Augmentation and Large Language Model for Legal Case Retrieval and Entailment
Bui, M.-Q., Do, D.-T., Le, N.-K., Nguyen, D.-H., Nguyen, K.-V.-H., Anh, T. P. N., & Nguyen, L.-M. (2024) · The Review of Socionetwork Strategies · doi:10.1007/s12626-024-00158-2 · Cited by 15
“The models used in the experiments are as follows: – BM25, Dense/Hybrid Retrieval : We use the implementation from Pyserini [15].”
Zhang, D., Tahami, A. V., Abualsaud, M., & Smucker, M. D. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531812 · Cited by 15
GRM: Generative Relevance Modeling Using Relevance-Aware Sample Estimation for Document RetrievalPreprint
Mackie, I., Sekulić, I., Chatterjee, S., Dalton, J., & Crestani, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.09938 · Cited by 15
“We use Pyserini’s [15] “impact” searcher, max-passage aggregatation, and naver/splade-cocondenser-ensembledistil.”
MethodThe Illusion of Certainty: Uncertainty Quantification for LLMs Fails under AmbiguityPreprint
Tomov, T., Fuchsgruber, D., Wollschläger, T., & Günnemann, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.04418 · Cited by 15
“This prevents the model from producing multiple answers in a single generation.”
Meng, S., Liu, J., Chen, Y., Mao, S., Cai, P., Yan, G., Shi, B., & Wang, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.09497 · Cited by 15
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · arXiv · doi:10.48550/arxiv.2305.14251 · Cited by 14
RAGE Against the Machine: Retrieval-Augmented LLM Explanations
Rorseth, J., Godfrey, P., Golab, L., Srivastava, D., & Szlichta, J. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00430 · Cited by 14
“All knowledge sources (documents) are retrieved from our locally-configured document indexes, using a BM25 retrieval model from the Pyserini retrieval toolkit [5].”
MethodUncovering Competing Poisoning Attacks in Retrieval-Augmented Generation
Chen, L., Yang, X., Lu, Y., Zhang, J., Sun, X., Liu, Q., Wu, S., Dong, J., & Wang, L. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818119 · Cited by 14
Axiomatic Retrieval Experimentation with ir_axioms
Bondarenko, A., Fröbe, M., Reimer, J. H., Stein, B., Völske, M., & Hagen, M. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531743 · Cited by 14
“Both Pyserini’s underlying Lucene index [3] and PyTerrier’s underlying Terrier index [29] can be directly accessed by axioms through a unified interface.”
BackgroundMethodInfluentialDeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM DistillationPreprint
Abdallah, A., Mozafari, J., Piryani, B., & Jatowt, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.16998 · Cited by 14
“For each query q i , we retrieve the top-20 candidate passages { d i 1 , d i 2 , . . . , d i 20 } using GPT-4 via RankZephyr (Pradeep et al., 2023b; Lin et al., 2021b).”
CIRAL: A Test Collection for CLIR Evaluations in African Languages
Adeyemi, M., Oladipo, A., Zhang, X., Alfonso-Hermelo, D., Rezagholizadeh, M., Chen, B., Omotayo, A.-H., Abdulmumin, I., Etori, N. A., Musa, T. B., et al. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657884 · Cited by 13
“1 as the default value in Pyserini, and the top-20 passages in the hybrid system are annotated.”
InfluentialSelf-citationBuilding efficient and effective OpenQA systems for low-resource languages
Budur, E., Özçelik, R., Soylu, D., Khattab, O., Güngör, T., & Potts, C. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112243 · Cited by 14
“These results show that ColBERT-QA performs significantly better than BM25, which also outperforms DPR in the task of ranking relevant passages.”
BackgroundHyQE: Ranking Contexts with Hypothetical Query Embeddings
Zhou, W., Zhang, J., Hasson, H., Singh, A. T., & Li, W. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.761 · Cited by 13
Patient2Trial: From patient to participant in clinical trials using large language models
Datta, S., Lee, K., Huang, L., Paek, H., Gildersleeve, R., Gold, J., Pillai, D., Wang, J., Higashi, M. K., Shi, L., et al. (2025) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2025.101615 · Cited by 13
Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs
Wang, L., Zhu, T., Qin, L., Gao, L., & Zhou, W. (2026) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2026.3709319 · Cited by 13
“This dual-stage approach addresses not only the potential bias introduced during the document retrieval process but also any residual bias during the generation phase, creating a comprehensive defense against the multi-faceted nature of BRRA attacks.”
DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation
Abdallah, A., Mozafari, J., Piryani, B., & Jatowt, A. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.306 · Cited by 13
Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation
Zhang, H., Tang, M., Bi, K., Guo, J., Liu, S., Shi, D., Yin, D., & Cheng, X. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.88 · Cited by 13
“Initially, the first-stage retrieval models were predominantly classical term-based models, such as BM25 (Robertson et al., 2009), which combines term matching with TF-IDF weighting.”
InfluentialGroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMs
Long, M., Sun, D., Yang, D., Jiao, Y., Liu, L., Wang, J., Hu, B., Shen, Y., Feng, J., Tan, Z., et al. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1260 · Cited by 13
“We integrate these two signals using a weighted sum of normalized scores: , where S BM25 ( q, d ) and S dense ( q, d ) denote the raw scores from the respective models.”
Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation
Zhang, H., Bi, K., Guo, J., Zhang, J., Wang, S., Yin, D., & Cheng, X. (2025) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3767695.3769496 · Cited by 13
Li, H., Mourad, A., Koopman, B., & Zuccon, G. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531822 · Cited by 13
An Encoder Attribution Analysis for Dense Passage Retriever in Open-Domain Question Answering
Li, M., Ma, X., & Lin, J. (2022) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2022.trustnlp-1.1 · Cited by 13
“Term-matching methods such as BM25 (Robertson and Zaragoza, 2009; Lin et al., 2021) have established strong baselines in the firststage retrieval of various QA tasks (Chen et al.”
MethodInfluentialSelf-citationPre-processing Matters! Improved Wikipedia Corpora for Open-Domain Question Answering
Tamber, M. S., Pradeep, R., & Lin, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28241-6_11 · Cited by 13
Self-citationThe Simplest Thing That Can Possibly Work: (Pseudo-)Relevance Feedback via Text Classification
Han, X., Liu, Y., & Lin, J. (2021) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3471158.3472261 · Cited by 13
“All our experiments were conducted using Pyserini [22], a Python toolkit for reproducible IR research using both sparse and dense representations.”
MethodSelf-citationBM25 Query Augmentation Learned End-to-EndPreprint
Chen, X., & Wiseman, S. M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.14087 · Cited by 13
“Documents were tokenized by BERT’s WordPiece tokenizer and indexed by Pyserini (Lin et al., 2021).”
MethodT1: Tool-integrated Verification for Test-time Compute Scaling in Small Language ModelsPreprint
Kang, M., Jeong, J., & Cho, J. W. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.04718 · Cited by 13
“In addition, we retrieve 3 documents for fact-checking from wikipedia abstracts using BM25 implemtented in Pyserini (Lin et al., 2021).”
Balanced Topic Aware Sampling for Effective Dense Retriever: A Reproducibility Study
Wang, S., & Zuccon, G. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591915 · Cited by 12
Gor, M., Daumé, H., Zhou, T., & Boyd-Graber, J. L. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1201 · Cited by 12
Predicting RAG Performance for Text Completion
Huly, O., Carmel, D., & Kurland, O. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730062 · Cited by 12
“The Pyserini implementation was used with default parameter values [29].”
Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information
Long, K., Li, S., Xu, C., Tang, J., & Wang, T. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730061 · Cited by 12
“Pyserini [17] is utilized with default settings to generate the initial top-100 BM25 first-stage ranking for all datasets.”
DECAF: A Modular and Extensible Conversational Search Framework
Alessio, M., Faggioli, G., & Ferro, N. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591913 · Cited by 12
“Pyserini [24] is a Python toolkit designed as a self-contained package providing a reproducible and easy-to-use first-stage retrieval module within a multi-stage architecture.”
MethodInfluentialHybrid Inverted Index Is a Robust Accelerator for Dense Retrieval
Zhang, P., Liu, Z., Xiao, S., Dou, Z., & Yao, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.116 · Cited by 12
“We compare with three types of retrieval models: 1) Sparse retrievers, including BM25 (Robertson and Zaragoza, 2009), DocT5 (Cheriton, 2019), DeepCT (Dai and Callan, 2019), UniCOIL (Lin and Ma, 2021), and Distil-SPLADE (Formal et al., 2021); 2) Dense retrievers with brute force search (denoted as Flat), including DPR…”
BackgroundMethodInfluentialQueryExplorer: An Interactive Query Generation Assistant for Search and Exploration
Dhole, K., Bajaj, S., Chandradevan, R., & Agichtein, E. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-demo.11 · Cited by 12
Pipeline and dataset generation for automated fact-checking in almost any language
Drchal, J., Ullrich, H., Mlynář, T., & Moravec, V. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10113-5 · Cited by 12
“We employed its Pyserini [62] Python implementation in all our experiments.”
Unveiling the Potential of Corporate Digital Responsibility (CDR)
Basu, S., Jha, S., Ooi, K., & Cham, T. (2024) · Journal of Computer Information Systems · doi:10.1080/08874417.2024.2445745 · Cited by 12
Analyzing and Improving Coherence of Large Language Models in Question Answering
Lauriola, I., Campese, S., & Moschitti, A. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.588 · Cited by 12
“We used an index of Wikipedia para-graphs (Lin et al., 2021) queried through a DPR model 3 .”
Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models
Doostmohammadi, E., Norlund, T., Kuhlmann, M., & Johansson, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.45 · Cited by 12
A Sensitivity Analysis of the MSMARCO Passage CollectionPreprint
Mackenzie, J., Petri, M., & Moffat, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.03396 · Cited by 12
“We use Pyserini [10] to conduct brute force retrieval via FAISS [7].”
MethodEmpowering Retrieval-based Conversational Recommendation with Contrasting User Preferences
Kook, H., Kim, J., Park, S., & Lee, J. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.392 · Cited by 11
“We implemented BM25 (Robertson and Walker, 1994) using Py-serini (Lin et al., 2021), and DPR (Karpukhin et al., 2020) was implemented with the BERT-base model (Devlin et al., 2019).”
TIREx Tracker: The Information Retrieval Experiment Tracker
Hagen, T., Fröbe, M., Reimer, J. H., Scells, H., Hagen, M., & Potthast, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730297 · Cited by 11
“The most popular information retrieval frameworks are written in Java (e.g., Lucene, Anserini, and Terrier [15, 20]) or Python (e.g., Pyserini and PyTerrier [12, 13]).”
Don’t “Overthink” Pointwise Reranking: Is Reasoning Truly Necessary?
Jedidi, N., Chuang, Y.-S., Glass, J. R., & Lin, J. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.2158 · Cited by 11
Self-citationInvestigating the Effects of Sparse Attention on Cross-Encoders
Schlatt, F., Fröbe, M., & Hagen, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56027-9_11 · Cited by 11
Enhancing Dataset Search with Compact Data Snippets
Chen, Q., Chen, J., Zhou, X., & Cheng, G. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657837 · Cited by 11
“For NTCIR-E, we reproduced the BM25 results with Pyserini library [27] 9 following official guidelines [22] 10 .”
Large Language Models Based Stemming for Information Retrieval: Promises, Pitfalls and Failures
Wang, S., Zhuang, S., & Zuccon, G. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657949 · Cited by 11
“The output of the stemming methods was used to index documents and prepare queries using Pyserini [17].”
Query Expansion and Verification with Large Language Model for Information Retrieval
Zhang, W., Liu, Z., Wang, K., & Lian, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5672-8_29 · Cited by 11
Fréchet Distance for Offline Evaluation of Information Retrieval Systems with Sparse Labels
Arabzadeh, N., & Clarke, C. L. A. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.26 · Cited by 12
XLMRQA: Open-Domain Question Answering on Vietnamese Wikipedia-Based Textual Knowledge Source
Nguyen, K. V., Nguyen-Thuan, P., Nguyen, N. D., Huynh, T. V., Nguyen, A. G.-T., & Nguyen, N. L.-T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21743-2_30 · Cited by 11
PyGaggle: A Gaggle of Resources for Open-Domain Question Answering
Pradeep, R., Chen, H., Gu, L., Tamber, M. S., & Lin, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28241-6_10 · Cited by 11
Self-citationLADER: Log-Augmented DEnse Retrieval for Biomedical Literature Search
Jin, Q., Shin, A., & Lu, Z. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592005 · Cited by 11
“We also replace the dense retriever in LADER with BM25, which we denote as LABM25 and implement with Pyserini [24].”
MethodRAVine: Reality-Aligned Evaluation for Agentic SearchPreprint
Xu, Y., Long, X., Zheng, Z., & Gao, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.16725 · Cited by 11
“Specifically, we use the gte-modernbert-base [41] model to build the dense index, implemented with FAISS [7], and adopt BM25 [29] for the lexical index, implemented using Py-serini [20].”
Lawrie, D., Mayfield, J., Yang, E., Yates, A., MacAvaney, S., Pradeep, R., Miller, S., McNamee, P., & Soldani, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.14758 · Cited by 11
Self-citationTwo Heads Are Better Than One: Improving Search Effectiveness Through LLM-Generated Query Variants
Ran, K., Alaofi, M., Sanderson, M., & Spina, D. (2025) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3698204.3716468 · Cited by 10
“We use Okapi’s BM25 [22] as the search retrieval model across all pipelines as implemented in Pyserini [18], with default parameters of 𝑘 1 = 0 .”
Cocktail: Chunk-Adaptive Mixed-Precision Quantization for Long-Context LLM Inference
Tao, W., Zhang, B., Qu, X., Wan, J., & Wang, J. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10992912 · Cited by 10
WebANNS: Fast and Efficient Approximate Nearest Neighbor Search in Web Browsers
Liu, M., Zhong, S., Yang, Q., Han, Y., Liu, X., & Ma, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730115 · Cited by 10
Lai, Y., Wu, J., Wang, Z., & Zhou, D. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.193 · Cited by 10
QueryGym: A Toolkit for Reproducible LLM-Based Query Reformulation
Bigdeli, A., Rad, R. H., Incesu, M., Arabzadeh, N., Clarke, C. L. A., & Bagheri, E. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3793135 · Cited by 10
MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation
Ji, Y., Kwak, M. G., Zhang, H., Wu, X., Li, C., & Wang, Y. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.399 · Cited by 10
Distillation versus Contrastive Learning: How to Train Your Rerankers
Xu, Z., Huang, Z., Zhuang, S., & Srikumar, V. (2025) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2025.findings-ijcnlp.33 · Cited by 10
HintEval: An Open-Source Python Toolkit for Hint Generation and Hint Evaluation
Mozafari, J., Piryani, B., Abdallah, A., & Jatowt, A. (2026) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/2026.emnlp-demos.2 · Cited by 11
Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labeling
Huang, C.-W., Li, C.-A., Hsu, T.-Y., Hsu, C.-Y., & Chen, Y.-N. (2024) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2024.findings-eacl.49 · Cited by 10
Chatty Goose: A Python Framework for Conversational Search
Zhang, E., Lin, S.-C., Yang, J.-H., Pradeep, R., Nogueira, R. F., & Lin, J. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462782 · Cited by 10
“Our framework takes advantage of popular open-source libraries such as Transformers by HuggingFace [20] and ParlAI by Face-book [11], and builds on additional components developed by our research group, including the Pyserini IR toolkit [8] and the Py-Gaggle library for text ranking.”
MethodSelf-citationCREDENCE: Counterfactual Explanations for Document Ranking
Rorseth, J., Godfrey, P., Golab, L., Kargar, M., Srivastava, D., & Szlichta, J. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00285 · Cited by 10
“To facilitate all retrieval functionality, we create a Lucene index using the Pyserini library [9], which is a Python interface for the Anserini retrieval toolkit [10].”
MethodSimple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information RetrievalPreprint
Lin, J., Alfonso-Hermelo, D., Jeronymo, V. A., Kamalloo, E., Lassance, C., Nogueira, R. F., Ogundepo, O., Rezagholizadeh, M., Thakur, N., Yang, J.-H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.01019 · Cited by 10
“paper are implemented in Anserini [47] and Pyserini [25].”
Self-citationText Retrieval with Multi-Stage Re-Ranking ModelsPreprint
Sasazawa, Y., Yokote, K.-I., Imaichi, O., & Sogawa, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.07994 · Cited by 10
“We uses the Pyserini (Lin et al., 2021) toolkit to calculate the similarity by BM25.”
Method
Metadata
- Authors
- Lin, J., Ma, X., Lin, S.-C., Yang, J.-H., Pradeep, R., & Nogueira, R.
- Affiliations
- University of Waterloo (CA)
- Venue
- Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2021-07-11
- Pages
- 2356-2362
- OpenAlex
- W3180230246
- Cited by
- 360
- References
- 41
- Citations by year
- 2026: 67 · 2025: 72 · 2024: 61 · 2023: 97 · 2022: 44 · 2021: 18
- FWCI
- 45.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion, cc-by)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Waterloo-Huawei Joint Innovation Laboratory
- Canada First Research Excellence Fund
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- reproducibility, sparse retrieval, dense retrieval, hybrid retrieval, BM25, transformer representations, nearest neighbor search, neural ranking, relevance assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.