Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations
Open accessThe DOI matches its Crossref record; 39 of the 41 references it cites were found in a registry or an index.
Abstract
Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations. It aims to provide effective, reproducible, and easy-to-use first-stage retrieval in a multi-stage ranking architecture. Our toolkit is self-contained as a standard Python package and comes with queries, relevance judgments, pre-built indexes, and evaluation scripts for many commonly used IR test collections. We aim to support, out of the box, the entire research lifecycle of efforts aimed at improving ranking with modern neural approaches. In particular, Pyserini supports sparse retrieval (e.g., BM25 scoring using bag-of-words representations), dense retrieval (e.g., nearest-neighbor search on transformer-encoded representations), as well as hybrid retrieval that integrates both approaches. This paper provides an overview of toolkit features and presents empirical results that illustrate its effectiveness on two popular ranking tasks. Around this toolkit, our group has built a culture of reproducibility through shared norms and tools that enable rigorous automated testing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Interestingly, the classic BM25 model [18] remains competitive, surpassing some semantic models trained on search behavior datasets.”
From Generative Agents Navigating Digital Libraries (2025) · cites this paper as [18]
In the citing paper
“Sparse lexical methods, from classical BM25 [Lin et al., 2021] to lightweight full-text indexes such as SQLite FTS5, complement dense retrieval by matching exact terms.”
From MemX: A Local-First Long-Term Memory System for AI Assistants (2026) · cites this paper as Lin et al., 2021
Citations
Cites
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“This is particularly the case for work on deep learning today, since the major toolkits (PyTorch [29] and Tensorflow [1]) have both adopted Python as their front-end This work is licensed under a Creative Commons Attribution International 4.”
BackgroundTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“also provides document expansion baselines using our doc2query method [28]; the latest model uses T5 [33] as described in Nogueira and Lin [26].”
MethodTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140 · cited 3×
“Row (2a) refers to brute-force scans over the document vectors in Faiss [12], which provides exact nearest-neighbor search.”
BackgroundMethodDense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Malkov, Y. A., & Yashunin, D. A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889473 · Cited by 1,904 · cited 3×
“To support dense and hybrid retrieval, Pyserini integrates Facebook’s Faiss library for efficient similarity search over dense vectors [12], which in turns integrates the HNSW library [23] to support low-latency querying.”
MethodColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Khattab, O., & Zaharia, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401075 · Cited by 1,301 · cited 8×
“Row (3a) shows the results of combining TCT-ColBERT with BM25 bag-of-words search over the original texts, while row (3b) shows results that combine document expansion using doc2query with the T5 model.”
MethodInfluential
Research works printed but not linked: 6
- Sebastian Hofstatter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2020). · arXiv
- Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2021. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. arXiv:2010.02666 (2021). · arXiv
- Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, and Nazli Goharian. 2021. Simplified Data Wrangling with textttir_datasets. arXiv:2103.02280 (2021). · arXiv
- Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. 2021 c. The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models. arXiv:2101.05667 (2021). · arXiv
- Chenyan Xiong, Zhenghao Liu, Si Sun, Zhuyun Dai, Kaitao Zhang, Shi Yu, Zhiyuan Liu, Hoifung Poon, Jianfeng Gao, and Paul Bennett. 2020 a. CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search. arXiv:2011.01580 (2020). · arXiv
- Yongze Yu, Jussi Karlgren, Hamed Bonab, Ann Clifton, Md Iftekhar Tanveer, and Rosie Jones. 2020. Spotify at the TREC 2020 Podcasts Track: Segment Retrieval. In Proceedings of the Twenty-Ninth Text REtrieval Conference (TREC 2020).
Cited by
Items 701–800 of 809
Closing the Lexical-Semantic Divide: A Unified Approach for Retrieval-Augmented Generation
Sinha, A., & Chakma, K. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-17181-8_13
EDITMEM: Enhancing Multi-Hop Fact Verification Via Editable Memory
Lan, Y., Liu, Z. B., Gu, Y., & Yu, G. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463827
STCALIR: Semi-Synthetic Test Collection for Algerian Legal Information RetrievalPreprint
Hatem, M. A., Mammasse, A., Azouaou, F., & Batata, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6621566
USLU, A. (2025) · Iconic Research and Engineering Journals · doi:10.64388/irev9i5-1716623
Resource-Lean Lexicon Induction for German Dialects
Litschko, R., Plank, B., & Frassinelli, D. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2feouaji2rxe
von Däniken, P., Cieliebak, M., & Deriu, J. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5a8nuzcc3cq3
“We benchmark four retrieval models, one sparse and three dense, using Pyserini 6 (Lin et al., 2021).”
Question answering dataset for information retrieval in Slovak
Hládek, D., Stromko, M., Sopkovič, K., Pleva, M., & Su, M.-H. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3831
CECOR: Correction-oriented synthetic data construction for factual error correctionPreprint
Zhu, L., Wang, X., Yang, J., Wang, C., He, D., Wang, L., & Dang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.02277
“For retrieval, we adopt BM25 [17], implemented via Pyserini [18], and select the top-3 retrieved paragraphs as evidence.”
Self-citationLan, J., Li, F., & Zhang, L. N. (2026) · International Conference on Communications, Information System and Computer Engineering (CISCE) · doi:10.1109/cisce69494.2026.11504788
“Standard paradigms, such as the classic “retrieve-then-read” approach, typically utilize sparse (e.g., BM25 [9]) or dense retrieval [10] to fetch top-k documents.”
Attention-Driven Keyword Extraction for Conversational Retrieval-Augmented Generation
Zhao, L., & Huang, D. (2026) · International Conference on Computer, Artificial Intelligence and Control Engineering (CAICE) · doi:10.1145/3804601.3804655
A Memory-Augmented Conversational AI Framework Using Vector Databases and Real-Time API IntegrationPreprint
Charuhasini, P. (2026) · Research Square · doi:10.21203/rs.3.rs-9809663/v1
Iterative conceptual query expansion for biomedical information retrieval
Zhou, C., Tu, X., Chen, G., He, T., Bai, X., & Liu, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133049
Exploring the Performance of Vespa Vector DB
Gill, S., Goyat, D. S., & Saini, M. (2026) · International Journal of Drug Delivery Technology · doi:10.25258/ijddt.16.34s.105
Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search
Cho, G.-H., Lee, Y., Jeong, K., Lee, S., Han, S., Déjean, H., Clinchant, S., & Hwang, S.-W. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840581
Darold, D. L. (2026) · Research Square · doi:10.21203/rs.3.rs-9589637/v1
Fröbe, M. (2026) · Common Library Network (Der Gemeinsame Bibliotheksverbund) · doi:10.22032/dbt.70453
ViQR-Decider: Lightweight Vietnamese Query Rewriting with Decider Tokens and Synthetic Data
Thai, P. K., & Quan, T. T. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-0263-8_2
UR-RAG: Unified Risk Calibration for Retrieval-Augmented GenerationPreprint
Mittal, S., FARAG, M. E., & Lu, M. (2026) · Research Square · doi:10.21203/rs.3.rs-9985651/v1
Orcheo: A Modular Full-Stack Platform for Conversational Search
Jiang, S., Vakulenko, S., & de Rijke, M. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808613
Dora, P. K. (2024) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.5337
“This complementarity has led to the increasing adoption of hybrid retrieval strategies, which integrate both approaches to leverage their respective strengths [9,10].”
RETRIEVAL-AUGMENTED GENERATION: СУЧАСНІ МЕТОДИ ПОКРАЩЕННЯ ЯКОСТІ ПОШУКУ В ГІБРИДНИХ АРХІТЕКТУРАХ
Лабудяк, Б., & Pereiaslavska, S. (2026) · ГРААЛЬ НАУКИ · doi:10.36074/grail-of-science.29.05.2026.121
The impact of using Large Language Models on the performance of Recommender Systems
Schmiedmayer, M. (2026) · reposiTUm (TU Wien) · doi:10.34726/hss.2026.118745
GPUSparse: Fast and Exact Learned Sparse Retrieval on a GPU-Resident Inverted IndexPreprint
Sharma, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.26441
ProofFusion: Improving Neural Theorem Proving via Adaptive Retrieval-Augmented Reasoning
Zhang, M., Dong, Y., Zhou, L., Xiao, B., & Liu, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797139
Wen, T., Chen, X., He, B., Cong, X., & Sun, L. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1840
“We utilized the BM25 algorithm from the Pyserini(Lin et al., 2021) with its default settings.”
Banaybanay, R. J., & Ricafort, R. B. (2026) · International Journal of Latest Technology in Engineering Management & Applied Science · doi:10.51583/ijltemas.2026.150600016
González-Alvarez, C., Fernández-Robles, L., Alegre, E., & Castejón‐Limas, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7068524
Ramesh, G., & Chaturvedula, S. K. (2026) · International Conference on Innovations in Emerging Technologies for Sustainable Development (ICIETSD) · doi:10.1109/icietsd68684.2026.11584705
“5 [4], likely reflecting the higher importance of semantic understanding in medical terminology.”
From Translation to Retrieval: Evaluating LLM-Based Information Retrieval for Hausa and Fongbe
Adjovi, M. P., Eiselen, R., & Mitra, P. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808531
Retrieve, Rerank, Answer, Experiment: Hands-On IR Research with PyTerrier
Macdonald, C., MacAvaney, S., Tonellotto, N., & Wang, X. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808640
“We show how PyTerrier supports multiple engines, specifically Ter-rier [39], PISA [21, 33], and Anserini [16, 51].”
Mikka-Muntuumo, J., Lazarus, W., & Indongo, N. N. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808527
Analysis of Reasoning-Intensive Retrieval in Japanese
Kurokawa, H., Kameko, H., & Mori, S. (2026) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3805713.3820433
Search Arena Meets Nuggets: Towards Explanations and Diagnostics in the Evaluation of LLM Responses
Sharifymoghaddam, S., Upadhyay, S., Thakur, N., Pradeep, R., & Lin, J. (2026) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3805713.3820418
Self-citationRanking Passages in Relevant Documents Using LLMs
Ani, E. E., Sheetrit, E., & Kurland, O. (2026) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3805713.3820407
“The Pyserini toolkit 9 is used for sparse retrieval experiments [36].”
Does Form Affect Function? An Extended Study of LLM Re-Ranking Behavior
Goli, R., & Moffat, A. (2026) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3805713.3820410
“For each query in each dataset, Sun et al. use BM25 similarity scoring (using Pyserini [21]) to identify the top-100 candidates, then apply the re-rankers to produce new orderings which are scored against the corresponding qrels to obtain per-query NDCG @10 values.”
Tool-Adaptive LLM RerankerPreprint
Liu, Z., & Hua, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.10555
Revisiting Poison Pills for Neural Information Retrieval
Menachem, M., Kurland, O., & Raiber, F. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809890
“The experiments were performed using Pyserini [18].”
Consensus-Anchored Expansion for Noise-Resilient Re-ranking
Subedi, N., & Shen, C. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809634
“Methods like learned hybridization [13] or post-hoc calibration [3] optimize score combinations but generally operate on the output distribution.”
The Relevance of (Relevant) Entity Presence
Boudens, N., Kamphuis, C., & de Vries, A. P. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808572
Nguyen, B.-G., & Le, Q.-H. (2026) · Journal of Communications Software and Systems · doi:10.24138/jcomss-2026-0036
“We use Pyserini [41] for BM25 retrieval implementation, while pre-processing and normalization employ spaCy [42], NLTK [43], and scispaCy [44].”
CIR at iKAT SCAI 2026: Exploring Clarification Need Prediction in Agentic Conversational SearchPreprint
Bernard, N., Keller, J., & Schaer, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.19801
“For the retrieval, we use the Pyserini [6] and PyTerrier t5 9 libraries.”
ADAPTIVE MULTI-STAGE VECTOR RETRIEVAL FOR RETRIEVAL-AUGMENTED GENERATION
Bankole, S. A., & Saheed, Y. K. (2026) · NLP & Big Data · doi:10.5121/csit.2026.1601307
Enhancing Arabic Document Re-ranking with BERT-Based Relevance Classification
Benali, M., & Lakhyar, Z. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-23844-3_41
Beyond the Window: Scaling Listwise LLM Reranking via Candidate Filtering
Remy, L., Bringay, S., Poncelet, P., & Servajean, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-34374-1_13
Contextual Tokenization for Graph Inverted Indices
Chakraborty, P., Roy, I., Chakrabarti, S., & De, A. (2025) · neural information processing systems · doi:10.52202/085713-1980
Evaluating Sparse-Dense Hybrid Fusion Strategies on the Indonesian Mr. TyDi Benchmark
Saragih, M. A. A. R., Yulianti, E., Alfina, I., & Budi, I. D. (2026) · International Conference on Advancement in Data Science, E-learning and Information System (ICADEIS) · doi:10.1109/icadeis71120.2026.11644578
“1-indonesia [13].”
Ghosh, U. K., & Chatterjee, S. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3841055
“BM25 is run via Pyserini [11]’s prebuilt Lucene indices with parameters 𝑘 1 = 0 .”
Albishre, K. (2026) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.18857
Azami, Y., Boulibya, H., Mahraz, M. A., Oulhadj, M., Tairi, H., Aboussaleh, I., Fazazy, K. E., & Riffi, J. (2026) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2026.101810
Enhancing Search Systems with Large Language Models
Peng, Z. (2026) · Scholar Commons (Santa Clara University)
Biancofiore, G. M., Palma, D. D., Pomo, C., Boratto, L., Noia, T. D., & Narducci, F. (2026) · Computer Science Review · doi:10.1016/j.cosrev.2026.101058
Tracing Query Expansion Effects through Sparse Autoencoder Features
Dong, F., Shi, W., Xu, Z., Ge, X., He, B., Xin, X., Chen, Z., & Zhou, Y. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1152
Daş, B., Savcı, P., Gökçimen, T., & Özek, M. B. (2026) · Applied Sciences · doi:10.3390/app16189003
Leite, J. A. (2026) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Route Me If You Can: A Benchmark for Query Reformulation Selection
Le, H. S., Arabzadeh, N., Bigdeli, A., Rad, R. H., Ebrahimi, S., Clarke, C. L. A., & Bagheri, E. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840215
dos Santos, B. L. M. (2026) · DSpace (Federal University of Santa Catarina)
Efficient training-free query routing framework for federated retrieval-augmented generation
Eoo, H., Yoo, Y., & Yang, G. (2026) · Scientific Reports · doi:10.1038/s41598-026-75918-x
Miyaguchi, A., Afrulbasha, I., & Pramov, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39150-6_17
No More Free Lunch: Corpus Task Complexity Matters as Corpora GrowPreprint
Singhal, P., Bertsch, A., Steinhardt, J., & Min, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.29245
“…(sourcing train and eval from existing train / eval set) [Izacard et al., 2022], with all documents — gold, hard negative and filler — drawn from the 100-word Wikipedia DPR corpus [Karpukhin et al., 2020] served by pyserini [Lin et al., 2021], so every document shares one surface format.”
A Systematic Multi-Domain Evaluation of Document RetrieversPreprint
Velev, V., & Spitz, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.29455
Laijun, Z. H. A. O., Gao, Y., Zhou, L., Li, H., & Yang, P. (2026) · Research Square · doi:10.21203/rs.3.rs-10417423/v1
Effective Dense Retrieval using Only In-Context ExamplesPreprint
Jedidi, N., Ali, A. B., Li, H., & Lin, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.38099
“We utilize vLLM (Kwon et al., 2023) for LLM inference, FAISS (Douze et al., 2025) for indexing and search, and Pyserini (Lin et al., 2021) for its BM25 implementation and retrieval evaluation.”
Self-citationСервісний семантичний пошук з ранжуванням на основі параметрів якості обслуговування
Zarichnyi, Y. (2026) · Системи обробки інформації · doi:10.30748/soi.2026.186.18
Fairness in information retrieval: algorithms, evaluation, and scalable annotation
Chen, F. (2025) · Library, Museums and Press - UDSpace (University of Delaware)
Conversational search : from Information retrieval to retrieval-augmented generation
Mo, F. (2026) · Papyrus : Institutional Repository (Université de Montréal) · doi:10.71781/36302
Anticipating and resolving ad hoc information needs during discourse
Ros, K. (2025) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic SearchPreprint
Lin, J., Sharifymoghaddam, S., Gu, L., & Jedidi, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.11922
Self-citationLearning Multi-Step Query Rewriting via Corpus Feedback for Conversational SearchPreprint
Coelho, J., Wang, H., Yuan, J., Wang, Z., Koelle, S., & Niu, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.10955
“68 for QReCC) as our sparse retriever through Py-serini [23].”
Etar, A., Valle, S., Soni, J., & Upadhyay, H. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-28454-9_15
LLM Evaluation is a Forecasting ProblemPreprint
Thomas, R., Block, A., & Goldblum, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.17018
XViewRank: Overlap-Conditioned Score Fusion for LLM-Augmented Sparse RetrievalPreprint
Lv, X., Li, M., & Zhou, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.20122
“Retrieval is performed with Pyserini BM25 [20].”
The Fusion Tax: A Diagnostic Analysis of When Hybrid Retrieval Hurts
Chauhan, A., & Agarwal, A. (2026) · IEEE Access · doi:10.1109/access.2026.3743502
“BM25 [1] remains a strong and cheap baseline, especially for keyword-and entity-heavy queries, and its Anserini/Pyserini implementation [11], [12] is the reference point for BEIR [13], [14].”
InfluentialCTS-CRL: Context Turn Selection for Conversational Rewriting with Joint Learning
Zhang, Y., Bei, G., Zhe, S., Li, J., & Wang, Y. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729850
“The evaluation is conducted using the Pyserini toolkit [22] to ensure reproducibility.”
Beyond Representation: Index Structure Limitations in Dense Retrieval
Magnani, A. (2026) · International Conference on Advanced Machine Learning and Data Science (AMLDS) · doi:10.1109/amlds69611.2026.11723632
“• BM25 : Anserini/Lucene via Pyserini [16] with default parameters. f) Metric and reproducibility.”
Optimal Fusion Weights Form Intervals, Not Points: A Latency-Aware Study of Query-Adaptive WRRFPreprint
Ma, I., Shekarchi, Z., Mcilroy, S., Farrokh, M., Zarrinkalam, F., & Stremmel, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.24341
“We pair two sparse and two dense retrievers into four hybrid configurations: • Sparse: BM25 (Robertson and Zaragoza, 2009) (via Pyserini (Lin et al., 2021)) and RM3 (Abdul-Jaleel et al., 2004; Lavrenko and Croft, 2017) (BM25 pseudo-relevance feed-back query expansion).”
Let Agents Design How They Search: Automatic Interface Optimization for Agentic SearchPreprint
An, S., Jung, J., Choi, Y., Baek, J., Cho, J., Hwang, S. J., & Lee, H. B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.29250
“On BRIGHT, Agentic Search (Dense) has the search tool only and encodes each query at search time against Pyserini’s (Lin et al., 2021) prebuilt ReasonEmbed-Qwen3-4B document embeddings.”
Interaction-Budget-Matched Analysis of Compact Cross-Encoder Reranking in Hybrid Retrieval
Jamadi, I. M., Anuar, S., Ijad, & Alkaoud, M. (2026) · IEEE Access · doi:10.1109/access.2026.3745756
“Pyserini supports sparse, dense, and hybrid retrieval in one environment [6].”
Acharya, P. (2026) · Dublin City University Open Access Institutional Repository (Dublin City University)
ACARE: Agent-Context-Aware Reranker for Iterative SearchPreprint
Sharifymoghaddam, S., Babakhin, Y., Esfandiarpoor, R., Holworthy, O., Lin, J., & Liu, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.35840
“4 (Lin et al., 2021).”
Self-citationWhen Retrieval Similarity Hurts: A Diagnostic Framework for Document Weighting in Black-Box RAG
Seo, D., & Jang, B. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.594
RAMP: Adapting One Agent to Multiple Retrievers via Behavioral Probing
Li, W., Yang, T., Yang, J., & Chao, F. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1977
Self-citationGutenBench: A Taxonomy-Based Dataset for Evaluation of Information Retrieval Models
Monteiro, M., Senkin, D., Ravichander, A., Kloft, M., & Fellenz, S. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1342
Mandiga, A., Li, G., & Lu, J. (2026) · doi:10.18653/2026.uncertainlp-main.34
Implicit Hierarchical Modeling for Cross-System Legal Case Retrieval
Zhong, Z., Lan, H., Chen, Y., Hua, S., Liang, K., Li, X., Lin, X., Li, H., & Ji, R. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.853
A Sequential Decision Framework for Evaluating LLM-Guided SearchPreprint
Huo, S., Parry, A., Ganguly, D., Smucker, M., & Clarke, C. L. A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.39231
“The toolkit supports a variety of LLM-based reformulation strategies, integrates with common retrieval backends such as Pyserini [35] and PyTerrier [42], and provides infrastructure for reproducible experimentation.”
Repairs for Whom? Evaluator-Executor Mismatch in Selecting Repairs for Failed LLM AgentsPreprint
Liao, Y., & Liang, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.38610
“…modification, with the 100,000-product configuration that the code supports (the first 100,000 products of the full product file and their attributes) and the 1,021 human-written instructions on those products; search uses the original BM25 index built with Pyserini 2.3.0 (Lin et al., 2021).”
InfluentialOrbit: Open Re-implementation of Biomedical Literature APIs
Ghosh, S., Okur, M., & Scells, H. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840238
On Non-Relevance Signals in Pointwise LLM-based Reranking
Katsuno, A., Okugawa, Y., Tanaka, T., Nagao, T., & Kawata, H. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3839921
Multi-Aspect Joint Retrieval for E-Commerce: Bridging Product Catalogs and Customer Reviews
Alessio, M., Muntean, C. I., Nardini, F. M., & Perego, R. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3839931
ReCaRe: A Bilingual Benchmark for Revision Candidate Retrieval in Evolving Legal Corpora
Ito, T., Kurokawa, Y., Kato, M. P., & Fujita, S. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840205
Query-Aware Index Pruning for Retrieval Under Budget Constraints
Campese, S., Lauriola, I., & Moschitti, A. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840963
Cite Unseen: Measuring Citation-Channel Vulnerabilities in Retrieval-Augmented Generation
Hur, M. G., Shin, J., Kim, D., & Min, M. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3839886
Attribution as Click: Understanding and Modeling Attribution Bias in RAG
Mousavian, M., Aliannejadi, M., Chen, X., & Crestani, F. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3841167
1ShotBench: Demonstrating One-Shot Prompt Reproducibility for Autonomous Coding Agents
Li, N., Ge, Y., Ma, X., & Lin, J. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3840271
Self-citationO2N2: Efficient Doubly Oblivious Approximate Nearest Neighbor SearchPreprint
Li, Z., Guo, Z., Zhou, M., Shi, C., & Hong, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.03415
User-Centered Design of Interactive Recommender Systems for Cancer Patients and Caregivers
Brusilovsky, P. L., Thaker, K. M., Hassany, M., Rahdari, B., Hui, V. C. C., Wang, Y., Donovan, H. S., Klem, M. L., He, D., & Lee, Y. J. (2026) · ACM Transactions on Recommender Systems · doi:10.1145/3849696
Do LLMs Have Social Sense? Evaluating Alignment of Conversation Understanding and Reasoning
Wu, Z., & Goldwasser, D. (2026) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/2026.aacl-long.261
Beyond Ranking: Are Decision Models Calibrated Relevance Scorers?Preprint
Zhuang, S., Ma, X., Petri, M., & Lin, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.11480
Self-citation
Metadata
- Authors
- Lin, J., Ma, X., Lin, S.-C., Yang, J.-H., Pradeep, R., & Nogueira, R.
- Affiliations
- University of Waterloo (CA)
- Venue
- Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2021-07-11
- Pages
- 2356-2362
- OpenAlex
- W3180230246
- Cited by
- 360
- References
- 41
- Citations by year
- 2026: 67 · 2025: 72 · 2024: 61 · 2023: 97 · 2022: 44 · 2021: 18
- FWCI
- 45.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion, cc-by)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Waterloo-Huawei Joint Innovation Laboratory
- Canada First Research Excellence Fund
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- reproducibility, sparse retrieval, dense retrieval, hybrid retrieval, BM25, transformer representations, nearest neighbor search, neural ranking, relevance assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.