Anserini: Enabling the Use of Lucene for Information Retrieval Research
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Software toolkits play an essential role in information retrieval research. Most open-source toolkits developed by academics are designed to facilitate the evaluation of retrieval models over standard test collections. Efforts are generally directed toward better ranking and less attention is usually given to scalability and other operational considerations. On the other hand, Lucene has become the de facto platform in industry for building search applications (outside a small number of companies that deploy custom infrastructure). Compared to academic IR toolkits, Lucene can handle heterogeneous web collections at scale, but lacks systematic support for evaluation over standard test collections. This paper introduces Anserini, a new information retrieval toolkit that aims to provide the best of both worlds, to better align information retrieval practice and research. Anserini provides wrappers and extensions on top of core Lucene libraries that allow researchers to use more intuitive APIs to accomplish common research tasks. Our initial efforts have focused on three functionalities: scalable, multi-threaded inverted indexing to handle modern web-scale collections, streamlined IR evaluation for ad hoc retrieval on standard test collections, and an extensible architecture for multi-stage ranking. Anserini ships with support for many TREC test collections, providing a convenient way to replicate competitive baselines right out of the box. Experiments verify that our system is both efficient and effective, providing a solid foundation to support future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Relevance-Based Language Models
Lavrenko, V., & Croft, W. B. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130376 · Cited by 1,422
Improvements that don't add up: ad-hoc retrieval results since 1998
Armstrong, T. G., Moffat, A., Webber, W., & Zobel, J. (2009) · Proceedings of the 18th ACM conference on Information and knowledge management · doi:10.1145/1645953.1646031 · Cited by 234
Neural Models for Information RetrievalPreprint
Mitra, B., & Craswell, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.01509 · Cited by 219
Improvements to BM25 and Language Models Examined
Trotman, A., Puurula, A., & Burgess, B. (2014) · Proceedings of the 2014 Australasian Document Computing Symposium · doi:10.1145/2682862.2682863 · Cited by 183
Effectiveness/efficiency tradeoffs for candidate generation in multi-stage retrieval architectures
Asadi, N., & Lin, J. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484132 · Cited by 96
Self-citationFerro, N., Fuhr, N., Järvelin, K., Kando, N., Lippold, M., & Zobel, J. (2016) · ACM SIGIR Forum · doi:10.1145/2964797.2964808 · Cited by 83
Arguello, J., Crane, M., Díaz, F., Lin, J., & Trotman, A. (2016) · ACM SIGIR Forum · doi:10.1145/2888422.2888439 · Cited by 68
Self-citationOverview of the TREC 2013 Session Track
Carterette, B., Bah, A., Kanoulas, E., Hall, M. M., & Clough, P. D. (2013) · Cited by 46
Clarke, C. L. A., Culpepper, J. S., & Moffat, A. (2016) · Information Retrieval · doi:10.1007/s10791-016-9279-1 · Cited by 34
Cited by
Latent retrieval for weakly supervised open domain question answering
Lee, K., Chang, M.-W., & Toutanova, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1612 · Cited by 771
“It has been shown to be robust for both traditional information retrieval tasks, and evidence retrieval for question answering (Yang et al., 2017 BM25 is not trainable, the retrieved evidence considered during fine-tuning is static.”
BackgroundDocument Ranking with a Pretrained Sequence-to-Sequence Model
Nogueira, R., Jiang, Z., Pradeep, R., & Lin, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.63 · Cited by 448
Self-citationEnd-to-End Open-Domain Question Answering with
Yang, W., Xie, Y., Lin, A., Li, X., Tan, L., Xiong, K., Li, M., & Lin, J. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-4013 · Cited by 391
Self-citationSparse, Dense, and Attentional Representations for Text Retrieval
Luan, Y., Eisenstein, J., Toutanova, K., & Collins, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00369 · Cited by 363
Lin, J., Ma, X., Lin, S.-C., Yang, J.-H., Pradeep, R., & Nogueira, R. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3463238 · Cited by 360
Self-citationQu, Y., Ding, Y., Liu, J., Liu, K., Ren, R., Zhao, W. X., Dong, D., Wu, H., & Wang, H. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.466 · Cited by 356
Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling
Hofstätter, S., Lin, S.-C., Yang, J.-H., Lin, J., & Hanbury, A. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462891 · Cited by 293
Self-citationOptimizing Dense Retrieval Model Training with Hard Negatives
Zhan, J., Mao, J., Liu, Y., Guo, J., Zhang, M., & Ma, S. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462880 · Cited by 248
Generation-Augmented Retrieval for Open-Domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.316 · Cited by 146
“We use Anserini (Yang et al., 2017) for text retrieval of BM25 and G AR with its default parameters.”
MethodOpen-Domain Question Answering Goes Conversational via Question Rewriting
Anantha, R., Vakulenko, S., Tu, Z., Longpre, S., Pulman, S., & Chappidi, S. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.44 · Cited by 130
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., & Gurevych, I. (2021) · arXiv · doi:10.48550/arxiv.2104.08663 · Cited by 45
Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selection
Arabzadeh, N., Yan, X., & Clarke, C. L. A. (2021) · Proceedings of the 30th ACM International Conference on Information & Knowledge Management · doi:10.1145/3459637.3482159 · Cited by 41
“As our sparse retriever, we employ BM25 as implemented by the open-source Anserini system from the University of Waterloo [25], which provides state-of-the-art performance for sparse retrievers.”
MethodLearning to retrieve passages without supervision
Ram, O., Shachaf, G., Levy, O., Berant, J., & Globerson, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.193 · Cited by 35
Generation-Augmented Retrieval for Open-domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2020) · arXiv · doi:10.48550/arxiv.2009.08553 · Cited by 31
Rawassizadeh (2023) · doi:10.1145/3569488 · Cited by 11
Metadata
- Authors
- Yang, P., Fang, H., & Lin, J.
- Venue
- Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2017-08-07
- Pages
- 1253-1256
- OpenAlex
- W2740321901
- Cited by
- 319
- References
- 9
- FWCI
- 19.06
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.