The Probabilistic Relevance Framework: BM25 and Beyond
The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
The Probabilistic Relevance Framework (PRF) is a formal framework for document retrieval, grounded in work done in the 1970–1980s, which led to the development of one of the most successful text-retrieval algo¬rithms, BM25. In recent years, research in the PRF has yielded new retrieval models capable of taking into account document meta-data (especially structure and link-graph information). Again, this has led to one of the most successful Web-search and corporate-search algo¬rithms, BM25F. This work presents the PRF from a conceptual point of view, describing the probabilistic modelling assumptions behind the framework and the different ranking algorithms that result from its application: the binary independence model, relevance feedback mod¬els, BM25 and BM25F. It also discusses the relation between the PRF and other statistical models for IR, and covers some related topics, such as the use of non-textual features, and parameter optimisation for models with free parameters.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Most flat baselines also lose ground as pressure rises; BM25 drops from 0 . 808 at 3 -hop to 0 . 425 at 5 / 6 -hop, confirming that the difficulty here is driven by the memory dimension rather than by task length per se.”
From MemGym: a Long-Horizon Memory Environment for LLM Agents (2026) · cites this paper as [38]
In the citing paper
“The combination of vector and BM25 [Robertson and Zaragoza, 2009] retrieval through RRF fusion [Cormack et al., 2009] consistently outperforms any single approach, consistent with findings in hybrid search literature [Ma et al., 2021].”
From Cognis: Context-Aware Memory for Conversational AI Agents (2026) · cites this paper as Robertson and Zaragoza, 2009
In the citing paper
“A streamlined dual-store architecture combining OpenSearch (for documents and native BM25 [Robertson and Zaragoza, 2009] search) with a vector database (for Matryoshka [Kusupati et al., 2022] embeddings at 768D and 256D)”
From Cognis: Context-Aware Memory for Conversational AI Agents (2026) · cites this paper as Robertson and Zaragoza, 2009
In the citing paper
“The BM25 retriever clearly outperformed all dense retrievers.”
From In-Context Retrieval-Augmented Language Models (2023) · cites this paper as Robertson and Zaragoza, 2009
In the citing paper
“a 345M parameter GPT-2 enhanced by In-Context RALM outperforms a 762M parameter GPT-2 when employing an off-the-shelf BM25 retriever (Robertson and Zaragoza, 2009)”
From In-Context Retrieval-Augmented Language Models (2023) · cites this paper as Robertson and Zaragoza, 2009
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Probabilistic latent semantic indexing
Hofmann, T. (1999) · Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/312624.312649 · Cited by 3,976
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
THE PROBABILITY RANKING PRINCIPLE IN IR
ROBERTSON, S. E. (1977) · Journal of Documentation · doi:10.1108/eb026647 · Cited by 1,103
Self-citationRelevance based language models
Lavrenko, V., & Croft, W. B. (2001) · Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/383952.383972 · Cited by 1,037
A probabilistic model of information retrieval: development and comparative experiments
Sparck Jones, K., Walker, S., & Robertson, S. E. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(00)00015-7 · Cited by 942
Self-citationOn Relevance, Probabilistic Indexing and Information Retrieval
Maron, M. E., & Kuhns, J. L. (1960) · Journal of the ACM · doi:10.1145/321033.321035 · Cited by 907
Probabilistic models of information retrieval based on measuring the divergence from randomness
Amati, G., & Van Rijsbergen, C. J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582416 · Cited by 883
A Markov random field model for term dependencies
Metzler, D., & Croft, W. B. (2005) · Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1076034.1076115 · Cited by 843
Document language models, query models, and risk minimization for information retrieval
Lafferty, J., & Zhai, C. (2001) · Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/383952.383970 · Cited by 530
USING PROBABILISTIC MODELS OF DOCUMENT RETRIEVAL WITHOUT RELEVANCE INFORMATION
CROFT, W. B., & HARPER, D. J. (1979) · Journal of Documentation · doi:10.1108/eb026683 · Cited by 433
Research works printed but not linked: 21
- Agarwal, 2005, Proceedings of the NIPS 2005 Workshop on Learning to Rank
- Beaulieu, 1997, Okapi at TREC-5, 143
- Berghen, 2004, CONDOR: A constrained, non-linear, derivative-free parallel optimizer for continuous, high computing load, noisy objective functions
- Blei, 2003, Latent dirichlet allocation, Journal of Machine Learning Research, 3, 993
- Boldi, 2005, MG4J at TREC 2005, 10.6028/NIST.SP.500-266.terabyte-umilano.vigna · doi:10.6028/nist.sp.500-266.terabyte-umilano.vigna
- Craswell, 2005, Microsoft Cambridge at TREC-14: Enterprise track, 10.6028/NIST.SP.500-266.enterprise-microsoft.robertson · doi:10.6028/nist.sp.500-266.enterprise-microsoft.robertson
- Lafferty, 2003, Language Modelling for Information Retrieval, 1
- Metzler, 2009, Information Retrieval in Practice
- Pérez-Agüera, 2008, UCM-Y!R at CLEF 2008 Robust and WSD tasks
- Pérez-Agüera, 2008, Exploiting morphological query structure using genetic optimization
- Robertson, 1983, The unified probabilistic model for IR, 108
- Robertson, 1977, Relevance weighting of search terms, Journal of the American Society for Information Science
- Robertson, 1981, Probabilistic models of indexing and searching, 35
- Robertson, 1994, Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval, 232
- Robertson, 2007, On relevance weights with little relevance information, 16
- Robertson, 1992, Okapi at TREC, 21
- Robertson, 2004, Simple BM25 extension to multiple weighted fields, 42
- Song, 2008, Viewing term proximity from a different perspective, 346
- Tao, 2007, An exploration of proximity measures in information retrieval, 295
- van Os, 2006, PF/Tijah: Text search in an XML database system, 12
- Zaragoza, 2005, Microsoft Cambridge at TREC 2004: Web and HARD track
Cited by
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2023) · arXiv · doi:10.48550/arxiv.2307.06435 · Cited by 1,259
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
Autonomous chemical research with large language models
Boiko, D. A., MacKnight, R., Kline, B., & Gomes, G. (2023) · Nature · doi:10.1038/s41586-023-06792-0 · Cited by 1,050
“At inference time, all documents containing words from a query are selected and ranked based on various manually defined formulas 42 .”
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876
“The development of Advanced RAG and Modular RAG was a response to these specific shortcomings in Naive RAG.”
BackgroundSiren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al. (2025) · Computational Linguistics · doi:10.1162/coli.a.16 · Cited by 534
“The evidence retrieval process typically employs various sparse (e.g., BM25 (Robertson et al., 2009)) or dense (e.g., PLM-based meth-ods (Zhao et al., 2022)) retrievers.”
Abdelrahman, G., Wang, Q., & Nunes, B. (2023) · ACM Computing Surveys · doi:10.1145/3569576 · Cited by 468
Active Retrieval Augmented Generation
Jiang, Z., Xu, F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.495 · Cited by 497
Short Text Similarity with Word Embeddings
Kenter, T., & de Rijke, M. (2015) · Proceedings of the 24th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2806416.2806475 · Cited by 440
In-Context Retrieval-Augmented Language Models
Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., & Shoham, Y. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00605 · Cited by 449
Sparse, Dense, and Attentional Representations for Text Retrieval
Luan, Y., Eisenstein, J., Toutanova, K., & Collins, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00369 · Cited by 363
Sentiment Analysis: Mining Opinions, Sentiments, and Emotions
Liu, B. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108639286 · Cited by 362
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022) · arXiv · doi:10.48550/arxiv.2212.10509 · Cited by 349
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.557 · Cited by 349
Learning To Retrieve Prompts for In-Context Learning
Rubin, O., Herzig, J., & Berant, J. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.191 · Cited by 374
Gupta, S., Ranjan, R., & Singh, S. N. (2024) · arXiv · doi:10.48550/arxiv.2410.12837 · Cited by 342
Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.546 · Cited by 340
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00530 · Cited by 337
“Traditionally sparse vector methods such as TF-IDF and BM25 are used for document retrieval (Robertson and Zaragoza, 2009).”
MethodLarge Language Models are Zero-Shot Rankers for Recommender Systems
Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J., & Zhao, W. X. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-56060-6_24 · Cited by 327
Matching patients to clinical trials with large language models
Jin, Q., Wang, Z., Floudas, C. S., Chen, F., Gong, C., Bracken-Clarke, D., Xue, E., Yang, Y., Sun, J., & Lu, Z. (2024) · Nature Communications · doi:10.1038/s41467-024-53081-z · Cited by 287
Matching Patients to Clinical Trials with Large Language Models
Jin, Q., Wang, Z., Floudas, C. S., Chen, F., Gong, C., Bracken-Clarke, D., Xue, E., Yang, Y., Sun, J., & Lu, Z. (2023) · arXiv · doi:10.48550/arxiv.2307.15051 · Cited by 287
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240
“The most prevalent application of retrieval lies in web search engines, which primarily focus on the task of document retrieval [19], [20].”
InfluentialThe power of noise: Redefining retrieval for rag systems
Cuconasu, F., Trappolini, G., Siciliano, F., Filice, S., Campagnano, C., Maarek, Y., Tonellotto, N., & Silvestri, F. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657834 · Cited by 236
Dense Text Retrieval Based on Pretrained Language Models: A Survey
Zhao, W. X., Liu, J., Ren, R., & Wen, J.-R. (2024) · ACM Transactions on Information Systems · doi:10.1145/3637870 · Cited by 211
“order to better understand the underlying retrieval mechanism, probabilistic relevance frameworks have been proposed for relevance modeling, exemplified by the classic BM25 model [16], [17].”
Early Detection of Depression: Social Network Analysis and Random Forest Techniques
Cacheda, F., Fernandez, D., Novoa, F. J., & Carneiro, V. (2019) · Journal of Medical Internet Research · doi:10.2196/12554 · Cited by 206
“The Okapi BM25 similarity between 2 subjects q and d was scored as equation c in Figure 4, following the study by Robertson and Zaragoza [41], where k 1 is a scaling factor for the term frequency, b is a scale factor on the document length, and l is the average number of terms in a document.”
(2022) · doi:10.18653/v1/2022.emnlp-main.669 · Cited by 177
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
“These methods, which include Boolean Retrieval [7], BM25 [8], SPLADE [9], and UniCOIL [10], establish connections between vocabulary and documents, offering high retrieval efficiency and robust system performance.”
Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Nashid, N., Sintaha, M., & Mesbah, A. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00205 · Cited by 178
Retrieval-augmented generation for educational application: A systematic survey
Li, Z., Wang, Z., Wang, W., Hung, K., Xie, H., & Wang, F. L. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100417 · Cited by 151
A survey on knowledge-oriented retrieval-augmented generation
Cheng, M., Luo, Y., Ouyang, J., Liu, Q., Liu, H., Li, L., Yu, S., Zhang, B., Cao, J., Ma, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.10677 · Cited by 140
Towards a Unified Multi-Dimensional Evaluator for Text Generation
Zhong, M., Liu, Y., Yin, D., Mao, Y., Jiao, Y., Liu, P., Zhu, C., Ji, H., & Han, J. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.131 · Cited by 117
(2021) · doi:10.18653/v1/2021.naacl-main.46 · Cited by 115
Parallel Context Windows for Large Language Models
Ratner, N., Levine, Y., Belinkov, Y., Ram, O., Magar, I., Abend, O., Karpas, E., Shashua, A., Leyton-Brown, K., & Shoham, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.352 · Cited by 111
Simple Entity-Centric Questions Challenge Dense Retrievers
Sciavolino, C., Zhong, Z., Lee, J., & Chen, D. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.496 · Cited by 98
AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems
Zhang, J., Hou, Y., Xie, R., Sun, W., McAuley, J., Zhao, W. X., Lin, L., & Wen, J.-R. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645537 · Cited by 94
Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
Ahmed, T., Pai, K. S., Devanbu, P., & Barr, E. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639183 · Cited by 112
“BM25 [55] is a frequency-based retrieval method which improves upon TF-IDF [54].”
MethodImproving Passage Retrieval with Zero-Shot Question Generation
Sachan, D., Lewis, M., Joshi, M., Aghajanyan, A., Yih, W.-T., Pineau, J., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.249 · Cited by 90
Luo, H., E, H., Tang, Z., Peng, S., Guo, Y., Zhang, W., Ma, C., Dong, G., Song, M., Lin, W., et al. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.122 · Cited by 88
Demystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 849
The Web of False Information: Rumors, Fake News, Hoaxes, Clickbait, and Various Other Shenanigans
Zannettou, S., Sirivianos, M., Blackburn, J., & Kourtellis, N. (2019) · Journal of Data and Information Quality · doi:10.1145/3309699 · Cited by 86
FAST2: An intelligent assistant for finding relevant papers
Yu, Z., & Menzies, T. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.11.021 · Cited by 80
SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval
Li, H., Ai, Q., Chen, J., Dong, Q., Wu, Y., Liu, Y., Chen, C., & Tian, Q. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591761 · Cited by 79
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.16789 · Cited by 71
On the Risk of Misinformation Pollution with Large Language Models
Pan, Y., Pan, L., Chen, W., Nakov, P., Kan, M.-Y., & Wang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.97 · Cited by 210
Han, B., Susnjak, T., & Mathrani, A. (2024) · Applied Sciences · doi:10.3390/app14199103 · Cited by 68
GERE: Generative Evidence Retrieval for Fact Verification
Chen, J., Zhang, R., Guo, J., Fan, Y., & Cheng, X. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531827 · Cited by 66
SkCoder: A Sketch-based Approach for Automatic Code Generation
Li, J., Li, Y., Li, G., Jin, Z., Hao, Y., & Hu, X. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00179 · Cited by 64
Do neural ranking models intensify gender bias?
Rekabsaz, N., & Schedl, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401280 · Cited by 64
A large-scale dataset of patient summaries for retrieval-based clinical decision support systems
Zhao, Z., Jin, Q., Chen, F., Peng, T., & Yu, S. (2023) · Scientific Data · doi:10.1038/s41597-023-02814-8 · Cited by 63
MedExpQA: Multilingual benchmarking of Large Language Models for Medical Question Answering
Alonso, I., Oronoz, M., & Agerri, R. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102938 · Cited by 62
An agentic system for rare disease diagnosis with traceable reasoning
Zhao, W., Wu, C., Fan, Y., Qiu, P., Zhang, X., Sun, Y., Zhou, X., Zhang, S., Peng, Y., Wang, Y., et al. (2026) · Nature · doi:10.1038/s41586-025-10097-9 · Cited by 62
Training language models with memory augmentation
Zhong, Z., Lei, T., & Chen, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.382 · Cited by 60
Diverse Demonstrations Improve In-context Compositional Generalization
Levy, I., Bogin, B., & Berant, J. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.78 · Cited by 66
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Craswell, N., Mitra, B., Yilmaz, E., Campos, D., Voorhees, E. M., & Soboroff, I. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3463249 · Cited by 58
CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks
Chen, J., Zhang, R., Guo, J., Liu, Y., Fan, Y., & Cheng, X. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · doi:10.1145/3511808.3557271 · Cited by 57
MultiVerS: Improving scientific claim verification with weak supervision and full-document context
Wadden, D., Lo, K., Wang, L., Cohan, A., Beltagy, I., & Hajishirzi, H. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.6 · Cited by 60
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2310.06770 · Cited by 72
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
Wang, Q., Fu, Y., Cao, Y., Wang, S., Tian, Z., & Ding, L. (2023) · arXiv · doi:10.48550/arxiv.2308.15022 · Cited by 55
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
Abootorabi, M. M., Zobeiri, A., Dehghani, M., Mohammadkhani, M., Mohammadi, B., Ghahroodi, O., Baghshah, M. S., & Asgari, E. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.861 · Cited by 76
Wu, J., Wu, X., Qiu, Z., Li, M., Lin, S., Zhang, Y., Zheng, Y., Yuan, C., & Yang, J. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae079 · Cited by 41
Recursively summarizing enables long-term dialogue memory in large language models
Wang, Q., Fu, Y., Cao, Y., Wang, S., Tian, Z., & Ding, L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130193 · Cited by 55
PeerQA: A Scientific Question Answering Dataset from Peer Reviews
Baumgärtner, T., Briscoe, T., & Gurevych, I. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.22 · Cited by 39
Continual Learning for Generative Retrieval over Dynamic Corpora
Chen, J., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan, Y., & Cheng, X. (2023) · Proceedings of the 32nd ACM International Conference on Information and Knowledge Management · doi:10.1145/3583780.3614821 · Cited by 40
Miao, Y., Zhao, Y., Luo, Y., Wang, H., & Wu, Y. (2025) · Journal of Medical Internet Research · doi:10.2196/80557 · Cited by 38
Lin, J., Zhao, D., Lu, S., Li, R., Xu, X., Wang, Z., Li, W., Ji, Y., Zhang, C., Shi, L., et al. (2025) · ACS Nano · doi:10.1021/acsnano.5c04880 · Cited by 36
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., & Manning, C. D. (2024) · arXiv · doi:10.48550/arxiv.2401.18059 · Cited by 36
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking
Zhuang, S., Liu, B., Koopman, B., & Zuccon, G. (2023) · arXiv · doi:10.48550/arxiv.2310.13243 · Cited by 36
Neural Retrievers are Biased Towards LLM-Generated Content
Dai, S., Zhou, Y., Pang, L., Liu, W., Hu, X., Liu, Y., Zhang, X., Wang, G., & Xu, J. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671882 · Cited by 35
Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction
Ma, X., Guo, J., Zhang, R., Fan, Y., & Cheng, X. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531772 · Cited by 35
SciFact-Open: Towards open-domain scientific claim verification
Wadden, D., Lo, K., Kuehl, B., Cohan, A., Beltagy, I., Wang, L. L., & Hajishirzi, H. (2022) · arXiv · doi:10.48550/arxiv.2210.13777 · Cited by 35
Retrieval-Augmented Generation with Graphs (GraphRAG)
Han, H., Wang, Y., Shomer, H., Guo, K., Ding, J., Lei, Y., Halappanavar, M., Rossi, R. A., Mukherjee, S., Tang, X., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.00309 · Cited by 34
A Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning
Chen, J., Zhang, R., Guo, J., de Rijke, M., Liu, Y., Fan, Y., & Cheng, X. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591631 · Cited by 32
UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation
Cheng, D., Huang, S., Bi, J., Zhan, Y., Liu, J., Wang, Y., Sun, H., Wei, F., Deng, W., & Zhang, Q. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.758 · Cited by 32
Scalable and Effective Generative Information Retrieval
Zeng, H., Luo, C., Jin, B., Sarwar, S. M., Wei, T., & Zamani, H. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645477 · Cited by 32
SciFact-Open: Towards open-domain scientific claim verification
Wadden, D., Lo, K., Kuehl, B., Cohan, A., Beltagy, I., Wang, L. L., & Hajishirzi, H. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.347 · Cited by 35
RA-DIT: Retrieval-Augmented Dual Instruction Tuning
Lin, X. V., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01352 · Cited by 26
Adaptive Information Seeking for Open-Domain Question Answering
Zhu, Y., Pang, L., Lan, Y., Shen, H., & Cheng, X. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.293 · Cited by 24
Liang, J., Su, G., Lin, H., Wu, Y., Zhao, R., & Li, Z. (2025) · arXiv · doi:10.48550/arxiv.2506.10408 · Cited by 23
Yang, H., Li, Z., Zhang, Y., Wang, J., Cheng, N., Li, M., & Xiao, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.326 · Cited by 23
Zeng, H., Luo, C., & Zamani, H. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657746 · Cited by 22
“We also select the following competitive sparse and dense retrieval methods as points of reference: BM25 [57], docT5query [12], ANCE [70], ADORE [76], RocketQA [51], TCT-ColBERT [39], MarginMSE [23], TAS-B [25], and CL-DRD [74].”
MethodResultInfluentialTang, H., Key, D., & Ellis, K. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2243 · Cited by 20
Reinforced Prompt Personalization for Recommendation with Large Language Models
Mao, W., Wu, J., Chen, W., Gao, C., Wang, X., & He, X. (2025) · ACM Transactions on Information Systems · doi:10.1145/3716320 · Cited by 20
Agentless: Demystifying LLM-based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2024) · arXiv · doi:10.48550/arxiv.2407.01489 · Cited by 24
“In this case, the agentless baseline uses the LLM to directly generate a patch file by providing it with the file content of the most relevant files, retrieved using BM25 [80].”
MethodSrba, I., Razuvayevskaya, O., Leite, J. A., Moro, R., Schlicht, I. B., Tonelli, S., García, F. M., Lottmann, S. B., Teyssou, D., Porcellini, V., et al. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3770077 · Cited by 19
Yang, H., Li, Z., Zhang, Y., Wang, J., Cheng, N., Li, M., & Xiao, J. (2023) · arXiv · doi:10.48550/arxiv.2310.18347 · Cited by 18
“We conducted experiments with five different retrievers, specifically BM25 (Robertson and Zaragoza, 2009), SentenceBert (Reimers and Gurevych, 2019), DPR (Karpukhin et al., 2020), SimCSE (Gao et al., 2021), and Contriver (Izac-ard et al., 2022a).”
MethodActive Retrieval Augmented Generation
Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · arXiv · doi:10.48550/arxiv.2305.06983 · Cited by 18
Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions
Cao, S., Zhang, J., Shi, J., Lv, X., Yao, Z., Tian, Q., Hou, L., & Li, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.835 · Cited by 17
Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review
Kuznetsov, I., Buchmann, J., Eichler, M., & Gurevych, I. (2022) · Computational Linguistics · doi:10.1162/coli_a_00455 · Cited by 17
Automation of Legal Precedents Retrieval: Findings from a Literature Review
Mentzingen, H., António, N., & Bacao, F. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/6660983 · Cited by 16
Generative Retrieval as Multi-Vector Dense Retrieval
Wu, S., Wei, W., Zhang, M., Chen, Z., Ma, J., Ren, Z., de Rijke, M., & Ren, P. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657697 · Cited by 15
TOME: A Two-stage Approach for Model-based Retrieval
Ren, R., Zhao, W. X., Liu, J., Wu, H., Wen, J.-R., & Wang, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.336 · Cited by 14
Query Rewriting for Retrieval-Augmented Large Language Models
Ma, X., Gong, Y., He, P., Zhao, H., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2305.14283 · Cited by 14
CompAct: Compressing Retrieved Documents Actively for Question Answering
Yoon, C., Lee, T., Hwang, H., Jeong, M., & Kang, J. (2024) · arXiv · doi:10.48550/arxiv.2407.09014 · Cited by 14
Using In-Context Learning to Improve Dialogue Safety
Meade, N., Gella, S., Hazarika, D., Gupta, P., Jin, D., Reddy, S., Liu, Y., & Hakkani-Tur, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.796 · Cited by 13
Assessing citation integrity in biomedical publications: corpus annotation and NLP models
Sarol, M. J., Ming, S., Radhakrishna, S., Schneider, J., & Kilicoglu, H. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae420 · Cited by 12
“� BM25 þ MonoT5 reranker : To identify the most semantically relevant sentences, we leverage the simple BM25 retrieval model (Robertson and Zaragoza 2009) to obtain the top 60 sentences and the MonoT5 reranker (Nogueira et al. 2020) to rerank these sentences.”
MethodLEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration
Cao, Y., Gao, Z., Li, Z., Xie, X., Zhou, S. K., & Xu, J. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3748191.3748194 · Cited by 35
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
Chen, Z., Xiang, Z., Xiao, C., Song, D., & Li, B. (2024) · arXiv · doi:10.48550/arxiv.2407.12784 · Cited by 246
“…can be categorized into sparse retrievers (e.g. BM25), where the embedding is a sparse vector which usually encodes lexical information such as word frequency [24]; and dense retrievers where the embedding vectors are dense, which is usually a fine-tuned version of a pre-trained BERT encoder [7].”
Predicting retracted research: a dataset and machine learning approaches
Fletcher, A. H. A., & Stevenson, M. (2025) · Research Integrity and Peer Review · doi:10.1186/s41073-025-00168-w · Cited by 9
“All other text features were represented using term weights produced by Best Matching 25 [29] with k = 2 , b = 0.3 and no maximum vocabulary length specified.”
InfluentialReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96
“The simplest technique for retrieving code is to build a sparse vector retriever like TF-IDF or BM25 (Robertson and Zaragoza, 2009) which are both based on keyword matching algorithms.”
Method
Metadata
- Authors
- Robertson, S., & Zaragoza, H.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- Yahoo (Spain) (ES)
- ORCID
- Robertson, Stephen
- Venue
- Foundations and Trends® in Information Retrieval
- Publisher
- Emerald
- Published
- 2009-09-17
- Volume
- 4
- Issue
- 1-2
- Pages
- 1-174
- ISSN
- 1554-0669 (ISSN-L)
- 1554-0677
- OpenAlex
- W4252076394
- Cited by
- 3,374
- References
- 27
- Citations by year
- 2026: 737 · 2025: 605 · 2024: 389 · 2023: 432 · 2022: 282 · 2021: 229 · 2020: 152 · 2019: 120 · 2018: 118 · 2017: 74 · 2016: 79 · 2015: 50 · 2014: 29 · 2013: 36 · 2012: 22
- FWCI
- 36.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Foundations and Trends® in Information Retrieval (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Mathematics › Theoretical Computer Science › Algorithms and Data Compression
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- BM25, probabilistic retrieval, document structure, link graphs, parameter optimization, web search
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.